2024-04-20T21:00:24.628 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T21:00:24.628 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414 --name rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --verbose --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} -- /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/orig.config.yaml 2024-04-20T21:00:24.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T21:00:24.770 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414 branch: wip-rishabh-testing-20240416.193735 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: null first_in_suite: false job_id: '7665414' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - Replacing daemon mds sha1: a8c84f5997950fd7d008204839253291d52e6eda ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9QeV7h0MaaSwHaphBUDlmNRalRKbj5cvKLE3E71dxdL1fda695dWpZEM3fByLnalQsaIUmEX/oi2MOCo8wV68= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAnUap0WmMc8ZX95t3l0FWneNQgTBlAOPaCQ5F7fbon/6yzNVg802gspoF17WsVJGSrplIgSAwTBiqbCHfDUXYA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - mds_thrash: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_20:50:37 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T21:00:24.770 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T21:00:24.771 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T21:00:24.771 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T21:00:24.772 INFO:teuthology.task.internal:Checking packages... 2024-04-20T21:00:24.796 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T21:00:24.796 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T21:00:24.796 INFO:teuthology.packaging:ref: None 2024-04-20T21:00:24.796 INFO:teuthology.packaging:tag: None 2024-04-20T21:00:24.796 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T21:00:24.796 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:00:24.796 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T21:00:25.121 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T21:00:25.123 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T21:00:25.130 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T21:00:25.130 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T21:00:25.169 INFO:teuthology.task.internal:Saving configuration 2024-04-20T21:00:25.188 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T21:00:25.236 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:00:25.260 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:47.456135', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9QeV7h0MaaSwHaphBUDlmNRalRKbj5cvKLE3E71dxdL1fda695dWpZEM3fByLnalQsaIUmEX/oi2MOCo8wV68='} 2024-04-20T21:00:25.282 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:47.454169', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAnUap0WmMc8ZX95t3l0FWneNQgTBlAOPaCQ5F7fbon/6yzNVg802gspoF17WsVJGSrplIgSAwTBiqbCHfDUXYA='} 2024-04-20T21:00:25.282 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T21:00:25.288 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-20T21:00:25.288 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-20T21:00:25.288 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T21:00:25.373 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2352dfb160>, signals=[15]) 2024-04-20T21:00:25.373 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T21:00:25.379 INFO:teuthology.task.internal:Opening connections... 2024-04-20T21:00:25.379 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:00:25.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:00:25.454 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:00:25.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:00:25.528 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T21:00:25.534 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-04-20T21:00:25.552 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-04-20T21:00:25.552 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-04-20T21:00:25.609 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T21:00:25.610 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T21:00:25.610 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T21:00:25.610 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-04-20T21:00:25.629 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-04-20T21:00:25.647 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-04-20T21:00:25.647 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-04-20T21:00:25.704 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T21:00:25.705 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-04-20T21:00:25.705 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T21:00:25.705 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T21:00:25.705 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T21:00:25.706 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-04-20T21:00:25.731 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T21:00:25.742 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T21:00:25.761 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T21:00:25.761 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-04-20T21:00:25.763 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-04-20T21:00:25.780 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T21:00:25.785 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T21:00:25.785 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-04-20T21:00:25.819 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-04-20T21:00:25.927 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T21:00:25.982 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T21:00:25.996 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T21:00:25.996 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-20T21:00:25.996 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-20T21:00:25.997 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T21:00:26.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:00:26.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-20T21:00:26.176 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T21:00:26.176 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T21:00:26.176 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-04-20T21:00:26.193 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-437.el9.x86_64 2024-04-20T21:00:26.194 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T21:00:26.194 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T21:00:26.194 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum upgrade -y linux-firmware 2024-04-20T21:00:29.153 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-04-20T21:00:52.268 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 884 kB/s | 19 MB 00:22 2024-04-20T21:00:56.785 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.2 MB 00:00 2024-04-20T21:00:58.295 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-20T21:01:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 21 MB 00:14 2024-04-20T21:01:17.894 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-20T21:01:19.950 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-20T21:01:19.951 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-04-20T21:01:19.951 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-20T21:01:20.062 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T21:01:20.063 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T21:01:20.358 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T21:01:20.358 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T21:01:20.360 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi102.front.sepia.ceph.com to complete... 2024-04-20T21:07:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20 21:07:59 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T21:07:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:FINISHED --2024-04-20 21:07:59-- 2024-04-20T21:07:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:Total wall clock time: 6m 39s 2024-04-20T21:07:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:Downloaded: 1 files, 341M in 6m 38s (876 KB/s) 2024-04-20T21:07:59.194 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T21:07:59.195 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T21:08:01.281 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying... ######################################## 2024-04-20T21:08:01.442 INFO:teuthology.orchestra.run.smithi102.stdout:Preparing... ######################################## 2024-04-20T21:08:01.548 INFO:teuthology.orchestra.run.smithi102.stdout:Updating / installing... 2024-04-20T21:08:09.476 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T21:08:46.779 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T21:08:46.780 DEBUG:teuthology.orchestra.run.smithi102:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T21:08:46.856 INFO:teuthology.orchestra.run.smithi102.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T21:08:46.856 INFO:teuthology.orchestra.run.smithi102.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T21:08:46.857 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T21:08:46.858 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T21:08:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T21:08:46.860 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T21:08:46.861 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T21:08:46.862 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T21:08:46.863 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T21:08:46.864 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T21:08:46.865 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T21:08:46.866 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T21:08:46.867 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T21:08:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T21:08:46.869 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T21:08:46.870 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T21:08:46.871 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T21:08:46.872 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T21:08:46.873 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T21:08:46.874 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T21:08:46.875 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T21:08:46.876 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T21:08:46.877 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T21:08:46.878 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T21:08:46.879 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T21:08:46.880 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T21:08:46.881 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T21:08:46.882 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T21:08:46.883 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T21:08:46.884 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T21:08:46.885 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T21:08:46.886 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T21:08:46.887 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T21:08:46.887 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T21:08:46.887 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T21:08:46.889 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T21:08:46.890 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T21:08:46.891 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T21:08:46.892 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T21:08:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T21:08:46.894 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T21:08:46.895 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T21:08:46.896 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T21:08:46.897 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T21:08:46.898 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T21:08:46.899 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T21:08:46.900 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T21:08:46.901 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T21:08:46.902 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T21:08:46.903 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T21:08:46.904 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T21:08:46.905 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T21:08:46.906 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T21:08:46.907 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T21:08:46.908 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T21:08:46.909 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T21:08:46.910 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T21:08:46.911 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T21:08:46.912 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T21:08:46.913 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T21:08:46.914 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T21:08:46.915 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T21:08:46.916 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T21:08:46.917 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T21:08:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T21:08:46.919 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T21:08:46.920 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T21:08:46.921 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T21:08:46.922 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T21:08:46.923 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T21:08:46.924 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T21:08:46.925 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T21:08:46.926 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T21:08:46.927 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T21:08:46.928 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T21:08:46.929 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T21:08:46.930 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T21:08:46.931 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T21:08:46.932 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T21:08:46.933 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T21:08:46.934 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T21:08:46.935 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T21:08:46.936 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T21:08:46.937 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T21:08:46.938 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T21:08:46.939 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T21:08:46.940 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T21:08:46.941 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T21:08:46.942 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T21:08:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T21:08:46.944 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T21:08:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T21:08:46.946 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T21:08:46.947 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T21:08:46.948 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T21:08:46.949 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T21:08:46.950 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T21:08:46.951 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T21:08:46.952 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T21:08:46.953 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T21:08:46.954 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T21:08:46.955 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T21:08:46.956 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T21:08:46.957 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T21:08:46.958 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T21:08:46.959 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T21:08:46.960 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T21:08:46.961 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T21:08:46.962 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T21:08:46.963 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T21:08:46.964 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T21:08:46.965 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T21:08:46.966 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T21:08:46.967 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T21:08:46.968 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T21:08:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T21:08:46.970 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T21:08:46.971 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T21:08:46.972 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T21:08:46.973 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T21:08:46.974 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T21:08:46.975 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T21:08:46.976 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T21:08:46.977 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T21:08:46.978 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T21:08:46.979 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T21:08:46.980 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T21:08:46.981 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T21:08:46.982 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T21:08:46.983 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T21:08:46.984 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T21:08:46.985 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T21:08:46.986 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T21:08:46.987 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T21:08:46.988 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T21:08:46.989 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T21:08:46.990 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T21:08:46.991 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T21:08:46.992 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T21:08:46.993 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T21:08:46.994 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T21:08:46.995 INFO:teuthology.orchestra.run.smithi102.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:08:46.999 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T21:08:46.999 DEBUG:teuthology.orchestra.run.smithi102:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T21:08:47.021 INFO:teuthology.orchestra.run.smithi102.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T21:08:47.021 INFO:teuthology.orchestra.run.smithi102.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:08:47.021 INFO:teuthology.orchestra.run.smithi102.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout:fi 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout:for file in vmlinuz System.map config; do 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout: fi 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout:done 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout:fi 2024-04-20T21:08:47.022 INFO:teuthology.orchestra.run.smithi102.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T21:08:47.023 INFO:teuthology.orchestra.run.smithi102.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T21:08:47.023 INFO:teuthology.orchestra.run.smithi102.stdout:fi 2024-04-20T21:08:47.023 INFO:teuthology.orchestra.run.smithi102.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T21:08:47.023 INFO:teuthology.orchestra.run.smithi102.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:08:47.023 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T21:08:47.023 INFO:teuthology.orchestra.run.smithi102.stdout:fi 2024-04-20T21:08:47.024 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-04-20T21:08:47.105 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.06 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 68.el9 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Group : Unspecified 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 8264254 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T21:08:47.106 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : builder@centos.org 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:08:47.107 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T21:08:47.108 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T21:08:47.108 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T21:08:47.108 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T21:08:47.648 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-04-20T21:08:49.085 INFO:teuthology.orchestra.run.smithi102.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T21:08:49.118 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-04-20T21:08:49.121 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-04-20T21:08:49.161 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.F9FSV5RWWP 2024-04-20T21:08:49.161 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.F9FSV5RWWP 2024-04-20T21:08:49.227 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.F9FSV5RWWP 2024-04-20T21:08:49.373 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.F9FSV5RWWP is 6KB 2024-04-20T21:08:49.421 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.F9FSV5RWWP 2024-04-20T21:08:49.434 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T21:08:49.496 INFO:teuthology.orchestra.run.smithi102.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T21:08:49.496 INFO:teuthology.orchestra.run.smithi102.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T21:08:49.496 INFO:teuthology.orchestra.run.smithi102.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T21:08:49.496 INFO:teuthology.orchestra.run.smithi102.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T21:08:49.497 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T21:08:49.646 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-04-20T21:09:19.649 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-20T21:09:19.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-20T21:09:19.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:09:54.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-04-20T21:10:03.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-20T21:10:03.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:10:21.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-04-20T21:10:33.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-20T21:10:33.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:10:36.241 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-04-20T21:10:36.858 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-20T21:10:36.858 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T21:10:36.858 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-04-20T21:10:36.897 INFO:teuthology.orchestra.run.smithi102.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:36.897 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:36.897 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T21:10:36.897 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T21:10:37.901 DEBUG:teuthology.parallel:result is None 2024-04-20T21:10:37.901 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T21:10:37.908 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T21:10:37.909 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:10:37.918 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:10:37.957 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T21:10:37.971 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T21:10:38.011 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T21:10:38.012 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:10:38.016 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:10:38.107 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T21:10:38.114 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T21:10:38.114 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T21:10:38.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:10:38.130 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:10:38.137 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:10:38.218 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:10:38.234 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:10:38.280 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:10:38.330 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:10:38.334 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T21:10:38.342 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T21:10:38.342 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:10:38.346 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:10:38.422 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T21:10:38.438 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T21:10:38.438 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:10:38.445 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:10:38.484 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:10:38.596 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:10:38.670 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:10:38.736 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:10:38.809 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:10:38.809 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:10:38.902 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:10:38.944 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:10:39.000 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:10:39.085 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:10:39.126 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:10:39.126 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:10:39.191 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-04-20T21:10:39.196 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-04-20T21:10:39.271 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:10:39.301 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:10:39.565 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T21:10:39.574 INFO:teuthology.task.internal:Starting timer... 2024-04-20T21:10:39.574 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T21:10:39.615 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T21:10:39.647 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-20T21:10:39.647 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-04-20T21:10:39.764 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-04-20T21:10:39.768 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-04-20T21:10:39.869 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-04-20T21:10:39.871 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T21:10:39.871 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-04-20T21:10:39.928 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-04-20T21:10:39.929 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-04-20T21:10:39.959 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-04-20T21:10:39.960 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-04-20T21:10:39.960 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:10:40.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:10:40.039 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:10:40.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:10:40.064 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T21:10:40.064 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:10:40.200 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:10:40.225 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T21:10:40.234 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-20T21:10:40.240 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-20T21:10:40.392 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T21:10:40.407 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T21:10:40.409 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi102.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T21:12:57.379 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-04-20T21:12:57.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-20T21:12:57.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:12:57.489 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-04-20T21:12:57.637 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-20T21:12:57.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-04-20T21:12:57.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:12:57.715 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-04-20T21:12:57.748 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-04-20T21:12:57.748 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T21:12:57.759 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T21:12:57.760 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:12:57.760 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:12:57.768 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:12:57.768 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:12:57.816 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:12:57.829 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:12:57.852 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-04-20T21:12:57.853 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:12:57.862 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-04-20T21:12:57.875 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:12:57.888 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:12:57.895 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:12:57.943 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:12:57.946 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:12:57.947 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-04-20T21:12:57.947 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:12:57.947 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:12:57.947 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:12:57.947 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:12:57.966 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-04-20T21:12:57.999 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-04-20T21:12:58.042 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:12:58.081 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:12:58.219 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:12:58.228 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:12:58.228 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-20T21:12:58.228 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:12:58.228 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:12:58.228 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:12:58.228 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:12:58.229 INFO:teuthology.run_tasks:Running task install... 2024-04-20T21:12:58.238 DEBUG:teuthology.task.install:project ceph 2024-04-20T21:12:58.238 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T21:12:58.238 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T21:12:58.238 INFO:teuthology.task.install:Using flavor: default 2024-04-20T21:12:58.244 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T21:12:58.245 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T21:12:58.245 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:12:58.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:12:58.248 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:12:58.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:12:58.400 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:12:58.400 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:12:58.477 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:12:58.477 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:12:58.706 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:12:58.707 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:12:58.707 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:12:58.733 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:12:58.733 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:12:58.799 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:12:58.805 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:12:58.805 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:12:58.805 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:12:58.872 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:12:58.893 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:12:58.893 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:12:58.935 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-04-20T21:12:58.937 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-04-20T21:12:59.021 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:12:59.129 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:12:59.159 INFO:teuthology.orchestra.run.smithi193.stdout:48 files removed 2024-04-20T21:12:59.186 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:12:59.203 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-04-20T21:12:59.210 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-04-20T21:12:59.530 INFO:teuthology.orchestra.run.smithi102.stdout:48 files removed 2024-04-20T21:12:59.569 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:13:00.411 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 87 kB/s | 84 kB 00:00 2024-04-20T21:13:00.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 86 kB/s | 84 kB 00:00 2024-04-20T21:13:01.212 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 15 kB/s | 12 kB 00:00 2024-04-20T21:13:02.021 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-04-20T21:13:02.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 12 kB/s | 12 kB 00:00 2024-04-20T21:13:03.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 2.0 kB/s | 1.9 kB 00:00 2024-04-20T21:13:05.554 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.0 MB 00:03 2024-04-20T21:13:07.946 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-20T21:13:12.105 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-20T21:13:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 865 kB/s | 8.0 MB 00:09 2024-04-20T21:13:13.456 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 16 kB 00:00 2024-04-20T21:13:14.932 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-20T21:13:16.024 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 19 MB 00:02 2024-04-20T21:13:19.924 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-20T21:13:21.259 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-04-20T21:13:22.107 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:13:22.108 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:13:22.119 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:13:22.119 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:13:22.120 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:13:22.121 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:13:22.122 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:13:22.125 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:13:22.126 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:13:22.185 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:13:22.204 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:13:22.205 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:13:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:13:22.207 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:13:22.208 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:13:22.208 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:13:22.208 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:13:22.208 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:13:22.208 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:13:22.208 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:13:22.208 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:13:22.208 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:13:22.209 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:13:22.210 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:13:22.211 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:13:22.212 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:13:22.213 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:13:22.213 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:13:22.213 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:13:22.213 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:13:22.213 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:13:22.213 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:13:22.213 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:13:22.213 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:13:22.214 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:13:22.215 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:13:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout:Install 148 Packages 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 2 Packages 2024-04-20T21:13:22.217 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:13:22.219 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 218 M 2024-04-20T21:13:22.219 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-20T21:13:22.959 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-20T21:13:23.248 INFO:teuthology.orchestra.run.smithi193.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-20T21:13:24.419 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-20T21:13:25.321 INFO:teuthology.orchestra.run.smithi193.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 430 kB/s | 893 kB 00:02 2024-04-20T21:13:25.608 INFO:teuthology.orchestra.run.smithi193.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 497 kB/s | 145 kB 00:00 2024-04-20T21:13:28.300 INFO:teuthology.orchestra.run.smithi193.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 1.0 MB/s | 5.4 MB 00:05 2024-04-20T21:13:29.658 INFO:teuthology.orchestra.run.smithi193.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 597 kB/s | 2.4 MB 00:04 2024-04-20T21:13:30.259 INFO:teuthology.orchestra.run.smithi193.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 783 kB/s | 1.5 MB 00:01 2024-04-20T21:13:30.344 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-20T21:13:32.839 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:13:32.840 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:13:32.860 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:13:32.861 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:13:32.863 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:13:32.864 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:13:32.866 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:13:32.870 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:13:32.871 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:13:32.941 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-20T21:13:32.956 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:13:32.957 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:13:32.958 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:13:32.959 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:13:32.960 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:13:32.961 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:13:32.961 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:13:32.961 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:13:32.962 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:13:32.963 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:13:32.964 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:13:32.965 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:13:32.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:13:32.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:13:32.968 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout:Install 148 Packages 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 2 Packages 2024-04-20T21:13:32.969 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:13:32.970 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 218 M 2024-04-20T21:13:32.970 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-04-20T21:13:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-20T21:13:35.158 INFO:teuthology.orchestra.run.smithi193.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 875 kB/s | 4.7 MB 00:05 2024-04-20T21:13:35.174 INFO:teuthology.orchestra.run.smithi102.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.0 MB/s | 893 kB 00:00 2024-04-20T21:13:35.374 INFO:teuthology.orchestra.run.smithi102.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 728 kB/s | 145 kB 00:00 2024-04-20T21:13:36.450 INFO:teuthology.orchestra.run.smithi102.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-04-20T21:13:37.092 INFO:teuthology.orchestra.run.smithi102.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-04-20T21:13:38.967 INFO:teuthology.orchestra.run.smithi102.stdout:(6/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-04-20T21:13:42.008 INFO:teuthology.orchestra.run.smithi102.stdout:(7/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 679 kB/s | 5.4 MB 00:08 2024-04-20T21:13:48.174 INFO:teuthology.orchestra.run.smithi193.stdout:(8/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 962 kB/s | 17 MB 00:17 2024-04-20T21:13:48.299 INFO:teuthology.orchestra.run.smithi193.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 202 kB/s | 25 kB 00:00 2024-04-20T21:13:54.591 INFO:teuthology.orchestra.run.smithi193.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 558 kB/s | 11 MB 00:19 2024-04-20T21:13:54.966 INFO:teuthology.orchestra.run.smithi102.stdout:(8/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 1.1 MB/s | 17 MB 00:15 2024-04-20T21:13:55.032 INFO:teuthology.orchestra.run.smithi193.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 509 kB/s | 225 kB 00:00 2024-04-20T21:13:55.140 INFO:teuthology.orchestra.run.smithi193.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 249 kB/s | 27 kB 00:00 2024-04-20T21:13:55.241 INFO:teuthology.orchestra.run.smithi193.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 317 kB/s | 32 kB 00:00 2024-04-20T21:13:55.250 INFO:teuthology.orchestra.run.smithi102.stdout:(9/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 1.0 MB/s | 21 MB 00:21 2024-04-20T21:13:55.274 INFO:teuthology.orchestra.run.smithi102.stdout:(10/150): ceph-selinux-19.0.0-3133.ga8c84f59.el 82 kB/s | 25 kB 00:00 2024-04-20T21:13:55.449 INFO:teuthology.orchestra.run.smithi102.stdout:(11/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 807 kB/s | 11 MB 00:13 2024-04-20T21:13:55.559 INFO:teuthology.orchestra.run.smithi102.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 251 kB/s | 27 kB 00:00 2024-04-20T21:13:55.655 INFO:teuthology.orchestra.run.smithi102.stdout:(13/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 592 kB/s | 225 kB 00:00 2024-04-20T21:13:55.675 INFO:teuthology.orchestra.run.smithi102.stdout:(14/150): libcephfs-devel-19.0.0-3133.ga8c84f59 272 kB/s | 32 kB 00:00 2024-04-20T21:13:55.984 INFO:teuthology.orchestra.run.smithi102.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 531 kB/s | 163 kB 00:00 2024-04-20T21:13:56.193 INFO:teuthology.orchestra.run.smithi102.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 609 kB/s | 126 kB 00:00 2024-04-20T21:13:56.199 INFO:teuthology.orchestra.run.smithi193.stdout:(14/150): ceph-common-19.0.0-3133.ga8c84f59.el9 648 kB/s | 21 MB 00:33 2024-04-20T21:13:56.271 INFO:teuthology.orchestra.run.smithi102.stdout:(17/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 1.2 MB/s | 748 kB 00:00 2024-04-20T21:13:56.499 INFO:teuthology.orchestra.run.smithi193.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 544 kB/s | 163 kB 00:00 2024-04-20T21:13:56.558 INFO:teuthology.orchestra.run.smithi193.stdout:(16/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 568 kB/s | 748 kB 00:01 2024-04-20T21:13:56.700 INFO:teuthology.orchestra.run.smithi193.stdout:(17/150): librados-devel-19.0.0-3133.ga8c84f59. 632 kB/s | 126 kB 00:00 2024-04-20T21:13:56.843 INFO:teuthology.orchestra.run.smithi102.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 771 kB/s | 501 kB 00:00 2024-04-20T21:13:56.993 INFO:teuthology.orchestra.run.smithi102.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 302 kB/s | 45 kB 00:00 2024-04-20T21:13:57.202 INFO:teuthology.orchestra.run.smithi102.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 649 kB/s | 135 kB 00:00 2024-04-20T21:13:57.333 INFO:teuthology.orchestra.run.smithi193.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 646 kB/s | 501 kB 00:00 2024-04-20T21:13:57.452 INFO:teuthology.orchestra.run.smithi102.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 650 kB/s | 162 kB 00:00 2024-04-20T21:13:57.458 INFO:teuthology.orchestra.run.smithi193.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 361 kB/s | 45 kB 00:00 2024-04-20T21:13:57.750 INFO:teuthology.orchestra.run.smithi193.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 462 kB/s | 135 kB 00:00 2024-04-20T21:13:57.852 INFO:teuthology.orchestra.run.smithi102.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 804 kB/s | 321 kB 00:00 2024-04-20T21:13:58.051 INFO:teuthology.orchestra.run.smithi193.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 540 kB/s | 162 kB 00:00 2024-04-20T21:13:58.228 INFO:teuthology.orchestra.run.smithi102.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 798 kB/s | 299 kB 00:00 2024-04-20T21:13:58.428 INFO:teuthology.orchestra.run.smithi102.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 499 kB/s | 99 kB 00:00 2024-04-20T21:13:58.535 INFO:teuthology.orchestra.run.smithi193.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 663 kB/s | 321 kB 00:00 2024-04-20T21:13:58.536 INFO:teuthology.orchestra.run.smithi102.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 790 kB/s | 85 kB 00:00 2024-04-20T21:13:59.010 INFO:teuthology.orchestra.run.smithi193.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 630 kB/s | 299 kB 00:00 2024-04-20T21:13:59.202 INFO:teuthology.orchestra.run.smithi193.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 518 kB/s | 99 kB 00:00 2024-04-20T21:13:59.410 INFO:teuthology.orchestra.run.smithi193.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 408 kB/s | 85 kB 00:00 2024-04-20T21:14:01.187 INFO:teuthology.orchestra.run.smithi102.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 1.1 MB/s | 5.4 MB 00:04 2024-04-20T21:14:01.437 INFO:teuthology.orchestra.run.smithi102.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 685 kB/s | 170 kB 00:00 2024-04-20T21:14:01.529 INFO:teuthology.orchestra.run.smithi102.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 375 kB/s | 34 kB 00:00 2024-04-20T21:14:01.696 INFO:teuthology.orchestra.run.smithi102.stdout:(29/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 1.0 MB/s | 3.1 MB 00:03 2024-04-20T21:14:01.721 INFO:teuthology.orchestra.run.smithi102.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 744 kB/s | 142 kB 00:00 2024-04-20T21:14:02.919 INFO:teuthology.orchestra.run.smithi193.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 885 kB/s | 5.4 MB 00:06 2024-04-20T21:14:03.136 INFO:teuthology.orchestra.run.smithi193.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 787 kB/s | 170 kB 00:00 2024-04-20T21:14:03.254 INFO:teuthology.orchestra.run.smithi193.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 294 kB/s | 34 kB 00:00 2024-04-20T21:14:03.586 INFO:teuthology.orchestra.run.smithi193.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 427 kB/s | 142 kB 00:00 2024-04-20T21:14:04.370 INFO:teuthology.orchestra.run.smithi193.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 638 kB/s | 3.1 MB 00:04 2024-04-20T21:14:04.597 INFO:teuthology.orchestra.run.smithi102.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 1.2 MB/s | 3.5 MB 00:02 2024-04-20T21:14:04.805 INFO:teuthology.orchestra.run.smithi102.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 1.2 MB/s | 249 kB 00:00 2024-04-20T21:14:04.947 INFO:teuthology.orchestra.run.smithi102.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 350 kB/s | 49 kB 00:00 2024-04-20T21:14:05.081 INFO:teuthology.orchestra.run.smithi102.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 124 kB/s | 16 kB 00:00 2024-04-20T21:14:05.306 INFO:teuthology.orchestra.run.smithi102.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.2 MB/s | 273 kB 00:00 2024-04-20T21:14:05.881 INFO:teuthology.orchestra.run.smithi102.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 1.3 MB/s | 763 kB 00:00 2024-04-20T21:14:06.019 INFO:teuthology.orchestra.run.smithi102.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 137 kB/s | 18 kB 00:00 2024-04-20T21:14:06.349 INFO:teuthology.orchestra.run.smithi102.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 242 kB/s | 80 kB 00:00 2024-04-20T21:14:06.866 INFO:teuthology.orchestra.run.smithi102.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-04-20T21:14:07.250 INFO:teuthology.orchestra.run.smithi102.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 511 kB/s | 195 kB 00:00 2024-04-20T21:14:07.474 INFO:teuthology.orchestra.run.smithi102.stdout:(41/150): ceph-mgr-diskprediction-local-19.0.0- 1.3 MB/s | 7.4 MB 00:05 2024-04-20T21:14:07.774 INFO:teuthology.orchestra.run.smithi102.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 483 kB/s | 253 kB 00:00 2024-04-20T21:14:07.992 INFO:teuthology.orchestra.run.smithi102.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-04-20T21:14:08.141 INFO:teuthology.orchestra.run.smithi102.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 292 kB/s | 106 kB 00:00 2024-04-20T21:14:08.286 INFO:teuthology.orchestra.run.smithi193.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 771 kB/s | 3.5 MB 00:04 2024-04-20T21:14:08.409 INFO:teuthology.orchestra.run.smithi102.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 325 kB/s | 135 kB 00:00 2024-04-20T21:14:08.559 INFO:teuthology.orchestra.run.smithi102.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 300 kB/s | 125 kB 00:00 2024-04-20T21:14:08.595 INFO:teuthology.orchestra.run.smithi193.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 807 kB/s | 249 kB 00:00 2024-04-20T21:14:08.712 INFO:teuthology.orchestra.run.smithi193.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 423 kB/s | 49 kB 00:00 2024-04-20T21:14:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 3.3 MB/s | 689 kB 00:00 2024-04-20T21:14:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:(48/150): python3-urllib3-1.26.5-5.el9.noarch.r 549 kB/s | 215 kB 00:00 2024-04-20T21:14:08.812 INFO:teuthology.orchestra.run.smithi193.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 165 kB/s | 16 kB 00:00 2024-04-20T21:14:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-20T21:14:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 442 kB/s | 21 kB 00:00 2024-04-20T21:14:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-20T21:14:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:(52/150): boost-program-options-1.75.0-8.el9.x8 559 kB/s | 107 kB 00:00 2024-04-20T21:14:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 660 kB/s | 33 kB 00:00 2024-04-20T21:14:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 245 kB/s | 18 kB 00:00 2024-04-20T21:14:09.145 INFO:teuthology.orchestra.run.smithi102.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 226 kB/s | 11 kB 00:00 2024-04-20T21:14:09.146 INFO:teuthology.orchestra.run.smithi193.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 819 kB/s | 273 kB 00:00 2024-04-20T21:14:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-20T21:14:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-20T21:14:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-04-20T21:14:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 662 kB 00:00 2024-04-20T21:14:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-04-20T21:14:09.406 INFO:teuthology.orchestra.run.smithi102.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 342 kB/s | 25 kB 00:00 2024-04-20T21:14:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 959 kB/s | 47 kB 00:00 2024-04-20T21:14:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-20T21:14:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-20T21:14:09.523 INFO:teuthology.orchestra.run.smithi102.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-04-20T21:14:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-20T21:14:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 16 MB/s | 4.1 MB 00:00 2024-04-20T21:14:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-04-20T21:14:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-04-20T21:14:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.8 MB/s | 133 kB 00:00 2024-04-20T21:14:09.971 INFO:teuthology.orchestra.run.smithi193.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 925 kB/s | 763 kB 00:00 2024-04-20T21:14:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-20T21:14:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-20T21:14:10.071 INFO:teuthology.orchestra.run.smithi193.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 183 kB/s | 18 kB 00:00 2024-04-20T21:14:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-20T21:14:10.196 INFO:teuthology.orchestra.run.smithi193.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 643 kB/s | 80 kB 00:00 2024-04-20T21:14:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.2 MB/s | 244 kB 00:00 2024-04-20T21:14:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-20T21:14:10.313 INFO:teuthology.orchestra.run.smithi193.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-20T21:14:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 521 kB/s | 26 kB 00:00 2024-04-20T21:14:10.364 INFO:teuthology.orchestra.run.smithi193.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-20T21:14:10.414 INFO:teuthology.orchestra.run.smithi193.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-20T21:14:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:(77/150): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-20T21:14:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 136 kB/s | 19 kB 00:00 2024-04-20T21:14:10.498 INFO:teuthology.orchestra.run.smithi193.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-20T21:14:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-20T21:14:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 602 kB/s | 35 kB 00:00 2024-04-20T21:14:10.548 INFO:teuthology.orchestra.run.smithi193.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-20T21:14:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-20T21:14:10.614 INFO:teuthology.orchestra.run.smithi193.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-20T21:14:10.665 INFO:teuthology.orchestra.run.smithi193.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-20T21:14:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-20T21:14:10.715 INFO:teuthology.orchestra.run.smithi193.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-20T21:14:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:(83/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-20T21:14:10.795 INFO:teuthology.orchestra.run.smithi102.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 619 kB/s | 77 kB 00:00 2024-04-20T21:14:10.821 INFO:teuthology.orchestra.run.smithi102.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 854 kB/s | 42 kB 00:00 2024-04-20T21:14:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-04-20T21:14:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-20T21:14:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.8 MB/s | 279 kB 00:00 2024-04-20T21:14:10.957 INFO:teuthology.orchestra.run.smithi193.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-04-20T21:14:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 772 kB/s | 51 kB 00:00 2024-04-20T21:14:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 927 kB/s | 54 kB 00:00 2024-04-20T21:14:11.072 INFO:teuthology.orchestra.run.smithi102.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 724 kB/s | 42 kB 00:00 2024-04-20T21:14:11.074 INFO:teuthology.orchestra.run.smithi193.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-04-20T21:14:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 768 kB/s | 44 kB 00:00 2024-04-20T21:14:11.141 INFO:teuthology.orchestra.run.smithi193.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-20T21:14:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-20T21:14:11.208 INFO:teuthology.orchestra.run.smithi193.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-04-20T21:14:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 732 kB/s | 36 kB 00:00 2024-04-20T21:14:11.276 INFO:teuthology.orchestra.run.smithi193.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-20T21:14:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 965 kB/s | 47 kB 00:00 2024-04-20T21:14:11.342 INFO:teuthology.orchestra.run.smithi193.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-04-20T21:14:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-20T21:14:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 58 kB 00:00 2024-04-20T21:14:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-20T21:14:11.468 INFO:teuthology.orchestra.run.smithi193.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-20T21:14:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-20T21:14:11.526 INFO:teuthology.orchestra.run.smithi193.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-20T21:14:11.586 INFO:teuthology.orchestra.run.smithi193.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-20T21:14:11.652 INFO:teuthology.orchestra.run.smithi193.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-20T21:14:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:(100/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-20T21:14:11.727 INFO:teuthology.orchestra.run.smithi193.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-04-20T21:14:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 192 kB/s | 45 kB 00:00 2024-04-20T21:14:11.802 INFO:teuthology.orchestra.run.smithi193.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-04-20T21:14:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 193 kB/s | 22 kB 00:00 2024-04-20T21:14:11.869 INFO:teuthology.orchestra.run.smithi193.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-20T21:14:11.936 INFO:teuthology.orchestra.run.smithi193.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 382 kB/s | 25 kB 00:00 2024-04-20T21:14:12.003 INFO:teuthology.orchestra.run.smithi193.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-04-20T21:14:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-04-20T21:14:12.070 INFO:teuthology.orchestra.run.smithi193.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-20T21:14:12.137 INFO:teuthology.orchestra.run.smithi193.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-20T21:14:12.212 INFO:teuthology.orchestra.run.smithi193.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 471 kB/s | 35 kB 00:00 2024-04-20T21:14:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-20T21:14:12.396 INFO:teuthology.orchestra.run.smithi193.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-20T21:14:12.443 INFO:teuthology.orchestra.run.smithi102.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 236 kB/s | 19 kB 00:00 2024-04-20T21:14:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 632 kB/s | 309 kB 00:00 2024-04-20T21:14:12.547 INFO:teuthology.orchestra.run.smithi193.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 28 MB/s | 4.1 MB 00:00 2024-04-20T21:14:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 286 kB/s | 26 kB 00:00 2024-04-20T21:14:12.630 INFO:teuthology.orchestra.run.smithi193.stdout:(67/150): ceph-mgr-diskprediction-local-19.0.0- 918 kB/s | 7.4 MB 00:08 2024-04-20T21:14:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 534 kB/s | 49 kB 00:00 2024-04-20T21:14:12.705 INFO:teuthology.orchestra.run.smithi193.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-20T21:14:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:(109/150): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-20T21:14:12.831 INFO:teuthology.orchestra.run.smithi193.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-20T21:14:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 405 kB/s | 67 kB 00:00 2024-04-20T21:14:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-20T21:14:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-20T21:14:13.014 INFO:teuthology.orchestra.run.smithi193.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 32 MB/s | 5.8 MB 00:00 2024-04-20T21:14:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-04-20T21:14:13.056 INFO:teuthology.orchestra.run.smithi193.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 2.1 MB/s | 911 kB 00:00 2024-04-20T21:14:13.071 INFO:teuthology.orchestra.run.smithi102.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 659 kB/s | 43 kB 00:00 2024-04-20T21:14:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 485 kB/s | 32 kB 00:00 2024-04-20T21:14:13.123 INFO:teuthology.orchestra.run.smithi193.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-20T21:14:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 226 kB/s | 15 kB 00:00 2024-04-20T21:14:13.148 INFO:teuthology.orchestra.run.smithi193.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 952 kB/s | 87 kB 00:00 2024-04-20T21:14:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-04-20T21:14:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-20T21:14:13.249 INFO:teuthology.orchestra.run.smithi193.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-20T21:14:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-04-20T21:14:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 473 kB/s | 35 kB 00:00 2024-04-20T21:14:13.324 INFO:teuthology.orchestra.run.smithi193.stdout:(75/150): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-20T21:14:13.364 INFO:teuthology.orchestra.run.smithi102.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-20T21:14:13.365 INFO:teuthology.orchestra.run.smithi193.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-20T21:14:13.391 INFO:teuthology.orchestra.run.smithi193.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 384 kB/s | 26 kB 00:00 2024-04-20T21:14:13.434 INFO:teuthology.orchestra.run.smithi193.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 285 kB/s | 19 kB 00:00 2024-04-20T21:14:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-20T21:14:13.478 INFO:teuthology.orchestra.run.smithi193.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-20T21:14:13.501 INFO:teuthology.orchestra.run.smithi193.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-04-20T21:14:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-20T21:14:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-20T21:14:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2024-04-20T21:14:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 310 kB/s | 23 kB 00:00 2024-04-20T21:14:13.634 INFO:teuthology.orchestra.run.smithi193.stdout:(81/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-20T21:14:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2024-04-20T21:14:13.659 INFO:teuthology.orchestra.run.smithi193.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-04-20T21:14:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2024-04-20T21:14:13.710 INFO:teuthology.orchestra.run.smithi193.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-20T21:14:13.735 INFO:teuthology.orchestra.run.smithi193.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-20T21:14:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 9.5 MB/s | 1.0 MB 00:00 2024-04-20T21:14:13.777 INFO:teuthology.orchestra.run.smithi193.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 630 kB/s | 42 kB 00:00 2024-04-20T21:14:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 429 kB/s | 46 kB 00:00 2024-04-20T21:14:13.811 INFO:teuthology.orchestra.run.smithi193.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-20T21:14:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-20T21:14:13.861 INFO:teuthology.orchestra.run.smithi193.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-20T21:14:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 774 kB/s | 58 kB 00:00 2024-04-20T21:14:13.911 INFO:teuthology.orchestra.run.smithi193.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-20T21:14:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-20T21:14:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-20T21:14:13.961 INFO:teuthology.orchestra.run.smithi193.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-04-20T21:14:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-20T21:14:14.003 INFO:teuthology.orchestra.run.smithi193.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 583 kB/s | 54 kB 00:00 2024-04-20T21:14:14.027 INFO:teuthology.orchestra.run.smithi102.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 378 kB/s | 28 kB 00:00 2024-04-20T21:14:14.077 INFO:teuthology.orchestra.run.smithi102.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 413 kB/s | 31 kB 00:00 2024-04-20T21:14:14.095 INFO:teuthology.orchestra.run.smithi193.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 456 kB/s | 42 kB 00:00 2024-04-20T21:14:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-20T21:14:14.164 INFO:teuthology.orchestra.run.smithi102.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-04-20T21:14:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 477 kB/s | 36 kB 00:00 2024-04-20T21:14:14.195 INFO:teuthology.orchestra.run.smithi193.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-04-20T21:14:14.237 INFO:teuthology.orchestra.run.smithi102.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 687 kB/s | 51 kB 00:00 2024-04-20T21:14:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-20T21:14:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-20T21:14:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-20T21:14:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 302 kB/s | 20 kB 00:00 2024-04-20T21:14:14.404 INFO:teuthology.orchestra.run.smithi193.stdout:(93/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-20T21:14:14.429 INFO:teuthology.orchestra.run.smithi193.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 945 kB/s | 221 kB 00:00 2024-04-20T21:14:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-20T21:14:14.471 INFO:teuthology.orchestra.run.smithi193.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-04-20T21:14:14.499 INFO:teuthology.orchestra.run.smithi193.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-04-20T21:14:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-20T21:14:14.547 INFO:teuthology.orchestra.run.smithi193.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-20T21:14:14.572 INFO:teuthology.orchestra.run.smithi193.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 769 kB/s | 58 kB 00:00 2024-04-20T21:14:14.614 INFO:teuthology.orchestra.run.smithi193.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-20T21:14:14.639 INFO:teuthology.orchestra.run.smithi193.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 953 kB/s | 64 kB 00:00 2024-04-20T21:14:14.681 INFO:teuthology.orchestra.run.smithi193.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-04-20T21:14:14.798 INFO:teuthology.orchestra.run.smithi193.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-04-20T21:14:14.965 INFO:teuthology.orchestra.run.smithi193.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-04-20T21:14:15.124 INFO:teuthology.orchestra.run.smithi193.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-20T21:14:15.174 INFO:teuthology.orchestra.run.smithi193.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-04-20T21:14:15.258 INFO:teuthology.orchestra.run.smithi193.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-20T21:14:15.310 INFO:teuthology.orchestra.run.smithi193.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-04-20T21:14:15.359 INFO:teuthology.orchestra.run.smithi193.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-04-20T21:14:15.434 INFO:teuthology.orchestra.run.smithi193.stdout:(109/150): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-20T21:14:15.459 INFO:teuthology.orchestra.run.smithi193.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-04-20T21:14:15.493 INFO:teuthology.orchestra.run.smithi193.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-20T21:14:15.584 INFO:teuthology.orchestra.run.smithi193.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-04-20T21:14:15.627 INFO:teuthology.orchestra.run.smithi193.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-20T21:14:15.652 INFO:teuthology.orchestra.run.smithi193.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-04-20T21:14:15.685 INFO:teuthology.orchestra.run.smithi193.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-04-20T21:14:15.711 INFO:teuthology.orchestra.run.smithi193.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-20T21:14:15.744 INFO:teuthology.orchestra.run.smithi193.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-20T21:14:15.778 INFO:teuthology.orchestra.run.smithi193.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-20T21:14:15.812 INFO:teuthology.orchestra.run.smithi193.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-20T21:14:15.837 INFO:teuthology.orchestra.run.smithi193.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 601 kB/s | 35 kB 00:00 2024-04-20T21:14:15.870 INFO:teuthology.orchestra.run.smithi193.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-20T21:14:15.929 INFO:teuthology.orchestra.run.smithi193.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-20T21:14:15.988 INFO:teuthology.orchestra.run.smithi193.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-20T21:14:16.013 INFO:teuthology.orchestra.run.smithi193.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-04-20T21:14:16.047 INFO:teuthology.orchestra.run.smithi193.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-04-20T21:14:16.072 INFO:teuthology.orchestra.run.smithi193.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-04-20T21:14:16.105 INFO:teuthology.orchestra.run.smithi193.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-20T21:14:16.131 INFO:teuthology.orchestra.run.smithi193.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-20T21:14:16.181 INFO:teuthology.orchestra.run.smithi193.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-20T21:14:16.206 INFO:teuthology.orchestra.run.smithi193.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-04-20T21:14:16.249 INFO:teuthology.orchestra.run.smithi193.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-20T21:14:16.274 INFO:teuthology.orchestra.run.smithi193.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-04-20T21:14:16.307 INFO:teuthology.orchestra.run.smithi193.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-20T21:14:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:(148/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 1.7 MB/s | 3.1 MB 00:01 2024-04-20T21:14:16.332 INFO:teuthology.orchestra.run.smithi193.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-20T21:14:16.366 INFO:teuthology.orchestra.run.smithi193.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-20T21:14:16.392 INFO:teuthology.orchestra.run.smithi193.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-04-20T21:14:16.436 INFO:teuthology.orchestra.run.smithi193.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-20T21:14:16.459 INFO:teuthology.orchestra.run.smithi193.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-04-20T21:14:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 1.7 MB/s | 3.4 MB 00:02 2024-04-20T21:14:16.492 INFO:teuthology.orchestra.run.smithi193.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-20T21:14:16.518 INFO:teuthology.orchestra.run.smithi193.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-04-20T21:14:16.551 INFO:teuthology.orchestra.run.smithi193.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 875 kB/s | 51 kB 00:00 2024-04-20T21:14:16.576 INFO:teuthology.orchestra.run.smithi193.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-20T21:14:16.610 INFO:teuthology.orchestra.run.smithi193.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-20T21:14:16.644 INFO:teuthology.orchestra.run.smithi193.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-20T21:14:16.669 INFO:teuthology.orchestra.run.smithi193.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-04-20T21:14:16.711 INFO:teuthology.orchestra.run.smithi193.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-20T21:14:16.761 INFO:teuthology.orchestra.run.smithi193.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-20T21:14:20.786 INFO:teuthology.orchestra.run.smithi193.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 853 kB/s | 3.4 MB 00:04 2024-04-20T21:14:20.836 INFO:teuthology.orchestra.run.smithi193.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 789 kB/s | 3.1 MB 00:04 2024-04-20T21:14:33.335 INFO:teuthology.orchestra.run.smithi193.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 1.1 MB/s | 49 MB 00:45 2024-04-20T21:14:33.341 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:14:33.341 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.1 MB/s | 218 MB 01:11 2024-04-20T21:14:34.382 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-20T21:14:34.509 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-20T21:14:34.509 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-20T21:14:36.217 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-20T21:14:36.218 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-20T21:14:37.129 INFO:teuthology.orchestra.run.smithi102.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 1.2 MB/s | 49 MB 00:41 2024-04-20T21:14:37.141 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:14:37.141 INFO:teuthology.orchestra.run.smithi102.stdout:Total 3.4 MB/s | 218 MB 01:04 2024-04-20T21:14:39.827 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-04-20T21:14:40.031 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-04-20T21:14:40.032 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-04-20T21:14:42.093 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-20T21:14:42.170 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:14:42.212 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:14:42.254 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:14:42.593 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:14:42.678 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-04-20T21:14:42.679 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-04-20T21:14:42.704 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:14:42.834 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:14:42.864 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:14:42.955 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:14:42.998 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:14:43.027 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:14:43.052 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:14:43.207 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:14:43.258 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:14:43.285 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:14:43.373 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:14:43.395 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:14:43.441 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:14:43.462 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:14:43.494 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:14:43.783 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:14:43.814 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:14:43.927 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:14:44.029 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:14:44.063 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:14:44.099 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:14:44.173 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:14:44.214 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:14:44.258 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:14:44.298 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:14:44.326 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:14:44.396 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:14:44.466 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:14:44.495 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:14:44.539 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:14:44.640 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:14:44.683 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:14:44.727 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:14:44.755 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:14:44.827 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:14:44.878 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:14:44.953 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:14:45.097 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:14:45.147 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:14:45.692 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:14:45.785 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:14:45.813 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:14:45.856 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:14:45.894 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:14:45.927 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:14:45.982 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:14:46.055 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:14:46.086 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:14:46.122 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:14:46.169 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:14:46.205 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:14:46.248 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:14:46.299 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:14:46.437 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:14:46.487 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:14:46.517 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:14:46.559 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:14:46.654 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:14:46.692 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:14:46.732 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:14:46.758 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:14:46.782 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:14:46.850 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:14:46.977 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:14:47.015 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:14:47.042 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:14:47.077 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:14:47.120 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:14:47.158 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:14:47.190 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:14:47.219 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:14:47.246 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:14:47.271 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:14:47.312 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:14:47.347 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:14:47.435 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:14:47.485 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:14:47.517 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:14:47.564 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:14:47.666 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:14:47.718 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:14:48.453 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:14:48.506 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:14:48.540 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:14:48.641 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:14:48.707 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:14:48.795 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:14:48.820 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-04-20T21:14:48.833 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:14:48.904 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:14:48.942 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:14:49.011 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:14:49.048 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:14:49.073 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:14:49.126 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:14:49.484 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:14:49.599 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:14:49.749 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:14:49.773 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:14:49.783 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:14:49.890 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:14:49.943 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:14:50.023 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:14:50.058 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:14:50.094 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:14:50.486 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:14:50.557 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:14:50.589 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:14:50.788 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:14:50.815 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:14:50.956 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:14:50.982 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:14:51.054 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:14:51.142 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:14:51.226 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:14:51.254 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:14:51.287 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:14:51.499 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:14:51.557 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:14:51.602 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:14:51.642 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:14:51.716 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:14:51.746 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:14:51.754 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:14:51.782 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:14:51.884 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:14:51.933 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:14:51.988 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:14:52.097 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:14:52.146 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:14:52.217 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:14:52.266 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:14:52.278 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:14:52.302 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:14:52.317 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:14:52.374 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:14:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:14:52.436 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:14:52.507 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:14:52.543 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:14:52.599 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:14:52.708 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:14:52.746 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:14:52.785 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:14:52.819 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:14:52.933 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:14:53.019 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:14:53.127 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:14:53.325 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:14:53.385 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:14:54.107 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:14:54.218 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:14:54.267 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:14:54.322 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:14:54.364 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:14:54.415 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:14:54.511 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:14:54.608 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:14:54.651 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:14:54.668 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:14:54.691 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:14:54.709 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:14:54.758 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:14:54.815 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:14:54.879 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:14:54.945 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:14:55.155 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:14:55.205 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:14:55.242 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:14:55.261 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:14:55.300 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:14:55.303 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:14:55.400 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:14:55.400 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:14:55.400 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:14:55.400 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:14:55.429 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:14:55.442 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:14:55.483 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:14:55.536 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:14:55.574 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:14:55.607 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:14:55.818 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:14:55.998 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:14:56.041 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:14:56.081 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:14:56.138 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:14:56.230 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:14:56.288 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:14:56.336 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:14:56.377 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:14:56.423 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:14:56.472 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:14:56.539 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:14:56.606 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:14:56.753 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:14:56.811 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:14:56.863 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:14:56.935 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:14:57.083 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:14:57.161 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:14:58.145 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:14:58.201 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:14:58.234 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:14:58.361 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:14:58.429 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:14:58.536 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:14:58.578 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:14:58.808 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:14:58.841 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:14:58.944 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:14:59.807 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:14:59.966 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:15:01.620 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:15:01.723 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:15:01.768 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:15:02.204 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:15:02.244 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:15:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:15:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:15:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:15:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:15:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:15:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:15:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:15:06.133 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:15:06.170 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:15:06.799 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:15:06.840 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:15:06.879 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:15:06.879 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-04-20T21:15:06.879 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-04-20T21:15:06.880 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-04-20T21:15:06.880 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-04-20T21:15:06.880 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-04-20T21:15:06.880 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-04-20T21:15:06.880 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-04-20T21:15:06.880 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:07.032 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:15:07.032 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:15:07.033 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:15:07.033 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:07.057 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:15:07.072 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:15:07.072 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:07.072 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:15:07.073 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:07.083 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:15:08.560 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:15:08.585 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:15:08.585 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:08.585 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:15:08.585 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:15:08.585 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:15:08.585 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:08.810 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:15:08.833 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:15:08.833 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:08.834 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:15:08.834 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:15:08.834 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:15:08.834 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:09.277 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:15:09.300 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:15:09.300 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:09.300 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:15:09.300 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:15:09.300 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:15:09.300 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:09.407 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:15:09.464 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:15:09.527 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:15:09.712 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:15:09.770 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:15:09.846 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:15:09.935 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:15:10.082 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:15:10.128 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:15:10.160 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:15:10.187 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:15:10.581 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:15:10.611 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:15:10.643 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:15:10.704 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:15:10.734 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:15:11.772 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:15:11.802 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:15:12.769 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:15:12.802 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:15:12.902 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:15:13.074 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:15:13.104 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:15:13.127 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:15:13.127 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:13.127 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:15:13.127 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:15:13.127 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:15:13.127 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:13.172 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:15:13.183 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:15:13.282 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:15:13.328 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:15:14.254 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:15:14.282 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:15:14.305 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:15:14.306 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:14.306 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:15:14.306 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:15:14.306 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:15:14.306 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:14.350 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:15:14.372 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:15:14.372 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:14.372 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:15:14.372 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:14.417 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:15:14.440 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:15:14.440 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:14.440 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:15:14.440 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:15:14.440 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:15:14.440 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:14.743 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:15:14.765 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:15:14.765 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:14.765 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:15:14.766 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:15:14.766 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:15:14.766 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:19.354 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:15:19.395 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:15:19.436 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:15:19.468 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:15:19.496 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:15:19.672 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:15:19.717 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:15:19.747 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-04-20T21:15:19.832 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:20.080 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:15:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:15:20.127 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:20.127 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:15:20.127 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:20.358 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:20.402 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:20.422 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:21.190 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:15:21.282 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:15:21.322 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:15:21.403 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:15:21.464 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:15:21.464 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:15:21.544 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:15:21.544 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:21.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:15:21.895 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:15:21.896 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:21.896 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:15:21.896 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:15:21.896 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:15:21.896 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:22.167 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:15:22.260 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:15:22.260 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:22.260 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:15:22.260 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:15:22.261 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:15:22.261 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:22.784 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:15:22.858 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:15:22.859 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:22.859 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:15:22.859 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:15:22.859 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:15:22.859 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:22.993 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:15:23.062 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:15:23.141 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:15:23.374 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:15:23.431 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:15:23.532 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:15:23.650 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:15:23.815 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:15:23.872 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:15:23.913 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:15:23.945 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:15:24.703 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:15:24.740 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:15:24.794 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:15:24.902 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:15:24.934 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:15:26.254 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:15:26.303 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:15:27.404 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:15:27.428 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:15:27.572 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:15:27.621 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:15:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:15:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:15:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:15:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:15:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:15:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:15:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:15:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:15:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:15:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:15:27.635 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:15:27.635 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:15:27.635 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:15:27.635 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:15:27.635 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:15:27.636 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:15:27.637 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:15:27.638 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:15:27.639 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:15:27.640 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:15:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:15:27.642 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:15:27.643 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:15:27.644 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:15:27.644 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:15:27.644 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:15:27.644 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:15:27.644 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:15:27.644 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:15:27.644 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:15:27.763 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:15:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:15:27.872 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:15:27.872 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:27.872 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:15:27.873 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:15:27.873 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:15:27.873 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:27.956 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:15:27.993 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:15:28.144 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:15:28.214 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:15:29.225 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:15:29.258 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:15:29.334 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:15:29.334 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:29.334 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:15:29.334 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:15:29.334 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:15:29.334 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:29.384 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:15:29.473 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:15:29.473 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:29.473 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:15:29.473 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:29.551 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:15:29.654 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:15:29.654 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:29.654 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:15:29.654 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:15:29.654 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:15:29.655 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:29.666 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-04-20T21:15:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.666 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-20T21:15:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:15:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:15:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:15:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:15:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:15:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:15:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:15:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:15:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:15:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:15:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:29.678 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-20T21:15:30.002 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:15:30.057 DEBUG:teuthology.parallel:result is None 2024-04-20T21:15:30.095 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:15:30.095 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:30.095 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:15:30.095 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:15:30.095 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:15:30.095 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:35.144 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:15:35.199 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:15:35.288 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:15:35.330 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:15:35.378 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:15:35.713 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:15:35.772 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:15:35.800 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:15:36.984 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:37.024 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:37.085 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:38.321 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:15:38.414 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:15:38.492 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:15:38.702 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:15:38.762 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:15:38.763 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:15:38.887 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:15:38.887 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:39.397 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:39.397 INFO:teuthology.orchestra.run.smithi102.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T21:15:39.397 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:45.249 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:15:45.250 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:15:45.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:15:45.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:15:45.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:15:45.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:15:45.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:15:45.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:15:45.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:15:45.265 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:15:45.266 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:15:45.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:15:45.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:15:45.269 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:15:45.271 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:15:45.271 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:15:45.271 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:15:45.271 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:15:45.272 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:15:45.273 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:15:45.274 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:15:45.275 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:15:45.276 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:15:45.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:15:45.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:15:47.610 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.611 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.612 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:15:47.613 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:15:47.614 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:15:47.615 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.616 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:15:47.617 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:15:47.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:15:47.619 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:15:47.620 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:15:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:15:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:15:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:15:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:15:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:15:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:15:47.622 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:47.622 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-20T21:15:48.219 DEBUG:teuthology.parallel:result is None 2024-04-20T21:15:48.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:15:48.442 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:15:48.498 INFO:teuthology.orchestra.run.smithi102.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:15:48.499 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:15:48.499 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:15:48.500 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:15:48.792 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:15:48.819 INFO:teuthology.orchestra.run.smithi193.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:15:48.819 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:15:48.819 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:15:48.821 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T21:15:48.822 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:48.822 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:15:48.896 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:15:48.896 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:15:48.927 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T21:15:48.928 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:48.928 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:15:49.003 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:15:49.110 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:15:49.110 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:15:49.139 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:15:49.204 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T21:15:49.205 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:49.205 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:15:49.274 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:15:49.377 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:15:49.377 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:15:49.401 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:15:49.465 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T21:15:49.465 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:49.466 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:15:49.536 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:15:49.646 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:15:49.646 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:15:49.670 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:15:49.734 INFO:teuthology.run_tasks:Running task ceph... 2024-04-20T21:15:49.823 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-20T21:15:49.823 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /var/log/ceph 2024-04-20T21:15:49.827 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /var/log/ceph 2024-04-20T21:15:49.890 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-20T21:15:49.890 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T21:15:49.894 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T21:15:49.957 INFO:tasks.ceph:Creating extra log directories... 2024-04-20T21:15:49.957 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T21:15:49.961 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T21:15:50.032 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-20T21:15:50.032 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-20T21:15:50.032 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-20T21:15:50.033 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:50.033 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T21:15:50.070 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T21:15:50.254 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T21:15:50.323 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:15:50.323 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T21:15:50.338 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T21:15:50.446 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T21:15:50.491 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-20T21:15:50.491 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-20T21:15:50.492 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414', 'branch': 'wip-rishabh-testing-20240416.193735', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': None, 'first_in_suite': False, 'job_id': '7665414', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 7954, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi102.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9QeV7h0MaaSwHaphBUDlmNRalRKbj5cvKLE3E71dxdL1fda695dWpZEM3fByLnalQsaIUmEX/oi2MOCo8wV68=', 'smithi193.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAnUap0WmMc8ZX95t3l0FWneNQgTBlAOPaCQ5F7fbon/6yzNVg802gspoF17WsVJGSrplIgSAwTBiqbCHfDUXYA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-20_20:50:37', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-20T21:15:50.492 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T21:15:50.496 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T21:15:50.540 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T21:15:50.578 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T21:15:50.639 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:50.639 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:15:50.678 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:15:50.678 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-04-20T21:15:50.760 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:15:50.760 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:15:50.760 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-20T21:15:50.761 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:15:50.761 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:15:50.761 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-20 21:15:39.850712361 +0000 2024-04-20T21:15:50.761 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-20 21:12:29.971982907 +0000 2024-04-20T21:15:50.761 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-20 21:12:29.971982907 +0000 2024-04-20T21:15:50.761 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-20 21:12:29.971982907 +0000 2024-04-20T21:15:50.761 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:15:50.863 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-20T21:15:50.863 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-20T21:15:50.863 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000411985 s, 1.2 MB/s 2024-04-20T21:15:50.866 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:15:50.908 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-20 21:15:39.851712340 +0000 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-20 21:12:30.589969596 +0000 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-20 21:12:30.589969596 +0000 2024-04-20T21:15:50.988 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-20 21:12:30.589969596 +0000 2024-04-20T21:15:50.989 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:15:51.087 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-20T21:15:51.087 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-20T21:15:51.087 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000284454 s, 1.8 MB/s 2024-04-20T21:15:51.090 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:15:51.139 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-04-20T21:15:51.221 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:15:51.222 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:15:51.222 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-20T21:15:51.222 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:15:51.222 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:15:51.222 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-20 21:15:39.851712340 +0000 2024-04-20T21:15:51.222 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-20 21:12:31.197956501 +0000 2024-04-20T21:15:51.222 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-20 21:12:31.197956501 +0000 2024-04-20T21:15:51.222 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-20 21:12:31.197956501 +0000 2024-04-20T21:15:51.222 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:15:51.331 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-20T21:15:51.331 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-20T21:15:51.331 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000355696 s, 1.4 MB/s 2024-04-20T21:15:51.334 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:15:51.379 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-04-20T21:15:51.459 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:15:51.459 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:15:51.459 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-20T21:15:51.459 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:15:51.460 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:15:51.460 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-20 21:15:39.852712320 +0000 2024-04-20T21:15:51.460 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-20 21:12:31.837942716 +0000 2024-04-20T21:15:51.460 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-20 21:12:31.837942716 +0000 2024-04-20T21:15:51.460 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-20 21:12:31.837942716 +0000 2024-04-20T21:15:51.461 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:15:51.558 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-20T21:15:51.559 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-20T21:15:51.559 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000311684 s, 1.6 MB/s 2024-04-20T21:15:51.561 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:15:51.601 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:15:51.601 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:15:51.601 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:15:51.618 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:15:51.618 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-04-20T21:15:51.675 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:15:51.675 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:15:51.675 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-20T21:15:51.676 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:15:51.676 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:15:51.676 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-20 21:15:22.032126787 +0000 2024-04-20T21:15:51.676 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-20 21:12:28.180396509 +0000 2024-04-20T21:15:51.676 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-20 21:12:28.180396509 +0000 2024-04-20T21:15:51.676 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-20 21:12:28.180396509 +0000 2024-04-20T21:15:51.676 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:15:51.740 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-20T21:15:51.740 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-20T21:15:51.740 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000326602 s, 1.6 MB/s 2024-04-20T21:15:51.741 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:15:51.798 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-04-20T21:15:51.855 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:15:51.855 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:15:51.855 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-20T21:15:51.856 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:15:51.856 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:15:51.856 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-20 21:15:22.032126787 +0000 2024-04-20T21:15:51.856 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-20 21:12:28.529390881 +0000 2024-04-20T21:15:51.856 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-20 21:12:28.529390881 +0000 2024-04-20T21:15:51.856 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-20 21:12:28.529390881 +0000 2024-04-20T21:15:51.856 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:15:51.919 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-20T21:15:51.919 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-20T21:15:51.919 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000201722 s, 2.5 MB/s 2024-04-20T21:15:51.921 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:15:51.978 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-04-20T21:15:52.036 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:15:52.036 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:15:52.036 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-20T21:15:52.036 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:15:52.037 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:15:52.037 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-20 21:15:22.033126771 +0000 2024-04-20T21:15:52.037 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-20 21:12:28.887385107 +0000 2024-04-20T21:15:52.037 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-20 21:12:28.887385107 +0000 2024-04-20T21:15:52.037 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-20 21:12:28.887385107 +0000 2024-04-20T21:15:52.038 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:15:52.099 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-20T21:15:52.099 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-20T21:15:52.100 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000206675 s, 2.5 MB/s 2024-04-20T21:15:52.101 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:15:52.159 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-04-20T21:15:52.215 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:15:52.216 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:15:52.216 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-20T21:15:52.216 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:15:52.216 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:15:52.216 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-20 21:15:22.033126771 +0000 2024-04-20T21:15:52.217 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-20 21:12:29.223379688 +0000 2024-04-20T21:15:52.217 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-20 21:12:29.223379688 +0000 2024-04-20T21:15:52.217 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-20 21:12:29.223379688 +0000 2024-04-20T21:15:52.217 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:15:52.279 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-20T21:15:52.279 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-20T21:15:52.279 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000213776 s, 2.4 MB/s 2024-04-20T21:15:52.280 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:15:52.336 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:15:52.336 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi193.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T21:15:52.336 INFO:tasks.ceph:Generating config... 2024-04-20T21:15:52.337 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-20T21:15:52.337 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-20T21:15:52.337 INFO:tasks.ceph:[client] debug client = 20 2024-04-20T21:15:52.337 INFO:tasks.ceph:[client] debug ms = 1 2024-04-20T21:15:52.338 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-20T21:15:52.338 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-20T21:15:52.338 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-20T21:15:52.338 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-20T21:15:52.339 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-20T21:15:52.340 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-20T21:15:52.340 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-20T21:15:52.340 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-20T21:15:52.340 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-20T21:15:52.340 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-20T21:15:52.340 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-20T21:15:52.340 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-20T21:15:52.340 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-20T21:15:52.340 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-20T21:15:52.340 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-20T21:15:52.340 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-20T21:15:52.340 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-20T21:15:52.340 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-20T21:15:52.340 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-20T21:15:52.341 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-04-20T21:15:52.341 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-20T21:15:52.341 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-20T21:15:52.341 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-20T21:15:52.341 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-04-20T21:15:52.341 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-04-20T21:15:52.341 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-04-20T21:15:52.341 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-04-20T21:15:52.342 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-20T21:15:52.342 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-20T21:15:52.342 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-20T21:15:52.342 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-20T21:15:52.342 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-20T21:15:52.342 INFO:tasks.ceph:Setting up mon.a... 2024-04-20T21:15:52.342 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-20T21:15:52.437 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.keyring 2024-04-20T21:15:52.443 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-20T21:15:52.531 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T21:15:52.591 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.102'), ('mon.b', '172.21.15.193'), ('mon.c', '[v2:172.21.15.193:3301,v1:172.21.15.193:6790]')] 2024-04-20T21:15:52.592 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.102,172.21.15.193,[v2:172.21.15.193:3301,v1:172.21.15.193:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-20T21:15:52.593 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:52.593 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T21:15:52.629 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.102 --add b 172.21.15.193 --addv c '[v2:172.21.15.193:3301,v1:172.21.15.193:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: generated fsid 1c72cacb-b320-4a89-8b72-ff890245905f 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:setting min_mon_release = quincy 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 0 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:fsid 1c72cacb-b320-4a89-8b72-ff890245905f 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:last_changed 2024-04-20T21:15:52.752662+0000 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-04-20T21:15:52.752662+0000 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:min_mon_release 17 (quincy) 2024-04-20T21:15:52.751 INFO:teuthology.orchestra.run.smithi102.stdout:election_strategy: 1 2024-04-20T21:15:52.752 INFO:teuthology.orchestra.run.smithi102.stdout:0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-04-20T21:15:52.752 INFO:teuthology.orchestra.run.smithi102.stdout:1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.b 2024-04-20T21:15:52.752 INFO:teuthology.orchestra.run.smithi102.stdout:2: [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] mon.c 2024-04-20T21:15:52.752 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-20T21:15:52.754 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T21:15:52.791 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1c72cacb-b320-4a89-8b72-ff890245905f... 2024-04-20T21:15:52.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:15:52.835 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:15:52.878 INFO:teuthology.orchestra.run.smithi193.stdout:[global] 2024-04-20T21:15:52.878 INFO:teuthology.orchestra.run.smithi193.stdout: chdir = "" 2024-04-20T21:15:52.878 INFO:teuthology.orchestra.run.smithi193.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:15:52.878 INFO:teuthology.orchestra.run.smithi193.stdout: auth supported = cephx 2024-04-20T21:15:52.878 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.879 INFO:teuthology.orchestra.run.smithi193.stdout: filestore xattr use omap = true 2024-04-20T21:15:52.879 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.879 INFO:teuthology.orchestra.run.smithi193.stdout: mon clock drift allowed = 1.000 2024-04-20T21:15:52.879 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.879 INFO:teuthology.orchestra.run.smithi193.stdout: osd crush chooseleaf type = 0 2024-04-20T21:15:52.879 INFO:teuthology.orchestra.run.smithi193.stdout: auth debug = true 2024-04-20T21:15:52.879 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on old message = true 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on bug = true 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: mon pg warn max object skew = 0 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: osd pool default size = 2 2024-04-20T21:15:52.880 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd allow primary affinity = true 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd allow pg remap = true 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on no sortbitwise = false 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on too few osds = false 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: mon_allow_pool_size_one = true 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.881 INFO:teuthology.orchestra.run.smithi193.stdout: osd default data pool replay window = 5 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: mon allow pool delete = true 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: mon cluster log file level = debug 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: debug asserts on shutdown = true 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: mon health detail to clog = false 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: mon host = "172.21.15.102,172.21.15.193,[v2:172.21.15.193:3301,v1:172.21.15.193:6790]" 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on skipped message = False 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: fsid = 1c72cacb-b320-4a89-8b72-ff890245905f 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout:[osd] 2024-04-20T21:15:52.882 INFO:teuthology.orchestra.run.smithi193.stdout: osd journal size = 100 2024-04-20T21:15:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: osd scrub load threshold = 5.0 2024-04-20T21:15:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: osd scrub max interval = 600 2024-04-20T21:15:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:15:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: osd recover clone overlap = true 2024-04-20T21:15:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: osd recovery max chunk = 1048576 2024-04-20T21:15:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug shutdown = true 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug op order = true 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug verify stray on activate = true 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug trim objects = true 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: osd open classes on start = true 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug pg log writeout = true 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: osd map max advance = 10 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.884 INFO:teuthology.orchestra.run.smithi193.stdout: journal zero on create = true 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: filestore apply finisher threads = 3 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: bdev debug aio = true 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug misdirected ops = true 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: bluestore block size = 96636764160 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: bluestore fsck on mount = True 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: debug bluefs = 20 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: debug bluestore = 20 2024-04-20T21:15:52.885 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: debug osd = 20 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: debug rocksdb = 10 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: mon client directed command retry = 5 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd full ratio = 0.9 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: ms inject delay max = 1 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: ms inject delay probability = 0.005 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: ms inject delay type = client mds 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: ms inject socket failures = 2500 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: osd blocked scrub grace period = 3600 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:15:52.886 INFO:teuthology.orchestra.run.smithi193.stdout: osd max markdown count = 1000 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: osd objectstore = bluestore 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: osd op complaint time = 180 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr] 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: debug mgr = 20 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: debug mon = 20 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: debug auth = 20 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: mgr/telemetry/nag = false 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: debug client = 20 2024-04-20T21:15:52.887 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout:[mon] 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: debug mon = 20 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: debug paxos = 20 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: debug auth = 20 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: mon data avail warn = 5 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: mon mgr mkfs grace = 240 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd reporter subtree level = osd 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd prime pg temp = true 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.888 INFO:teuthology.orchestra.run.smithi193.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: # 1m isn't quite enough 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: mon down mkfs grace = 300 2024-04-20T21:15:52.889 INFO:teuthology.orchestra.run.smithi193.stdout: mon op complaint time = 120 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout:[client] 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: rgw cache enabled = true 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: rgw enable ops log = true 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: rgw enable usage log = true 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: client mount timeout = 600 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: client_shutdown_timeout = 120 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: debug client = 20 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: rados mon op timeout = 900 2024-04-20T21:15:52.890 INFO:teuthology.orchestra.run.smithi193.stdout: rados osd op timeout = 900 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mon.a] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mon.b] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mon.c] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.a] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.c] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.e] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.b] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.d] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.f] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout:[mds] 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout: debug mds = 20 2024-04-20T21:15:52.891 INFO:teuthology.orchestra.run.smithi193.stdout: debug mds balancer = 20 2024-04-20T21:15:52.892 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-20T21:15:52.892 INFO:teuthology.orchestra.run.smithi193.stdout: mds bal fragment size max = 10000 2024-04-20T21:15:52.892 INFO:teuthology.orchestra.run.smithi193.stdout: mds bal merge size = 5 2024-04-20T21:15:52.892 INFO:teuthology.orchestra.run.smithi193.stdout: mds bal split bits = 3 2024-04-20T21:15:52.892 INFO:teuthology.orchestra.run.smithi193.stdout: mds bal split size = 100 2024-04-20T21:15:52.892 INFO:teuthology.orchestra.run.smithi193.stdout: mds debug frag = True 2024-04-20T21:15:52.892 INFO:teuthology.orchestra.run.smithi193.stdout: mds debug scatterstat = True 2024-04-20T21:15:52.893 INFO:teuthology.orchestra.run.smithi193.stdout: mds oft prefetch dirfrags = True 2024-04-20T21:15:52.893 INFO:teuthology.orchestra.run.smithi193.stdout: mds op complaint time = 180 2024-04-20T21:15:52.893 INFO:teuthology.orchestra.run.smithi193.stdout: mds verify scatter = True 2024-04-20T21:15:52.893 INFO:teuthology.orchestra.run.smithi193.stdout: mds_dir_prefetch = True 2024-04-20T21:15:52.893 INFO:teuthology.orchestra.run.smithi193.stdout: mds_sleep_rank_change = 5000000.0 2024-04-20T21:15:52.893 INFO:teuthology.orchestra.run.smithi193.stdout: mon client directed command retry = 5 2024-04-20T21:15:52.893 INFO:teuthology.orchestra.run.smithi193.stdout: ms inject delay max = 1 2024-04-20T21:15:52.893 INFO:teuthology.orchestra.run.smithi193.stdout: ms inject delay probability = 0.005 2024-04-20T21:15:52.894 INFO:teuthology.orchestra.run.smithi193.stdout: ms inject delay type = client mds osd 2024-04-20T21:15:52.894 INFO:teuthology.orchestra.run.smithi193.stdout: ms inject socket failures = 2500 2024-04-20T21:15:52.894 INFO:teuthology.orchestra.run.smithi193.stdout: osd op complaint time = 180 2024-04-20T21:15:52.894 INFO:teuthology.orchestra.run.smithi193.stdout: rados mon op timeout = 900 2024-04-20T21:15:52.894 INFO:teuthology.orchestra.run.smithi193.stdout: rados osd op timeout = 900 2024-04-20T21:15:52.938 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: chdir = "" 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: auth supported = cephx 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: filestore xattr use omap = true 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: mon clock drift allowed = 1.000 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: osd crush chooseleaf type = 0 2024-04-20T21:15:52.939 INFO:teuthology.orchestra.run.smithi102.stdout: auth debug = true 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on old message = true 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on bug = true 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn max object skew = 0 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default size = 2 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow primary affinity = true 2024-04-20T21:15:52.940 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow pg remap = true 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on no sortbitwise = false 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on too few osds = false 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: mon_allow_pool_size_one = true 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: osd default data pool replay window = 5 2024-04-20T21:15:52.941 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: mon allow pool delete = true 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: mon cluster log file level = debug 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: debug asserts on shutdown = true 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: mon health detail to clog = false 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: mon host = "172.21.15.102,172.21.15.193,[v2:172.21.15.193:3301,v1:172.21.15.193:6790]" 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on skipped message = False 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = 1c72cacb-b320-4a89-8b72-ff890245905f 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout:[osd] 2024-04-20T21:15:52.942 INFO:teuthology.orchestra.run.smithi102.stdout: osd journal size = 100 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub load threshold = 5.0 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub max interval = 600 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd recover clone overlap = true 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd recovery max chunk = 1048576 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug shutdown = true 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug op order = true 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify stray on activate = true 2024-04-20T21:15:52.943 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug trim objects = true 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: osd open classes on start = true 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug pg log writeout = true 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: osd map max advance = 10 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: journal zero on create = true 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: filestore apply finisher threads = 3 2024-04-20T21:15:52.944 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: bdev debug aio = true 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug misdirected ops = true 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore block size = 96636764160 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore fsck on mount = True 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluefs = 20 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluestore = 20 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: debug osd = 20 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: debug rocksdb = 10 2024-04-20T21:15:52.945 INFO:teuthology.orchestra.run.smithi102.stdout: mon client directed command retry = 5 2024-04-20T21:15:52.946 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:15:52.946 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd full ratio = 0.9 2024-04-20T21:15:52.946 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:15:52.946 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay max = 1 2024-04-20T21:15:52.946 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay probability = 0.005 2024-04-20T21:15:52.946 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay type = client mds 2024-04-20T21:15:52.946 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject socket failures = 2500 2024-04-20T21:15:52.947 INFO:teuthology.orchestra.run.smithi102.stdout: osd blocked scrub grace period = 3600 2024-04-20T21:15:52.947 INFO:teuthology.orchestra.run.smithi102.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:15:52.947 INFO:teuthology.orchestra.run.smithi102.stdout: osd max markdown count = 1000 2024-04-20T21:15:52.947 INFO:teuthology.orchestra.run.smithi102.stdout: osd objectstore = bluestore 2024-04-20T21:15:52.947 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2024-04-20T21:15:52.947 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.948 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr] 2024-04-20T21:15:52.948 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-20T21:15:52.948 INFO:teuthology.orchestra.run.smithi102.stdout: debug mgr = 20 2024-04-20T21:15:52.948 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-04-20T21:15:52.948 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-04-20T21:15:52.948 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:15:52.949 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:15:52.949 INFO:teuthology.orchestra.run.smithi102.stdout: mgr/telemetry/nag = false 2024-04-20T21:15:52.949 INFO:teuthology.orchestra.run.smithi102.stdout: debug client = 20 2024-04-20T21:15:52.949 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.949 INFO:teuthology.orchestra.run.smithi102.stdout:[mon] 2024-04-20T21:15:52.949 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-20T21:15:52.950 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-04-20T21:15:52.950 INFO:teuthology.orchestra.run.smithi102.stdout: debug paxos = 20 2024-04-20T21:15:52.950 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-04-20T21:15:52.950 INFO:teuthology.orchestra.run.smithi102.stdout: mon data avail warn = 5 2024-04-20T21:15:52.950 INFO:teuthology.orchestra.run.smithi102.stdout: mon mgr mkfs grace = 240 2024-04-20T21:15:52.950 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:15:52.951 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd reporter subtree level = osd 2024-04-20T21:15:52.951 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd prime pg temp = true 2024-04-20T21:15:52.951 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:15:52.951 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.951 INFO:teuthology.orchestra.run.smithi102.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:15:52.951 INFO:teuthology.orchestra.run.smithi102.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:15:52.952 INFO:teuthology.orchestra.run.smithi102.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:15:52.952 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.952 INFO:teuthology.orchestra.run.smithi102.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:15:52.952 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:15:52.952 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:15:52.952 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.953 INFO:teuthology.orchestra.run.smithi102.stdout: # 1m isn't quite enough 2024-04-20T21:15:52.953 INFO:teuthology.orchestra.run.smithi102.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:15:52.953 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.953 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:15:52.953 INFO:teuthology.orchestra.run.smithi102.stdout: mon down mkfs grace = 300 2024-04-20T21:15:52.953 INFO:teuthology.orchestra.run.smithi102.stdout: mon op complaint time = 120 2024-04-20T21:15:52.954 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:15:52.954 INFO:teuthology.orchestra.run.smithi102.stdout:[client] 2024-04-20T21:15:52.954 INFO:teuthology.orchestra.run.smithi102.stdout: rgw cache enabled = true 2024-04-20T21:15:52.954 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable ops log = true 2024-04-20T21:15:52.954 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable usage log = true 2024-04-20T21:15:52.955 INFO:teuthology.orchestra.run.smithi102.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:15:52.955 INFO:teuthology.orchestra.run.smithi102.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:15:52.955 INFO:teuthology.orchestra.run.smithi102.stdout: client mount timeout = 600 2024-04-20T21:15:52.955 INFO:teuthology.orchestra.run.smithi102.stdout: client_shutdown_timeout = 120 2024-04-20T21:15:52.955 INFO:teuthology.orchestra.run.smithi102.stdout: debug client = 20 2024-04-20T21:15:52.955 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-20T21:15:52.956 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2024-04-20T21:15:52.956 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2024-04-20T21:15:52.956 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.a] 2024-04-20T21:15:52.956 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.b] 2024-04-20T21:15:52.956 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.c] 2024-04-20T21:15:52.956 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2024-04-20T21:15:52.956 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.c] 2024-04-20T21:15:52.956 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.e] 2024-04-20T21:15:52.957 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.b] 2024-04-20T21:15:52.957 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.d] 2024-04-20T21:15:52.957 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.f] 2024-04-20T21:15:52.957 INFO:teuthology.orchestra.run.smithi102.stdout:[mds] 2024-04-20T21:15:52.957 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds = 20 2024-04-20T21:15:52.957 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds balancer = 20 2024-04-20T21:15:52.958 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-04-20T21:15:52.958 INFO:teuthology.orchestra.run.smithi102.stdout: mds bal fragment size max = 10000 2024-04-20T21:15:52.958 INFO:teuthology.orchestra.run.smithi102.stdout: mds bal merge size = 5 2024-04-20T21:15:52.958 INFO:teuthology.orchestra.run.smithi102.stdout: mds bal split bits = 3 2024-04-20T21:15:52.958 INFO:teuthology.orchestra.run.smithi102.stdout: mds bal split size = 100 2024-04-20T21:15:52.958 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug frag = True 2024-04-20T21:15:52.959 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug scatterstat = True 2024-04-20T21:15:52.959 INFO:teuthology.orchestra.run.smithi102.stdout: mds oft prefetch dirfrags = True 2024-04-20T21:15:52.959 INFO:teuthology.orchestra.run.smithi102.stdout: mds op complaint time = 180 2024-04-20T21:15:52.959 INFO:teuthology.orchestra.run.smithi102.stdout: mds verify scatter = True 2024-04-20T21:15:52.959 INFO:teuthology.orchestra.run.smithi102.stdout: mds_dir_prefetch = True 2024-04-20T21:15:52.959 INFO:teuthology.orchestra.run.smithi102.stdout: mds_sleep_rank_change = 5000000.0 2024-04-20T21:15:52.960 INFO:teuthology.orchestra.run.smithi102.stdout: mon client directed command retry = 5 2024-04-20T21:15:52.960 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay max = 1 2024-04-20T21:15:52.960 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay probability = 0.005 2024-04-20T21:15:52.960 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject delay type = client mds osd 2024-04-20T21:15:52.960 INFO:teuthology.orchestra.run.smithi102.stdout: ms inject socket failures = 2500 2024-04-20T21:15:52.960 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2024-04-20T21:15:52.961 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2024-04-20T21:15:52.961 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2024-04-20T21:15:52.966 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-20T21:15:52.967 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-20T21:15:53.081 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-20T21:15:53.081 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:53.081 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-20T21:15:53.117 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:53.117 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-20T21:15:53.195 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:15:53.195 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:53.196 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T21:15:53.196 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T21:15:53.324 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:15:53.324 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:15:53.405 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:15:53.405 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:15:53.405 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T21:15:53.405 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T21:15:53.441 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:15:53.441 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:15:53.497 INFO:tasks.ceph:Setting up mon nodes... 2024-04-20T21:15:53.497 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-20T21:15:53.497 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T21:15:53.607 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T21:15:53.615 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T21:15:53.676 INFO:teuthology.orchestra.run.smithi193.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T21:15:53.678 INFO:tasks.ceph:Setting up mds nodes... 2024-04-20T21:15:53.678 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T21:15:53.794 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T21:15:53.800 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-20T21:15:53.861 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T21:15:53.981 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T21:15:53.989 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-20T21:15:54.061 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-20T21:15:54.200 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-20T21:15:54.211 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-20T21:15:54.280 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T21:15:54.337 INFO:teuthology.orchestra.run.smithi193.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T21:15:54.339 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-20T21:15:54.365 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T21:15:54.455 INFO:teuthology.orchestra.run.smithi193.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T21:15:54.458 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-20T21:15:54.481 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-20T21:15:54.569 INFO:teuthology.orchestra.run.smithi193.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-20T21:15:54.572 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-20T21:15:54.595 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-20T21:15:54.595 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T21:15:54.705 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-20T21:15:54.740 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-20T21:15:54.740 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi193.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T21:15:54.741 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-20T21:15:54.823 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:15:54.823 INFO:tasks.ceph:role: osd.0 2024-04-20T21:15:54.823 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:15:54.823 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:15:54.899 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:15:55.317 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-04-20T21:15:55.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-04-20T21:15:55.324 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-20T21:15:55.459 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-20T21:15:55.538 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-20T21:15:55.607 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:15:55.607 INFO:tasks.ceph:role: osd.1 2024-04-20T21:15:55.607 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:15:55.607 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T21:15:55.677 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:15:55.677 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:15:55.677 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:15:55.677 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:15:55.677 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:15:55.677 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:15:55.677 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:15:55.677 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:15:55.678 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:15:55.678 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:15:56.032 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-04-20T21:15:56.038 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-04-20T21:15:56.038 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-20T21:15:56.144 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-20T21:15:56.229 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-20T21:15:56.295 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:15:56.295 INFO:tasks.ceph:role: osd.2 2024-04-20T21:15:56.295 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:15:56.296 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T21:15:56.377 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:15:56.377 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:15:56.377 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:15:56.378 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:15:56.378 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:15:56.378 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:15:56.378 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:15:56.378 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:15:56.378 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:15:56.378 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:15:56.762 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-04-20T21:15:56.768 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-04-20T21:15:56.768 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-20T21:15:56.892 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-20T21:15:56.980 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-20T21:15:57.059 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:15:57.059 INFO:tasks.ceph:role: osd.3 2024-04-20T21:15:57.059 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:15:57.059 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T21:15:57.137 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:15:57.138 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:15:57.560 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-04-20T21:15:57.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-04-20T21:15:57.567 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-20T21:15:57.734 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-20T21:15:57.819 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:15:57.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:15:57.953+0000 7fb939118640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-20T21:15:57.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:15:57.954+0000 7fb939118640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-20T21:15:57.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:15:57.954+0000 7fb939118640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T21:15:57.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:15:57.954+0000 7fb939118640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T21:15:57.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:15:57.954+0000 7fb939118640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T21:15:57.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:15:57.958+0000 7fb939118640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-20T21:16:01.204 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T21:16:01.288 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:01.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:01.417+0000 7f5797611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-20T21:16:01.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:01.418+0000 7f5797611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-20T21:16:01.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:01.418+0000 7f5797611640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T21:16:01.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:01.418+0000 7f5797611640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T21:16:01.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:01.419+0000 7f5797611640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T21:16:01.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:01.421+0000 7f5797611640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-20T21:16:04.626 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T21:16:04.712 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:04.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:04.833+0000 7f7ff096e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-20T21:16:04.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:04.834+0000 7f7ff096e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-20T21:16:04.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:04.834+0000 7f7ff096e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T21:16:04.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:04.834+0000 7f7ff096e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T21:16:04.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:04.834+0000 7f7ff096e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T21:16:04.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:04.836+0000 7f7ff096e640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-20T21:16:07.808 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T21:16:07.883 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:08.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:08.007+0000 7fdcefdd6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-20T21:16:08.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:08.008+0000 7fdcefdd6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-20T21:16:08.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:08.008+0000 7fdcefdd6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T21:16:08.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:08.008+0000 7fdcefdd6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T21:16:08.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:08.008+0000 7fdcefdd6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T21:16:08.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:08.010+0000 7fdcefdd6640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-20T21:16:10.967 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T21:16:11.051 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-20T21:16:11.079 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:11.080 INFO:tasks.ceph:role: osd.4 2024-04-20T21:16:11.080 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:16:11.080 DEBUG:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T21:16:11.155 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:16:11.656 INFO:teuthology.orchestra.run.smithi193.stdout:Discarding blocks...Done. 2024-04-20T21:16:11.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2024-04-20T21:16:11.658 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-20T21:16:11.800 DEBUG:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-20T21:16:11.834 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-20T21:16:11.900 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:11.900 INFO:tasks.ceph:role: osd.5 2024-04-20T21:16:11.900 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:16:11.900 DEBUG:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T21:16:11.972 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:16:11.972 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:16:11.973 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:16:11.973 INFO:teuthology.orchestra.run.smithi193.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:16:11.973 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:16:11.973 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:16:11.973 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:16:11.973 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:16:11.973 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:16:11.973 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:16:12.427 INFO:teuthology.orchestra.run.smithi193.stdout:Discarding blocks...Done. 2024-04-20T21:16:12.429 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2024-04-20T21:16:12.429 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-20T21:16:12.520 DEBUG:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-20T21:16:12.550 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-20T21:16:12.616 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:12.616 INFO:tasks.ceph:role: osd.6 2024-04-20T21:16:12.617 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:16:12.617 DEBUG:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T21:16:12.688 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:16:12.688 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:16:12.688 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:16:12.688 INFO:teuthology.orchestra.run.smithi193.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:16:12.688 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:16:12.688 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:16:12.688 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:16:12.688 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:16:12.689 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:16:12.689 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:16:13.178 INFO:teuthology.orchestra.run.smithi193.stdout:Discarding blocks...Done. 2024-04-20T21:16:13.180 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2024-04-20T21:16:13.180 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-20T21:16:13.310 DEBUG:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-20T21:16:13.340 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-20T21:16:13.403 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:13.403 INFO:tasks.ceph:role: osd.7 2024-04-20T21:16:13.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:16:13.404 DEBUG:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T21:16:13.474 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:16:13.474 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:16:13.475 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:16:13.475 INFO:teuthology.orchestra.run.smithi193.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:16:13.475 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:16:13.475 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:16:13.475 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:16:13.475 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:16:13.475 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:16:13.475 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:16:13.990 INFO:teuthology.orchestra.run.smithi193.stdout:Discarding blocks...Done. 2024-04-20T21:16:13.992 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2024-04-20T21:16:13.992 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-20T21:16:14.119 DEBUG:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-20T21:16:14.150 DEBUG:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:14.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:14.248+0000 7f0078263640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-20T21:16:14.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:14.248+0000 7f0078263640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-20T21:16:14.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:14.248+0000 7f0078263640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T21:16:14.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:14.248+0000 7f0078263640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T21:16:14.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:14.248+0000 7f0078263640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T21:16:14.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:14.248+0000 7f0078263640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-20T21:16:17.093 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T21:16:17.121 DEBUG:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:17.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:17.211+0000 7fd335605640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-20T21:16:17.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:17.211+0000 7fd335605640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-20T21:16:17.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:17.211+0000 7fd335605640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T21:16:17.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:17.211+0000 7fd335605640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T21:16:17.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:17.211+0000 7fd335605640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T21:16:17.211 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:17.212+0000 7fd335605640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-20T21:16:20.057 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T21:16:20.084 DEBUG:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:20.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:20.173+0000 7f6584625640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-20T21:16:20.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:20.173+0000 7f6584625640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-20T21:16:20.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:20.173+0000 7f6584625640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T21:16:20.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:20.173+0000 7f6584625640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T21:16:20.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:20.173+0000 7f6584625640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T21:16:20.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:20.174+0000 7f6584625640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-20T21:16:20.493 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:20.501 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:23.024 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-20T21:16:23.053 DEBUG:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:23.143 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:23.143+0000 7ff90f8d7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-20T21:16:23.143 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:23.144+0000 7ff90f8d7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-20T21:16:23.144 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:23.144+0000 7ff90f8d7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T21:16:23.144 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:23.144+0000 7ff90f8d7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T21:16:23.144 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:23.144+0000 7ff90f8d7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T21:16:23.144 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:23.144+0000 7ff90f8d7640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-20T21:16:25.995 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-20T21:16:26.022 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-20T21:16:26.022 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:26.022 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-20T21:16:26.112 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:26.112 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-20T21:16:26.192 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:26.193 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-20T21:16:26.275 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:26.275 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-20T21:16:26.353 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:26.354 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-20T21:16:26.428 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:26.428 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-20T21:16:26.546 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:26.546 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-20T21:16:26.627 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:26.627 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-20T21:16:26.708 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:26.709 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-20T21:16:26.736 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:26.737 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-20T21:16:26.803 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:26.803 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-20T21:16:26.869 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:26.869 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-20T21:16:26.935 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:26.935 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-20T21:16:27.002 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:27.002 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-20T21:16:27.067 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:27.067 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-20T21:16:27.131 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:27.132 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-20T21:16:27.195 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:27.195 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-20T21:16:27.253 INFO:tasks.ceph:Adding keys to all mons... 2024-04-20T21:16:27.254 DEBUG:teuthology.orchestra.run.smithi102:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T21:16:27.259 DEBUG:teuthology.orchestra.run.smithi193:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T21:16:27.281 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr.y] 2024-04-20T21:16:27.281 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCJMCRmAuZPJBAABuTaI0yAhQEgZYsz1iraZg== 2024-04-20T21:16:27.282 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.a] 2024-04-20T21:16:27.282 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCJMCRm1cN1LxAARh1upkCGnHTJgloZo46qUA== 2024-04-20T21:16:27.282 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.c] 2024-04-20T21:16:27.282 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCJMCRmcOOdOhAAOA/r4MiyTZ25KcThrIPKLQ== 2024-04-20T21:16:27.282 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.e] 2024-04-20T21:16:27.282 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCKMCRmPgoDDBAAJNbDY4rcnhHdzkFSlQSrSA== 2024-04-20T21:16:27.282 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.0] 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCNMCRmGsLsOBAAfhnBNO3+dRDqwMcYiet6hg== 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.1] 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCRMCRmPg75GBAA2UI/5UOaxQfQy2xBP7KnoQ== 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.2] 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCUMCRmFRjCMRAA/t1w6vexu+xdtl9TFblNzQ== 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.3] 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCYMCRmpHaGABAABP0Xfp9FuqsOMLcGt5xzfw== 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr.x] 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCJMCRmLjVjKBAAmS8begZJFsz+jMW/YtVdOQ== 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.b] 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCKMCRmXrItFBAAO0X8hSHWrPSHCejAySnnHg== 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.d] 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCKMCRmm4tAGxAAl4aORJJSVJrGhB6vjhkbrg== 2024-04-20T21:16:27.283 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.f] 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCKMCRmMRAQIhAAoBf1nsh8R4GjDWLRpPzIew== 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.4] 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCeMCRmS43YDhAAQHsEJ3Gf0CJ+ARhlzwdzlw== 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.5] 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQChMCRmN1anDBAAnrfxXXsADY0MLrb3d8t4Hg== 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.6] 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCkMCRmKF5gChAAPqZwxFy5U7jDA3kVkOtPrw== 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.7] 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCnMCRmXemiCBAARL3D9eBpFUdCiyUEXi63UA== 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout:[client.0] 2024-04-20T21:16:27.284 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCKMCRm2n0iKhAA5KYFmxM4raEhiZ188hFSNQ== 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr.y] 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCJMCRmAuZPJBAABuTaI0yAhQEgZYsz1iraZg== 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCJMCRm1cN1LxAARh1upkCGnHTJgloZo46qUA== 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.c] 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCJMCRmcOOdOhAAOA/r4MiyTZ25KcThrIPKLQ== 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.e] 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCKMCRmPgoDDBAAJNbDY4rcnhHdzkFSlQSrSA== 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.0] 2024-04-20T21:16:27.334 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCNMCRmGsLsOBAAfhnBNO3+dRDqwMcYiet6hg== 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.1] 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCRMCRmPg75GBAA2UI/5UOaxQfQy2xBP7KnoQ== 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.2] 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCUMCRmFRjCMRAA/t1w6vexu+xdtl9TFblNzQ== 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.3] 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCYMCRmpHaGABAABP0Xfp9FuqsOMLcGt5xzfw== 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr.x] 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCJMCRmLjVjKBAAmS8begZJFsz+jMW/YtVdOQ== 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.b] 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCKMCRmXrItFBAAO0X8hSHWrPSHCejAySnnHg== 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.d] 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCKMCRmm4tAGxAAl4aORJJSVJrGhB6vjhkbrg== 2024-04-20T21:16:27.335 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.f] 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCKMCRmMRAQIhAAoBf1nsh8R4GjDWLRpPzIew== 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.4] 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCeMCRmS43YDhAAQHsEJ3Gf0CJ+ARhlzwdzlw== 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.5] 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQChMCRmN1anDBAAnrfxXXsADY0MLrb3d8t4Hg== 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.6] 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCkMCRmKF5gChAAPqZwxFy5U7jDA3kVkOtPrw== 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.7] 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCnMCRmXemiCBAARL3D9eBpFUdCiyUEXi63UA== 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout:[client.0] 2024-04-20T21:16:27.336 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCKMCRm2n0iKhAA5KYFmxM4raEhiZ188hFSNQ== 2024-04-20T21:16:27.338 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T21:16:27.386 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T21:16:27.514 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:27.518 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:27.647 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:27.652 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:27.772 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:27.776 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:27.899 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:27.906 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:28.033 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:28.039 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:28.157 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:28.162 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:28.281 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:28.286 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:28.411 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T21:16:28.417 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T21:16:28.542 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:28.549 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:28.669 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:28.676 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:28.796 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:28.802 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:16:28.926 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:28.932 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:29.060 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:29.064 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:29.190 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:29.195 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:29.323 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:29.327 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:16:29.441 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:16:29.446 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:16:29.569 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-20T21:16:29.570 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-20T21:16:29.644 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T21:16:29.790 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-20T21:16:29.863 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-20T21:16:29.887 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T21:16:29.983 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-20T21:16:30.006 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-20T21:16:30.069 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T21:16:30.162 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-20T21:16:30.185 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:30.193 DEBUG:teuthology.orchestra.run.smithi193:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:30.242 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-20T21:16:30.243 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T21:16:30.243 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T21:16:30.248 INFO:tasks.ceph.mon.a:Started 2024-04-20T21:16:30.248 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T21:16:30.248 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T21:16:30.282 INFO:tasks.ceph.mon.b:Started 2024-04-20T21:16:30.282 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T21:16:30.283 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T21:16:30.284 INFO:tasks.ceph.mon.c:Started 2024-04-20T21:16:30.284 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-20T21:16:30.285 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-20T21:16:30.285 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-20T21:16:30.289 INFO:tasks.ceph.mgr.y:Started 2024-04-20T21:16:30.289 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-20T21:16:30.289 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-20T21:16:30.291 INFO:tasks.ceph.mgr.x:Started 2024-04-20T21:16:30.291 INFO:tasks.ceph:Setting crush tunables to default 2024-04-20T21:16:30.291 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd crush tunables default 2024-04-20T21:16:30.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.464+0000 7fc680e00640 1 Processor -- start 2024-04-20T21:16:30.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.464+0000 7fc680e00640 1 -- start start 2024-04-20T21:16:30.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc680e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc67c155550 0x7fc67c15f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:30.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc680e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:30.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc680e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc67c15b2c0 con 0x7fc67c0cc130 2024-04-20T21:16:30.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc680e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc67c15f800 con 0x7fc67c155550 2024-04-20T21:16:30.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc680e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc67c15c150 con 0x7fc67c0cbd10 2024-04-20T21:16:30.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc679e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc67c155550 0x7fc67c15f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:30.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc671400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:30.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc67a800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc67c0cc130 0x7fc67c15c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50318/0 (socket says 172.21.15.102:50318) 2024-04-20T21:16:30.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.465+0000 7fc67a800640 1 -- 172.21.15.102:0/623144482 learned_addr learned my addr 172.21.15.102:0/623144482 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:30.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.466+0000 7fc671400640 1 -- 172.21.15.102:0/623144482 >> v2:172.21.15.193:3300/0 conn(0x7fc67c155550 msgr2=0x7fc67c15f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:30.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.466+0000 7fc671400640 1 --2- 172.21.15.102:0/623144482 >> v2:172.21.15.193:3300/0 conn(0x7fc67c155550 0x7fc67c15f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:30.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.466+0000 7fc671400640 1 -- 172.21.15.102:0/623144482 >> v1:172.21.15.102:6789/0 conn(0x7fc67c0cc130 legacy=0x7fc67c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:30.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.466+0000 7fc671400640 1 -- 172.21.15.102:0/623144482 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c15b540 con 0x7fc67c0cbd10 2024-04-20T21:16:30.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.466+0000 7fc679400640 1 -- 172.21.15.102:0/623144482 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700990588 0 0) 0x7fc674001110 con 0x7fc67c0cc130 2024-04-20T21:16:30.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.467+0000 7fc671400640 1 --2- 172.21.15.102:0/623144482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c0d0ab0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6640097a0 tx=0x7fc66402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c214fa8ed99d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:30.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.467+0000 7fc679400640 1 -- 172.21.15.102:0/623144482 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664034070 con 0x7fc67c0cbd10 2024-04-20T21:16:30.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.467+0000 7fc679400640 1 -- 172.21.15.102:0/623144482 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc664030800 con 0x7fc67c0cbd10 2024-04-20T21:16:30.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.467+0000 7fc679400640 1 -- 172.21.15.102:0/623144482 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664030bd0 con 0x7fc67c0cbd10 2024-04-20T21:16:30.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.468+0000 7fc680e00640 1 -- 172.21.15.102:0/623144482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 msgr2=0x7fc67c0d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:30.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.468+0000 7fc680e00640 1 --2- 172.21.15.102:0/623144482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c0d0ab0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6640097a0 tx=0x7fc66402b940 comp rx=0 tx=0).stop 2024-04-20T21:16:30.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.468+0000 7fc680e00640 1 -- 172.21.15.102:0/623144482 shutdown_connections 2024-04-20T21:16:30.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.468+0000 7fc680e00640 1 --2- 172.21.15.102:0/623144482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c0d0ab0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:30.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.468+0000 7fc680e00640 1 --2- 172.21.15.102:0/623144482 >> v2:172.21.15.193:3300/0 conn(0x7fc67c155550 0x7fc67c15f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:30.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.468+0000 7fc680e00640 1 -- 172.21.15.102:0/623144482 >> 172.21.15.102:0/623144482 conn(0x7fc67c06d2f0 msgr2=0x7fc67c072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:30.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.468+0000 7fc680e00640 1 -- 172.21.15.102:0/623144482 shutdown_connections 2024-04-20T21:16:30.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.468+0000 7fc680e00640 1 -- 172.21.15.102:0/623144482 wait complete. 2024-04-20T21:16:30.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.470+0000 7fc680e00640 1 Processor -- start 2024-04-20T21:16:30.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.470+0000 7fc680e00640 1 -- start start 2024-04-20T21:16:30.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc680e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c1e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc680e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc67c155550 0x7fc67c1e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc680e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c15e740 0x7fc67c154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc680e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c15bde0 con 0x7fc67c15e740 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc680e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c15bb00 con 0x7fc67c155550 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc680e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc67c155930 con 0x7fc67c0cbd10 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc679e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c1e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc679e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c1e47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:45376/0 (socket says 172.21.15.102:45376) 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc679e00640 1 -- 172.21.15.102:0/585829024 learned_addr learned my addr 172.21.15.102:0/585829024 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.471+0000 7fc671400640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc67c155550 0x7fc67c1e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc67a800640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c15e740 0x7fc67c154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc679e00640 1 -- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc67c155550 msgr2=0x7fc67c1e4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:30.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc679e00640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc67c155550 0x7fc67c1e4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc679e00640 1 -- 172.21.15.102:0/585829024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c15e740 msgr2=0x7fc67c154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc679e00640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c15e740 0x7fc67c154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc679e00640 1 -- 172.21.15.102:0/585829024 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c15ecf0 con 0x7fc67c0cbd10 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc671400640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc67c155550 0x7fc67c1e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc679e00640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c1e47a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009d10 tx=0x7fc66c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc672a00640 1 -- 172.21.15.102:0/585829024 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c018020 con 0x7fc67c0cbd10 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc672a00640 1 -- 172.21.15.102:0/585829024 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66c014480 con 0x7fc67c0cbd10 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc672a00640 1 -- 172.21.15.102:0/585829024 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c014830 con 0x7fc67c0cbd10 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67c15d4a0 con 0x7fc67c0cbd10 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.472+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67c15da60 con 0x7fc67c0cbd10 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.473+0000 7fc672a00640 1 -- 172.21.15.102:0/585829024 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc66c005d10 con 0x7fc67c0cbd10 2024-04-20T21:16:30.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.473+0000 7fc672a00640 1 -- 172.21.15.102:0/585829024 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc66c0133e0 con 0x7fc67c0cbd10 2024-04-20T21:16:30.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.474+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc67c04fdd0 con 0x7fc67c0cbd10 2024-04-20T21:16:30.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.477+0000 7fc672a00640 1 -- 172.21.15.102:0/585829024 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fc66c022030 con 0x7fc67c0cbd10 2024-04-20T21:16:30.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.566+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc67c0cc670 con 0x7fc67c0cbd10 2024-04-20T21:16:30.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.570+0000 7fc672a00640 1 -- 172.21.15.102:0/585829024 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fc66c010040 con 0x7fc67c0cbd10 2024-04-20T21:16:30.570 INFO:teuthology.orchestra.run.smithi102.stderr:adjusted tunables profile to default 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 msgr2=0x7fc67c1e47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c1e47a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009d10 tx=0x7fc66c00c490 comp rx=0 tx=0).stop 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 shutdown_connections 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c15e740 0x7fc67c154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc67c155550 0x7fc67c1e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 --2- 172.21.15.102:0/585829024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc67c0cbd10 0x7fc67c1e47a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 >> 172.21.15.102:0/585829024 conn(0x7fc67c06d2f0 msgr2=0x7fc67c0c5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 shutdown_connections 2024-04-20T21:16:30.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:30.572+0000 7fc680e00640 1 -- 172.21.15.102:0/585829024 wait complete. 2024-04-20T21:16:30.588 INFO:tasks.ceph:check_enable_crimson: False 2024-04-20T21:16:30.588 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-20T21:16:30.589 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:30.589 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-20T21:16:30.612 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:30.612+0000 7fad1d211040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:16:30.657 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:30.657 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-20T21:16:30.663 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:30.664+0000 7f96a744d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:16:30.774 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:30.774 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-20T21:16:30.810 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:30.810+0000 7f96a744d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:16:30.815 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:30.815+0000 7fad1d211040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:16:30.862 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:30.863+0000 7f96a744d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:16:30.882 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:30.882+0000 7fad1d211040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:16:30.893 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:16:30.893 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-20T21:16:30.962 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:30.962 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-20T21:16:30.985 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:30.985 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-20T21:16:31.048 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:31.048 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-20T21:16:31.108 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:31.109+0000 7f96a744d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:16:31.111 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-20T21:16:31.111 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-20T21:16:31.154 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:31.155+0000 7f96a744d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:16:31.174 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new cc15372f-7389-4989-b1f9-32dce20f8e38 0 2024-04-20T21:16:31.191 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:31.191+0000 7fad1d211040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:16:31.243 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:31.244+0000 7fad1d211040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:16:31.312 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.312+0000 7f459a94b640 1 Processor -- start 2024-04-20T21:16:31.312 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.312+0000 7f459a94b640 1 -- start start 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f459a94b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4594153210 0x7f45940c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f459a94b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f459415c5e0 0x7f4594159a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f459a94b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45940730f0 con 0x7f4594073a20 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f459a94b640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f4594159570 con 0x7f4594153210 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f459a94b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f45940733d0 con 0x7f459415c5e0 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f45937fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f459415c5e0 0x7f4594159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f45937fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f459415c5e0 0x7f4594159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:38156/0 (socket says 172.21.15.193:38156) 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f45937fe640 1 -- 172.21.15.193:0/977702653 learned_addr learned my addr 172.21.15.193:0/977702653 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f45937fe640 1 -- 172.21.15.193:0/977702653 >> v2:172.21.15.102:3300/0 conn(0x7f4594153210 msgr2=0x7f45940c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f45937fe640 1 --2- 172.21.15.193:0/977702653 >> v2:172.21.15.102:3300/0 conn(0x7f4594153210 0x7f45940c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.314 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f45937fe640 1 -- 172.21.15.193:0/977702653 >> v1:172.21.15.102:6789/0 conn(0x7f4594073a20 legacy=0x7f45940c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.315 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.313+0000 7f45937fe640 1 -- 172.21.15.193:0/977702653 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4594159200 con 0x7f459415c5e0 2024-04-20T21:16:31.315 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.314+0000 7f45937fe640 1 --2- 172.21.15.193:0/977702653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f459415c5e0 0x7f4594159a00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4584009870 tx=0x7f458402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5fa1d9f57789a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:31.315 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.314+0000 7f4592ffd640 1 -- 172.21.15.193:0/977702653 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584034070 con 0x7f459415c5e0 2024-04-20T21:16:31.315 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.314+0000 7f4592ffd640 1 -- 172.21.15.193:0/977702653 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4584030840 con 0x7f459415c5e0 2024-04-20T21:16:31.315 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.314+0000 7f4592ffd640 1 -- 172.21.15.193:0/977702653 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584030c10 con 0x7f459415c5e0 2024-04-20T21:16:31.318 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.315+0000 7f459a94b640 1 -- 172.21.15.193:0/977702653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f459415c5e0 msgr2=0x7f4594159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.318 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.315+0000 7f459a94b640 1 --2- 172.21.15.193:0/977702653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f459415c5e0 0x7f4594159a00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4584009870 tx=0x7f458402b980 comp rx=0 tx=0).stop 2024-04-20T21:16:31.318 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.315+0000 7f459a94b640 1 -- 172.21.15.193:0/977702653 shutdown_connections 2024-04-20T21:16:31.318 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.315+0000 7f459a94b640 1 --2- 172.21.15.193:0/977702653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f459415c5e0 0x7f4594159a00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.318 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.315+0000 7f459a94b640 1 --2- 172.21.15.193:0/977702653 >> v2:172.21.15.102:3300/0 conn(0x7f4594153210 0x7f45940c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.318 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.315+0000 7f459a94b640 1 -- 172.21.15.193:0/977702653 >> 172.21.15.193:0/977702653 conn(0x7f4594075080 msgr2=0x7f4594075460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.315+0000 7f459a94b640 1 -- 172.21.15.193:0/977702653 shutdown_connections 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.315+0000 7f459a94b640 1 -- 172.21.15.193:0/977702653 wait complete. 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.316+0000 7f459a94b640 1 Processor -- start 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.316+0000 7f459a94b640 1 -- start start 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f459a94b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4594073a20 0x7f45941f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f459a94b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4594153210 0x7f45941e4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f459a94b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f459415c5e0 0x7f4594147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f459a94b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45940c5d80 con 0x7f4594073a20 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f459a94b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45940c5c00 con 0x7f459415c5e0 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f459a94b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f45940c9600 con 0x7f4594153210 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f45937fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4594153210 0x7f45941e4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f45937fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4594153210 0x7f45941e4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:38164/0 (socket says 172.21.15.193:38164) 2024-04-20T21:16:31.319 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f45937fe640 1 -- 172.21.15.193:0/3478479264 learned_addr learned my addr 172.21.15.193:0/3478479264 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.317+0000 7f4598ee8640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f459415c5e0 0x7f4594147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4598ee8640 1 -- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4594153210 msgr2=0x7f45941e4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4598ee8640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4594153210 0x7f45941e4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4598ee8640 1 -- 172.21.15.193:0/3478479264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4594073a20 msgr2=0x7f45941f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4598ee8640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4594073a20 0x7f45941f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4598ee8640 1 -- 172.21.15.193:0/3478479264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459414f6e0 con 0x7f459415c5e0 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f45937fe640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4594153210 0x7f45941e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4598ee8640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f459415c5e0 0x7f4594147330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f458c00c7b0 tx=0x7f458c00cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4590ff9640 1 -- 172.21.15.193:0/3478479264 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c017070 con 0x7f459415c5e0 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4590ff9640 1 -- 172.21.15.193:0/3478479264 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458c003b90 con 0x7f459415c5e0 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4594158ae0 con 0x7f459415c5e0 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4590ff9640 1 -- 172.21.15.193:0/3478479264 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c005070 con 0x7f459415c5e0 2024-04-20T21:16:31.320 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459414fc30 con 0x7f459415c5e0 2024-04-20T21:16:31.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4590ff9640 1 -- 172.21.15.193:0/3478479264 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f458c01a070 con 0x7f459415c5e0 2024-04-20T21:16:31.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.318+0000 7f4590ff9640 1 -- 172.21.15.193:0/3478479264 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f458c01c780 con 0x7f459415c5e0 2024-04-20T21:16:31.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.319+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459406a0c0 con 0x7f459415c5e0 2024-04-20T21:16:31.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.321+0000 7f4590ff9640 1 -- 172.21.15.193:0/3478479264 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f459406a0c0 con 0x7f459415c5e0 2024-04-20T21:16:31.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.393+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cc15372f-7389-4989-b1f9-32dce20f8e38", "id": 0} v 0) -- 0x7f4594074340 con 0x7f459415c5e0 2024-04-20T21:16:31.397 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.398+0000 7f4590ff9640 1 -- 172.21.15.193:0/3478479264 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cc15372f-7389-4989-b1f9-32dce20f8e38", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f458c021020 con 0x7f459415c5e0 2024-04-20T21:16:31.398 INFO:teuthology.orchestra.run.smithi193.stdout:0 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f459415c5e0 msgr2=0x7f4594147330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f459415c5e0 0x7f4594147330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f458c00c7b0 tx=0x7f458c00cc70 comp rx=0 tx=0).stop 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 shutdown_connections 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f459415c5e0 0x7f4594147330 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4594153210 0x7f45941e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 --2- 172.21.15.193:0/3478479264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4594073a20 0x7f45941f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 >> 172.21.15.193:0/3478479264 conn(0x7f4594075080 msgr2=0x7f459414d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 shutdown_connections 2024-04-20T21:16:31.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.399+0000 7f459a94b640 1 -- 172.21.15.193:0/3478479264 wait complete. 2024-04-20T21:16:31.409 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5 1 2024-04-20T21:16:31.509 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.510+0000 7f8e5d5f4640 1 Processor -- start 2024-04-20T21:16:31.509 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.510+0000 7f8e5d5f4640 1 -- start start 2024-04-20T21:16:31.511 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.510+0000 7f8e5d5f4640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8e58075110 0x7f8e580c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.511 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.510+0000 7f8e5d5f4640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8e580759a0 0x7f8e580c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.511 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.510+0000 7f8e5d5f4640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5815e960 con 0x7f8e580754f0 2024-04-20T21:16:31.511 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.510+0000 7f8e5d5f4640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8e5815ec20 con 0x7f8e58075110 2024-04-20T21:16:31.511 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.510+0000 7f8e5d5f4640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8e5815ef00 con 0x7f8e580759a0 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.511+0000 7f8e4ffff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8e580759a0 0x7f8e580c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.511+0000 7f8e4ffff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8e580759a0 0x7f8e580c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54394/0 (socket says 172.21.15.193:54394) 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.511+0000 7f8e4ffff640 1 -- 172.21.15.193:0/2436479818 learned_addr learned my addr 172.21.15.193:0/2436479818 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.511+0000 7f8e4ffff640 1 -- 172.21.15.193:0/2436479818 >> v2:172.21.15.102:3300/0 conn(0x7f8e58075110 msgr2=0x7f8e580c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.511+0000 7f8e4ffff640 1 --2- 172.21.15.193:0/2436479818 >> v2:172.21.15.102:3300/0 conn(0x7f8e58075110 0x7f8e580c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.511+0000 7f8e4ffff640 1 -- 172.21.15.193:0/2436479818 >> v1:172.21.15.102:6789/0 conn(0x7f8e580754f0 legacy=0x7f8e5806fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.511+0000 7f8e4ffff640 1 -- 172.21.15.193:0/2436479818 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5815f1e0 con 0x7f8e580759a0 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e4ffff640 1 --2- 172.21.15.193:0/2436479818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580759a0 0x7f8e580c79e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e5806d300 tx=0x7f8e4002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f945f3bed9f8cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e567fc640 1 -- 172.21.15.193:0/2436479818 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40034070 con 0x7f8e580759a0 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e567fc640 1 -- 172.21.15.193:0/2436479818 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e40030860 con 0x7f8e580759a0 2024-04-20T21:16:31.512 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e567fc640 1 -- 172.21.15.193:0/2436479818 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40030c30 con 0x7f8e580759a0 2024-04-20T21:16:31.515 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/2436479818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580759a0 msgr2=0x7f8e580c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e5d5f4640 1 --2- 172.21.15.193:0/2436479818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580759a0 0x7f8e580c79e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e5806d300 tx=0x7f8e4002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/2436479818 shutdown_connections 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e5d5f4640 1 --2- 172.21.15.193:0/2436479818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580759a0 0x7f8e580c79e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e5d5f4640 1 --2- 172.21.15.193:0/2436479818 >> v2:172.21.15.102:3300/0 conn(0x7f8e58075110 0x7f8e580c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/2436479818 >> 172.21.15.193:0/2436479818 conn(0x7f8e580cd8a0 msgr2=0x7f8e580cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.512+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/2436479818 shutdown_connections 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.513+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/2436479818 wait complete. 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e5d5f4640 1 Processor -- start 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e5d5f4640 1 -- start start 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e5d5f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e58075110 0x7f8e581e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e5d5f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580754f0 0x7f8e581f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e5d5f4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e580759a0 0x7f8e581e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e5d5f4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5806f7f0 con 0x7f8e580759a0 2024-04-20T21:16:31.516 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e5d5f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5815f7a0 con 0x7f8e580754f0 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e5d5f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8e58072880 con 0x7f8e58075110 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580754f0 0x7f8e581f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580754f0 0x7f8e581f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54400/0 (socket says 172.21.15.193:54400) 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e4ffff640 1 -- 172.21.15.193:0/1266859394 learned_addr learned my addr 172.21.15.193:0/1266859394 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.514+0000 7f8e56ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e58075110 0x7f8e581e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4ffff640 1 -- 172.21.15.193:0/1266859394 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e58075110 msgr2=0x7f8e581e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4ffff640 1 --2- 172.21.15.193:0/1266859394 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e58075110 0x7f8e581e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4ffff640 1 -- 172.21.15.193:0/1266859394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e580759a0 msgr2=0x7f8e581e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4ffff640 1 --2- 172.21.15.193:0/1266859394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e580759a0 0x7f8e581e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4ffff640 1 -- 172.21.15.193:0/1266859394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e581590f0 con 0x7f8e580754f0 2024-04-20T21:16:31.517 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4ffff640 1 --2- 172.21.15.193:0/1266859394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580754f0 0x7f8e581f4340 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009960 tx=0x7f8e400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4f7fe640 1 -- 172.21.15.193:0/1266859394 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40034040 con 0x7f8e580754f0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4f7fe640 1 -- 172.21.15.193:0/1266859394 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e40039d90 con 0x7f8e580754f0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e581593d0 con 0x7f8e580754f0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4f7fe640 1 -- 172.21.15.193:0/1266859394 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40043630 con 0x7f8e580754f0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581f4880 con 0x7f8e580754f0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4f7fe640 1 -- 172.21.15.193:0/1266859394 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8e40037070 con 0x7f8e580754f0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.515+0000 7f8e4f7fe640 1 -- 172.21.15.193:0/1266859394 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f8e4004c950 con 0x7f8e580754f0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.516+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e5804fdd0 con 0x7f8e580754f0 2024-04-20T21:16:31.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.518+0000 7f8e4f7fe640 1 -- 172.21.15.193:0/1266859394 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f8e4003e070 con 0x7f8e580754f0 2024-04-20T21:16:31.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.591+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5", "id": 1} v 0) -- 0x7f8e58070120 con 0x7f8e580754f0 2024-04-20T21:16:31.597 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.597+0000 7f8e4f7fe640 1 -- 172.21.15.193:0/1266859394 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8e4004cb50 con 0x7f8e580754f0 2024-04-20T21:16:31.597 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-04-20T21:16:31.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580754f0 msgr2=0x7f8e581f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 --2- 172.21.15.193:0/1266859394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580754f0 0x7f8e581f4340 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009960 tx=0x7f8e400046c0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 shutdown_connections 2024-04-20T21:16:31.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 --2- 172.21.15.193:0/1266859394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e580759a0 0x7f8e581e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 --2- 172.21.15.193:0/1266859394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e580754f0 0x7f8e581f4340 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 --2- 172.21.15.193:0/1266859394 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e58075110 0x7f8e581e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 >> 172.21.15.193:0/1266859394 conn(0x7f8e580cd8a0 msgr2=0x7f8e580ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:31.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 shutdown_connections 2024-04-20T21:16:31.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.598+0000 7f8e5d5f4640 1 -- 172.21.15.193:0/1266859394 wait complete. 2024-04-20T21:16:31.608 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 4582fd67-4f82-4bee-9055-5905a095036b 2 2024-04-20T21:16:31.705 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.706+0000 7f4a6c011640 1 Processor -- start 2024-04-20T21:16:31.705 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.706+0000 7f4a6c011640 1 -- start start 2024-04-20T21:16:31.709 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a6c011640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4a640728b0 0x7f4a64163840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.709 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a6c011640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a64157f50 con 0x7f4a6406e430 2024-04-20T21:16:31.709 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a6c011640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a641581f0 con 0x7f4a641523a0 2024-04-20T21:16:31.709 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a6c011640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f4a640c3de0 con 0x7f4a640728b0 2024-04-20T21:16:31.709 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a69dad640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f4a641523a0 0x7f4a6406d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:43516/0 (socket says 172.21.15.193:43516) 2024-04-20T21:16:31.709 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a69dad640 1 -- 172.21.15.193:0/1190227575 learned_addr learned my addr 172.21.15.193:0/1190227575 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-20T21:16:31.709 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4200381769 0 0) 0x7f4a641581f0 con 0x7f4a641523a0 2024-04-20T21:16:31.709 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a50003520 con 0x7f4a641523a0 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 601042700 0 0) 0x7f4a50003520 con 0x7f4a641523a0 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 >> v2:172.21.15.102:3300/0 conn(0x7f4a640728b0 msgr2=0x7f4a64163840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 --2- 172.21.15.193:0/1190227575 >> v2:172.21.15.102:3300/0 conn(0x7f4a640728b0 0x7f4a64163840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 >> v1:172.21.15.102:6789/0 conn(0x7f4a6406e430 legacy=0x7f4a640c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 --> v1:172.21.15.193:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a641584b0 con 0x7f4a641523a0 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 <== mon.1 v1:172.21.15.193:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7f4a60001670 con 0x7f4a641523a0 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 >> v1:172.21.15.193:6789/0 conn(0x7f4a641523a0 legacy=0x7f4a6406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a500039b0 0x7f4a50003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a50004800 0x7f4a50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a500052e0 0x7f4a5000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5000e3c0 con 0x7f4a500052e0 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5000e570 con 0x7f4a500039b0 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.707+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5000e770 con 0x7f4a50004800 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a69dad640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a500039b0 0x7f4a50003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a695ac640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a50004800 0x7f4a50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a69dad640 1 -- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a50004800 msgr2=0x7f4a50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a69dad640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a50004800 0x7f4a50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a69dad640 1 -- 172.21.15.193:0/1190227575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a500052e0 msgr2=0x7f4a5000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a69dad640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a500052e0 0x7f4a5000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.712 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a69dad640 1 -- 172.21.15.193:0/1190227575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a50003520 con 0x7f4a500039b0 2024-04-20T21:16:31.712 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a6a5ae640 1 -- 172.21.15.193:0/1190227575 reap_dead start 2024-04-20T21:16:31.712 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.708+0000 7f4a695ac640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a50004800 0x7f4a50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:31.712 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.709+0000 7f4a69dad640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a500039b0 0x7f4a50003db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a60007fd0 tx=0x7f4a50005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a3a6cbb5aafefb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:31.712 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.709+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60007d50 con 0x7f4a500039b0 2024-04-20T21:16:31.712 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.709+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a60030070 con 0x7f4a500039b0 2024-04-20T21:16:31.712 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.709+0000 7f4a68dab640 1 -- 172.21.15.193:0/1190227575 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6002cd80 con 0x7f4a500039b0 2024-04-20T21:16:31.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.710+0000 7f4a6c011640 1 -- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a500039b0 msgr2=0x7f4a50003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.710+0000 7f4a6c011640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a500039b0 0x7f4a50003db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a60007fd0 tx=0x7f4a50005830 comp rx=0 tx=0).stop 2024-04-20T21:16:31.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.710+0000 7f4a6c011640 1 -- 172.21.15.193:0/1190227575 shutdown_connections 2024-04-20T21:16:31.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.710+0000 7f4a6c011640 1 --2- 172.21.15.193:0/1190227575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a500039b0 0x7f4a50003db0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.710+0000 7f4a6c011640 1 -- 172.21.15.193:0/1190227575 >> 172.21.15.193:0/1190227575 conn(0x7f4a640cdbf0 msgr2=0x7f4a640ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:31.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.710+0000 7f4a6c011640 1 -- 172.21.15.193:0/1190227575 shutdown_connections 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.710+0000 7f4a6c011640 1 -- 172.21.15.193:0/1190227575 wait complete. 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.711+0000 7f4a6c011640 1 Processor -- start 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.711+0000 7f4a6c011640 1 -- start start 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a6c011640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a500039b0 0x7f4a64150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a6c011640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a64157b90 0x7f4a64149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a6c011640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6414a2d0 0x7f4a6414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a6c011640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64158d50 con 0x7f4a500039b0 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a6c011640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64158a70 con 0x7f4a6414a2d0 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a6c011640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4a640b3a50 con 0x7f4a64157b90 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a64157b90 0x7f4a64149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a6a5ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6414a2d0 0x7f4a6414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a64157b90 0x7f4a64149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:38180/0 (socket says 172.21.15.193:38180) 2024-04-20T21:16:31.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 -- 172.21.15.193:0/1583199806 learned_addr learned my addr 172.21.15.193:0/1583199806 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 -- 172.21.15.193:0/1583199806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6414a2d0 msgr2=0x7f4a6414f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 --2- 172.21.15.193:0/1583199806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6414a2d0 0x7f4a6414f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 -- 172.21.15.193:0/1583199806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a500039b0 msgr2=0x7f4a64150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a69dad640 1 --2- 172.21.15.193:0/1583199806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a500039b0 0x7f4a64150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 --2- 172.21.15.193:0/1583199806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a500039b0 0x7f4a64150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 -- 172.21.15.193:0/1583199806 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a641f0450 con 0x7f4a64157b90 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a695ac640 1 --2- 172.21.15.193:0/1583199806 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a64157b90 0x7f4a64149d90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400f7a0 tx=0x7f4a5400fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.712+0000 7f4a5a7fc640 1 -- 172.21.15.193:0/1583199806 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54001240 con 0x7f4a64157b90 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.713+0000 7f4a5a7fc640 1 -- 172.21.15.193:0/1583199806 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a54018070 con 0x7f4a64157b90 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.713+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a641f0730 con 0x7f4a64157b90 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.713+0000 7f4a5a7fc640 1 -- 172.21.15.193:0/1583199806 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54005310 con 0x7f4a64157b90 2024-04-20T21:16:31.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.713+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a641f0cf0 con 0x7f4a64157b90 2024-04-20T21:16:31.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.713+0000 7f4a5a7fc640 1 -- 172.21.15.193:0/1583199806 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a5400d730 con 0x7f4a64157b90 2024-04-20T21:16:31.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.713+0000 7f4a5a7fc640 1 -- 172.21.15.193:0/1583199806 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f4a5401d460 con 0x7f4a64157b90 2024-04-20T21:16:31.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.713+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a641f10d0 con 0x7f4a64157b90 2024-04-20T21:16:31.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.716+0000 7f4a5a7fc640 1 -- 172.21.15.193:0/1583199806 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f4a641f10d0 con 0x7f4a64157b90 2024-04-20T21:16:31.788 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.788+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4582fd67-4f82-4bee-9055-5905a095036b", "id": 2} v 0) -- 0x7f4a64156f90 con 0x7f4a64157b90 2024-04-20T21:16:31.794 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.794+0000 7f4a5a7fc640 1 -- 172.21.15.193:0/1583199806 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4582fd67-4f82-4bee-9055-5905a095036b", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f4a540035f0 con 0x7f4a64157b90 2024-04-20T21:16:31.794 INFO:teuthology.orchestra.run.smithi193.stdout:2 2024-04-20T21:16:31.795 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a64157b90 msgr2=0x7f4a64149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.795 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 --2- 172.21.15.193:0/1583199806 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a64157b90 0x7f4a64149d90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400f7a0 tx=0x7f4a5400fc60 comp rx=0 tx=0).stop 2024-04-20T21:16:31.795 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 shutdown_connections 2024-04-20T21:16:31.795 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 --2- 172.21.15.193:0/1583199806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a6414a2d0 0x7f4a6414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.795 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 --2- 172.21.15.193:0/1583199806 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a64157b90 0x7f4a64149d90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.795 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 --2- 172.21.15.193:0/1583199806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a500039b0 0x7f4a64150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.796 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 >> 172.21.15.193:0/1583199806 conn(0x7f4a640cdbf0 msgr2=0x7f4a640c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:31.796 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 shutdown_connections 2024-04-20T21:16:31.796 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.795+0000 7f4a6c011640 1 -- 172.21.15.193:0/1583199806 wait complete. 2024-04-20T21:16:31.805 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 32da37a5-4aee-4279-a79a-2b689b1ebd48 3 2024-04-20T21:16:31.883 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:31.884+0000 7f96a744d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:16:31.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.904+0000 7f1e8e0d2640 1 Processor -- start 2024-04-20T21:16:31.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.904+0000 7f1e8e0d2640 1 -- start start 2024-04-20T21:16:31.906 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e8e0d2640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1e880cf1b0 0x7f1e880ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e8e0d2640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8815b3b0 con 0x7f1e880c3ec0 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e8e0d2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e880c4390 con 0x7f1e880cedd0 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e8e0d2640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8815b9a0 con 0x7f1e880cf1b0 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e877fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f1e880cedd0 0x7f1e88155130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:43522/0 (socket says 172.21.15.193:43522) 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e877fe640 1 -- 172.21.15.193:0/417178850 learned_addr learned my addr 172.21.15.193:0/417178850 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3353149680 0 0) 0x7f1e880c4390 con 0x7f1e880cedd0 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e74003520 con 0x7f1e880cedd0 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e87fff640 1 --2- 172.21.15.193:0/417178850 >> v2:172.21.15.102:3300/0 conn(0x7f1e880cf1b0 0x7f1e880ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 475321416 0 0) 0x7f1e74003520 con 0x7f1e880cedd0 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 >> v2:172.21.15.102:3300/0 conn(0x7f1e880cf1b0 msgr2=0x7f1e880ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 --2- 172.21.15.193:0/417178850 >> v2:172.21.15.102:3300/0 conn(0x7f1e880cf1b0 0x7f1e880ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 >> v1:172.21.15.102:6789/0 conn(0x7f1e880c3ec0 legacy=0x7f1e88163c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 --> v1:172.21.15.193:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e880c4670 con 0x7f1e880cedd0 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 <== mon.1 v1:172.21.15.193:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7f1e700019f0 con 0x7f1e880cedd0 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 >> v1:172.21.15.193:6789/0 conn(0x7f1e880cedd0 legacy=0x7f1e88155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e74004800 0x7f1e74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e74005370 0x7f1e7400b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8815b9a0 con 0x7f1e74004800 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7400c2c0 con 0x7f1e74005370 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.905+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7400c4c0 con 0x7f1e74003a00 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.906+0000 7f1e877fe640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.906+0000 7f1e87fff640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e74005370 0x7f1e7400b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.906+0000 7f1e877fe640 1 -- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e74005370 msgr2=0x7f1e7400b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.906+0000 7f1e877fe640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e74005370 0x7f1e7400b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.906+0000 7f1e877fe640 1 -- 172.21.15.193:0/417178850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e74004800 msgr2=0x7f1e74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.906+0000 7f1e877fe640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e74004800 0x7f1e74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.906+0000 7f1e877fe640 1 -- 172.21.15.193:0/417178850 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e74003520 con 0x7f1e74003a00 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.906+0000 7f1e87fff640 1 -- 172.21.15.193:0/417178850 reap_dead start 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.907+0000 7f1e877fe640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e74003e00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000a7b0 tx=0x7f1e74005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1dcb881bc3b53bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.907+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000a3d0 con 0x7f1e74003a00 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.907+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70034630 con 0x7f1e74003a00 2024-04-20T21:16:31.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.907+0000 7f1e867fc640 1 -- 172.21.15.193:0/417178850 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70034a60 con 0x7f1e74003a00 2024-04-20T21:16:31.912 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.908+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 msgr2=0x7f1e74003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.912 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.908+0000 7f1e8e0d2640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e74003e00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000a7b0 tx=0x7f1e74005830 comp rx=0 tx=0).stop 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.908+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/417178850 shutdown_connections 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.908+0000 7f1e8e0d2640 1 --2- 172.21.15.193:0/417178850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e74003e00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.908+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/417178850 >> 172.21.15.193:0/417178850 conn(0x7f1e880736e0 msgr2=0x7f1e8806f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.908+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/417178850 shutdown_connections 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.908+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/417178850 wait complete. 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.909+0000 7f1e8e0d2640 1 Processor -- start 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.909+0000 7f1e8e0d2640 1 -- start start 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e8e0d2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e88158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e8e0d2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88151670 0x7f1e881572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e8e0d2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e881577f0 0x7f1e881527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e8e0d2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1e880cad70 con 0x7f1e881577f0 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e8e0d2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8815b630 con 0x7f1e88151670 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e8e0d2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1e880caef0 con 0x7f1e74003a00 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88151670 0x7f1e881572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e877fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e88158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88151670 0x7f1e881572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54448/0 (socket says 172.21.15.193:54448) 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e877fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e88158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:38192/0 (socket says 172.21.15.193:38192) 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 -- 172.21.15.193:0/1395957320 learned_addr learned my addr 172.21.15.193:0/1395957320 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 -- 172.21.15.193:0/1395957320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 msgr2=0x7f1e88158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 --2- 172.21.15.193:0/1395957320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e88158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 -- 172.21.15.193:0/1395957320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e881577f0 msgr2=0x7f1e881527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 --2- 172.21.15.193:0/1395957320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e881577f0 0x7f1e881527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 -- 172.21.15.193:0/1395957320 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e88151c20 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.910+0000 7f1e86ffd640 1 --2- 172.21.15.193:0/1395957320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88151670 0x7f1e881572b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c4e0 tx=0x7f1e7800c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.911+0000 7f1e7ffff640 1 -- 172.21.15.193:0/1395957320 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78011790 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.911+0000 7f1e7ffff640 1 -- 172.21.15.193:0/1395957320 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e78017070 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.911+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8814e5e0 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.911+0000 7f1e7ffff640 1 -- 172.21.15.193:0/1395957320 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78011bb0 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.911+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e881f42e0 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.911+0000 7f1e7ffff640 1 -- 172.21.15.193:0/1395957320 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e7801c460 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.911+0000 7f1e7ffff640 1 -- 172.21.15.193:0/1395957320 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1e78012c60 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.912+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8804fdd0 con 0x7f1e88151670 2024-04-20T21:16:31.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.914+0000 7f1e7ffff640 1 -- 172.21.15.193:0/1395957320 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f1e78011d50 con 0x7f1e88151670 2024-04-20T21:16:31.975 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:31.976+0000 7f96a744d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:16:31.985 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.986+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd new", "uuid": "32da37a5-4aee-4279-a79a-2b689b1ebd48", "id": 3} v 0) -- 0x7f1e880b4cb0 con 0x7f1e88151670 2024-04-20T21:16:31.992 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.993+0000 7f1e7ffff640 1 -- 172.21.15.193:0/1395957320 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "32da37a5-4aee-4279-a79a-2b689b1ebd48", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1e78012790 con 0x7f1e88151670 2024-04-20T21:16:31.992 INFO:teuthology.orchestra.run.smithi193.stdout:3 2024-04-20T21:16:31.993 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.993+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88151670 msgr2=0x7f1e881572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:31.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.993+0000 7f1e8e0d2640 1 --2- 172.21.15.193:0/1395957320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88151670 0x7f1e881572b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c4e0 tx=0x7f1e7800c9a0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.994+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 shutdown_connections 2024-04-20T21:16:31.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.994+0000 7f1e8e0d2640 1 --2- 172.21.15.193:0/1395957320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e881577f0 0x7f1e881527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.994+0000 7f1e8e0d2640 1 --2- 172.21.15.193:0/1395957320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88151670 0x7f1e881572b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.994+0000 7f1e8e0d2640 1 --2- 172.21.15.193:0/1395957320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e74003a00 0x7f1e88158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:31.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.994+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 >> 172.21.15.193:0/1395957320 conn(0x7f1e880736e0 msgr2=0x7f1e880ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:31.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.994+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 shutdown_connections 2024-04-20T21:16:31.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:31.994+0000 7f1e8e0d2640 1 -- 172.21.15.193:0/1395957320 wait complete. 2024-04-20T21:16:32.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 32489a66-07e4-4062-a82b-4183f3b727e5 4 2024-04-20T21:16:32.021 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:32.021+0000 7f96a744d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:16:32.101 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.102+0000 7f959b315640 1 Processor -- start 2024-04-20T21:16:32.101 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.102+0000 7f959b315640 1 -- start start 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f959b315640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9594153150 0x7f95940c9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f959b315640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95940c9b80 con 0x7f9594153530 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f959b315640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95940ca410 con 0x7f95940b4d90 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f959b315640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f959415c5b0 con 0x7f9594153150 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f95990b1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9594153150 0x7f95940c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f95988b0640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f95940b4d90 0x7f95940c42a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:43538/0 (socket says 172.21.15.193:43538) 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f95988b0640 1 -- 172.21.15.193:0/3715276231 learned_addr learned my addr 172.21.15.193:0/3715276231 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f95990b1640 1 -- 172.21.15.193:0/3715276231 >> v1:172.21.15.193:6789/0 conn(0x7f95940b4d90 legacy=0x7f95940c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f95990b1640 1 -- 172.21.15.193:0/3715276231 >> v1:172.21.15.102:6789/0 conn(0x7f9594153530 legacy=0x7f95940cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:32.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.103+0000 7f95990b1640 1 -- 172.21.15.193:0/3715276231 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95940ca6f0 con 0x7f9594153150 2024-04-20T21:16:32.104 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.104+0000 7f95990b1640 1 --2- 172.21.15.193:0/3715276231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153150 0x7f95940c9560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f95900097a0 tx=0x7f959002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace313d42ccbf5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:32.104 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.104+0000 7f958bfff640 1 -- 172.21.15.193:0/3715276231 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590035020 con 0x7f9594153150 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.104+0000 7f958bfff640 1 -- 172.21.15.193:0/3715276231 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9590031830 con 0x7f9594153150 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.104+0000 7f958bfff640 1 -- 172.21.15.193:0/3715276231 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590031c00 con 0x7f9594153150 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.104+0000 7f959b315640 1 -- 172.21.15.193:0/3715276231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153150 msgr2=0x7f95940c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.104+0000 7f959b315640 1 --2- 172.21.15.193:0/3715276231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153150 0x7f95940c9560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f95900097a0 tx=0x7f959002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.105+0000 7f959b315640 1 -- 172.21.15.193:0/3715276231 shutdown_connections 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.105+0000 7f959b315640 1 --2- 172.21.15.193:0/3715276231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153150 0x7f95940c9560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.105+0000 7f959b315640 1 -- 172.21.15.193:0/3715276231 >> 172.21.15.193:0/3715276231 conn(0x7f959406f110 msgr2=0x7f959406f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.105+0000 7f959b315640 1 -- 172.21.15.193:0/3715276231 shutdown_connections 2024-04-20T21:16:32.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.105+0000 7f959b315640 1 -- 172.21.15.193:0/3715276231 wait complete. 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.106+0000 7f959b315640 1 Processor -- start 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.106+0000 7f959b315640 1 -- start start 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.106+0000 7f959b315640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95940b4d90 0x7f959406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.106+0000 7f959b315640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9594153150 0x7f9594155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.106+0000 7f959b315640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153530 0x7f959406bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.106+0000 7f959b315640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f95940c49b0 con 0x7f95940b4d90 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.106+0000 7f959b315640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95940ca1c0 con 0x7f9594153530 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.106+0000 7f959b315640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f95940c4b30 con 0x7f9594153150 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95988b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9594153150 0x7f9594155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153530 0x7f959406bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153530 0x7f959406bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54476/0 (socket says 172.21.15.193:54476) 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 -- 172.21.15.193:0/3319757199 learned_addr learned my addr 172.21.15.193:0/3319757199 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:32.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 -- 172.21.15.193:0/3319757199 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9594153150 msgr2=0x7f9594155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 --2- 172.21.15.193:0/3319757199 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9594153150 0x7f9594155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 -- 172.21.15.193:0/3319757199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95940b4d90 msgr2=0x7f959406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 --2- 172.21.15.193:0/3319757199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95940b4d90 0x7f959406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 -- 172.21.15.193:0/3319757199 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95941f2200 con 0x7f9594153530 2024-04-20T21:16:32.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95988b0640 1 --2- 172.21.15.193:0/3319757199 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9594153150 0x7f9594155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:32.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f95998b2640 1 --2- 172.21.15.193:0/3319757199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153530 0x7f959406bf90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c780 tx=0x7f958c00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:32.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f9589ffb640 1 -- 172.21.15.193:0/3319757199 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c017070 con 0x7f9594153530 2024-04-20T21:16:32.110 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f9589ffb640 1 -- 172.21.15.193:0/3319757199 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c003da0 con 0x7f9594153530 2024-04-20T21:16:32.110 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95941f1680 con 0x7f9594153530 2024-04-20T21:16:32.110 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f9589ffb640 1 -- 172.21.15.193:0/3319757199 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c0052b0 con 0x7f9594153530 2024-04-20T21:16:32.110 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95941f1c40 con 0x7f9594153530 2024-04-20T21:16:32.110 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.107+0000 7f9589ffb640 1 -- 172.21.15.193:0/3319757199 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f958c01a070 con 0x7f9594153530 2024-04-20T21:16:32.110 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.108+0000 7f9589ffb640 1 -- 172.21.15.193:0/3319757199 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f958c00a5e0 con 0x7f9594153530 2024-04-20T21:16:32.110 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.108+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95940c9240 con 0x7f9594153530 2024-04-20T21:16:32.110 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.110+0000 7f9589ffb640 1 -- 172.21.15.193:0/3319757199 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f958c00a7e0 con 0x7f9594153530 2024-04-20T21:16:32.163 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:32.163+0000 7fad1d211040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:16:32.182 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.183+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd new", "uuid": "32489a66-07e4-4062-a82b-4183f3b727e5", "id": 4} v 0) -- 0x7f959404fdd0 con 0x7f9594153530 2024-04-20T21:16:32.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.189+0000 7f9589ffb640 1 -- 172.21.15.193:0/3319757199 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "32489a66-07e4-4062-a82b-4183f3b727e5", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f958c005450 con 0x7f9594153530 2024-04-20T21:16:32.189 INFO:teuthology.orchestra.run.smithi193.stdout:4 2024-04-20T21:16:32.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153530 msgr2=0x7f959406bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 --2- 172.21.15.193:0/3319757199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153530 0x7f959406bf90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c780 tx=0x7f958c00cc40 comp rx=0 tx=0).stop 2024-04-20T21:16:32.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 shutdown_connections 2024-04-20T21:16:32.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 --2- 172.21.15.193:0/3319757199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9594153530 0x7f959406bf90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 --2- 172.21.15.193:0/3319757199 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9594153150 0x7f9594155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 --2- 172.21.15.193:0/3319757199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95940b4d90 0x7f959406d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 >> 172.21.15.193:0/3319757199 conn(0x7f959406f110 msgr2=0x7f9594159130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:32.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 shutdown_connections 2024-04-20T21:16:32.191 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.190+0000 7f959b315640 1 -- 172.21.15.193:0/3319757199 wait complete. 2024-04-20T21:16:32.194 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:32.195+0000 7f96a744d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:16:32.200 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new bc20150c-0f8e-498b-996c-83f43f44964a 5 2024-04-20T21:16:32.243 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:32.244+0000 7f96a744d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:16:32.265 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:32.266+0000 7fad1d211040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:16:32.298 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:32.299+0000 7f96a744d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:16:32.315 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:32.315+0000 7fad1d211040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:16:32.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.338+0000 7f6cf617e640 1 Processor -- start 2024-04-20T21:16:32.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.338+0000 7f6cf617e640 1 -- start start 2024-04-20T21:16:32.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.339+0000 7f6cf617e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0074f10 0x7f6cf0071e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.339+0000 7f6cf617e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf0072a90 con 0x7f6cf00723b0 2024-04-20T21:16:32.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.339+0000 7f6cf617e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf015fb90 con 0x7f6cf0159a20 2024-04-20T21:16:32.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.339+0000 7f6cf617e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf0072840 con 0x7f6cf0074f10 2024-04-20T21:16:32.340 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.339+0000 7f6ceeffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0074f10 0x7f6cf0071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.340 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.339+0000 7f6cef7fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6cf0159a20 0x7f6cf00c82a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:43554/0 (socket says 172.21.15.193:43554) 2024-04-20T21:16:32.340 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.339+0000 7f6cef7fe640 1 -- 172.21.15.193:0/2949897563 learned_addr learned my addr 172.21.15.193:0/2949897563 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-20T21:16:32.340 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.340+0000 7f6ceeffd640 1 -- 172.21.15.193:0/2949897563 >> v1:172.21.15.193:6789/0 conn(0x7f6cf0159a20 legacy=0x7f6cf00c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.340 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.340+0000 7f6cee7fc640 1 -- 172.21.15.193:0/2949897563 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620735032 0 0) 0x7f6cf015fb90 con 0x7f6cf0159a20 2024-04-20T21:16:32.340 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.340+0000 7f6ceeffd640 1 -- 172.21.15.193:0/2949897563 >> v1:172.21.15.102:6789/0 conn(0x7f6cf00723b0 legacy=0x7f6cf006f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:32.340 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.340+0000 7f6ceeffd640 1 -- 172.21.15.193:0/2949897563 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf015fe70 con 0x7f6cf0074f10 2024-04-20T21:16:32.341 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.340+0000 7f6ceeffd640 1 --2- 172.21.15.193:0/2949897563 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0074f10 0x7f6cf0071e70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00097d0 tx=0x7f6ce002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b3285ca173b244 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:32.341 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cee7fc640 1 -- 172.21.15.193:0/2949897563 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0035020 con 0x7f6cf0074f10 2024-04-20T21:16:32.341 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cee7fc640 1 -- 172.21.15.193:0/2949897563 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce0031890 con 0x7f6cf0074f10 2024-04-20T21:16:32.341 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cee7fc640 1 -- 172.21.15.193:0/2949897563 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0031c60 con 0x7f6cf0074f10 2024-04-20T21:16:32.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cf617e640 1 -- 172.21.15.193:0/2949897563 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0074f10 msgr2=0x7f6cf0071e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cf617e640 1 --2- 172.21.15.193:0/2949897563 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0074f10 0x7f6cf0071e70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00097d0 tx=0x7f6ce002b930 comp rx=0 tx=0).stop 2024-04-20T21:16:32.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cf617e640 1 -- 172.21.15.193:0/2949897563 shutdown_connections 2024-04-20T21:16:32.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cf617e640 1 --2- 172.21.15.193:0/2949897563 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0074f10 0x7f6cf0071e70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cf617e640 1 -- 172.21.15.193:0/2949897563 >> 172.21.15.193:0/2949897563 conn(0x7f6cf00cd8a0 msgr2=0x7f6cf00cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:32.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cf617e640 1 -- 172.21.15.193:0/2949897563 shutdown_connections 2024-04-20T21:16:32.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.341+0000 7f6cf617e640 1 -- 172.21.15.193:0/2949897563 wait complete. 2024-04-20T21:16:32.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.342+0000 7f6cf617e640 1 Processor -- start 2024-04-20T21:16:32.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6cf617e640 1 -- start start 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6cf617e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf00723b0 0x7f6cf01f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6cf617e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf0074f10 0x7f6cf015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6cf617e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0159a20 0x7f6cf00c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6cf617e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf015f840 con 0x7f6cf00723b0 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6cf617e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0160430 con 0x7f6cf0074f10 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6cf617e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf00c89b0 con 0x7f6cf0159a20 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6ceeffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf0074f10 0x7f6cf015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6ceffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0159a20 0x7f6cf00c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6ceffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0159a20 0x7f6cf00c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:38210/0 (socket says 172.21.15.193:38210) 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.343+0000 7f6ceffff640 1 -- 172.21.15.193:0/571536370 learned_addr learned my addr 172.21.15.193:0/571536370 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6ceffff640 1 -- 172.21.15.193:0/571536370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf0074f10 msgr2=0x7f6cf015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6ceffff640 1 --2- 172.21.15.193:0/571536370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf0074f10 0x7f6cf015ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6ceffff640 1 -- 172.21.15.193:0/571536370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf00723b0 msgr2=0x7f6cf01f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6ceffff640 1 --2- 172.21.15.193:0/571536370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf00723b0 0x7f6cf01f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6ceffff640 1 -- 172.21.15.193:0/571536370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf00ca990 con 0x7f6cf0159a20 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6ceffff640 1 --2- 172.21.15.193:0/571536370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0159a20 0x7f6cf00c9550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800c7b0 tx=0x7f6ce800cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6cd7fff640 1 -- 172.21.15.193:0/571536370 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8017070 con 0x7f6cf0159a20 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6cd7fff640 1 -- 172.21.15.193:0/571536370 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce8003da0 con 0x7f6cf0159a20 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf015b3f0 con 0x7f6cf0159a20 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6cd7fff640 1 -- 172.21.15.193:0/571536370 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80052b0 con 0x7f6cf0159a20 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf00cac30 con 0x7f6cf0159a20 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6cd7fff640 1 -- 172.21.15.193:0/571536370 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ce801a070 con 0x7f6cf0159a20 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.344+0000 7f6cd7fff640 1 -- 172.21.15.193:0/571536370 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f6ce800ac70 con 0x7f6cf0159a20 2024-04-20T21:16:32.346 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.345+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf0072e60 con 0x7f6cf0159a20 2024-04-20T21:16:32.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.347+0000 7f6cd7fff640 1 -- 172.21.15.193:0/571536370 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f6cf0072e60 con 0x7f6cf0159a20 2024-04-20T21:16:32.416 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:32.417+0000 7f96a744d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:16:32.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.420+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bc20150c-0f8e-498b-996c-83f43f44964a", "id": 5} v 0) -- 0x7f6cf004fdd0 con 0x7f6cf0159a20 2024-04-20T21:16:32.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.426+0000 7f6cd7fff640 1 -- 172.21.15.193:0/571536370 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bc20150c-0f8e-498b-996c-83f43f44964a", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f6ce8021020 con 0x7f6cf0159a20 2024-04-20T21:16:32.426 INFO:teuthology.orchestra.run.smithi193.stdout:5 2024-04-20T21:16:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.427+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0159a20 msgr2=0x7f6cf00c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.427+0000 7f6cf617e640 1 --2- 172.21.15.193:0/571536370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0159a20 0x7f6cf00c9550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800c7b0 tx=0x7f6ce800cc70 comp rx=0 tx=0).stop 2024-04-20T21:16:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.427+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 shutdown_connections 2024-04-20T21:16:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.427+0000 7f6cf617e640 1 --2- 172.21.15.193:0/571536370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf0159a20 0x7f6cf00c9550 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.427+0000 7f6cf617e640 1 --2- 172.21.15.193:0/571536370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf0074f10 0x7f6cf015ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.427+0000 7f6cf617e640 1 --2- 172.21.15.193:0/571536370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf00723b0 0x7f6cf01f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.427+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 >> 172.21.15.193:0/571536370 conn(0x7f6cf00cd8a0 msgr2=0x7f6cf00cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.428+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 shutdown_connections 2024-04-20T21:16:32.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.428+0000 7f6cf617e640 1 -- 172.21.15.193:0/571536370 wait complete. 2024-04-20T21:16:32.437 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new cbdceaf5-95a2-481b-831a-3edbfe220712 6 2024-04-20T21:16:32.505 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:32.505+0000 7fad1d211040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:16:32.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.534+0000 7f5407881640 1 Processor -- start 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.535+0000 7f5407881640 1 -- start start 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.535+0000 7f5407881640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f540014e9d0 0x7f5400157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.535+0000 7f5407881640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5400071fd0 0x7f54000cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.535+0000 7f5407881640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5400072860 con 0x7f54000723d0 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.535+0000 7f5407881640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f5400072b00 con 0x7f540014e9d0 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.535+0000 7f5407881640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5400072dc0 con 0x7f5400071fd0 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f540561d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f540014e9d0 0x7f5400157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f5404e1c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5400071fd0 0x7f54000cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f540561d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f540014e9d0 0x7f5400157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54490/0 (socket says 172.21.15.193:54490) 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f540561d640 1 -- 172.21.15.193:0/481354299 learned_addr learned my addr 172.21.15.193:0/481354299 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f5405e1e640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f54000723d0 0x7f54000c6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:43562/0 (socket says 172.21.15.193:43562) 2024-04-20T21:16:32.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f540561d640 1 -- 172.21.15.193:0/481354299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5400071fd0 msgr2=0x7f54000cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:32.537 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f540561d640 1 --2- 172.21.15.193:0/481354299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5400071fd0 0x7f54000cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.537 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f540561d640 1 -- 172.21.15.193:0/481354299 >> v1:172.21.15.193:6789/0 conn(0x7f54000723d0 legacy=0x7f54000c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.537 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f540561d640 1 -- 172.21.15.193:0/481354299 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54000730a0 con 0x7f540014e9d0 2024-04-20T21:16:32.537 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f53f7fff640 1 -- 172.21.15.193:0/481354299 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349532140 0 0) 0x7f5400072860 con 0x7f54000723d0 2024-04-20T21:16:32.537 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.536+0000 7f5404e1c640 1 --2- 172.21.15.193:0/481354299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5400071fd0 0x7f54000cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:32.538 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.537+0000 7f540561d640 1 --2- 172.21.15.193:0/481354299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f540014e9d0 0x7f5400157fb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53f00098a0 tx=0x7f53f002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51956d6f0c679d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.537+0000 7f53f7fff640 1 -- 172.21.15.193:0/481354299 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f002f830 con 0x7f540014e9d0 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.537+0000 7f53f7fff640 1 -- 172.21.15.193:0/481354299 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f002f9d0 con 0x7f540014e9d0 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.537+0000 7f53f7fff640 1 -- 172.21.15.193:0/481354299 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f002fda0 con 0x7f540014e9d0 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.538+0000 7f5407881640 1 -- 172.21.15.193:0/481354299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f540014e9d0 msgr2=0x7f5400157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.538+0000 7f5407881640 1 --2- 172.21.15.193:0/481354299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f540014e9d0 0x7f5400157fb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53f00098a0 tx=0x7f53f002ba10 comp rx=0 tx=0).stop 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.538+0000 7f5407881640 1 -- 172.21.15.193:0/481354299 shutdown_connections 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.538+0000 7f5407881640 1 --2- 172.21.15.193:0/481354299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5400071fd0 0x7f54000cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.538+0000 7f5407881640 1 --2- 172.21.15.193:0/481354299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f540014e9d0 0x7f5400157fb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.538+0000 7f5407881640 1 -- 172.21.15.193:0/481354299 >> 172.21.15.193:0/481354299 conn(0x7f540006bf70 msgr2=0x7f540006daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.538+0000 7f5407881640 1 -- 172.21.15.193:0/481354299 shutdown_connections 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.538+0000 7f5407881640 1 -- 172.21.15.193:0/481354299 wait complete. 2024-04-20T21:16:32.539 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.539+0000 7f5407881640 1 Processor -- start 2024-04-20T21:16:32.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.539+0000 7f5407881640 1 -- start start 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5407881640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5400071fd0 0x7f54001f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5407881640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f540014e9d0 0x7f5400149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5407881640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f540015c170 0x7f54001f1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5407881640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f540014edb0 con 0x7f540015c170 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5407881640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54000c64f0 con 0x7f5400071fd0 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5407881640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f54001525c0 con 0x7f540014e9d0 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f540561d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5400071fd0 0x7f54001f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f540561d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5400071fd0 0x7f54001f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54492/0 (socket says 172.21.15.193:54492) 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f540561d640 1 -- 172.21.15.193:0/3976666268 learned_addr learned my addr 172.21.15.193:0/3976666268 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5404e1c640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f540014e9d0 0x7f5400149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5404e1c640 1 -- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5400071fd0 msgr2=0x7f54001f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5404e1c640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5400071fd0 0x7f54001f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5404e1c640 1 -- 172.21.15.193:0/3976666268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f540015c170 msgr2=0x7f54001f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5404e1c640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f540015c170 0x7f54001f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5404e1c640 1 -- 172.21.15.193:0/3976666268 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5400153fe0 con 0x7f540014e9d0 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f540561d640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5400071fd0 0x7f54001f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5404e1c640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f540014e9d0 0x7f5400149c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00e900 tx=0x7f53fc00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f53f5ffb640 1 -- 172.21.15.193:0/3976666268 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc017070 con 0x7f540014e9d0 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f53f5ffb640 1 -- 172.21.15.193:0/3976666268 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53fc00ca00 con 0x7f540014e9d0 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f53f5ffb640 1 -- 172.21.15.193:0/3976666268 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00cdf0 con 0x7f540014e9d0 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.540+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54001f4f40 con 0x7f540014e9d0 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.541+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5400152d90 con 0x7f540014e9d0 2024-04-20T21:16:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.541+0000 7f53f5ffb640 1 -- 172.21.15.193:0/3976666268 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53fc01a070 con 0x7f540014e9d0 2024-04-20T21:16:32.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.541+0000 7f53f5ffb640 1 -- 172.21.15.193:0/3976666268 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f53fc013470 con 0x7f540014e9d0 2024-04-20T21:16:32.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.542+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540004fdd0 con 0x7f540014e9d0 2024-04-20T21:16:32.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.544+0000 7f53f5ffb640 1 -- 172.21.15.193:0/3976666268 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f53fc009330 con 0x7f540014e9d0 2024-04-20T21:16:32.570 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:32.571+0000 7fad1d211040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:16:32.616 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.616+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cbdceaf5-95a2-481b-831a-3edbfe220712", "id": 6} v 0) -- 0x7f5400155fa0 con 0x7f540014e9d0 2024-04-20T21:16:32.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.623+0000 7f53f5ffb640 1 -- 172.21.15.193:0/3976666268 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cbdceaf5-95a2-481b-831a-3edbfe220712", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f53fc006550 con 0x7f540014e9d0 2024-04-20T21:16:32.622 INFO:teuthology.orchestra.run.smithi193.stdout:6 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f540014e9d0 msgr2=0x7f5400149c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f540014e9d0 0x7f5400149c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00e900 tx=0x7f53fc00edc0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 shutdown_connections 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f540015c170 0x7f54001f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f540014e9d0 0x7f5400149c80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 --2- 172.21.15.193:0/3976666268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5400071fd0 0x7f54001f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 >> 172.21.15.193:0/3976666268 conn(0x7f540006bf70 msgr2=0x7f540014d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 shutdown_connections 2024-04-20T21:16:32.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.624+0000 7f5407881640 1 -- 172.21.15.193:0/3976666268 wait complete. 2024-04-20T21:16:32.631 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:32.631+0000 7fad1d211040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:16:32.634 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 4f7cd742-6ea8-43ff-a38b-f68c893a5c07 7 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.731+0000 7faec8ea5640 1 Processor -- start 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.731+0000 7faec8ea5640 1 -- start start 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec8ea5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec4072300 0x7faec4158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec8ea5640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faec40c8e30 con 0x7faec4070350 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec8ea5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faec40c90f0 con 0x7faec41580f0 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec8ea5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faec40c9640 con 0x7faec4072300 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec2d9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec4072300 0x7faec4158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec259c640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7faec41580f0 0x7faec406fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:43574/0 (socket says 172.21.15.193:43574) 2024-04-20T21:16:32.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec259c640 1 -- 172.21.15.193:0/423378646 learned_addr learned my addr 172.21.15.193:0/423378646 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec1d9b640 1 -- 172.21.15.193:0/423378646 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099267211 0 0) 0x7faec40c90f0 con 0x7faec41580f0 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.732+0000 7faec1d9b640 1 -- 172.21.15.193:0/423378646 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faeb0003520 con 0x7faec41580f0 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.733+0000 7faec2d9d640 1 -- 172.21.15.193:0/423378646 >> v1:172.21.15.193:6789/0 conn(0x7faec41580f0 legacy=0x7faec406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.733+0000 7faec2d9d640 1 -- 172.21.15.193:0/423378646 >> v1:172.21.15.102:6789/0 conn(0x7faec4070350 legacy=0x7faec41584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.733+0000 7faec2d9d640 1 -- 172.21.15.193:0/423378646 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec40c9920 con 0x7faec4072300 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.733+0000 7faec2d9d640 1 --2- 172.21.15.193:0/423378646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec4072300 0x7faec4158be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faebc009dc0 tx=0x7faebc02be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7c1c3a3a522f79f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.733+0000 7faec1d9b640 1 -- 172.21.15.193:0/423378646 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc039070 con 0x7faec4072300 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.733+0000 7faec1d9b640 1 -- 172.21.15.193:0/423378646 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faebc02cbb0 con 0x7faec4072300 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.733+0000 7faec1d9b640 1 -- 172.21.15.193:0/423378646 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc034440 con 0x7faec4072300 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.734+0000 7faec8ea5640 1 -- 172.21.15.193:0/423378646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec4072300 msgr2=0x7faec4158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.734+0000 7faec8ea5640 1 --2- 172.21.15.193:0/423378646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec4072300 0x7faec4158be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faebc009dc0 tx=0x7faebc02be00 comp rx=0 tx=0).stop 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.734+0000 7faec8ea5640 1 -- 172.21.15.193:0/423378646 shutdown_connections 2024-04-20T21:16:32.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.734+0000 7faec8ea5640 1 --2- 172.21.15.193:0/423378646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec4072300 0x7faec4158be0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.734+0000 7faec8ea5640 1 -- 172.21.15.193:0/423378646 >> 172.21.15.193:0/423378646 conn(0x7faec40cd8a0 msgr2=0x7faec40cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.734+0000 7faec8ea5640 1 -- 172.21.15.193:0/423378646 shutdown_connections 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.734+0000 7faec8ea5640 1 -- 172.21.15.193:0/423378646 wait complete. 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.735+0000 7faec8ea5640 1 Processor -- start 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.735+0000 7faec8ea5640 1 -- start start 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec8ea5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec4070350 0x7faec40cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec8ea5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec4072300 0x7faec41e1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec8ea5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec41580f0 0x7faec414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec8ea5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faec41527f0 con 0x7faec4072300 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec8ea5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faec4152670 con 0x7faec4070350 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec8ea5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faec406eb40 con 0x7faec41580f0 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec4070350 0x7faec40cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec4070350 0x7faec40cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54494/0 (socket says 172.21.15.193:54494) 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec259c640 1 -- 172.21.15.193:0/3895147967 learned_addr learned my addr 172.21.15.193:0/3895147967 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec2d9d640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec41580f0 0x7faec414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec259c640 1 -- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec41580f0 msgr2=0x7faec414d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec259c640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec41580f0 0x7faec414d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec259c640 1 -- 172.21.15.193:0/3895147967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec4072300 msgr2=0x7faec41e1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.736+0000 7faec259c640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec4072300 0x7faec41e1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faec259c640 1 -- 172.21.15.193:0/3895147967 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec41edf30 con 0x7faec4070350 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faec2d9d640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec41580f0 0x7faec414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faec259c640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec4070350 0x7faec40cba10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c390 tx=0x7faeb400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faebaffd640 1 -- 172.21.15.193:0/3895147967 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4017070 con 0x7faec4070350 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faebaffd640 1 -- 172.21.15.193:0/3895147967 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeb4005ce0 con 0x7faec4070350 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faebaffd640 1 -- 172.21.15.193:0/3895147967 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb40072d0 con 0x7faec4070350 2024-04-20T21:16:32.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec41f1f40 con 0x7faec4070350 2024-04-20T21:16:32.742 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec41ee4b0 con 0x7faec4070350 2024-04-20T21:16:32.742 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faebaffd640 1 -- 172.21.15.193:0/3895147967 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faeb401a070 con 0x7faec4070350 2024-04-20T21:16:32.742 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.737+0000 7faebaffd640 1 -- 172.21.15.193:0/3895147967 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7faeb401cc10 con 0x7faec4070350 2024-04-20T21:16:32.742 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.738+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec40ca180 con 0x7faec4070350 2024-04-20T21:16:32.742 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.740+0000 7faebaffd640 1 -- 172.21.15.193:0/3895147967 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7faeb40074d0 con 0x7faec4070350 2024-04-20T21:16:32.757 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:32.758+0000 7fad1d211040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:16:32.812 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.812+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4f7cd742-6ea8-43ff-a38b-f68c893a5c07", "id": 7} v 0) -- 0x7faec404fdd0 con 0x7faec4070350 2024-04-20T21:16:32.818 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:32.819+0000 7f96a744d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:16:32.821 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.821+0000 7faebaffd640 1 -- 172.21.15.193:0/3895147967 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f7cd742-6ea8-43ff-a38b-f68c893a5c07", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7faeb401c440 con 0x7faec4070350 2024-04-20T21:16:32.821 INFO:teuthology.orchestra.run.smithi193.stdout:7 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec4070350 msgr2=0x7faec40cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec4070350 0x7faec40cba10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c390 tx=0x7faeb400c850 comp rx=0 tx=0).stop 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 shutdown_connections 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faec41580f0 0x7faec414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faec4072300 0x7faec41e1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 --2- 172.21.15.193:0/3895147967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec4070350 0x7faec40cba10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 >> 172.21.15.193:0/3895147967 conn(0x7faec40cd8a0 msgr2=0x7faec40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 shutdown_connections 2024-04-20T21:16:32.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-20T21:16:32.822+0000 7faec8ea5640 1 -- 172.21.15.193:0/3895147967 wait complete. 2024-04-20T21:16:32.832 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-20T21:16:32.832 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-20T21:16:32.840 INFO:tasks.ceph.osd.0:Started 2024-04-20T21:16:32.840 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-20T21:16:32.840 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-20T21:16:32.847 INFO:tasks.ceph.osd.1:Started 2024-04-20T21:16:32.848 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-20T21:16:32.848 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-20T21:16:32.855 INFO:tasks.ceph.osd.2:Started 2024-04-20T21:16:32.855 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-20T21:16:32.855 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-20T21:16:32.859 INFO:tasks.ceph.osd.3:Started 2024-04-20T21:16:32.860 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-20T21:16:32.860 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-20T21:16:32.862 INFO:tasks.ceph.osd.4:Started 2024-04-20T21:16:32.862 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-20T21:16:32.862 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-20T21:16:32.864 INFO:tasks.ceph.osd.5:Started 2024-04-20T21:16:32.864 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-20T21:16:32.864 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-20T21:16:32.866 INFO:tasks.ceph.osd.6:Started 2024-04-20T21:16:32.866 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-20T21:16:32.867 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-20T21:16:32.869 INFO:tasks.ceph.osd.7:Started 2024-04-20T21:16:32.869 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:16:33.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.084+0000 7fa5d9c00640 1 Processor -- start 2024-04-20T21:16:33.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.084+0000 7fa5d9c00640 1 -- start start 2024-04-20T21:16:33.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.086+0000 7fa5d9c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa5d4072940 0x7fa5d415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:33.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.086+0000 7fa5d9c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d415f940 con 0x7fa5d40c7900 2024-04-20T21:16:33.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.086+0000 7fa5d9c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d415eb60 con 0x7fa5d40cd940 2024-04-20T21:16:33.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.086+0000 7fa5d9c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d415ee00 con 0x7fa5d4072940 2024-04-20T21:16:33.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.086+0000 7fa5d2a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa5d40c7900 0x7fa5d4074ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50378/0 (socket says 172.21.15.102:50378) 2024-04-20T21:16:33.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.086+0000 7fa5d2a00640 1 -- 172.21.15.102:0/2126942161 learned_addr learned my addr 172.21.15.102:0/2126942161 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:33.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.087+0000 7fa5d3400640 1 --2- 172.21.15.102:0/2126942161 >> v2:172.21.15.102:3300/0 conn(0x7fa5d4072940 0x7fa5d415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:33.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.087+0000 7fa5d2000640 1 -- 172.21.15.102:0/2126942161 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030626300 0 0) 0x7fa5d415eb60 con 0x7fa5d40cd940 2024-04-20T21:16:33.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.087+0000 7fa5d2000640 1 -- 172.21.15.102:0/2126942161 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5b8003520 con 0x7fa5d40cd940 2024-04-20T21:16:33.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.087+0000 7fa5d2000640 1 -- 172.21.15.102:0/2126942161 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262774942 0 0) 0x7fa5d415f940 con 0x7fa5d40c7900 2024-04-20T21:16:33.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.087+0000 7fa5d2000640 1 -- 172.21.15.102:0/2126942161 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d415eb60 con 0x7fa5d40c7900 2024-04-20T21:16:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.087+0000 7fa5d3400640 1 -- 172.21.15.102:0/2126942161 >> v1:172.21.15.193:6789/0 conn(0x7fa5d40cd940 legacy=0x7fa5d4072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.087+0000 7fa5d3400640 1 -- 172.21.15.102:0/2126942161 >> v1:172.21.15.102:6789/0 conn(0x7fa5d40c7900 legacy=0x7fa5d4074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.087+0000 7fa5d3400640 1 -- 172.21.15.102:0/2126942161 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d415f0a0 con 0x7fa5d4072940 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.088+0000 7fa5d3400640 1 --2- 172.21.15.102:0/2126942161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d4072940 0x7fa5d415bee0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009d90 tx=0x7fa5cc02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf47e5a5155860c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d2000640 1 -- 172.21.15.102:0/2126942161 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc039070 con 0x7fa5d4072940 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d2000640 1 -- 172.21.15.102:0/2126942161 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5cc02cd80 con 0x7fa5d4072940 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d2000640 1 -- 172.21.15.102:0/2126942161 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc034660 con 0x7fa5d4072940 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2126942161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d4072940 msgr2=0x7fa5d415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d9c00640 1 --2- 172.21.15.102:0/2126942161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d4072940 0x7fa5d415bee0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009d90 tx=0x7fa5cc02bfd0 comp rx=0 tx=0).stop 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2126942161 shutdown_connections 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d9c00640 1 --2- 172.21.15.102:0/2126942161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d4072940 0x7fa5d415bee0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2126942161 >> 172.21.15.102:0/2126942161 conn(0x7fa5d406dad0 msgr2=0x7fa5d406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:33.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2126942161 shutdown_connections 2024-04-20T21:16:33.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.089+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2126942161 wait complete. 2024-04-20T21:16:33.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.091+0000 7fa5d9c00640 1 Processor -- start 2024-04-20T21:16:33.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.091+0000 7fa5d9c00640 1 -- start start 2024-04-20T21:16:33.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d4072940 0x7fa5d414a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:33.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d40c7900 0x7fa5d414a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:33.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d9c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40cd940 0x7fa5d415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:33.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d9c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d40755b0 con 0x7fa5d40c7900 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d9c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d415f660 con 0x7fa5d4072940 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d9c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d4075730 con 0x7fa5d40cd940 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d4072940 0x7fa5d414a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d3400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40cd940 0x7fa5d415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d40c7900 0x7fa5d414a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d3400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40cd940 0x7fa5d415a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:45402/0 (socket says 172.21.15.102:45402) 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d4072940 0x7fa5d414a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:39634/0 (socket says 172.21.15.102:39634) 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.092+0000 7fa5d3400640 1 -- 172.21.15.102:0/2784324254 learned_addr learned my addr 172.21.15.102:0/2784324254 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5d3400640 1 -- 172.21.15.102:0/2784324254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d4072940 msgr2=0x7fa5d414a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5d3400640 1 --2- 172.21.15.102:0/2784324254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d4072940 0x7fa5d414a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5d3400640 1 -- 172.21.15.102:0/2784324254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d40c7900 msgr2=0x7fa5d414a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5d3400640 1 --2- 172.21.15.102:0/2784324254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d40c7900 0x7fa5d414a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:33.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5d3400640 1 -- 172.21.15.102:0/2784324254 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d40ccf30 con 0x7fa5d40cd940 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5d3400640 1 --2- 172.21.15.102:0/2784324254 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40cd940 0x7fa5d415a200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009d90 tx=0x7fa5cc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5caa00640 1 -- 172.21.15.102:0/2784324254 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc039050 con 0x7fa5d40cd940 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5caa00640 1 -- 172.21.15.102:0/2784324254 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5cc045070 con 0x7fa5d40cd940 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5caa00640 1 -- 172.21.15.102:0/2784324254 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc009450 con 0x7fa5d40cd940 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.093+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d41f1a60 con 0x7fa5d40cd940 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.094+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d415ad30 con 0x7fa5d40cd940 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.094+0000 7fa5caa00640 1 -- 172.21.15.102:0/2784324254 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa5cc03e850 con 0x7fa5d40cd940 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.094+0000 7fa5caa00640 1 -- 172.21.15.102:0/2784324254 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa5cc04a460 con 0x7fa5d40cd940 2024-04-20T21:16:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.095+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d40cd100 con 0x7fa5d40cd940 2024-04-20T21:16:33.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.097+0000 7fa5caa00640 1 -- 172.21.15.102:0/2784324254 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fa5cc03eb40 con 0x7fa5d40cd940 2024-04-20T21:16:33.097 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:33.096+0000 7f96a744d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:16:33.149 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:33.149+0000 7f96a744d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:16:33.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.164+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5d414eab0 con 0x7fa5d40cd940 2024-04-20T21:16:33.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.165+0000 7fa5caa00640 1 -- 172.21.15.102:0/2784324254 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa5cc03ed20 con 0x7fa5d40cd940 2024-04-20T21:16:33.165 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:16:33.165 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":10,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:32.814840+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:16:33.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.166+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40cd940 msgr2=0x7fa5d415a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:33.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.166+0000 7fa5d9c00640 1 --2- 172.21.15.102:0/2784324254 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40cd940 0x7fa5d415a200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009d90 tx=0x7fa5cc004a40 comp rx=0 tx=0).stop 2024-04-20T21:16:33.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.166+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 shutdown_connections 2024-04-20T21:16:33.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.166+0000 7fa5d9c00640 1 --2- 172.21.15.102:0/2784324254 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40cd940 0x7fa5d415a200 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:33.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.166+0000 7fa5d9c00640 1 --2- 172.21.15.102:0/2784324254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d40c7900 0x7fa5d414a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:33.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.166+0000 7fa5d9c00640 1 --2- 172.21.15.102:0/2784324254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d4072940 0x7fa5d414a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:33.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.166+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 >> 172.21.15.102:0/2784324254 conn(0x7fa5d406dad0 msgr2=0x7fa5d406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:33.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.167+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 shutdown_connections 2024-04-20T21:16:33.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:33.167+0000 7fa5d9c00640 1 -- 172.21.15.102:0/2784324254 wait complete. 2024-04-20T21:16:33.181 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-20T21:16:33.182 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-20T21:16:33.266 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:33.266+0000 7f96a744d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:16:33.279 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:33.279+0000 7fad1d211040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:16:33.470 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-20T21:16:33.469+0000 7f4af022c640 -1 Falling back to public interface 2024-04-20T21:16:33.471 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-20T21:16:33.470+0000 7f5facba3640 -1 Falling back to public interface 2024-04-20T21:16:33.486 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-20T21:16:33.486+0000 7f5a2f40f640 -1 Falling back to public interface 2024-04-20T21:16:33.487 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-20T21:16:33.487+0000 7f6692542640 -1 Falling back to public interface 2024-04-20T21:16:33.531 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-20T21:16:33.531+0000 7f9ab889a640 -1 Falling back to public interface 2024-04-20T21:16:33.538 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-20T21:16:33.539+0000 7f08914d9640 -1 Falling back to public interface 2024-04-20T21:16:33.549 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:33.549+0000 7f96a744d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:16:33.580 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:33.580+0000 7fad1d211040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:16:33.583 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-20T21:16:33.584+0000 7f5307e05640 -1 Falling back to public interface 2024-04-20T21:16:33.584 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-20T21:16:33.585+0000 7ff6e8011640 -1 Falling back to public interface 2024-04-20T21:16:33.595 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:33.595+0000 7f96a744d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:16:33.648 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:33.648+0000 7fad1d211040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:16:33.751 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:33.750+0000 7f96a744d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:16:33.778 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:33.779+0000 7fad1d211040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:16:33.886 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:33.885+0000 7f96a744d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:16:33.936 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:33.935+0000 7f96a744d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:16:33.988 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T21:16:34.004 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:34.004+0000 7f96a744d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:16:34.051 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:34.051+0000 7f96a744d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:16:34.123 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:34.124+0000 7fad1d211040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:16:34.152 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.152+0000 7f9457200640 1 Processor -- start 2024-04-20T21:16:34.152 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.152+0000 7f9457200640 1 -- start start 2024-04-20T21:16:34.156 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9457200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 0x7f94580c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:34.156 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9457200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9458071e00 0x7f945806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:34.156 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9457200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f945806fb20 0x7f94580c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:34.156 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9457200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9458072710 con 0x7f9458071e00 2024-04-20T21:16:34.156 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9457200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f945806df70 con 0x7f945806fb20 2024-04-20T21:16:34.157 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9457200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9458158ed0 con 0x7f9458153230 2024-04-20T21:16:34.157 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9458071e00 0x7f945806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:34.157 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 0x7f94580c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:34.157 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9458071e00 0x7f945806e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50030/0 (socket says 172.21.15.102:50030) 2024-04-20T21:16:34.157 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455400640 1 -- 172.21.15.102:0/2591325145 learned_addr learned my addr 172.21.15.102:0/2591325145 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:34.157 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9456800640 1 --2- 172.21.15.102:0/2591325145 >> v2:172.21.15.193:3300/0 conn(0x7f945806fb20 0x7f94580c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:34.157 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455400640 1 -- 172.21.15.102:0/2591325145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 msgr2=0x7f94580c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:34.158 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455400640 1 --2- 172.21.15.102:0/2591325145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 0x7f94580c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.158 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455400640 1 -- 172.21.15.102:0/2591325145 >> v2:172.21.15.193:3300/0 conn(0x7f945806fb20 msgr2=0x7f94580c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:34.158 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455400640 1 --2- 172.21.15.102:0/2591325145 >> v2:172.21.15.193:3300/0 conn(0x7f945806fb20 0x7f94580c6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.158 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.154+0000 7f9455400640 1 -- 172.21.15.102:0/2591325145 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9458153610 con 0x7f9458071e00 2024-04-20T21:16:34.158 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.155+0000 7f9455400640 1 --2- 172.21.15.102:0/2591325145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9458071e00 0x7f945806e400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f945806d4b0 tx=0x7f944402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574ee72f9d1d8262 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:34.158 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.156+0000 7f9454a00640 1 -- 172.21.15.102:0/2591325145 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444034070 con 0x7f9458071e00 2024-04-20T21:16:34.158 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.156+0000 7f9454a00640 1 -- 172.21.15.102:0/2591325145 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9444030860 con 0x7f9458071e00 2024-04-20T21:16:34.159 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.156+0000 7f9454a00640 1 -- 172.21.15.102:0/2591325145 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444030c30 con 0x7f9458071e00 2024-04-20T21:16:34.166 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.156+0000 7f9457200640 1 -- 172.21.15.102:0/2591325145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9458071e00 msgr2=0x7f945806e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:34.166 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.156+0000 7f9457200640 1 --2- 172.21.15.102:0/2591325145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9458071e00 0x7f945806e400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f945806d4b0 tx=0x7f944402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.166 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.157+0000 7f9457200640 1 -- 172.21.15.102:0/2591325145 shutdown_connections 2024-04-20T21:16:34.166 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.157+0000 7f9457200640 1 --2- 172.21.15.102:0/2591325145 >> v2:172.21.15.193:3300/0 conn(0x7f945806fb20 0x7f94580c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.166 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.157+0000 7f9457200640 1 --2- 172.21.15.102:0/2591325145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9458071e00 0x7f945806e400 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.157+0000 7f9457200640 1 --2- 172.21.15.102:0/2591325145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 0x7f94580c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.157+0000 7f9457200640 1 -- 172.21.15.102:0/2591325145 >> 172.21.15.102:0/2591325145 conn(0x7f94580cd8c0 msgr2=0x7f94580cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.157+0000 7f9457200640 1 -- 172.21.15.102:0/2591325145 shutdown_connections 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.157+0000 7f9457200640 1 -- 172.21.15.102:0/2591325145 wait complete. 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.159+0000 7f9457200640 1 Processor -- start 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.159+0000 7f9457200640 1 -- start start 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9457200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f945806fb20 0x7f945814c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9457200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9458071e00 0x7f945815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9457200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 0x7f945815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9457200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9458072f20 con 0x7f945806fb20 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9457200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9458072c40 con 0x7f9458071e00 2024-04-20T21:16:34.167 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9457200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9458073200 con 0x7f9458153230 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9455e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f945806fb20 0x7f945814c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9455e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f945806fb20 0x7f945814c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50036/0 (socket says 172.21.15.102:50036) 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9456800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 0x7f945815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9455e00640 1 -- 172.21.15.102:0/3936338029 learned_addr learned my addr 172.21.15.102:0/3936338029 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.161+0000 7f9455400640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9458071e00 0x7f945815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9455e00640 1 -- 172.21.15.102:0/3936338029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 msgr2=0x7f945815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9455e00640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 0x7f945815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9455e00640 1 -- 172.21.15.102:0/3936338029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9458071e00 msgr2=0x7f945815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9455e00640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9458071e00 0x7f945815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9455e00640 1 -- 172.21.15.102:0/3936338029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94581f1310 con 0x7f945806fb20 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9455400640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9458071e00 0x7f945815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9455e00640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f945806fb20 0x7f945814c340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f944800e9e0 tx=0x7f944800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:34.168 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f944e000640 1 -- 172.21.15.102:0/3936338029 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448017070 con 0x7f945806fb20 2024-04-20T21:16:34.169 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f944e000640 1 -- 172.21.15.102:0/3936338029 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944800c970 con 0x7f945806fb20 2024-04-20T21:16:34.169 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f944e000640 1 -- 172.21.15.102:0/3936338029 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944800cd60 con 0x7f945806fb20 2024-04-20T21:16:34.169 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945815cf20 con 0x7f945806fb20 2024-04-20T21:16:34.169 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.162+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945806f770 con 0x7f945806fb20 2024-04-20T21:16:34.169 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.163+0000 7f944e000640 1 -- 172.21.15.102:0/3936338029 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f944801a070 con 0x7f945806fb20 2024-04-20T21:16:34.169 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.163+0000 7f944e000640 1 -- 172.21.15.102:0/3936338029 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9448013b40 con 0x7f945806fb20 2024-04-20T21:16:34.169 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.163+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9458076390 con 0x7f945806fb20 2024-04-20T21:16:34.169 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.166+0000 7f944e000640 1 -- 172.21.15.102:0/3936338029 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f94581f1310 con 0x7f945806fb20 2024-04-20T21:16:34.185 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:34.185+0000 7fad1d211040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:16:34.233 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.234+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9458070680 con 0x7f945806fb20 2024-04-20T21:16:34.235 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.235+0000 7f944e000640 1 -- 172.21.15.102:0/3936338029 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f9448013470 con 0x7f945806fb20 2024-04-20T21:16:34.235 INFO:teuthology.misc.health.smithi102.stdout: 2024-04-20T21:16:34.235 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":10,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:32.814840+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:16:34.236 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.236+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f945806fb20 msgr2=0x7f945814c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:34.237 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.236+0000 7f9457200640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f945806fb20 0x7f945814c340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f944800e9e0 tx=0x7f944800eea0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.237 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.237+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 shutdown_connections 2024-04-20T21:16:34.237 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.237+0000 7f9457200640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9458153230 0x7f945815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.237 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.237+0000 7f9457200640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9458071e00 0x7f945815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.237 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.237+0000 7f9457200640 1 --2- 172.21.15.102:0/3936338029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f945806fb20 0x7f945814c340 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:34.237 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.237+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 >> 172.21.15.102:0/3936338029 conn(0x7f94580cd8c0 msgr2=0x7f94580c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:34.237 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.237+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 shutdown_connections 2024-04-20T21:16:34.237 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:34.237+0000 7f9457200640 1 -- 172.21.15.102:0/3936338029 wait complete. 2024-04-20T21:16:34.249 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T21:16:34.366 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:34.367+0000 7fad1d211040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:16:34.528 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:34.527+0000 7f96a744d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:16:34.536 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:34.536+0000 7fad1d211040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:16:34.594 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:34.594+0000 7fad1d211040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:16:34.644 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:34.643+0000 7f96a744d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:16:34.679 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:34.679+0000 7fad1d211040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:16:34.736 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:34.736+0000 7fad1d211040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:16:35.277 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:35.277+0000 7fad1d211040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:16:35.411 INFO:tasks.ceph.mgr.y.smithi102.stderr:2024-04-20T21:16:35.411+0000 7fad1d211040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:16:35.671 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:35.670+0000 7f9647faf640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T21:16:35.671 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:16:35.671+0000 7f9647faf640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T21:16:36.961 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-20T21:16:36.962+0000 7f5facba3640 -1 osd.5 0 log_to_monitors true 2024-04-20T21:16:36.967 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-20T21:16:36.968+0000 7f9ab889a640 -1 osd.1 0 log_to_monitors true 2024-04-20T21:16:36.999 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-20T21:16:36.999+0000 7ff6e8011640 -1 osd.2 0 log_to_monitors true 2024-04-20T21:16:37.004 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-20T21:16:37.003+0000 7f5a2f40f640 -1 osd.6 0 log_to_monitors true 2024-04-20T21:16:37.008 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-20T21:16:37.008+0000 7f6692542640 -1 osd.7 0 log_to_monitors true 2024-04-20T21:16:37.008 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-20T21:16:37.008+0000 7f4af022c640 -1 osd.4 0 log_to_monitors true 2024-04-20T21:16:37.185 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-20T21:16:37.185+0000 7f08914d9640 -1 osd.0 0 log_to_monitors true 2024-04-20T21:16:37.252 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-20T21:16:37.252+0000 7f5307e05640 -1 osd.3 0 log_to_monitors true 2024-04-20T21:16:40.773 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-20T21:16:40.774+0000 7f4aeb7c3640 -1 osd.4 0 waiting for initial osdmap 2024-04-20T21:16:40.786 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-20T21:16:40.787+0000 7f4ae35e3640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:16:40.796 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-20T21:16:40.797+0000 7f5fa8b39640 -1 osd.5 0 waiting for initial osdmap 2024-04-20T21:16:40.808 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-20T21:16:40.809+0000 7f5fa0959640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:16:40.824 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-20T21:16:40.825+0000 7f5a2adaa640 -1 osd.6 0 waiting for initial osdmap 2024-04-20T21:16:40.840 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-20T21:16:40.841+0000 7f668e4d8640 -1 osd.7 0 waiting for initial osdmap 2024-04-20T21:16:40.842 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-20T21:16:40.843+0000 7f5a22bca640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:16:40.855 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-20T21:16:40.856+0000 7f66862f8640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:16:41.011 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-20T21:16:41.012+0000 7f9ab4200640 -1 osd.1 0 waiting for initial osdmap 2024-04-20T21:16:41.025 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-20T21:16:41.025+0000 7f9aa9800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:16:41.055 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T21:16:41.080 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-20T21:16:41.080+0000 7ff6e3400640 -1 osd.2 0 waiting for initial osdmap 2024-04-20T21:16:41.085 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-20T21:16:41.085+0000 7f088c400640 -1 osd.0 0 waiting for initial osdmap 2024-04-20T21:16:41.086 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-20T21:16:41.087+0000 7ff6d8a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:16:41.091 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-20T21:16:41.092+0000 7f5302c00640 -1 osd.3 0 waiting for initial osdmap 2024-04-20T21:16:41.093 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-20T21:16:41.094+0000 7f0882400640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:16:41.110 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-20T21:16:41.111+0000 7f52f8c00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:16:41.206 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.206+0000 7fd911400640 1 Processor -- start 2024-04-20T21:16:41.206 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.206+0000 7fd911400640 1 -- start start 2024-04-20T21:16:41.207 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd911400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0ccbe0 0x7fd90c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:41.208 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd911400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd90c0b4720 0x7fd90c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:41.208 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd911400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90c15bcd0 con 0x7fd90c0b54a0 2024-04-20T21:16:41.208 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd911400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fd90c0b4bb0 con 0x7fd90c0b4720 2024-04-20T21:16:41.208 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd911400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c0b3cf0 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.208 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd909e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0ccbe0 0x7fd90c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:41.208 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd90a800640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fd90c0b54a0 0x7fd90c15b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:60530/0 (socket says 172.21.15.102:60530) 2024-04-20T21:16:41.208 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd90a800640 1 -- 172.21.15.102:0/2253857504 learned_addr learned my addr 172.21.15.102:0/2253857504 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:41.208 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd90b200640 1 --2- 172.21.15.102:0/2253857504 >> v2:172.21.15.102:3300/0 conn(0x7fd90c0b4720 0x7fd90c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:41.209 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd909400640 1 -- 172.21.15.102:0/2253857504 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199954691 0 0) 0x7fd90c15bcd0 con 0x7fd90c0b54a0 2024-04-20T21:16:41.209 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd909e00640 1 -- 172.21.15.102:0/2253857504 >> v2:172.21.15.102:3300/0 conn(0x7fd90c0b4720 msgr2=0x7fd90c0cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:41.209 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd909e00640 1 --2- 172.21.15.102:0/2253857504 >> v2:172.21.15.102:3300/0 conn(0x7fd90c0b4720 0x7fd90c0cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.209 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd909e00640 1 -- 172.21.15.102:0/2253857504 >> v1:172.21.15.193:6789/0 conn(0x7fd90c0b54a0 legacy=0x7fd90c15b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:41.209 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.208+0000 7fd909e00640 1 -- 172.21.15.102:0/2253857504 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c15c270 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.209 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.209+0000 7fd909e00640 1 --2- 172.21.15.102:0/2253857504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0ccbe0 0x7fd90c0b4180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80097a0 tx=0x7fd8f802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515fb05d7cea2d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:41.209 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd909400640 1 -- 172.21.15.102:0/2253857504 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8035020 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.210 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd909400640 1 -- 172.21.15.102:0/2253857504 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f8031830 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.210 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd909400640 1 -- 172.21.15.102:0/2253857504 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8031c00 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.210 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd911400640 1 -- 172.21.15.102:0/2253857504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0ccbe0 msgr2=0x7fd90c0b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:41.211 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd911400640 1 --2- 172.21.15.102:0/2253857504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0ccbe0 0x7fd90c0b4180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80097a0 tx=0x7fd8f802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.211 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd911400640 1 -- 172.21.15.102:0/2253857504 shutdown_connections 2024-04-20T21:16:41.211 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd911400640 1 --2- 172.21.15.102:0/2253857504 >> v2:172.21.15.102:3300/0 conn(0x7fd90c0b4720 0x7fd90c0cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.211 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd911400640 1 --2- 172.21.15.102:0/2253857504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0ccbe0 0x7fd90c0b4180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.211 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd911400640 1 -- 172.21.15.102:0/2253857504 >> 172.21.15.102:0/2253857504 conn(0x7fd90c067a20 msgr2=0x7fd90c067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:41.211 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd911400640 1 -- 172.21.15.102:0/2253857504 shutdown_connections 2024-04-20T21:16:41.211 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.210+0000 7fd911400640 1 -- 172.21.15.102:0/2253857504 wait complete. 2024-04-20T21:16:41.211 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.212+0000 7fd911400640 1 Processor -- start 2024-04-20T21:16:41.212 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.212+0000 7fd911400640 1 -- start start 2024-04-20T21:16:41.213 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd911400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0b4720 0x7fd90c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:41.213 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd911400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd90c0b54a0 0x7fd90c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:41.213 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd911400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90c0ccbe0 0x7fd90c1f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:41.213 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd911400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c0b36a0 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.213 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd911400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c15ba70 con 0x7fd90c0b54a0 2024-04-20T21:16:41.213 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd911400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c0b3980 con 0x7fd90c0b4720 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90c0ccbe0 0x7fd90c1f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90c0ccbe0 0x7fd90c1f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57504/0 (socket says 172.21.15.102:57504) 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90b200640 1 -- 172.21.15.102:0/1329711142 learned_addr learned my addr 172.21.15.102:0/1329711142 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90a800640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0b4720 0x7fd90c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd909e00640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd90c0b54a0 0x7fd90c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90b200640 1 -- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0b4720 msgr2=0x7fd90c156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90b200640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0b4720 0x7fd90c156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90b200640 1 -- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd90c0b54a0 msgr2=0x7fd90c158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90b200640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd90c0b54a0 0x7fd90c158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.213+0000 7fd90b200640 1 -- 172.21.15.102:0/1329711142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c1e1aa0 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.214 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.214+0000 7fd90b200640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90c0ccbe0 0x7fd90c1f32d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e960 tx=0x7fd90400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:41.215 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.214+0000 7fd902a00640 1 -- 172.21.15.102:0/1329711142 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904017070 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.215 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.214+0000 7fd902a00640 1 -- 172.21.15.102:0/1329711142 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd90400c980 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.215 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.214+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c157280 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.215 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.214+0000 7fd902a00640 1 -- 172.21.15.102:0/1329711142 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90400cd70 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.216 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.214+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c1f27f0 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.216 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.215+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c068c60 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.216 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.216+0000 7fd902a00640 1 -- 172.21.15.102:0/1329711142 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fd90401a070 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.216 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.217+0000 7fd902a00640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8ec06c890 0x7fd8ec06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:41.216 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.217+0000 7fd902a00640 1 -- 172.21.15.102:0/1329711142 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fd90408ad30 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.216 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.217+0000 7fd90a800640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8ec06c890 0x7fd8ec06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:41.217 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.217+0000 7fd90a800640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8ec06c890 0x7fd8ec06ed50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0012e0 tx=0x7fd8fc001700 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:41.222 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.222+0000 7fd902a00640 1 -- 172.21.15.102:0/1329711142 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd90c068c60 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.407 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.407+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd90c0ce5d0 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.408 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.408+0000 7fd902a00640 1 -- 172.21.15.102:0/1329711142 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fd90c0ce5d0 con 0x7fd90c0ccbe0 2024-04-20T21:16:41.408 INFO:teuthology.misc.health.smithi102.stdout: 2024-04-20T21:16:41.408 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":12,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:38.675971+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:16:41.411 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8ec06c890 msgr2=0x7fd8ec06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:41.411 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8ec06c890 0x7fd8ec06ed50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0012e0 tx=0x7fd8fc001700 comp rx=0 tx=0).stop 2024-04-20T21:16:41.411 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90c0ccbe0 msgr2=0x7fd90c1f32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:41.411 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90c0ccbe0 0x7fd90c1f32d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e960 tx=0x7fd90400ee20 comp rx=0 tx=0).stop 2024-04-20T21:16:41.411 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 shutdown_connections 2024-04-20T21:16:41.411 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90c0ccbe0 0x7fd90c1f32d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.412 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8ec06c890 0x7fd8ec06ed50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.412 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd90c0b54a0 0x7fd90c158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.412 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 --2- 172.21.15.102:0/1329711142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd90c0b4720 0x7fd90c156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:41.412 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 >> 172.21.15.102:0/1329711142 conn(0x7fd90c067a20 msgr2=0x7fd90c067e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:41.412 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 shutdown_connections 2024-04-20T21:16:41.412 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:41.411+0000 7fd911400640 1 -- 172.21.15.102:0/1329711142 wait complete. 2024-04-20T21:16:41.424 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T21:16:48.232 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T21:16:48.417 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.417+0000 7f9657e00640 1 Processor -- start 2024-04-20T21:16:48.417 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.418+0000 7f9657e00640 1 -- start start 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.419+0000 7f9657e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f96580c7810 0x7f965815be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.419+0000 7f9657e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f96580678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.419+0000 7f9657e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9658067e20 0x7f965815e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.419+0000 7f9657e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f96580682b0 con 0x7f9658067e20 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.419+0000 7f9657e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f965815c360 con 0x7f96580c7810 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.419+0000 7f9657e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9658071e80 con 0x7f96580c8640 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9656a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f96580c7810 0x7f965815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9656a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f96580c7810 0x7f965815be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41414/0 (socket says 172.21.15.102:41414) 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9656a00640 1 -- 172.21.15.102:0/933993503 learned_addr learned my addr 172.21.15.102:0/933993503 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:48.420 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9657400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9658067e20 0x7f965815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.421 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f964e000640 1 --2- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f96580678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.421 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9656a00640 1 -- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 msgr2=0x7f96580678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:48.421 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9656a00640 1 --2- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f96580678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.421 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9656a00640 1 -- 172.21.15.102:0/933993503 >> v2:172.21.15.102:3300/0 conn(0x7f9658067e20 msgr2=0x7f965815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:48.421 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9656a00640 1 --2- 172.21.15.102:0/933993503 >> v2:172.21.15.102:3300/0 conn(0x7f9658067e20 0x7f965815e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.421 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9656a00640 1 -- 172.21.15.102:0/933993503 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9658072120 con 0x7f96580c7810 2024-04-20T21:16:48.421 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f964e000640 1 --2- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f96580678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:48.421 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.420+0000 7f9657400640 1 --2- 172.21.15.102:0/933993503 >> v2:172.21.15.102:3300/0 conn(0x7f9658067e20 0x7f965815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:48.422 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.421+0000 7f9656a00640 1 --2- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 0x7f965815be20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9648009e50 tx=0x7f964802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61f970c9f6d3637c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:48.422 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.422+0000 7f9656000640 1 -- 172.21.15.102:0/933993503 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964802cb00 con 0x7f96580c7810 2024-04-20T21:16:48.422 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.422+0000 7f9656000640 1 -- 172.21.15.102:0/933993503 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964802cca0 con 0x7f96580c7810 2024-04-20T21:16:48.422 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.422+0000 7f9656000640 1 -- 172.21.15.102:0/933993503 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648034610 con 0x7f96580c7810 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.422+0000 7f9657e00640 1 -- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 msgr2=0x7f965815be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.422+0000 7f9657e00640 1 --2- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 0x7f965815be20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9648009e50 tx=0x7f964802bf60 comp rx=0 tx=0).stop 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.422+0000 7f9657e00640 1 -- 172.21.15.102:0/933993503 shutdown_connections 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.423+0000 7f9657e00640 1 --2- 172.21.15.102:0/933993503 >> v2:172.21.15.102:3300/0 conn(0x7f9658067e20 0x7f965815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.423+0000 7f9657e00640 1 --2- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f96580678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.423+0000 7f9657e00640 1 --2- 172.21.15.102:0/933993503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 0x7f965815be20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.423+0000 7f9657e00640 1 -- 172.21.15.102:0/933993503 >> 172.21.15.102:0/933993503 conn(0x7f96580cd8c0 msgr2=0x7f96580cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.423+0000 7f9657e00640 1 -- 172.21.15.102:0/933993503 shutdown_connections 2024-04-20T21:16:48.423 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.423+0000 7f9657e00640 1 -- 172.21.15.102:0/933993503 wait complete. 2024-04-20T21:16:48.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.425+0000 7f9657e00640 1 Processor -- start 2024-04-20T21:16:48.424 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.425+0000 7f9657e00640 1 -- start start 2024-04-20T21:16:48.427 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.426+0000 7f9657e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658067e20 0x7f96581409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.427 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.426+0000 7f9657e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 0x7f965813f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.427 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.426+0000 7f9657e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f965813fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.426+0000 7f9657e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f96580729c0 con 0x7f9658067e20 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.426+0000 7f9657e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96580726e0 con 0x7f96580c7810 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.426+0000 7f9657e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9658072d30 con 0x7f96580c8640 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f9656a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658067e20 0x7f96581409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f964e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 0x7f965813f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f9656a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658067e20 0x7f96581409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57558/0 (socket says 172.21.15.102:57558) 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f9656a00640 1 -- 172.21.15.102:0/919773481 learned_addr learned my addr 172.21.15.102:0/919773481 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f9657400640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f965813fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f964e000640 1 -- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 msgr2=0x7f965813fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f964e000640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f965813fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f964e000640 1 -- 172.21.15.102:0/919773481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658067e20 msgr2=0x7f96581409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f964e000640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658067e20 0x7f96581409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f964e000640 1 -- 172.21.15.102:0/919773481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965813f0b0 con 0x7f96580c7810 2024-04-20T21:16:48.428 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f9656a00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658067e20 0x7f96581409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:48.429 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.427+0000 7f964e000640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 0x7f965813f840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f964000e8d0 tx=0x7f964000ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:48.429 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.428+0000 7f964f400640 1 -- 172.21.15.102:0/919773481 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640017070 con 0x7f96580c7810 2024-04-20T21:16:48.429 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.428+0000 7f964f400640 1 -- 172.21.15.102:0/919773481 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964000c940 con 0x7f96580c7810 2024-04-20T21:16:48.429 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.428+0000 7f964f400640 1 -- 172.21.15.102:0/919773481 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964000cd30 con 0x7f96580c7810 2024-04-20T21:16:48.429 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.428+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96581f3aa0 con 0x7f96580c7810 2024-04-20T21:16:48.429 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.428+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965813f320 con 0x7f96580c7810 2024-04-20T21:16:48.429 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.429+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9658076390 con 0x7f96580c7810 2024-04-20T21:16:48.429 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.430+0000 7f964f400640 1 -- 172.21.15.102:0/919773481 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f964001a070 con 0x7f96580c7810 2024-04-20T21:16:48.430 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.431+0000 7f964f400640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f963406c8f0 0x7f963406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.430 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.431+0000 7f964f400640 1 -- 172.21.15.102:0/919773481 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f964008b980 con 0x7f96580c7810 2024-04-20T21:16:48.431 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.431+0000 7f9656a00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f963406c8f0 0x7f963406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.431 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.432+0000 7f9656a00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f963406c8f0 0x7f963406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9648009e50 tx=0x7f9648030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:48.435 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.436+0000 7f964f400640 1 -- 172.21.15.102:0/919773481 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9640059a10 con 0x7f96580c7810 2024-04-20T21:16:48.635 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.635+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9658067c00 con 0x7f96580c7810 2024-04-20T21:16:48.636 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.637+0000 7f964f400640 1 -- 172.21.15.102:0/919773481 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f964005dd10 con 0x7f96580c7810 2024-04-20T21:16:48.637 INFO:teuthology.misc.health.smithi102.stdout: 2024-04-20T21:16:48.637 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":16,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:45.024434+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:16:48.639 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.639+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f963406c8f0 msgr2=0x7f963406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.639 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.639+0000 7f9657e00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f963406c8f0 0x7f963406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9648009e50 tx=0x7f9648030000 comp rx=0 tx=0).stop 2024-04-20T21:16:48.639 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.639+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 msgr2=0x7f965813f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.639+0000 7f9657e00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 0x7f965813f840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f964000e8d0 tx=0x7f964000ed90 comp rx=0 tx=0).stop 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.639+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 shutdown_connections 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.640+0000 7f9657e00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96580c8640 0x7f965813fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.640+0000 7f9657e00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f963406c8f0 0x7f963406edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.640+0000 7f9657e00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96580c7810 0x7f965813f840 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.640+0000 7f9657e00640 1 --2- 172.21.15.102:0/919773481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658067e20 0x7f96581409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.640+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 >> 172.21.15.102:0/919773481 conn(0x7f96580cd8c0 msgr2=0x7f96580c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.640+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 shutdown_connections 2024-04-20T21:16:48.640 INFO:teuthology.misc.health.smithi102.stderr:2024-04-20T21:16:48.640+0000 7f9657e00640 1 -- 172.21.15.102:0/919773481 wait complete. 2024-04-20T21:16:48.654 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-20T21:16:48.654 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-20T21:16:48.654 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:16:48.654 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:16:48.661 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:16:48.661 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:16:48.661 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:16:48.668 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:16:48.668 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-20T21:16:48.668 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-20T21:16:48.674 INFO:tasks.ceph.mds.e:Started 2024-04-20T21:16:48.674 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:16:48.674 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:16:48.677 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:16:48.677 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:16:48.677 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:16:48.679 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:16:48.679 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-20T21:16:48.679 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-20T21:16:48.681 INFO:tasks.ceph.mds.f:Started 2024-04-20T21:16:48.682 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-20T21:16:48.682 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:16:48.761 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-20T21:16:48.761 INFO:tasks.ceph.mds.f.smithi193.stdout:starting mds.f at 2024-04-20T21:16:48.761 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-20T21:16:48.814 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-20T21:16:48.817 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:16:48.821 INFO:tasks.ceph.mds.e.smithi102.stdout:starting mds.e at 2024-04-20T21:16:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.865+0000 7ff95c800640 1 Processor -- start 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.865+0000 7ff95c800640 1 -- start start 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff95c800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff9540cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff95c800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff95415bfb0 con 0x7ff954155a10 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff95c800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff95415c290 con 0x7ff954155630 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff95c800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff9540b4560 con 0x7ff95415c900 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff959e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff954155630 0x7ff9540b5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49630/0 (socket says 172.21.15.102:49630) 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff959e00640 1 -- 172.21.15.102:0/3066757573 learned_addr learned my addr 172.21.15.102:0/3066757573 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff95a800640 1 --2- 172.21.15.102:0/3066757573 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff9540cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff958a00640 1 -- 172.21.15.102:0/3066757573 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2991650394 0 0) 0x7ff95415c290 con 0x7ff954155630 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff958a00640 1 -- 172.21.15.102:0/3066757573 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff93c003520 con 0x7ff954155630 2024-04-20T21:16:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff958a00640 1 -- 172.21.15.102:0/3066757573 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3114577856 0 0) 0x7ff95415bfb0 con 0x7ff954155a10 2024-04-20T21:16:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff958a00640 1 -- 172.21.15.102:0/3066757573 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff95415c290 con 0x7ff954155a10 2024-04-20T21:16:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff95a800640 1 -- 172.21.15.102:0/3066757573 >> v1:172.21.15.193:6789/0 conn(0x7ff954155630 legacy=0x7ff9540b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff95a800640 1 -- 172.21.15.102:0/3066757573 >> v1:172.21.15.102:6789/0 conn(0x7ff954155a10 legacy=0x7ff95415b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.867+0000 7ff95a800640 1 -- 172.21.15.102:0/3066757573 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9540b47b0 con 0x7ff95415c900 2024-04-20T21:16:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.868+0000 7ff95a800640 1 --2- 172.21.15.102:0/3066757573 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff9540cbc10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff950009dc0 tx=0x7ff95002bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67cb758f1e242b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.869+0000 7ff958a00640 1 -- 172.21.15.102:0/3066757573 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950039070 con 0x7ff95415c900 2024-04-20T21:16:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.869+0000 7ff958a00640 1 -- 172.21.15.102:0/3066757573 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95002cd50 con 0x7ff95415c900 2024-04-20T21:16:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.869+0000 7ff958a00640 1 -- 172.21.15.102:0/3066757573 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950034660 con 0x7ff95415c900 2024-04-20T21:16:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.869+0000 7ff95c800640 1 -- 172.21.15.102:0/3066757573 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 msgr2=0x7ff9540cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.869+0000 7ff95c800640 1 --2- 172.21.15.102:0/3066757573 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff9540cbc10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff950009dc0 tx=0x7ff95002bfa0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.869+0000 7ff95c800640 1 -- 172.21.15.102:0/3066757573 shutdown_connections 2024-04-20T21:16:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.869+0000 7ff95c800640 1 --2- 172.21.15.102:0/3066757573 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff9540cbc10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.869+0000 7ff95c800640 1 -- 172.21.15.102:0/3066757573 >> 172.21.15.102:0/3066757573 conn(0x7ff954075080 msgr2=0x7ff954075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.870+0000 7ff95c800640 1 -- 172.21.15.102:0/3066757573 shutdown_connections 2024-04-20T21:16:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.870+0000 7ff95c800640 1 -- 172.21.15.102:0/3066757573 wait complete. 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.871+0000 7ff95c800640 1 Processor -- start 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.871+0000 7ff95c800640 1 -- start start 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.872+0000 7ff95c800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954155630 0x7ff9540c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.872+0000 7ff95c800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff95415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.872+0000 7ff95c800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9540ca550 0x7ff9541f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.872+0000 7ff95c800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff95415bba0 con 0x7ff9540ca550 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.872+0000 7ff95c800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9540b4e00 con 0x7ff954155630 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.872+0000 7ff95c800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff95415bd20 con 0x7ff95415c900 2024-04-20T21:16:48.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.872+0000 7ff959e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954155630 0x7ff9540c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.872+0000 7ff959400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff95415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff95a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9540ca550 0x7ff9541f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954155630 0x7ff9540c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41492/0 (socket says 172.21.15.102:41492) 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff95415abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56214/0 (socket says 172.21.15.102:56214) 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff95a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9540ca550 0x7ff9541f42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57594/0 (socket says 172.21.15.102:57594) 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959e00640 1 -- 172.21.15.102:0/256880424 learned_addr learned my addr 172.21.15.102:0/256880424 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959e00640 1 -- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 msgr2=0x7ff95415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959e00640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff95415abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959e00640 1 -- 172.21.15.102:0/256880424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9540ca550 msgr2=0x7ff9541f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959e00640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9540ca550 0x7ff9541f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959e00640 1 -- 172.21.15.102:0/256880424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9541520a0 con 0x7ff954155630 2024-04-20T21:16:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959400640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff95415abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff959e00640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954155630 0x7ff9540c8a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c500 tx=0x7ff94800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff94e000640 1 -- 172.21.15.102:0/256880424 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948017070 con 0x7ff954155630 2024-04-20T21:16:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff94e000640 1 -- 172.21.15.102:0/256880424 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff948005ce0 con 0x7ff954155630 2024-04-20T21:16:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.873+0000 7ff94e000640 1 -- 172.21.15.102:0/256880424 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9480072f0 con 0x7ff954155630 2024-04-20T21:16:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.874+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95415a3f0 con 0x7ff954155630 2024-04-20T21:16:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.874+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff954154b40 con 0x7ff954155630 2024-04-20T21:16:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.875+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9540d1690 con 0x7ff954155630 2024-04-20T21:16:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.875+0000 7ff94e000640 1 -- 172.21.15.102:0/256880424 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff94801a070 con 0x7ff954155630 2024-04-20T21:16:48.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.876+0000 7ff94e000640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff93806c920 0x7ff93806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:48.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.876+0000 7ff94e000640 1 -- 172.21.15.102:0/256880424 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff94808ab20 con 0x7ff954155630 2024-04-20T21:16:48.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.877+0000 7ff959400640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff93806c920 0x7ff93806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:48.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.877+0000 7ff959400640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff93806c920 0x7ff93806ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9540cbb10 tx=0x7ff944009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:48.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:48.881+0000 7ff94e000640 1 -- 172.21.15.102:0/256880424 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff9540d1690 con 0x7ff954155630 2024-04-20T21:16:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.077+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff954074340 con 0x7ff954155630 2024-04-20T21:16:49.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.078+0000 7ff94e000640 1 -- 172.21.15.102:0/256880424 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7ff948058d00 con 0x7ff954155630 2024-04-20T21:16:49.078 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:16:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:45.024434+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:16:49.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff93806c920 msgr2=0x7ff93806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff93806c920 0x7ff93806ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9540cbb10 tx=0x7ff944009040 comp rx=0 tx=0).stop 2024-04-20T21:16:49.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954155630 msgr2=0x7ff9540c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954155630 0x7ff9540c8a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c500 tx=0x7ff94800c9c0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 shutdown_connections 2024-04-20T21:16:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9540ca550 0x7ff9541f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff95415c900 0x7ff95415abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff93806c920 0x7ff93806ede0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 --2- 172.21.15.102:0/256880424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954155630 0x7ff9540c8a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 >> 172.21.15.102:0/256880424 conn(0x7ff954075080 msgr2=0x7ff9541542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:49.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 shutdown_connections 2024-04-20T21:16:49.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.081+0000 7ff95c800640 1 -- 172.21.15.102:0/256880424 wait complete. 2024-04-20T21:16:49.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:16:49.100 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:16:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.296+0000 7faf26400640 1 Processor -- start 2024-04-20T21:16:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.296+0000 7faf26400640 1 -- start start 2024-04-20T21:16:49.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf26400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf200cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf26400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faf200c7ff0 0x7faf200ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf26400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faf200c8540 0x7faf2006c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf26400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7faf2015c350 con 0x7faf200c8540 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf26400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7faf200cb450 con 0x7faf200c7ff0 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf26400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faf200cb730 con 0x7faf20155930 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf1f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf200cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf16a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faf200c7ff0 0x7faf200ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf1f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf200cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56222/0 (socket says 172.21.15.102:56222) 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf1fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faf200c8540 0x7faf2006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.298+0000 7faf1f400640 1 -- 172.21.15.102:0/3943752994 learned_addr learned my addr 172.21.15.102:0/3943752994 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.299+0000 7faf1f400640 1 -- 172.21.15.102:0/3943752994 >> v2:172.21.15.193:3300/0 conn(0x7faf200c7ff0 msgr2=0x7faf200ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.299+0000 7faf1f400640 1 --2- 172.21.15.102:0/3943752994 >> v2:172.21.15.193:3300/0 conn(0x7faf200c7ff0 0x7faf200ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.299+0000 7faf1f400640 1 -- 172.21.15.102:0/3943752994 >> v2:172.21.15.102:3300/0 conn(0x7faf200c8540 msgr2=0x7faf2006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:49.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.299+0000 7faf1f400640 1 --2- 172.21.15.102:0/3943752994 >> v2:172.21.15.102:3300/0 conn(0x7faf200c8540 0x7faf2006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.299+0000 7faf1f400640 1 -- 172.21.15.102:0/3943752994 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf200cba10 con 0x7faf20155930 2024-04-20T21:16:49.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.299+0000 7faf1fe00640 1 --2- 172.21.15.102:0/3943752994 >> v2:172.21.15.102:3300/0 conn(0x7faf200c8540 0x7faf2006c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.300+0000 7faf1f400640 1 --2- 172.21.15.102:0/3943752994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf200cc7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faf20067680 tx=0x7faf1002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570409d1b832ac8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.300+0000 7faf1ea00640 1 -- 172.21.15.102:0/3943752994 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10035020 con 0x7faf20155930 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.300+0000 7faf1ea00640 1 -- 172.21.15.102:0/3943752994 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf10031870 con 0x7faf20155930 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.300+0000 7faf1ea00640 1 -- 172.21.15.102:0/3943752994 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10031c40 con 0x7faf20155930 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 -- 172.21.15.102:0/3943752994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 msgr2=0x7faf200cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 --2- 172.21.15.102:0/3943752994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf200cc7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faf20067680 tx=0x7faf1002b910 comp rx=0 tx=0).stop 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 -- 172.21.15.102:0/3943752994 shutdown_connections 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 --2- 172.21.15.102:0/3943752994 >> v2:172.21.15.102:3300/0 conn(0x7faf200c8540 0x7faf2006c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 --2- 172.21.15.102:0/3943752994 >> v2:172.21.15.193:3300/0 conn(0x7faf200c7ff0 0x7faf200ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 --2- 172.21.15.102:0/3943752994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf200cc7c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 -- 172.21.15.102:0/3943752994 >> 172.21.15.102:0/3943752994 conn(0x7faf200d0cb0 msgr2=0x7faf200d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:49.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 -- 172.21.15.102:0/3943752994 shutdown_connections 2024-04-20T21:16:49.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.301+0000 7faf26400640 1 -- 172.21.15.102:0/3943752994 wait complete. 2024-04-20T21:16:49.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.303+0000 7faf26400640 1 Processor -- start 2024-04-20T21:16:49.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.303+0000 7faf26400640 1 -- start start 2024-04-20T21:16:49.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf26400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf200c7ff0 0x7faf201f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf26400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf200c8540 0x7faf2015da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf26400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf2015e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf26400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faf2015bf60 con 0x7faf200c8540 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf26400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf200cbfd0 con 0x7faf200c7ff0 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf26400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faf2015c0e0 con 0x7faf20155930 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf16a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf200c8540 0x7faf2015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf1fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf2015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf16a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf200c8540 0x7faf2015da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57614/0 (socket says 172.21.15.102:57614) 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf1f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf200c7ff0 0x7faf201f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.304+0000 7faf16a00640 1 -- 172.21.15.102:0/2909662348 learned_addr learned my addr 172.21.15.102:0/2909662348 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf1fe00640 1 -- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf200c7ff0 msgr2=0x7faf201f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf1fe00640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf200c7ff0 0x7faf201f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf1fe00640 1 -- 172.21.15.102:0/2909662348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf200c8540 msgr2=0x7faf2015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf1fe00640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf200c8540 0x7faf2015da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf1fe00640 1 -- 172.21.15.102:0/2909662348 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf201f2610 con 0x7faf20155930 2024-04-20T21:16:49.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf16a00640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf200c8540 0x7faf2015da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:49.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf1fe00640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf2015e740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf18009db0 tx=0x7faf1800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:49.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf17e00640 1 -- 172.21.15.102:0/2909662348 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18018030 con 0x7faf20155930 2024-04-20T21:16:49.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf17e00640 1 -- 172.21.15.102:0/2909662348 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf1800cd30 con 0x7faf20155930 2024-04-20T21:16:49.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf17e00640 1 -- 172.21.15.102:0/2909662348 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18003bb0 con 0x7faf20155930 2024-04-20T21:16:49.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.305+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf201e6f30 con 0x7faf20155930 2024-04-20T21:16:49.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.306+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faf201e71d0 con 0x7faf20155930 2024-04-20T21:16:49.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.307+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf200cae20 con 0x7faf20155930 2024-04-20T21:16:49.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.308+0000 7faf17e00640 1 -- 172.21.15.102:0/2909662348 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf18005d10 con 0x7faf20155930 2024-04-20T21:16:49.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.309+0000 7faf17e00640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faefc06c8f0 0x7faefc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.309+0000 7faf17e00640 1 -- 172.21.15.102:0/2909662348 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7faf1808c0b0 con 0x7faf20155930 2024-04-20T21:16:49.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.309+0000 7faf1f400640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faefc06c8f0 0x7faefc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.309+0000 7faf1f400640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faefc06c8f0 0x7faefc06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf10036000 tx=0x7faf1003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:49.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.314+0000 7faf17e00640 1 -- 172.21.15.102:0/2909662348 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faf200cae20 con 0x7faf20155930 2024-04-20T21:16:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.515+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf2004fd60 con 0x7faf20155930 2024-04-20T21:16:49.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.516+0000 7faf17e00640 1 -- 172.21.15.102:0/2909662348 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7faf18022070 con 0x7faf20155930 2024-04-20T21:16:49.516 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:16:49.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faefc06c8f0 msgr2=0x7faefc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faefc06c8f0 0x7faefc06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf10036000 tx=0x7faf1003c040 comp rx=0 tx=0).stop 2024-04-20T21:16:49.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 msgr2=0x7faf2015e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf2015e740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf18009db0 tx=0x7faf1800c490 comp rx=0 tx=0).stop 2024-04-20T21:16:49.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 shutdown_connections 2024-04-20T21:16:49.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf20155930 0x7faf2015e740 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf200c8540 0x7faf2015da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faefc06c8f0 0x7faefc06edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 --2- 172.21.15.102:0/2909662348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf200c7ff0 0x7faf201f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 >> 172.21.15.102:0/2909662348 conn(0x7faf200d0cb0 msgr2=0x7faf2014efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:49.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 shutdown_connections 2024-04-20T21:16:49.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.519+0000 7faf26400640 1 -- 172.21.15.102:0/2909662348 wait complete. 2024-04-20T21:16:49.538 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:16:49.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.740+0000 7ffba4c00640 1 Processor -- start 2024-04-20T21:16:49.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.740+0000 7ffba4c00640 1 -- start start 2024-04-20T21:16:49.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.741+0000 7ffba4c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba01516d0 0x7ffba00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.741+0000 7ffba4c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba0158480 con 0x7ffba0151ab0 2024-04-20T21:16:49.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.741+0000 7ffba4c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba006ef60 con 0x7ffba0157ff0 2024-04-20T21:16:49.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.741+0000 7ffba4c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00c4dd0 con 0x7ffba01516d0 2024-04-20T21:16:49.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.741+0000 7ffb96a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ffba0151ab0 0x7ffba00c3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47294/0 (socket says 172.21.15.102:47294) 2024-04-20T21:16:49.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.741+0000 7ffb96a00640 1 -- 172.21.15.102:0/183623128 learned_addr learned my addr 172.21.15.102:0/183623128 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:49.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.741+0000 7ffb9f400640 1 --2- 172.21.15.102:0/183623128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba01516d0 0x7ffba00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.742+0000 7ffb9ea00640 1 -- 172.21.15.102:0/183623128 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1232407746 0 0) 0x7ffba0158480 con 0x7ffba0151ab0 2024-04-20T21:16:49.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.742+0000 7ffb9ea00640 1 -- 172.21.15.102:0/183623128 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb84003520 con 0x7ffba0151ab0 2024-04-20T21:16:49.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.742+0000 7ffb9ea00640 1 -- 172.21.15.102:0/183623128 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058006477 0 0) 0x7ffba006ef60 con 0x7ffba0157ff0 2024-04-20T21:16:49.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.742+0000 7ffb9ea00640 1 -- 172.21.15.102:0/183623128 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffba0158480 con 0x7ffba0157ff0 2024-04-20T21:16:49.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.742+0000 7ffb9f400640 1 -- 172.21.15.102:0/183623128 >> v1:172.21.15.193:6789/0 conn(0x7ffba0157ff0 legacy=0x7ffba00c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.742+0000 7ffb9f400640 1 -- 172.21.15.102:0/183623128 >> v1:172.21.15.102:6789/0 conn(0x7ffba0151ab0 legacy=0x7ffba00c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.742+0000 7ffb9f400640 1 -- 172.21.15.102:0/183623128 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba006f240 con 0x7ffba01516d0 2024-04-20T21:16:49.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.743+0000 7ffb9f400640 1 --2- 172.21.15.102:0/183623128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba01516d0 0x7ffba00cd310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009dc0 tx=0x7ffb9002bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb57c872931d572 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.744+0000 7ffb9ea00640 1 -- 172.21.15.102:0/183623128 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90039070 con 0x7ffba01516d0 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.744+0000 7ffb9ea00640 1 -- 172.21.15.102:0/183623128 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9002cd50 con 0x7ffba01516d0 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.744+0000 7ffb9ea00640 1 -- 172.21.15.102:0/183623128 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90034640 con 0x7ffba01516d0 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.744+0000 7ffba4c00640 1 -- 172.21.15.102:0/183623128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba01516d0 msgr2=0x7ffba00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.744+0000 7ffba4c00640 1 --2- 172.21.15.102:0/183623128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba01516d0 0x7ffba00cd310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009dc0 tx=0x7ffb9002bfa0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.745+0000 7ffba4c00640 1 -- 172.21.15.102:0/183623128 shutdown_connections 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.745+0000 7ffba4c00640 1 --2- 172.21.15.102:0/183623128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba01516d0 0x7ffba00cd310 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.745+0000 7ffba4c00640 1 -- 172.21.15.102:0/183623128 >> 172.21.15.102:0/183623128 conn(0x7ffba00cc820 msgr2=0x7ffba00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.745+0000 7ffba4c00640 1 -- 172.21.15.102:0/183623128 shutdown_connections 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.745+0000 7ffba4c00640 1 -- 172.21.15.102:0/183623128 wait complete. 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.746+0000 7ffba4c00640 1 Processor -- start 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.746+0000 7ffba4c00640 1 -- start start 2024-04-20T21:16:49.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.747+0000 7ffba4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffba01516d0 0x7ffba014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.747+0000 7ffba4c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0151ab0 0x7ffba00be320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.747+0000 7ffba4c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba0157ff0 0x7ffba00be860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.747+0000 7ffba4c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffba00c5300 con 0x7ffba01516d0 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.747+0000 7ffba4c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffba00c5020 con 0x7ffba0151ab0 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.747+0000 7ffba4c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00c4a40 con 0x7ffba0157ff0 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffba01516d0 0x7ffba014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffba01516d0 0x7ffba014d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57624/0 (socket says 172.21.15.102:57624) 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 -- 172.21.15.102:0/4283052914 learned_addr learned my addr 172.21.15.102:0/4283052914 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb96a00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0151ab0 0x7ffba00be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9fe00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba0157ff0 0x7ffba00be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 -- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba0157ff0 msgr2=0x7ffba00be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba0157ff0 0x7ffba00be860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 -- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0151ab0 msgr2=0x7ffba00be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:49.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0151ab0 0x7ffba00be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:49.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 -- 172.21.15.102:0/4283052914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba0156700 con 0x7ffba01516d0 2024-04-20T21:16:49.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.748+0000 7ffb9f400640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffba01516d0 0x7ffba014d470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009eb0 tx=0x7ffb90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:49.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.749+0000 7ffb97e00640 1 -- 172.21.15.102:0/4283052914 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90042020 con 0x7ffba01516d0 2024-04-20T21:16:49.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.749+0000 7ffb97e00640 1 -- 172.21.15.102:0/4283052914 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9003a040 con 0x7ffba01516d0 2024-04-20T21:16:49.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.749+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba01569e0 con 0x7ffba01516d0 2024-04-20T21:16:49.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.749+0000 7ffb97e00640 1 -- 172.21.15.102:0/4283052914 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90039070 con 0x7ffba01516d0 2024-04-20T21:16:49.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.749+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba0156fa0 con 0x7ffba01516d0 2024-04-20T21:16:49.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.750+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba004fd60 con 0x7ffba01516d0 2024-04-20T21:16:49.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.751+0000 7ffb97e00640 1 -- 172.21.15.102:0/4283052914 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb90045020 con 0x7ffba01516d0 2024-04-20T21:16:49.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.752+0000 7ffb97e00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ffb7c06c8f0 0x7ffb7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:49.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.752+0000 7ffb97e00640 1 -- 172.21.15.102:0/4283052914 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ffb900ad970 con 0x7ffba01516d0 2024-04-20T21:16:49.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.752+0000 7ffb96a00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ffb7c06c8f0 0x7ffb7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:49.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.753+0000 7ffb96a00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ffb7c06c8f0 0x7ffb7c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffba0150490 tx=0x7ffb88007d70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:49.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:49.757+0000 7ffb97e00640 1 -- 172.21.15.102:0/4283052914 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffb9004c030 con 0x7ffba01516d0 2024-04-20T21:16:50.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.001+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffba006dcb0 con 0x7ffba01516d0 2024-04-20T21:16:50.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.002+0000 7ffb97e00640 1 -- 172.21.15.102:0/4283052914 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5524 (secure 0 0 0) 0x7ffb9007ba10 con 0x7ffba01516d0 2024-04-20T21:16:50.002 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:16:50.002 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-20T21:16:50.002 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 3 2024-04-20T21:16:50.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.004+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ffb7c06c8f0 msgr2=0x7ffb7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.004+0000 7ffba4c00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ffb7c06c8f0 0x7ffb7c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffba0150490 tx=0x7ffb88007d70 comp rx=0 tx=0).stop 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffba01516d0 msgr2=0x7ffba014d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffba01516d0 0x7ffba014d470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009eb0 tx=0x7ffb90004a40 comp rx=0 tx=0).stop 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 shutdown_connections 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffba0157ff0 0x7ffba00be860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ffb7c06c8f0 0x7ffb7c06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffba0151ab0 0x7ffba00be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 --2- 172.21.15.102:0/4283052914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffba01516d0 0x7ffba014d470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 >> 172.21.15.102:0/4283052914 conn(0x7ffba00cc820 msgr2=0x7ffba0150970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 shutdown_connections 2024-04-20T21:16:50.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.005+0000 7ffba4c00640 1 -- 172.21.15.102:0/4283052914 wait complete. 2024-04-20T21:16:50.023 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:16:50.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.224+0000 7fda77e00640 1 Processor -- start 2024-04-20T21:16:50.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.224+0000 7fda77e00640 1 -- start start 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.225+0000 7fda77e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda78153180 0x7fda78158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.225+0000 7fda77e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fda78159380 0x7fda780c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.225+0000 7fda77e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda78153560 con 0x7fda780c75a0 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.225+0000 7fda77e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fda780c7a70 con 0x7fda78159380 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.225+0000 7fda77e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fda780c7d30 con 0x7fda78153180 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.226+0000 7fda6e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fda78159380 0x7fda780c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.226+0000 7fda77400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fda780c75a0 0x7fda78070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47300/0 (socket says 172.21.15.102:47300) 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.226+0000 7fda77400640 1 -- 172.21.15.102:0/2849713769 learned_addr learned my addr 172.21.15.102:0/2849713769 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.226+0000 7fda76a00640 1 --2- 172.21.15.102:0/2849713769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda78153180 0x7fda78158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.226+0000 7fda6e000640 1 -- 172.21.15.102:0/2849713769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda78153180 msgr2=0x7fda78158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:50.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.226+0000 7fda6e000640 1 --2- 172.21.15.102:0/2849713769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda78153180 0x7fda78158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.226+0000 7fda6e000640 1 -- 172.21.15.102:0/2849713769 >> v1:172.21.15.102:6789/0 conn(0x7fda780c75a0 legacy=0x7fda78070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.226+0000 7fda6e000640 1 -- 172.21.15.102:0/2849713769 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7815c4d0 con 0x7fda78159380 2024-04-20T21:16:50.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.227+0000 7fda6e000640 1 --2- 172.21.15.102:0/2849713769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78159380 0x7fda780c5f30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda600097a0 tx=0x7fda6002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e22855740386d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:50.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda76000640 1 -- 172.21.15.102:0/2849713769 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60035020 con 0x7fda78159380 2024-04-20T21:16:50.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda76000640 1 -- 172.21.15.102:0/2849713769 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda60031810 con 0x7fda78159380 2024-04-20T21:16:50.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda76000640 1 -- 172.21.15.102:0/2849713769 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60031be0 con 0x7fda78159380 2024-04-20T21:16:50.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda77e00640 1 -- 172.21.15.102:0/2849713769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78159380 msgr2=0x7fda780c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda77e00640 1 --2- 172.21.15.102:0/2849713769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78159380 0x7fda780c5f30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda600097a0 tx=0x7fda6002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda77e00640 1 -- 172.21.15.102:0/2849713769 shutdown_connections 2024-04-20T21:16:50.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda77e00640 1 --2- 172.21.15.102:0/2849713769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78159380 0x7fda780c5f30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda77e00640 1 --2- 172.21.15.102:0/2849713769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda78153180 0x7fda78158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda77e00640 1 -- 172.21.15.102:0/2849713769 >> 172.21.15.102:0/2849713769 conn(0x7fda780cc820 msgr2=0x7fda780ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:50.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.228+0000 7fda77e00640 1 -- 172.21.15.102:0/2849713769 shutdown_connections 2024-04-20T21:16:50.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.229+0000 7fda77e00640 1 -- 172.21.15.102:0/2849713769 wait complete. 2024-04-20T21:16:50.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.230+0000 7fda77e00640 1 Processor -- start 2024-04-20T21:16:50.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.230+0000 7fda77e00640 1 -- start start 2024-04-20T21:16:50.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda77e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda780c75a0 0x7fda781e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78153180 0x7fda781f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda77e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda78159380 0x7fda781f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda77e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fda7815d290 con 0x7fda78153180 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda77e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda7815ca90 con 0x7fda78159380 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda77e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fda780c6be0 con 0x7fda780c75a0 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda6e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78153180 0x7fda781f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda6e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78153180 0x7fda781f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57644/0 (socket says 172.21.15.102:57644) 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.231+0000 7fda6e000640 1 -- 172.21.15.102:0/2790934681 learned_addr learned my addr 172.21.15.102:0/2790934681 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda76a00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda780c75a0 0x7fda781e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda77400640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda78159380 0x7fda781f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6e000640 1 -- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda780c75a0 msgr2=0x7fda781e7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6e000640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda780c75a0 0x7fda781e7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6e000640 1 -- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda78159380 msgr2=0x7fda781f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6e000640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda78159380 0x7fda781f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6e000640 1 -- 172.21.15.102:0/2790934681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7815b5a0 con 0x7fda78153180 2024-04-20T21:16:50.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6e000640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78153180 0x7fda781f4460 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fda60009890 tx=0x7fda600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:50.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6f400640 1 -- 172.21.15.102:0/2790934681 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60035040 con 0x7fda78153180 2024-04-20T21:16:50.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6f400640 1 -- 172.21.15.102:0/2790934681 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6003ad30 con 0x7fda78153180 2024-04-20T21:16:50.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda6f400640 1 -- 172.21.15.102:0/2790934681 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda600466a0 con 0x7fda78153180 2024-04-20T21:16:50.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.232+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7815b810 con 0x7fda78153180 2024-04-20T21:16:50.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.233+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7815bdd0 con 0x7fda78153180 2024-04-20T21:16:50.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.234+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7804fd60 con 0x7fda78153180 2024-04-20T21:16:50.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.235+0000 7fda6f400640 1 -- 172.21.15.102:0/2790934681 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda60038020 con 0x7fda78153180 2024-04-20T21:16:50.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.236+0000 7fda6f400640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.236+0000 7fda6f400640 1 -- 172.21.15.102:0/2790934681 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fda600ad830 con 0x7fda78153180 2024-04-20T21:16:50.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.236+0000 7fda76a00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.237+0000 7fda76a00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda5406c8f0 0x7fda5406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda68009aa0 tx=0x7fda68009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:50.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.240+0000 7fda6f400640 1 -- 172.21.15.102:0/2790934681 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fda6007b960 con 0x7fda78153180 2024-04-20T21:16:50.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.445+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda7806b9b0 con 0x7fda78153180 2024-04-20T21:16:50.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.447+0000 7fda6f400640 1 -- 172.21.15.102:0/2790934681 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7fda6007fc60 con 0x7fda78153180 2024-04-20T21:16:50.447 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:16:50.447 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:45.024434+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:16:50.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda5406c8f0 msgr2=0x7fda5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda5406c8f0 0x7fda5406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda68009aa0 tx=0x7fda68009190 comp rx=0 tx=0).stop 2024-04-20T21:16:50.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78153180 msgr2=0x7fda781f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78153180 0x7fda781f4460 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fda60009890 tx=0x7fda600046c0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 shutdown_connections 2024-04-20T21:16:50.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda5406c8f0 0x7fda5406edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda78159380 0x7fda781f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda78153180 0x7fda781f4460 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 --2- 172.21.15.102:0/2790934681 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda780c75a0 0x7fda781e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 >> 172.21.15.102:0/2790934681 conn(0x7fda780cc820 msgr2=0x7fda780cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:50.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 shutdown_connections 2024-04-20T21:16:50.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.450+0000 7fda77e00640 1 -- 172.21.15.102:0/2790934681 wait complete. 2024-04-20T21:16:50.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:16:50.469 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:16:50.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.674+0000 7ff407200640 1 Processor -- start 2024-04-20T21:16:50.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.674+0000 7ff407200640 1 -- start start 2024-04-20T21:16:50.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.676+0000 7ff407200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff4001522e0 0x7ff400158990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.676+0000 7ff407200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff40015b520 con 0x7ff40015c120 2024-04-20T21:16:50.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.676+0000 7ff407200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4000712b0 con 0x7ff40015bce0 2024-04-20T21:16:50.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.676+0000 7ff407200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff400071060 con 0x7ff4001522e0 2024-04-20T21:16:50.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.676+0000 7ff3ffe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff4001522e0 0x7ff400158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.676+0000 7ff405200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff40015c120 0x7ff4000c8ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47302/0 (socket says 172.21.15.102:47302) 2024-04-20T21:16:50.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.676+0000 7ff405200640 1 -- 172.21.15.102:0/3332226071 learned_addr learned my addr 172.21.15.102:0/3332226071 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:50.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.677+0000 7ff3ffe00640 1 -- 172.21.15.102:0/3332226071 >> v1:172.21.15.193:6789/0 conn(0x7ff40015bce0 legacy=0x7ff4000c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.677+0000 7ff3ffe00640 1 -- 172.21.15.102:0/3332226071 >> v1:172.21.15.102:6789/0 conn(0x7ff40015c120 legacy=0x7ff4000c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.677+0000 7ff3ffe00640 1 -- 172.21.15.102:0/3332226071 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff400071590 con 0x7ff4001522e0 2024-04-20T21:16:50.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.678+0000 7ff3ffe00640 1 --2- 172.21.15.102:0/3332226071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4001522e0 0x7ff400158990 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00097a0 tx=0x7ff3f002b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f292d831433e151 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:50.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.678+0000 7ff3fea00640 1 -- 172.21.15.102:0/3332226071 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0037070 con 0x7ff4001522e0 2024-04-20T21:16:50.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.678+0000 7ff3fea00640 1 -- 172.21.15.102:0/3332226071 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f0030510 con 0x7ff4001522e0 2024-04-20T21:16:50.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.678+0000 7ff3fea00640 1 -- 172.21.15.102:0/3332226071 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f00308e0 con 0x7ff4001522e0 2024-04-20T21:16:50.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.679+0000 7ff407200640 1 -- 172.21.15.102:0/3332226071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4001522e0 msgr2=0x7ff400158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.679+0000 7ff407200640 1 --2- 172.21.15.102:0/3332226071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4001522e0 0x7ff400158990 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00097a0 tx=0x7ff3f002b600 comp rx=0 tx=0).stop 2024-04-20T21:16:50.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.679+0000 7ff407200640 1 -- 172.21.15.102:0/3332226071 shutdown_connections 2024-04-20T21:16:50.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.679+0000 7ff407200640 1 --2- 172.21.15.102:0/3332226071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4001522e0 0x7ff400158990 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.679+0000 7ff407200640 1 -- 172.21.15.102:0/3332226071 >> 172.21.15.102:0/3332226071 conn(0x7ff4000c9390 msgr2=0x7ff4000c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:50.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.679+0000 7ff407200640 1 -- 172.21.15.102:0/3332226071 shutdown_connections 2024-04-20T21:16:50.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.679+0000 7ff407200640 1 -- 172.21.15.102:0/3332226071 wait complete. 2024-04-20T21:16:50.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.681+0000 7ff407200640 1 Processor -- start 2024-04-20T21:16:50.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.681+0000 7ff407200640 1 -- start start 2024-04-20T21:16:50.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff407200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4001522e0 0x7ff40014dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff407200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff40015bce0 0x7ff4000c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff407200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff40015c120 0x7ff4000c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff407200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff400071870 con 0x7ff40015bce0 2024-04-20T21:16:50.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff407200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001526c0 con 0x7ff40015c120 2024-04-20T21:16:50.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff407200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff4000c5060 con 0x7ff4001522e0 2024-04-20T21:16:50.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff3ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff40015bce0 0x7ff4000c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff3ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff40015bce0 0x7ff4000c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57656/0 (socket says 172.21.15.102:57656) 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff3ff400640 1 -- 172.21.15.102:0/1443949967 learned_addr learned my addr 172.21.15.102:0/1443949967 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff3ffe00640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4001522e0 0x7ff40014dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.682+0000 7ff405200640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff40015c120 0x7ff4000c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3ff400640 1 -- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4001522e0 msgr2=0x7ff40014dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3ff400640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4001522e0 0x7ff40014dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3ff400640 1 -- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff40015c120 msgr2=0x7ff4000c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3ff400640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff40015c120 0x7ff4000c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3ff400640 1 -- 172.21.15.102:0/1443949967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff400159700 con 0x7ff40015bce0 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3ff400640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff40015bce0 0x7ff4000c6de0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c870 tx=0x7ff3e800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3f7e00640 1 -- 172.21.15.102:0/1443949967 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8017070 con 0x7ff40015bce0 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3f7e00640 1 -- 172.21.15.102:0/1443949967 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e8003dc0 con 0x7ff40015bce0 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff3f7e00640 1 -- 172.21.15.102:0/1443949967 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80052b0 con 0x7ff40015bce0 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4001599e0 con 0x7ff40015bce0 2024-04-20T21:16:50.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.683+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff400159fa0 con 0x7ff40015bce0 2024-04-20T21:16:50.685 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:50.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.685+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40004fd60 con 0x7ff40015bce0 2024-04-20T21:16:50.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.685+0000 7ff3f7e00640 1 -- 172.21.15.102:0/1443949967 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3e8005450 con 0x7ff40015bce0 2024-04-20T21:16:50.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.686+0000 7ff3f7e00640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:50.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.686+0000 7ff3f7e00640 1 -- 172.21.15.102:0/1443949967 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff3e808adf0 con 0x7ff40015bce0 2024-04-20T21:16:50.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.687+0000 7ff3ffe00640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:50.689 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:50.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.687+0000 7ff3ffe00640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0034000 tx=0x7ff3f0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:50.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.692+0000 7ff3f7e00640 1 -- 172.21.15.102:0/1443949967 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff3e8058e90 con 0x7ff40015bce0 2024-04-20T21:16:50.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.893+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4000734b0 con 0x7ff40015bce0 2024-04-20T21:16:50.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.894+0000 7ff3f7e00640 1 -- 172.21.15.102:0/1443949967 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7ff3e805d190 con 0x7ff40015bce0 2024-04-20T21:16:50.894 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3dc06c8f0 msgr2=0x7ff3dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0034000 tx=0x7ff3f0008030 comp rx=0 tx=0).stop 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff40015bce0 msgr2=0x7ff4000c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff40015bce0 0x7ff4000c6de0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c870 tx=0x7ff3e800cd30 comp rx=0 tx=0).stop 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 shutdown_connections 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3dc06c8f0 0x7ff3dc06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff40015c120 0x7ff4000c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff40015bce0 0x7ff4000c6de0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 --2- 172.21.15.102:0/1443949967 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4001522e0 0x7ff40014dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:50.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 >> 172.21.15.102:0/1443949967 conn(0x7ff4000c9390 msgr2=0x7ff400072410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:50.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 shutdown_connections 2024-04-20T21:16:50.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:50.897+0000 7ff407200640 1 -- 172.21.15.102:0/1443949967 wait complete. 2024-04-20T21:16:50.916 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:16:51.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.115+0000 7f56b6800640 1 Processor -- start 2024-04-20T21:16:51.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.115+0000 7f56b6800640 1 -- start start 2024-04-20T21:16:51.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.117+0000 7f56b6800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f56b015f0c0 0x7f56b00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.117+0000 7f56b6800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b00c7730 con 0x7f56b00716d0 2024-04-20T21:16:51.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.117+0000 7f56b6800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b015b460 con 0x7f56b00c7350 2024-04-20T21:16:51.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.117+0000 7f56b6800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f56b00ca5a0 con 0x7f56b015f0c0 2024-04-20T21:16:51.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.117+0000 7f56aea00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f56b00716d0 0x7f56b015e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47314/0 (socket says 172.21.15.102:47314) 2024-04-20T21:16:51.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.117+0000 7f56aea00640 1 -- 172.21.15.102:0/1642193705 learned_addr learned my addr 172.21.15.102:0/1642193705 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:51.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.117+0000 7f56afe00640 1 --2- 172.21.15.102:0/1642193705 >> v2:172.21.15.102:3300/0 conn(0x7f56b015f0c0 0x7f56b00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.118+0000 7f56ae000640 1 -- 172.21.15.102:0/1642193705 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443761951 0 0) 0x7f56b00c7730 con 0x7f56b00716d0 2024-04-20T21:16:51.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.118+0000 7f56ae000640 1 -- 172.21.15.102:0/1642193705 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5694003520 con 0x7f56b00716d0 2024-04-20T21:16:51.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.118+0000 7f56ae000640 1 -- 172.21.15.102:0/1642193705 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 332023519 0 0) 0x7f56b015b460 con 0x7f56b00c7350 2024-04-20T21:16:51.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.118+0000 7f56ae000640 1 -- 172.21.15.102:0/1642193705 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56b00c7730 con 0x7f56b00c7350 2024-04-20T21:16:51.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.118+0000 7f56afe00640 1 -- 172.21.15.102:0/1642193705 >> v1:172.21.15.193:6789/0 conn(0x7f56b00c7350 legacy=0x7f56b006b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.118+0000 7f56afe00640 1 -- 172.21.15.102:0/1642193705 >> v1:172.21.15.102:6789/0 conn(0x7f56b00716d0 legacy=0x7f56b015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.118+0000 7f56afe00640 1 -- 172.21.15.102:0/1642193705 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b015b740 con 0x7f56b015f0c0 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.119+0000 7f56afe00640 1 --2- 172.21.15.102:0/1642193705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b015f0c0 0x7f56b00c9ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f56b00ca720 tx=0x7f56a802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774a76e996619e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.119+0000 7f56ae000640 1 -- 172.21.15.102:0/1642193705 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8039070 con 0x7f56b015f0c0 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.119+0000 7f56ae000640 1 -- 172.21.15.102:0/1642193705 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a802cd80 con 0x7f56b015f0c0 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.119+0000 7f56ae000640 1 -- 172.21.15.102:0/1642193705 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8034640 con 0x7f56b015f0c0 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.120+0000 7f56b6800640 1 -- 172.21.15.102:0/1642193705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b015f0c0 msgr2=0x7f56b00c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.120+0000 7f56b6800640 1 --2- 172.21.15.102:0/1642193705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b015f0c0 0x7f56b00c9ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f56b00ca720 tx=0x7f56a802bfd0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.120+0000 7f56b6800640 1 -- 172.21.15.102:0/1642193705 shutdown_connections 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.120+0000 7f56b6800640 1 --2- 172.21.15.102:0/1642193705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b015f0c0 0x7f56b00c9ef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.120+0000 7f56b6800640 1 -- 172.21.15.102:0/1642193705 >> 172.21.15.102:0/1642193705 conn(0x7f56b006dd40 msgr2=0x7f56b0074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.120+0000 7f56b6800640 1 -- 172.21.15.102:0/1642193705 shutdown_connections 2024-04-20T21:16:51.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.120+0000 7f56b6800640 1 -- 172.21.15.102:0/1642193705 wait complete. 2024-04-20T21:16:51.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.122+0000 7f56b6800640 1 Processor -- start 2024-04-20T21:16:51.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.122+0000 7f56b6800640 1 -- start start 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56b6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56b00716d0 0x7f56b0142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56b6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b00c7350 0x7f56b00cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56b6800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56b015f0c0 0x7f56b00ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56b6800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56b015bfe0 con 0x7f56b00c7350 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56b6800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56b015bd00 con 0x7f56b00716d0 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56b6800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f56b006c050 con 0x7f56b015f0c0 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56aea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b00c7350 0x7f56b00cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56af400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56b00716d0 0x7f56b0142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56afe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56b015f0c0 0x7f56b00ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56aea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b00c7350 0x7f56b00cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57676/0 (socket says 172.21.15.102:57676) 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56af400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56b00716d0 0x7f56b0142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41522/0 (socket says 172.21.15.102:41522) 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56aea00640 1 -- 172.21.15.102:0/3903212648 learned_addr learned my addr 172.21.15.102:0/3903212648 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56afe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56b015f0c0 0x7f56b00ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56284/0 (socket says 172.21.15.102:56284) 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56afe00640 1 -- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56b00716d0 msgr2=0x7f56b0142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56afe00640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56b00716d0 0x7f56b0142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56afe00640 1 -- 172.21.15.102:0/3903212648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b00c7350 msgr2=0x7f56b00cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.123+0000 7f56afe00640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b00c7350 0x7f56b00cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.124+0000 7f56afe00640 1 -- 172.21.15.102:0/3903212648 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b00c5e70 con 0x7f56b015f0c0 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.124+0000 7f56aea00640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b00c7350 0x7f56b00cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.124+0000 7f56afe00640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56b015f0c0 0x7f56b00ccb10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56b00c7920 tx=0x7f56a8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.124+0000 7f56a7400640 1 -- 172.21.15.102:0/3903212648 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8042020 con 0x7f56b015f0c0 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.124+0000 7f56a7400640 1 -- 172.21.15.102:0/3903212648 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a803a040 con 0x7f56b015f0c0 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.124+0000 7f56a7400640 1 -- 172.21.15.102:0/3903212648 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8039070 con 0x7f56b015f0c0 2024-04-20T21:16:51.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.124+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b01f4b80 con 0x7f56b015f0c0 2024-04-20T21:16:51.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.124+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b00c6410 con 0x7f56b015f0c0 2024-04-20T21:16:51.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.125+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b0074340 con 0x7f56b015f0c0 2024-04-20T21:16:51.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.126+0000 7f56a7400640 1 -- 172.21.15.102:0/3903212648 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56a8045020 con 0x7f56b015f0c0 2024-04-20T21:16:51.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.127+0000 7f56a7400640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f569006c8f0 0x7f569006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.127+0000 7f56a7400640 1 -- 172.21.15.102:0/3903212648 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f56a80adad0 con 0x7f56b015f0c0 2024-04-20T21:16:51.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.128+0000 7f56af400640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f569006c8f0 0x7f569006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.128+0000 7f56af400640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f569006c8f0 0x7f569006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56a0007fd0 tx=0x7f56a0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:51.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.131+0000 7f56a7400640 1 -- 172.21.15.102:0/3903212648 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f56a804c030 con 0x7f56b015f0c0 2024-04-20T21:16:51.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.331+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f56b00ca9b0 con 0x7f56b015f0c0 2024-04-20T21:16:51.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.331+0000 7f56a7400640 1 -- 172.21.15.102:0/3903212648 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f56a807bd00 con 0x7f56b015f0c0 2024-04-20T21:16:51.331 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:16:51.331 INFO:teuthology.orchestra.run.smithi102.stdout:[ 2024-04-20T21:16:51.331 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-20T21:16:51.331 INFO:teuthology.orchestra.run.smithi102.stdout: "poolnum": 1, 2024-04-20T21:16:51.332 INFO:teuthology.orchestra.run.smithi102.stdout: "poolname": ".mgr" 2024-04-20T21:16:51.332 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-20T21:16:51.332 INFO:teuthology.orchestra.run.smithi102.stdout:] 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f569006c8f0 msgr2=0x7f569006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f569006c8f0 0x7f569006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56a0007fd0 tx=0x7f56a0009040 comp rx=0 tx=0).stop 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56b015f0c0 msgr2=0x7f56b00ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56b015f0c0 0x7f56b00ccb10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56b00c7920 tx=0x7f56a8004a40 comp rx=0 tx=0).stop 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 shutdown_connections 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56b015f0c0 0x7f56b00ccb10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56b00c7350 0x7f56b00cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f569006c8f0 0x7f569006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 --2- 172.21.15.102:0/3903212648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56b00716d0 0x7f56b0142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 >> 172.21.15.102:0/3903212648 conn(0x7f56b006dd40 msgr2=0x7f56b0074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:51.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 shutdown_connections 2024-04-20T21:16:51.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.335+0000 7f56b6800640 1 -- 172.21.15.102:0/3903212648 wait complete. 2024-04-20T21:16:51.354 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:16:51.354 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:16:51.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.563+0000 7f4b56200640 1 Processor -- start 2024-04-20T21:16:51.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.563+0000 7f4b56200640 1 -- start start 2024-04-20T21:16:51.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b56200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50153200 0x7f4b500c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b56200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4b5006f100 0x7f4b500c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b56200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4b50073390 0x7f4b500737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b56200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f4b50073d00 con 0x7f4b5006f100 2024-04-20T21:16:51.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b56200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f4b500c7fe0 con 0x7f4b50073390 2024-04-20T21:16:51.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b56200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4b50073f40 con 0x7f4b50153200 2024-04-20T21:16:51.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b4ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4b5006f100 0x7f4b500c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b4fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4b50073390 0x7f4b500737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b4f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50153200 0x7f4b500c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b4fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4b50073390 0x7f4b500737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41536/0 (socket says 172.21.15.102:41536) 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.565+0000 7f4b4fe00640 1 -- 172.21.15.102:0/346703864 learned_addr learned my addr 172.21.15.102:0/346703864 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.566+0000 7f4b4fe00640 1 -- 172.21.15.102:0/346703864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50153200 msgr2=0x7f4b500c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.566+0000 7f4b4fe00640 1 --2- 172.21.15.102:0/346703864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50153200 0x7f4b500c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.566+0000 7f4b4fe00640 1 -- 172.21.15.102:0/346703864 >> v2:172.21.15.102:3300/0 conn(0x7f4b5006f100 msgr2=0x7f4b500c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.566+0000 7f4b4fe00640 1 --2- 172.21.15.102:0/346703864 >> v2:172.21.15.102:3300/0 conn(0x7f4b5006f100 0x7f4b500c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.566+0000 7f4b4fe00640 1 -- 172.21.15.102:0/346703864 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b500c73d0 con 0x7f4b50073390 2024-04-20T21:16:51.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.566+0000 7f4b4fe00640 1 --2- 172.21.15.102:0/346703864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50073390 0x7f4b500737c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b5006dee0 tx=0x7f4b4802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f73925bdb4f1d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:51.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.567+0000 7f4b4e000640 1 -- 172.21.15.102:0/346703864 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4802cb00 con 0x7f4b50073390 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.567+0000 7f4b4e000640 1 -- 172.21.15.102:0/346703864 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4802cca0 con 0x7f4b50073390 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.567+0000 7f4b4e000640 1 -- 172.21.15.102:0/346703864 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48034610 con 0x7f4b50073390 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 -- 172.21.15.102:0/346703864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50073390 msgr2=0x7f4b500737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 --2- 172.21.15.102:0/346703864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50073390 0x7f4b500737c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b5006dee0 tx=0x7f4b4802bf60 comp rx=0 tx=0).stop 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 -- 172.21.15.102:0/346703864 shutdown_connections 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 --2- 172.21.15.102:0/346703864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50073390 0x7f4b500737c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 --2- 172.21.15.102:0/346703864 >> v2:172.21.15.102:3300/0 conn(0x7f4b5006f100 0x7f4b500c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 --2- 172.21.15.102:0/346703864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50153200 0x7f4b500c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 -- 172.21.15.102:0/346703864 >> 172.21.15.102:0/346703864 conn(0x7f4b50074ea0 msgr2=0x7f4b50075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 -- 172.21.15.102:0/346703864 shutdown_connections 2024-04-20T21:16:51.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.568+0000 7f4b56200640 1 -- 172.21.15.102:0/346703864 wait complete. 2024-04-20T21:16:51.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.569+0000 7f4b56200640 1 Processor -- start 2024-04-20T21:16:51.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.570+0000 7f4b56200640 1 -- start start 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.570+0000 7f4b56200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b5006f100 0x7f4b500caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.570+0000 7f4b56200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50073390 0x7f4b50145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.570+0000 7f4b56200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50153200 0x7f4b50145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.570+0000 7f4b56200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500c7c70 con 0x7f4b5006f100 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.570+0000 7f4b56200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500c7990 con 0x7f4b50153200 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.570+0000 7f4b56200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4b501535e0 con 0x7f4b50073390 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b5006f100 0x7f4b500caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50153200 0x7f4b50145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50073390 0x7f4b50145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b5006f100 0x7f4b500caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57688/0 (socket says 172.21.15.102:57688) 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50153200 0x7f4b50145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41538/0 (socket says 172.21.15.102:41538) 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4f400640 1 -- 172.21.15.102:0/757572826 learned_addr learned my addr 172.21.15.102:0/757572826 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:51.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50073390 0x7f4b50145830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56302/0 (socket says 172.21.15.102:56302) 2024-04-20T21:16:51.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4fe00640 1 -- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50073390 msgr2=0x7f4b50145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4fe00640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50073390 0x7f4b50145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4fe00640 1 -- 172.21.15.102:0/757572826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b5006f100 msgr2=0x7f4b500caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:51.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4fe00640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b5006f100 0x7f4b500caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:51.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4fe00640 1 -- 172.21.15.102:0/757572826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b501453f0 con 0x7f4b50153200 2024-04-20T21:16:51.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4f400640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b5006f100 0x7f4b500caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:51.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.571+0000 7f4b4fe00640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50153200 0x7f4b50145d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b48032000 tx=0x7f4b48004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:51.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.572+0000 7f4b47400640 1 -- 172.21.15.102:0/757572826 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48043020 con 0x7f4b50153200 2024-04-20T21:16:51.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.572+0000 7f4b47400640 1 -- 172.21.15.102:0/757572826 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4802ce50 con 0x7f4b50153200 2024-04-20T21:16:51.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.572+0000 7f4b47400640 1 -- 172.21.15.102:0/757572826 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48009510 con 0x7f4b50153200 2024-04-20T21:16:51.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.572+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b500cb410 con 0x7f4b50153200 2024-04-20T21:16:51.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.572+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501482f0 con 0x7f4b50153200 2024-04-20T21:16:51.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.573+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b50148120 con 0x7f4b50153200 2024-04-20T21:16:51.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.574+0000 7f4b47400640 1 -- 172.21.15.102:0/757572826 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b48046020 con 0x7f4b50153200 2024-04-20T21:16:51.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.575+0000 7f4b47400640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4b3006c8f0 0x7f4b3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:51.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.575+0000 7f4b47400640 1 -- 172.21.15.102:0/757572826 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4b480adbd0 con 0x7f4b50153200 2024-04-20T21:16:51.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.575+0000 7f4b4f400640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4b3006c8f0 0x7f4b3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:51.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.575+0000 7f4b4f400640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4b3006c8f0 0x7f4b3006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b400135a0 tx=0x7f4b4000f040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.580+0000 7f4b47400640 1 -- 172.21.15.102:0/757572826 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4b4804d030 con 0x7f4b50153200 2024-04-20T21:16:51.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:51.778+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b50074340 con 0x7f4b50153200 2024-04-20T21:16:52.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.522+0000 7f4b47400640 1 -- 172.21.15.102:0/757572826 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f4b48082070 con 0x7f4b50153200 2024-04-20T21:16:52.523 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' created 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.528+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4b3006c8f0 msgr2=0x7f4b3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.528+0000 7f4b56200640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4b3006c8f0 0x7f4b3006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b400135a0 tx=0x7f4b4000f040 comp rx=0 tx=0).stop 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.528+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50153200 msgr2=0x7f4b50145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.528+0000 7f4b56200640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50153200 0x7f4b50145d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b48032000 tx=0x7f4b48004a40 comp rx=0 tx=0).stop 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.529+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 shutdown_connections 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.529+0000 7f4b56200640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4b3006c8f0 0x7f4b3006edb0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b400135a0 tx=0x7f4b4000f040 comp rx=0 tx=0).stop 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.529+0000 7f4b56200640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b50153200 0x7f4b50145d70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.529+0000 7f4b56200640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b50073390 0x7f4b50145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.529+0000 7f4b56200640 1 --2- 172.21.15.102:0/757572826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b5006f100 0x7f4b500caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:52.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.529+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 >> 172.21.15.102:0/757572826 conn(0x7f4b50074ea0 msgr2=0x7f4b50146850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:52.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.530+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 shutdown_connections 2024-04-20T21:16:52.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.530+0000 7f4b56200640 1 -- 172.21.15.102:0/757572826 wait complete. 2024-04-20T21:16:52.554 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-20T21:16:52.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.749+0000 7f132ae00640 1 Processor -- start 2024-04-20T21:16:52.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.749+0000 7f132ae00640 1 -- start start 2024-04-20T21:16:52.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.750+0000 7f132ae00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f13241555d0 0x7f132415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:52.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f132ae00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f13240cea10 0x7f13240cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:52.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f132ae00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1324073ac0 con 0x7f13240ceee0 2024-04-20T21:16:52.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f132ae00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f132415eaf0 con 0x7f13240cea10 2024-04-20T21:16:52.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f132ae00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f132415edd0 con 0x7f13241555d0 2024-04-20T21:16:52.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1323400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f13240ceee0 0x7f13240d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49668/0 (socket says 172.21.15.102:49668) 2024-04-20T21:16:52.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1323e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f13241555d0 0x7f132415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:52.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1323400640 1 -- 172.21.15.102:0/1703652031 learned_addr learned my addr 172.21.15.102:0/1703652031 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:52.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1328e00640 1 --2- 172.21.15.102:0/1703652031 >> v2:172.21.15.102:3300/0 conn(0x7f13240cea10 0x7f13240cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:52.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1322a00640 1 -- 172.21.15.102:0/1703652031 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 876468879 0 0) 0x7f1324073ac0 con 0x7f13240ceee0 2024-04-20T21:16:52.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1322a00640 1 -- 172.21.15.102:0/1703652031 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1308003520 con 0x7f13240ceee0 2024-04-20T21:16:52.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1323e00640 1 -- 172.21.15.102:0/1703652031 >> v2:172.21.15.102:3300/0 conn(0x7f13240cea10 msgr2=0x7f13240cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:52.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1323e00640 1 --2- 172.21.15.102:0/1703652031 >> v2:172.21.15.102:3300/0 conn(0x7f13240cea10 0x7f13240cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:52.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1323e00640 1 -- 172.21.15.102:0/1703652031 >> v1:172.21.15.193:6789/0 conn(0x7f13240ceee0 legacy=0x7f13240d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:52.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.751+0000 7f1323e00640 1 -- 172.21.15.102:0/1703652031 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132415f0b0 con 0x7f13241555d0 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f1323e00640 1 --2- 172.21.15.102:0/1703652031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 0x7f132415b380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f13240731e0 tx=0x7f131402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9bbe11614aa4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f1322a00640 1 -- 172.21.15.102:0/1703652031 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314034070 con 0x7f13241555d0 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f1322a00640 1 -- 172.21.15.102:0/1703652031 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1314030610 con 0x7f13241555d0 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f1322a00640 1 -- 172.21.15.102:0/1703652031 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13140309e0 con 0x7f13241555d0 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f132ae00640 1 -- 172.21.15.102:0/1703652031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 msgr2=0x7f132415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f132ae00640 1 --2- 172.21.15.102:0/1703652031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 0x7f132415b380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f13240731e0 tx=0x7f131402b960 comp rx=0 tx=0).stop 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f132ae00640 1 -- 172.21.15.102:0/1703652031 shutdown_connections 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f132ae00640 1 --2- 172.21.15.102:0/1703652031 >> v2:172.21.15.102:3300/0 conn(0x7f13240cea10 0x7f13240cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f132ae00640 1 --2- 172.21.15.102:0/1703652031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 0x7f132415b380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.753+0000 7f132ae00640 1 -- 172.21.15.102:0/1703652031 >> 172.21.15.102:0/1703652031 conn(0x7f1324075080 msgr2=0x7f1324075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.754+0000 7f132ae00640 1 -- 172.21.15.102:0/1703652031 shutdown_connections 2024-04-20T21:16:52.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.754+0000 7f132ae00640 1 -- 172.21.15.102:0/1703652031 wait complete. 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.755+0000 7f132ae00640 1 Processor -- start 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.755+0000 7f132ae00640 1 -- start start 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.756+0000 7f132ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13240cea10 0x7f132415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.756+0000 7f132ae00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13240ceee0 0x7f1324154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.756+0000 7f132ae00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 0x7f13240b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.756+0000 7f132ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13241559b0 con 0x7f13240cea10 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.756+0000 7f132ae00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f132415f670 con 0x7f13241555d0 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.756+0000 7f132ae00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f13240d1650 con 0x7f13240ceee0 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13240cea10 0x7f132415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13240cea10 0x7f132415e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57706/0 (socket says 172.21.15.102:57706) 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 -- 172.21.15.102:0/2702277035 learned_addr learned my addr 172.21.15.102:0/2702277035 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323400640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13240ceee0 0x7f1324154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1328e00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 0x7f13240b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 -- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13240ceee0 msgr2=0x7f1324154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:52.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13240ceee0 0x7f1324154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 -- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 msgr2=0x7f13240b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 0x7f13240b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 -- 172.21.15.102:0/2702277035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1324154d80 con 0x7f13240cea10 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.757+0000 7f1323e00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13240cea10 0x7f132415e320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f131402bf70 tx=0x7f13140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.758+0000 7f131be00640 1 -- 172.21.15.102:0/2702277035 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314034040 con 0x7f13240cea10 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.758+0000 7f131be00640 1 -- 172.21.15.102:0/2702277035 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f131403aaa0 con 0x7f13240cea10 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.758+0000 7f131be00640 1 -- 172.21.15.102:0/2702277035 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314006e10 con 0x7f13240cea10 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.758+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1324150b90 con 0x7f13240cea10 2024-04-20T21:16:52.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.758+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1324151150 con 0x7f13240cea10 2024-04-20T21:16:52.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.759+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1324074340 con 0x7f13240cea10 2024-04-20T21:16:52.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.760+0000 7f131be00640 1 -- 172.21.15.102:0/2702277035 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1314037070 con 0x7f13240cea10 2024-04-20T21:16:52.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.761+0000 7f131be00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f130406c8f0 0x7f130406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:52.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.761+0000 7f131be00640 1 -- 172.21.15.102:0/2702277035 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f13140ad210 con 0x7f13240cea10 2024-04-20T21:16:52.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.761+0000 7f1323400640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f130406c8f0 0x7f130406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:52.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.762+0000 7f1323400640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f130406c8f0 0x7f130406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f132415e860 tx=0x7f1310009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:52.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.766+0000 7f131be00640 1 -- 172.21.15.102:0/2702277035 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f131404a060 con 0x7f13240cea10 2024-04-20T21:16:52.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:52.966+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f13240b6110 con 0x7f13240cea10 2024-04-20T21:16:53.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.532+0000 7f131be00640 1 -- 172.21.15.102:0/2702277035 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f131403f070 con 0x7f13240cea10 2024-04-20T21:16:53.532 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' created 2024-04-20T21:16:53.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f130406c8f0 msgr2=0x7f130406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:53.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f130406c8f0 0x7f130406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f132415e860 tx=0x7f1310009040 comp rx=0 tx=0).stop 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13240cea10 msgr2=0x7f132415e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13240cea10 0x7f132415e320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f131402bf70 tx=0x7f13140046c0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 shutdown_connections 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f130406c8f0 0x7f130406edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13241555d0 0x7f13240b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13240ceee0 0x7f1324154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 --2- 172.21.15.102:0/2702277035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13240cea10 0x7f132415e320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.537+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 >> 172.21.15.102:0/2702277035 conn(0x7f1324075080 msgr2=0x7f13240cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.538+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 shutdown_connections 2024-04-20T21:16:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.539+0000 7f132ae00640 1 -- 172.21.15.102:0/2702277035 wait complete. 2024-04-20T21:16:53.580 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:16:53.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.753+0000 7f5fc5000640 1 Processor -- start 2024-04-20T21:16:53.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.754+0000 7f5fc5000640 1 -- start start 2024-04-20T21:16:53.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fc5000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006f0f0 0x7f5fc015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fc5000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5fc0152250 0x7f5fc00c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fc5000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc006e370 con 0x7f5fc0151e70 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fc5000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc00c4a50 con 0x7f5fc0152250 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fc5000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc00c4d30 con 0x7f5fc006f0f0 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fbf400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5fc0151e70 0x7f5fc00c5010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47318/0 (socket says 172.21.15.102:47318) 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fbf400640 1 -- 172.21.15.102:0/975490896 learned_addr learned my addr 172.21.15.102:0/975490896 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fb7e00640 1 --2- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006f0f0 0x7f5fc015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.755+0000 7f5fbfe00640 1 --2- 172.21.15.102:0/975490896 >> v2:172.21.15.193:3300/0 conn(0x7f5fc0152250 0x7f5fc00c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.756+0000 7f5fbea00640 1 -- 172.21.15.102:0/975490896 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203117196 0 0) 0x7f5fc006e370 con 0x7f5fc0151e70 2024-04-20T21:16:53.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.756+0000 7f5fbfe00640 1 -- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006f0f0 msgr2=0x7f5fc015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:53.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.756+0000 7f5fbfe00640 1 --2- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006f0f0 0x7f5fc015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.756+0000 7f5fbfe00640 1 -- 172.21.15.102:0/975490896 >> v1:172.21.15.102:6789/0 conn(0x7f5fc0151e70 legacy=0x7f5fc00c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:53.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.756+0000 7f5fbfe00640 1 -- 172.21.15.102:0/975490896 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc006e610 con 0x7f5fc0152250 2024-04-20T21:16:53.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.756+0000 7f5fb7e00640 1 --2- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006f0f0 0x7f5fc015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:53.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.757+0000 7f5fbfe00640 1 --2- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc0152250 0x7f5fc00c8130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009d90 tx=0x7f5fb802e0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9dd39782e5eb64c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:53.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.757+0000 7f5fbea00640 1 -- 172.21.15.102:0/975490896 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb803b070 con 0x7f5fc0152250 2024-04-20T21:16:53.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.757+0000 7f5fbea00640 1 -- 172.21.15.102:0/975490896 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb8036440 con 0x7f5fc0152250 2024-04-20T21:16:53.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.757+0000 7f5fbea00640 1 -- 172.21.15.102:0/975490896 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8036810 con 0x7f5fc0152250 2024-04-20T21:16:53.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.758+0000 7f5fc5000640 1 -- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc0152250 msgr2=0x7f5fc00c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:53.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.758+0000 7f5fc5000640 1 --2- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc0152250 0x7f5fc00c8130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009d90 tx=0x7f5fb802e0c0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.758+0000 7f5fc5000640 1 -- 172.21.15.102:0/975490896 shutdown_connections 2024-04-20T21:16:53.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.758+0000 7f5fc5000640 1 --2- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc0152250 0x7f5fc00c8130 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.758+0000 7f5fc5000640 1 --2- 172.21.15.102:0/975490896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006f0f0 0x7f5fc015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.758+0000 7f5fc5000640 1 -- 172.21.15.102:0/975490896 >> 172.21.15.102:0/975490896 conn(0x7f5fc00cc620 msgr2=0x7f5fc00cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:53.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.758+0000 7f5fc5000640 1 -- 172.21.15.102:0/975490896 shutdown_connections 2024-04-20T21:16:53.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.758+0000 7f5fc5000640 1 -- 172.21.15.102:0/975490896 wait complete. 2024-04-20T21:16:53.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.760+0000 7f5fc5000640 1 Processor -- start 2024-04-20T21:16:53.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.760+0000 7f5fc5000640 1 -- start start 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fc5000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc006f0f0 0x7f5fc014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fc5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0151e70 0x7f5fc014c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fc5000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc0152250 0x7f5fc0145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fc5000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc00cd370 con 0x7f5fc0151e70 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fc5000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc006ebd0 con 0x7f5fc006f0f0 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fc5000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc00cd4f0 con 0x7f5fc0152250 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fb7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0151e70 0x7f5fc014c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc006f0f0 0x7f5fc014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc0152250 0x7f5fc0145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc0152250 0x7f5fc0145190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56320/0 (socket says 172.21.15.102:56320) 2024-04-20T21:16:53.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc006f0f0 0x7f5fc014dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41566/0 (socket says 172.21.15.102:41566) 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbfe00640 1 -- 172.21.15.102:0/3180776319 learned_addr learned my addr 172.21.15.102:0/3180776319 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbfe00640 1 -- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc006f0f0 msgr2=0x7f5fc014dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbfe00640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc006f0f0 0x7f5fc014dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbfe00640 1 -- 172.21.15.102:0/3180776319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0151e70 msgr2=0x7f5fc014c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbfe00640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0151e70 0x7f5fc014c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.761+0000 7f5fbfe00640 1 -- 172.21.15.102:0/3180776319 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc014e110 con 0x7f5fc0152250 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.762+0000 7f5fbfe00640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc0152250 0x7f5fc0145190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009d90 tx=0x7f5fb8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.762+0000 7f5fb7400640 1 -- 172.21.15.102:0/3180776319 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8046020 con 0x7f5fc0152250 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.762+0000 7f5fb7400640 1 -- 172.21.15.102:0/3180776319 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb803c040 con 0x7f5fc0152250 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.762+0000 7f5fb7400640 1 -- 172.21.15.102:0/3180776319 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb803b050 con 0x7f5fc0152250 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.762+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc01592d0 con 0x7f5fc0152250 2024-04-20T21:16:53.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.762+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc0159890 con 0x7f5fc0152250 2024-04-20T21:16:53.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.763+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc00c88d0 con 0x7f5fc0152250 2024-04-20T21:16:53.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.764+0000 7f5fb7400640 1 -- 172.21.15.102:0/3180776319 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fb8004030 con 0x7f5fc0152250 2024-04-20T21:16:53.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.765+0000 7f5fb7400640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f9c06c920 0x7f5f9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:53.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.765+0000 7f5fb7400640 1 -- 172.21.15.102:0/3180776319 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f5fb80b0350 con 0x7f5fc0152250 2024-04-20T21:16:53.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.765+0000 7f5fbf400640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f9c06c920 0x7f5f9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:53.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.766+0000 7f5fbf400640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f9c06c920 0x7f5f9c06ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0000f60 tx=0x7f5fb0007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:53.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:53.769+0000 7f5fb7400640 1 -- 172.21.15.102:0/3180776319 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5fc00c88d0 con 0x7f5fc0152250 2024-04-20T21:16:54.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.014+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5fc004fcf0 con 0x7f5fc0152250 2024-04-20T21:16:54.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.559+0000 7f5fb7400640 1 -- 172.21.15.102:0/3180776319 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f5fb8050060 con 0x7f5fc0152250 2024-04-20T21:16:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T21:16:54.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.570+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f9c06c920 msgr2=0x7f5f9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:54.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.570+0000 7f5fc5000640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f9c06c920 0x7f5f9c06ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0000f60 tx=0x7f5fb0007d50 comp rx=0 tx=0).stop 2024-04-20T21:16:54.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.570+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc0152250 msgr2=0x7f5fc0145190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:54.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.570+0000 7f5fc5000640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc0152250 0x7f5fc0145190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009d90 tx=0x7f5fb8004a40 comp rx=0 tx=0).stop 2024-04-20T21:16:54.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.571+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 shutdown_connections 2024-04-20T21:16:54.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.571+0000 7f5fc5000640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc0152250 0x7f5fc0145190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:54.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.571+0000 7f5fc5000640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0151e70 0x7f5fc014c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:54.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.571+0000 7f5fc5000640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f9c06c920 0x7f5f9c06ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:54.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.571+0000 7f5fc5000640 1 --2- 172.21.15.102:0/3180776319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc006f0f0 0x7f5fc014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:54.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.571+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 >> 172.21.15.102:0/3180776319 conn(0x7f5fc00cc620 msgr2=0x7f5fc00c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:54.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.571+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 shutdown_connections 2024-04-20T21:16:54.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.571+0000 7f5fc5000640 1 -- 172.21.15.102:0/3180776319 wait complete. 2024-04-20T21:16:54.596 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-20T21:16:54.596 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-20T21:16:54.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.773+0000 7f31f7c00640 1 Processor -- start 2024-04-20T21:16:54.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.773+0000 7f31f7c00640 1 -- start start 2024-04-20T21:16:54.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.775+0000 7f31f7c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f31f00c5ec0 0x7f31f0166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:54.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.775+0000 7f31f7c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f015f980 con 0x7f31f0155500 2024-04-20T21:16:54.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.775+0000 7f31f7c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f01558e0 con 0x7f31f00c5a80 2024-04-20T21:16:54.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.775+0000 7f31f7c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f31f015f4f0 con 0x7f31f00c5ec0 2024-04-20T21:16:54.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.775+0000 7f31f5200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f31f0155500 0x7f31f015beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47326/0 (socket says 172.21.15.102:47326) 2024-04-20T21:16:54.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.775+0000 7f31f5200640 1 -- 172.21.15.102:0/62870979 learned_addr learned my addr 172.21.15.102:0/62870979 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:54.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.775+0000 7f31f5c00640 1 --2- 172.21.15.102:0/62870979 >> v2:172.21.15.193:3300/0 conn(0x7f31f00c5ec0 0x7f31f0166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:54.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.776+0000 7f31ef400640 1 -- 172.21.15.102:0/62870979 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1493097890 0 0) 0x7f31f015f980 con 0x7f31f0155500 2024-04-20T21:16:54.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.776+0000 7f31ef400640 1 -- 172.21.15.102:0/62870979 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d4003520 con 0x7f31f0155500 2024-04-20T21:16:54.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.776+0000 7f31ef400640 1 -- 172.21.15.102:0/62870979 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1045207777 0 0) 0x7f31f01558e0 con 0x7f31f00c5a80 2024-04-20T21:16:54.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.776+0000 7f31f5c00640 1 -- 172.21.15.102:0/62870979 >> v1:172.21.15.193:6789/0 conn(0x7f31f00c5a80 legacy=0x7f31f00d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:54.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.776+0000 7f31f5c00640 1 -- 172.21.15.102:0/62870979 >> v1:172.21.15.102:6789/0 conn(0x7f31f0155500 legacy=0x7f31f015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:54.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.776+0000 7f31f5c00640 1 -- 172.21.15.102:0/62870979 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f00cbf00 con 0x7f31f00c5ec0 2024-04-20T21:16:54.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.777+0000 7f31f5c00640 1 --2- 172.21.15.102:0/62870979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f00c5ec0 0x7f31f0166dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f31e80097a0 tx=0x7f31e802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d922616d5d3357 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.777+0000 7f31ef400640 1 -- 172.21.15.102:0/62870979 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e80315a0 con 0x7f31f00c5ec0 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.777+0000 7f31ef400640 1 -- 172.21.15.102:0/62870979 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e8037070 con 0x7f31f00c5ec0 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.777+0000 7f31ef400640 1 -- 172.21.15.102:0/62870979 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8031960 con 0x7f31f00c5ec0 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.778+0000 7f31f7c00640 1 -- 172.21.15.102:0/62870979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f00c5ec0 msgr2=0x7f31f0166dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.778+0000 7f31f7c00640 1 --2- 172.21.15.102:0/62870979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f00c5ec0 0x7f31f0166dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f31e80097a0 tx=0x7f31e802b840 comp rx=0 tx=0).stop 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.778+0000 7f31f7c00640 1 -- 172.21.15.102:0/62870979 shutdown_connections 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.778+0000 7f31f7c00640 1 --2- 172.21.15.102:0/62870979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f00c5ec0 0x7f31f0166dc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.778+0000 7f31f7c00640 1 -- 172.21.15.102:0/62870979 >> 172.21.15.102:0/62870979 conn(0x7f31f0075080 msgr2=0x7f31f0075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.778+0000 7f31f7c00640 1 -- 172.21.15.102:0/62870979 shutdown_connections 2024-04-20T21:16:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.778+0000 7f31f7c00640 1 -- 172.21.15.102:0/62870979 wait complete. 2024-04-20T21:16:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.780+0000 7f31f7c00640 1 Processor -- start 2024-04-20T21:16:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.780+0000 7f31f7c00640 1 -- start start 2024-04-20T21:16:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31f7c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00c5a80 0x7f31f01588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31f7c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f00c5ec0 0x7f31f014e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31f7c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f0155500 0x7f31f014ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31f7c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31f015f180 con 0x7f31f00c5ec0 2024-04-20T21:16:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31f7c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31f015eea0 con 0x7f31f0155500 2024-04-20T21:16:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31f7c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f31f00b4870 con 0x7f31f00c5a80 2024-04-20T21:16:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f00c5ec0 0x7f31f014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f00c5ec0 0x7f31f014e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57722/0 (socket says 172.21.15.102:57722) 2024-04-20T21:16:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31efe00640 1 -- 172.21.15.102:0/3411077729 learned_addr learned my addr 172.21.15.102:0/3411077729 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31f5c00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f0155500 0x7f31f014ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.781+0000 7f31f5200640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00c5a80 0x7f31f01588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31efe00640 1 -- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00c5a80 msgr2=0x7f31f01588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31efe00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00c5a80 0x7f31f01588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31efe00640 1 -- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f0155500 msgr2=0x7f31f014ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31efe00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f0155500 0x7f31f014ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31efe00640 1 -- 172.21.15.102:0/3411077729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f00b3890 con 0x7f31f00c5ec0 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31efe00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f00c5ec0 0x7f31f014e730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c4b0 tx=0x7f31d800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31ecc00640 1 -- 172.21.15.102:0/3411077729 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8017070 con 0x7f31f00c5ec0 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31ecc00640 1 -- 172.21.15.102:0/3411077729 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d8005480 con 0x7f31f00c5ec0 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31ecc00640 1 -- 172.21.15.102:0/3411077729 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8003830 con 0x7f31f00c5ec0 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.782+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f0158140 con 0x7f31f00c5ec0 2024-04-20T21:16:54.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.783+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f01e6ec0 con 0x7f31f00c5ec0 2024-04-20T21:16:54.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.784+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f00cc1e0 con 0x7f31f00c5ec0 2024-04-20T21:16:54.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.785+0000 7f31ecc00640 1 -- 172.21.15.102:0/3411077729 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31d801a070 con 0x7f31f00c5ec0 2024-04-20T21:16:54.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.786+0000 7f31ecc00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31cc06c8f0 0x7f31cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:54.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.786+0000 7f31ecc00640 1 -- 172.21.15.102:0/3411077729 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f31d808aeb0 con 0x7f31f00c5ec0 2024-04-20T21:16:54.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.786+0000 7f31f5200640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31cc06c8f0 0x7f31cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:54.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.787+0000 7f31f5200640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31cc06c8f0 0x7f31cc06edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f31e0005c00 tx=0x7f31e0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:54.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.792+0000 7f31ecc00640 1 -- 172.21.15.102:0/3411077729 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f31f00cc1e0 con 0x7f31f00c5ec0 2024-04-20T21:16:54.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:54.992+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f31f0075890 con 0x7f31f00c5ec0 2024-04-20T21:16:55.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.545+0000 7f31ecc00640 1 -- 172.21.15.102:0/3411077729 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f31f0075890 con 0x7f31f00c5ec0 2024-04-20T21:16:55.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.549+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31cc06c8f0 msgr2=0x7f31cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:55.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.549+0000 7f31f7c00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31cc06c8f0 0x7f31cc06edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f31e0005c00 tx=0x7f31e0009040 comp rx=0 tx=0).stop 2024-04-20T21:16:55.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.549+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f00c5ec0 msgr2=0x7f31f014e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:55.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.549+0000 7f31f7c00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f00c5ec0 0x7f31f014e730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c4b0 tx=0x7f31d800c970 comp rx=0 tx=0).stop 2024-04-20T21:16:55.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.550+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 shutdown_connections 2024-04-20T21:16:55.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.550+0000 7f31f7c00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31cc06c8f0 0x7f31cc06edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:55.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.550+0000 7f31f7c00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31f0155500 0x7f31f014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:55.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.550+0000 7f31f7c00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f00c5ec0 0x7f31f014e730 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c4b0 tx=0x7f31d800c970 comp rx=0 tx=0).stop 2024-04-20T21:16:55.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.550+0000 7f31f7c00640 1 --2- 172.21.15.102:0/3411077729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00c5a80 0x7f31f01588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:55.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.550+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 >> 172.21.15.102:0/3411077729 conn(0x7f31f0075080 msgr2=0x7f31f0072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:55.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.551+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 shutdown_connections 2024-04-20T21:16:55.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.551+0000 7f31f7c00640 1 -- 172.21.15.102:0/3411077729 wait complete. 2024-04-20T21:16:55.572 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:16:55.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.770+0000 7ff006e00640 1 Processor -- start 2024-04-20T21:16:55.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.770+0000 7ff006e00640 1 -- start start 2024-04-20T21:16:55.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.772+0000 7ff006e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff000162980 0x7ff0000c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:55.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.772+0000 7ff006e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 0x7ff000169d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:55.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.772+0000 7ff006e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff00006e5e0 con 0x7ff000162d60 2024-04-20T21:16:55.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.772+0000 7ff006e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff00006f1f0 con 0x7ff000162980 2024-04-20T21:16:55.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.772+0000 7ff006e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff00006e8c0 con 0x7ff000160880 2024-04-20T21:16:55.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.772+0000 7ff005000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff000162d60 0x7ff0000c7a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47330/0 (socket says 172.21.15.102:47330) 2024-04-20T21:16:55.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.772+0000 7ff005000640 1 -- 172.21.15.102:0/1336555810 learned_addr learned my addr 172.21.15.102:0/1336555810 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:55.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.772+0000 7ff005a00640 1 --2- 172.21.15.102:0/1336555810 >> v2:172.21.15.102:3300/0 conn(0x7ff000162980 0x7ff0000c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:55.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.773+0000 7ff006400640 1 --2- 172.21.15.102:0/1336555810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 0x7ff000169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:55.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.773+0000 7fefffe00640 1 -- 172.21.15.102:0/1336555810 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 765331078 0 0) 0x7ff00006e5e0 con 0x7ff000162d60 2024-04-20T21:16:55.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.773+0000 7ff005a00640 1 -- 172.21.15.102:0/1336555810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 msgr2=0x7ff000169d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:55.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.773+0000 7ff005a00640 1 --2- 172.21.15.102:0/1336555810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 0x7ff000169d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:55.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.773+0000 7ff005a00640 1 -- 172.21.15.102:0/1336555810 >> v1:172.21.15.102:6789/0 conn(0x7ff000162d60 legacy=0x7ff0000c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:55.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.773+0000 7ff005a00640 1 -- 172.21.15.102:0/1336555810 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00006eba0 con 0x7ff000162980 2024-04-20T21:16:55.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.774+0000 7ff005a00640 1 --2- 172.21.15.102:0/1336555810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162980 0x7ff0000c8950 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feff40097a0 tx=0x7feff402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c33c417e450be95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:55.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.774+0000 7fefffe00640 1 -- 172.21.15.102:0/1336555810 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4034070 con 0x7ff000162980 2024-04-20T21:16:55.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.774+0000 7fefffe00640 1 -- 172.21.15.102:0/1336555810 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff4030610 con 0x7ff000162980 2024-04-20T21:16:55.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.775+0000 7fefffe00640 1 -- 172.21.15.102:0/1336555810 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff40309e0 con 0x7ff000162980 2024-04-20T21:16:55.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.775+0000 7ff006e00640 1 -- 172.21.15.102:0/1336555810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162980 msgr2=0x7ff0000c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:55.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.775+0000 7ff006e00640 1 --2- 172.21.15.102:0/1336555810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162980 0x7ff0000c8950 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feff40097a0 tx=0x7feff402b960 comp rx=0 tx=0).stop 2024-04-20T21:16:55.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.775+0000 7ff006e00640 1 -- 172.21.15.102:0/1336555810 shutdown_connections 2024-04-20T21:16:55.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.775+0000 7ff006e00640 1 --2- 172.21.15.102:0/1336555810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 0x7ff000169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:55.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.775+0000 7ff006e00640 1 --2- 172.21.15.102:0/1336555810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162980 0x7ff0000c8950 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:55.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.775+0000 7ff006e00640 1 -- 172.21.15.102:0/1336555810 >> 172.21.15.102:0/1336555810 conn(0x7ff0000cc620 msgr2=0x7ff0000cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:55.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.775+0000 7ff006e00640 1 -- 172.21.15.102:0/1336555810 shutdown_connections 2024-04-20T21:16:55.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.776+0000 7ff006e00640 1 -- 172.21.15.102:0/1336555810 wait complete. 2024-04-20T21:16:55.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.777+0000 7ff006e00640 1 Processor -- start 2024-04-20T21:16:55.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.777+0000 7ff006e00640 1 -- start start 2024-04-20T21:16:55.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 0x7ff0001f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:55.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff000162980 0x7ff000071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:55.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162d60 0x7ff00015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:55.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001631a0 con 0x7ff000162d60 2024-04-20T21:16:55.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff00006f680 con 0x7ff000162980 2024-04-20T21:16:55.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff0001635d0 con 0x7ff000160880 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162d60 0x7ff00015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162d60 0x7ff00015b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57744/0 (socket says 172.21.15.102:57744) 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff006400640 1 -- 172.21.15.102:0/338134954 learned_addr learned my addr 172.21.15.102:0/338134954 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff005000640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff000162980 0x7ff000071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.778+0000 7ff005a00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 0x7ff0001f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7ff006400640 1 -- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 msgr2=0x7ff0001f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7ff006400640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 0x7ff0001f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7ff006400640 1 -- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff000162980 msgr2=0x7ff000071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7ff006400640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff000162980 0x7ff000071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7ff006400640 1 -- 172.21.15.102:0/338134954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00015d820 con 0x7ff000162d60 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7ff006400640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162d60 0x7ff00015b290 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feff8009d30 tx=0x7feff800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7feffd600640 1 -- 172.21.15.102:0/338134954 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8018020 con 0x7ff000162d60 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7feffd600640 1 -- 172.21.15.102:0/338134954 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff8014480 con 0x7ff000162d60 2024-04-20T21:16:55.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7feffd600640 1 -- 172.21.15.102:0/338134954 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8014870 con 0x7ff000162d60 2024-04-20T21:16:55.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00015ca60 con 0x7ff000162d60 2024-04-20T21:16:55.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.779+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00015c1c0 con 0x7ff000162d60 2024-04-20T21:16:55.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.780+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00004fcf0 con 0x7ff000162d60 2024-04-20T21:16:55.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.781+0000 7feffd600640 1 -- 172.21.15.102:0/338134954 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feff8005d10 con 0x7ff000162d60 2024-04-20T21:16:55.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.782+0000 7feffd600640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefe006c8f0 0x7fefe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:55.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.782+0000 7feffd600640 1 -- 172.21.15.102:0/338134954 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7feff808c9b0 con 0x7ff000162d60 2024-04-20T21:16:55.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.783+0000 7ff005a00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefe006c8f0 0x7fefe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:55.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.783+0000 7ff005a00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefe006c8f0 0x7fefe006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feff4036000 tx=0x7feff403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:55.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.787+0000 7feffd600640 1 -- 172.21.15.102:0/338134954 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feff8022030 con 0x7ff000162d60 2024-04-20T21:16:55.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:55.989+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff0000c81e0 con 0x7ff000162d60 2024-04-20T21:16:57.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.572+0000 7feffd600640 1 -- 172.21.15.102:0/338134954 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7feff8010040 con 0x7ff000162d60 2024-04-20T21:16:57.572 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data_ec' created 2024-04-20T21:16:57.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefe006c8f0 msgr2=0x7fefe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:57.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefe006c8f0 0x7fefe006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feff4036000 tx=0x7feff403b040 comp rx=0 tx=0).stop 2024-04-20T21:16:57.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162d60 msgr2=0x7ff00015b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:57.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162d60 0x7ff00015b290 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feff8009d30 tx=0x7feff800c490 comp rx=0 tx=0).stop 2024-04-20T21:16:57.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 shutdown_connections 2024-04-20T21:16:57.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000162d60 0x7ff00015b290 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefe006c8f0 0x7fefe006edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff000162980 0x7ff000071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 --2- 172.21.15.102:0/338134954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff000160880 0x7ff0001f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.576+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 >> 172.21.15.102:0/338134954 conn(0x7ff0000cc620 msgr2=0x7ff0000cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:57.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.578+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 shutdown_connections 2024-04-20T21:16:57.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.578+0000 7ff006e00640 1 -- 172.21.15.102:0/338134954 wait complete. 2024-04-20T21:16:57.606 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-20T21:16:57.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.786+0000 7fad14200640 1 Processor -- start 2024-04-20T21:16:57.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.786+0000 7fad14200640 1 -- start start 2024-04-20T21:16:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.787+0000 7fad14200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fad0c153200 0x7fad0c0c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.787+0000 7fad14200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fad0c073aa0 0x7fad0c0c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad14200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad0c073f40 con 0x7fad0c06f100 2024-04-20T21:16:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad14200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fad0c0c7fe0 con 0x7fad0c153200 2024-04-20T21:16:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad14200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fad0c0c73d0 con 0x7fad0c073aa0 2024-04-20T21:16:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad11800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fad0c153200 0x7fad0c0c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad10e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fad0c06f100 0x7fad0c073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47336/0 (socket says 172.21.15.102:47336) 2024-04-20T21:16:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad10e00640 1 -- 172.21.15.102:0/325784850 learned_addr learned my addr 172.21.15.102:0/325784850 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad12200640 1 --2- 172.21.15.102:0/325784850 >> v2:172.21.15.193:3300/0 conn(0x7fad0c073aa0 0x7fad0c0c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad0be00640 1 -- 172.21.15.102:0/325784850 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705595617 0 0) 0x7fad0c073f40 con 0x7fad0c06f100 2024-04-20T21:16:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.788+0000 7fad0be00640 1 -- 172.21.15.102:0/325784850 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facf4003520 con 0x7fad0c06f100 2024-04-20T21:16:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.789+0000 7fad12200640 1 -- 172.21.15.102:0/325784850 >> v2:172.21.15.102:3300/0 conn(0x7fad0c153200 msgr2=0x7fad0c0c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.789+0000 7fad12200640 1 --2- 172.21.15.102:0/325784850 >> v2:172.21.15.102:3300/0 conn(0x7fad0c153200 0x7fad0c0c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.789+0000 7fad12200640 1 -- 172.21.15.102:0/325784850 >> v1:172.21.15.102:6789/0 conn(0x7fad0c06f100 legacy=0x7fad0c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.789+0000 7fad12200640 1 -- 172.21.15.102:0/325784850 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c0c76b0 con 0x7fad0c073aa0 2024-04-20T21:16:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.790+0000 7fad12200640 1 --2- 172.21.15.102:0/325784850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c073aa0 0x7fad0c0c6580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0c6fd0 tx=0x7fad0402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c1f63b0d1181b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:57.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.790+0000 7fad0be00640 1 -- 172.21.15.102:0/325784850 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04039070 con 0x7fad0c073aa0 2024-04-20T21:16:57.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.790+0000 7fad0be00640 1 -- 172.21.15.102:0/325784850 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0402cd00 con 0x7fad0c073aa0 2024-04-20T21:16:57.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.790+0000 7fad0be00640 1 -- 172.21.15.102:0/325784850 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04034660 con 0x7fad0c073aa0 2024-04-20T21:16:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.791+0000 7fad14200640 1 -- 172.21.15.102:0/325784850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c073aa0 msgr2=0x7fad0c0c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.791+0000 7fad14200640 1 --2- 172.21.15.102:0/325784850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c073aa0 0x7fad0c0c6580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0c6fd0 tx=0x7fad0402bf50 comp rx=0 tx=0).stop 2024-04-20T21:16:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.791+0000 7fad14200640 1 -- 172.21.15.102:0/325784850 shutdown_connections 2024-04-20T21:16:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.791+0000 7fad14200640 1 --2- 172.21.15.102:0/325784850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c073aa0 0x7fad0c0c6580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.791+0000 7fad14200640 1 --2- 172.21.15.102:0/325784850 >> v2:172.21.15.102:3300/0 conn(0x7fad0c153200 0x7fad0c0c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.791+0000 7fad14200640 1 -- 172.21.15.102:0/325784850 >> 172.21.15.102:0/325784850 conn(0x7fad0c074ea0 msgr2=0x7fad0c075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.791+0000 7fad14200640 1 -- 172.21.15.102:0/325784850 shutdown_connections 2024-04-20T21:16:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.791+0000 7fad14200640 1 -- 172.21.15.102:0/325784850 wait complete. 2024-04-20T21:16:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.793+0000 7fad14200640 1 Processor -- start 2024-04-20T21:16:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.793+0000 7fad14200640 1 -- start start 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad14200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad0c06f100 0x7fad0c146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad14200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad0c073aa0 0x7fad0c144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad14200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c153200 0x7fad0c145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad14200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1535e0 con 0x7fad0c06f100 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad14200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c0c7c70 con 0x7fad0c153200 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad14200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c153760 con 0x7fad0c073aa0 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad11800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad0c06f100 0x7fad0c146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad11800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad0c06f100 0x7fad0c146cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57758/0 (socket says 172.21.15.102:57758) 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad11800640 1 -- 172.21.15.102:0/1610747335 learned_addr learned my addr 172.21.15.102:0/1610747335 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad12200640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c153200 0x7fad0c145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad10e00640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad0c073aa0 0x7fad0c144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad11800640 1 -- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad0c073aa0 msgr2=0x7fad0c144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad11800640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad0c073aa0 0x7fad0c144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad11800640 1 -- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c153200 msgr2=0x7fad0c145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad11800640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c153200 0x7fad0c145510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad11800640 1 -- 172.21.15.102:0/1610747335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c0cbd70 con 0x7fad0c06f100 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.794+0000 7fad10e00640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad0c073aa0 0x7fad0c144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.795+0000 7fad11800640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad0c06f100 0x7fad0c146cc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad0000e8a0 tx=0x7fad0000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.795+0000 7fad09600640 1 -- 172.21.15.102:0/1610747335 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00017070 con 0x7fad0c06f100 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.795+0000 7fad09600640 1 -- 172.21.15.102:0/1610747335 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0000c940 con 0x7fad0c06f100 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.795+0000 7fad09600640 1 -- 172.21.15.102:0/1610747335 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0000cd30 con 0x7fad0c06f100 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.795+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c14acc0 con 0x7fad0c06f100 2024-04-20T21:16:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.795+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c14c9e0 con 0x7fad0c06f100 2024-04-20T21:16:57.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.796+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0c074340 con 0x7fad0c06f100 2024-04-20T21:16:57.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.797+0000 7fad09600640 1 -- 172.21.15.102:0/1610747335 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad0001a070 con 0x7fad0c06f100 2024-04-20T21:16:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.799+0000 7fad09600640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7facec06c8f0 0x7facec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.799+0000 7fad09600640 1 -- 172.21.15.102:0/1610747335 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fad0008c080 con 0x7fad0c06f100 2024-04-20T21:16:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.799+0000 7fad10e00640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7facec06c8f0 0x7facec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.799+0000 7fad10e00640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7facec06c8f0 0x7facec06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad0c1563e0 tx=0x7facf8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:57.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:57.804+0000 7fad09600640 1 -- 172.21.15.102:0/1610747335 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fad00059ab0 con 0x7fad0c06f100 2024-04-20T21:16:58.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.001+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fad0c0c3720 con 0x7fad0c06f100 2024-04-20T21:16:58.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.575+0000 7fad09600640 1 -- 172.21.15.102:0/1610747335 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fad0005ddb0 con 0x7fad0c06f100 2024-04-20T21:16:58.575 INFO:teuthology.orchestra.run.smithi102.stderr:set pool 4 allow_ec_overwrites to true 2024-04-20T21:16:58.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7facec06c8f0 msgr2=0x7facec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:58.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7facec06c8f0 0x7facec06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad0c1563e0 tx=0x7facf8009040 comp rx=0 tx=0).stop 2024-04-20T21:16:58.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad0c06f100 msgr2=0x7fad0c146cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:58.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad0c06f100 0x7fad0c146cc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad0000e8a0 tx=0x7fad0000ed60 comp rx=0 tx=0).stop 2024-04-20T21:16:58.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 shutdown_connections 2024-04-20T21:16:58.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7facec06c8f0 0x7facec06edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:58.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad0c153200 0x7fad0c145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad0c073aa0 0x7fad0c144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 --2- 172.21.15.102:0/1610747335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad0c06f100 0x7fad0c146cc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:58.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.580+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 >> 172.21.15.102:0/1610747335 conn(0x7fad0c074ea0 msgr2=0x7fad0c1523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:58.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.586+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 shutdown_connections 2024-04-20T21:16:58.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.587+0000 7fad14200640 1 -- 172.21.15.102:0/1610747335 wait complete. 2024-04-20T21:16:58.621 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-20T21:16:58.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.797+0000 7f21ac600640 1 Processor -- start 2024-04-20T21:16:58.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.798+0000 7f21ac600640 1 -- start start 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.799+0000 7f21ac600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a4155550 0x7f21a415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.799+0000 7f21ac600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a415b2c0 con 0x7f21a40cbd10 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.799+0000 7f21ac600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a415b540 con 0x7f21a40cc130 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.799+0000 7f21ac600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f21a415f800 con 0x7f21a4155550 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.799+0000 7f21a9200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f21a40cbd10 0x7f21a40d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43322/0 (socket says 172.21.15.102:43322) 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.799+0000 7f21a9200640 1 -- 172.21.15.102:0/835576345 learned_addr learned my addr 172.21.15.102:0/835576345 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.799+0000 7f21a9c00640 1 --2- 172.21.15.102:0/835576345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a4155550 0x7f21a415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.800+0000 7f21a9c00640 1 -- 172.21.15.102:0/835576345 >> v1:172.21.15.193:6789/0 conn(0x7f21a40cc130 legacy=0x7f21a415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:58.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.800+0000 7f21a3e00640 1 -- 172.21.15.102:0/835576345 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3232922302 0 0) 0x7f2190001110 con 0x7f21a40cbd10 2024-04-20T21:16:58.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.800+0000 7f21a9c00640 1 -- 172.21.15.102:0/835576345 >> v1:172.21.15.102:6789/0 conn(0x7f21a40cbd10 legacy=0x7f21a40d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:58.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.800+0000 7f21a9c00640 1 -- 172.21.15.102:0/835576345 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a415b820 con 0x7f21a4155550 2024-04-20T21:16:58.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.801+0000 7f21a9c00640 1 --2- 172.21.15.102:0/835576345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a4155550 0x7f21a415f2c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2198009750 tx=0x7f219802b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11fe213f3e796ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:58.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.801+0000 7f21a3e00640 1 -- 172.21.15.102:0/835576345 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198038070 con 0x7f21a4155550 2024-04-20T21:16:58.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.801+0000 7f21a3e00640 1 -- 172.21.15.102:0/835576345 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2198031710 con 0x7f21a4155550 2024-04-20T21:16:58.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.801+0000 7f21a3e00640 1 -- 172.21.15.102:0/835576345 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198031ae0 con 0x7f21a4155550 2024-04-20T21:16:58.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.802+0000 7f21ac600640 1 -- 172.21.15.102:0/835576345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a4155550 msgr2=0x7f21a415f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:58.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.802+0000 7f21ac600640 1 --2- 172.21.15.102:0/835576345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a4155550 0x7f21a415f2c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2198009750 tx=0x7f219802b7b0 comp rx=0 tx=0).stop 2024-04-20T21:16:58.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.802+0000 7f21ac600640 1 -- 172.21.15.102:0/835576345 shutdown_connections 2024-04-20T21:16:58.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.802+0000 7f21ac600640 1 --2- 172.21.15.102:0/835576345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a4155550 0x7f21a415f2c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:58.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.802+0000 7f21ac600640 1 -- 172.21.15.102:0/835576345 >> 172.21.15.102:0/835576345 conn(0x7f21a406d2f0 msgr2=0x7f21a4072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:58.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.802+0000 7f21ac600640 1 -- 172.21.15.102:0/835576345 shutdown_connections 2024-04-20T21:16:58.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.802+0000 7f21ac600640 1 -- 172.21.15.102:0/835576345 wait complete. 2024-04-20T21:16:58.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.803+0000 7f21ac600640 1 Processor -- start 2024-04-20T21:16:58.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.804+0000 7f21ac600640 1 -- start start 2024-04-20T21:16:58.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.804+0000 7f21ac600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a40cbd10 0x7f21a41e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:58.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.804+0000 7f21ac600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21a4155550 0x7f21a40cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:58.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.804+0000 7f21ac600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a41e54d0 0x7f21a414b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:58.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.804+0000 7f21ac600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f21a415bde0 con 0x7f21a4155550 2024-04-20T21:16:58.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.804+0000 7f21ac600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21a415c150 con 0x7f21a40cbd10 2024-04-20T21:16:58.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.804+0000 7f21ac600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f21a4155930 con 0x7f21a41e54d0 2024-04-20T21:16:58.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a40cbd10 0x7f21a41e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:58.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a40cbd10 0x7f21a41e46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:38694/0 (socket says 172.21.15.102:38694) 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 -- 172.21.15.102:0/1345759114 learned_addr learned my addr 172.21.15.102:0/1345759114 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21aa600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a41e54d0 0x7f21a414b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9200640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21a4155550 0x7f21a40cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 -- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a41e54d0 msgr2=0x7f21a414b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a41e54d0 0x7f21a414b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 -- 172.21.15.102:0/1345759114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21a4155550 msgr2=0x7f21a40cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21a4155550 0x7f21a40cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 -- 172.21.15.102:0/1345759114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a40d0220 con 0x7f21a40cbd10 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21aa600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a41e54d0 0x7f21a414b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:58.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.805+0000 7f21a9c00640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a40cbd10 0x7f21a41e46b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2198009870 tx=0x7f21980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:58.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.806+0000 7f21a1600640 1 -- 172.21.15.102:0/1345759114 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198043070 con 0x7f21a40cbd10 2024-04-20T21:16:58.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.806+0000 7f21a1600640 1 -- 172.21.15.102:0/1345759114 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21980339c0 con 0x7f21a40cbd10 2024-04-20T21:16:58.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.806+0000 7f21a1600640 1 -- 172.21.15.102:0/1345759114 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198038040 con 0x7f21a40cbd10 2024-04-20T21:16:58.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.806+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a40d04c0 con 0x7f21a40cbd10 2024-04-20T21:16:58.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.806+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a40b38a0 con 0x7f21a40cbd10 2024-04-20T21:16:58.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.807+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a40d0690 con 0x7f21a40cbd10 2024-04-20T21:16:58.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.808+0000 7f21a1600640 1 -- 172.21.15.102:0/1345759114 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2198046070 con 0x7f21a40cbd10 2024-04-20T21:16:58.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.809+0000 7f21a1600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f218406c8f0 0x7f218406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:58.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.809+0000 7f21a1600640 1 -- 172.21.15.102:0/1345759114 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f21980adff0 con 0x7f21a40cbd10 2024-04-20T21:16:58.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.809+0000 7f21a9200640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f218406c8f0 0x7f218406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:58.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.810+0000 7f21a9200640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f218406c8f0 0x7f218406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2190007df0 tx=0x7f2190007d80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:58.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:58.813+0000 7f21a1600640 1 -- 172.21.15.102:0/1345759114 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f219807bbd0 con 0x7f21a40cbd10 2024-04-20T21:16:59.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.050+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f21a404fdd0 con 0x7f21a40cbd10 2024-04-20T21:16:59.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.616+0000 7f21a1600640 1 -- 172.21.15.102:0/1345759114 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) ==== 137+0+0 (secure 0 0 0) 0x7f219807fed0 con 0x7f21a40cbd10 2024-04-20T21:16:59.616 INFO:teuthology.orchestra.run.smithi102.stderr:added data pool 4 to fsmap 2024-04-20T21:16:59.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f218406c8f0 msgr2=0x7f218406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:59.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f218406c8f0 0x7f218406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2190007df0 tx=0x7f2190007d80 comp rx=0 tx=0).stop 2024-04-20T21:16:59.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a40cbd10 msgr2=0x7f21a41e46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:59.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a40cbd10 0x7f21a41e46b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2198009870 tx=0x7f21980046c0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 shutdown_connections 2024-04-20T21:16:59.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21a41e54d0 0x7f21a414b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21a4155550 0x7f21a40cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f218406c8f0 0x7f218406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 --2- 172.21.15.102:0/1345759114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a40cbd10 0x7f21a41e46b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.619+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 >> 172.21.15.102:0/1345759114 conn(0x7f21a406d2f0 msgr2=0x7f21a40cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:59.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.620+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 shutdown_connections 2024-04-20T21:16:59.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.620+0000 7f21ac600640 1 -- 172.21.15.102:0/1345759114 wait complete. 2024-04-20T21:16:59.641 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:16:59.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.828+0000 7f49af600640 1 Processor -- start 2024-04-20T21:16:59.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.828+0000 7f49af600640 1 -- start start 2024-04-20T21:16:59.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.829+0000 7f49af600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f49a80c8db0 0x7f49a8070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:59.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.829+0000 7f49af600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8071330 0x7f49a81628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:59.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.829+0000 7f49af600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49a8158e80 con 0x7f49a8152170 2024-04-20T21:16:59.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.829+0000 7f49af600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f49a80c6dc0 con 0x7f49a80c8db0 2024-04-20T21:16:59.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.829+0000 7f49af600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f49a8071f70 con 0x7f49a8071330 2024-04-20T21:16:59.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49acc00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f49a8152170 0x7f49a8158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43326/0 (socket says 172.21.15.102:43326) 2024-04-20T21:16:59.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49acc00640 1 -- 172.21.15.102:0/1774496948 learned_addr learned my addr 172.21.15.102:0/1774496948 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:16:59.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49a7e00640 1 --2- 172.21.15.102:0/1774496948 >> v2:172.21.15.193:3300/0 conn(0x7f49a80c8db0 0x7f49a8070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:59.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49ad600640 1 --2- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8071330 0x7f49a81628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:59.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49a7400640 1 -- 172.21.15.102:0/1774496948 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481157873 0 0) 0x7f49a8158e80 con 0x7f49a8152170 2024-04-20T21:16:59.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49a7400640 1 -- 172.21.15.102:0/1774496948 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4990003520 con 0x7f49a8152170 2024-04-20T21:16:59.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49a7e00640 1 -- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8071330 msgr2=0x7f49a81628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:16:59.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49a7e00640 1 --2- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8071330 0x7f49a81628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49a7e00640 1 -- 172.21.15.102:0/1774496948 >> v1:172.21.15.102:6789/0 conn(0x7f49a8152170 legacy=0x7f49a8158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:59.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49a7e00640 1 -- 172.21.15.102:0/1774496948 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a80c91c0 con 0x7f49a80c8db0 2024-04-20T21:16:59.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.830+0000 7f49ad600640 1 --2- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8071330 0x7f49a81628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:16:59.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.831+0000 7f49a7e00640 1 --2- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a80c8db0 0x7f49a8070df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49940097a0 tx=0x7f499402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f580f025f22caaf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:59.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49a7400640 1 -- 172.21.15.102:0/1774496948 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994035020 con 0x7f49a80c8db0 2024-04-20T21:16:59.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49a7400640 1 -- 172.21.15.102:0/1774496948 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4994031810 con 0x7f49a80c8db0 2024-04-20T21:16:59.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49a7400640 1 -- 172.21.15.102:0/1774496948 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994031be0 con 0x7f49a80c8db0 2024-04-20T21:16:59.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49af600640 1 -- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a80c8db0 msgr2=0x7f49a8070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:59.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49af600640 1 --2- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a80c8db0 0x7f49a8070df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49940097a0 tx=0x7f499402b8b0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49af600640 1 -- 172.21.15.102:0/1774496948 shutdown_connections 2024-04-20T21:16:59.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49af600640 1 --2- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8071330 0x7f49a81628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49af600640 1 --2- 172.21.15.102:0/1774496948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a80c8db0 0x7f49a8070df0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.832+0000 7f49af600640 1 -- 172.21.15.102:0/1774496948 >> 172.21.15.102:0/1774496948 conn(0x7f49a80c9390 msgr2=0x7f49a80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:16:59.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.833+0000 7f49af600640 1 -- 172.21.15.102:0/1774496948 shutdown_connections 2024-04-20T21:16:59.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.833+0000 7f49af600640 1 -- 172.21.15.102:0/1774496948 wait complete. 2024-04-20T21:16:59.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.835+0000 7f49af600640 1 Processor -- start 2024-04-20T21:16:59.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.835+0000 7f49af600640 1 -- start start 2024-04-20T21:16:59.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49af600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a8071330 0x7f49a81e1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:59.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49af600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49a80c8db0 0x7f49a80cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:59.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49af600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8152170 0x7f49a81566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:59.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49af600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f49a80724b0 con 0x7f49a80c8db0 2024-04-20T21:16:59.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49af600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8072210 con 0x7f49a8071330 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49af600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f49a8158250 con 0x7f49a8152170 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49a80c8db0 0x7f49a80cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49a80c8db0 0x7f49a80cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32964/0 (socket says 172.21.15.102:32964) 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49a7e00640 1 -- 172.21.15.102:0/3423112969 learned_addr learned my addr 172.21.15.102:0/3423112969 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49ad600640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8152170 0x7f49a81566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49acc00640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a8071330 0x7f49a81e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49a7e00640 1 -- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8152170 msgr2=0x7f49a81566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.836+0000 7f49a7e00640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8152170 0x7f49a81566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49a7e00640 1 -- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a8071330 msgr2=0x7f49a81e1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49a7e00640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a8071330 0x7f49a81e1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49a7e00640 1 -- 172.21.15.102:0/3423112969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a81e1e80 con 0x7f49a80c8db0 2024-04-20T21:16:59.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49a7e00640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49a80c8db0 0x7f49a80cbae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f499402bec0 tx=0x7f49940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:59.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49a4c00640 1 -- 172.21.15.102:0/3423112969 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994041070 con 0x7f49a80c8db0 2024-04-20T21:16:59.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49a4c00640 1 -- 172.21.15.102:0/3423112969 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4994031e40 con 0x7f49a80c8db0 2024-04-20T21:16:59.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a814c180 con 0x7f49a80c8db0 2024-04-20T21:16:59.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49a4c00640 1 -- 172.21.15.102:0/3423112969 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994035040 con 0x7f49a80c8db0 2024-04-20T21:16:59.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.837+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81ed170 con 0x7f49a80c8db0 2024-04-20T21:16:59.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.838+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a8074340 con 0x7f49a80c8db0 2024-04-20T21:16:59.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.839+0000 7f49a4c00640 1 -- 172.21.15.102:0/3423112969 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f499404c060 con 0x7f49a80c8db0 2024-04-20T21:16:59.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.840+0000 7f49a4c00640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f498806c920 0x7f498806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:16:59.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.840+0000 7f49a4c00640 1 -- 172.21.15.102:0/3423112969 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f49940ad980 con 0x7f49a80c8db0 2024-04-20T21:16:59.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.840+0000 7f49acc00640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f498806c920 0x7f498806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:16:59.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.841+0000 7f49acc00640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f498806c920 0x7f498806ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f499c005880 tx=0x7f499c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:16:59.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:16:59.844+0000 7f49a4c00640 1 -- 172.21.15.102:0/3423112969 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f499407b940 con 0x7f49a80c8db0 2024-04-20T21:17:00.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.089+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49a80c3460 con 0x7f49a80c8db0 2024-04-20T21:17:00.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.090+0000 7f49a4c00640 1 -- 172.21.15.102:0/3423112969 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+6850 (secure 0 0 0) 0x7f499407b240 con 0x7f49a80c8db0 2024-04-20T21:17:00.090 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:00.090 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:16:59.053371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:17:00.091 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-20T21:17:00.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.092+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f498806c920 msgr2=0x7f498806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.092+0000 7f49af600640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f498806c920 0x7f498806ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f499c005880 tx=0x7f499c009040 comp rx=0 tx=0).stop 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49a80c8db0 msgr2=0x7f49a80cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49a80c8db0 0x7f49a80cbae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f499402bec0 tx=0x7f49940046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 shutdown_connections 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49a8152170 0x7f49a81566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49a80c8db0 0x7f49a80cbae0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f498806c920 0x7f498806ede0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 --2- 172.21.15.102:0/3423112969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a8071330 0x7f49a81e1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 >> 172.21.15.102:0/3423112969 conn(0x7f49a80c9390 msgr2=0x7f49a806ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 shutdown_connections 2024-04-20T21:17:00.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.093+0000 7f49af600640 1 -- 172.21.15.102:0/3423112969 wait complete. 2024-04-20T21:17:00.112 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:00.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.311+0000 7f1b21200640 1 Processor -- start 2024-04-20T21:17:00.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.311+0000 7f1b21200640 1 -- start start 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b21200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1b1c15ea20 0x7f1b1c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b21200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1b1c15f2b0 0x7f1b1c0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b21200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c0c8f20 con 0x7f1b1c15ee00 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b21200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1c06cb80 con 0x7f1b1c15f2b0 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b21200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1c06ce60 con 0x7f1b1c15ea20 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b19e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1b1c15ea20 0x7f1b1c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b19400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f1b1c15ee00 0x7f1b1c0c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:50640/0 (socket says 172.21.15.102:50640) 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b19400640 1 -- 172.21.15.102:0/4146690255 learned_addr learned my addr 172.21.15.102:0/4146690255 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:00.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.313+0000 7f1b1a800640 1 --2- 172.21.15.102:0/4146690255 >> v2:172.21.15.102:3300/0 conn(0x7f1b1c15f2b0 0x7f1b1c0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.314+0000 7f1b18a00640 1 -- 172.21.15.102:0/4146690255 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272753306 0 0) 0x7f1b1c0c8f20 con 0x7f1b1c15ee00 2024-04-20T21:17:00.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.314+0000 7f1b18a00640 1 -- 172.21.15.102:0/4146690255 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b00003520 con 0x7f1b1c15ee00 2024-04-20T21:17:00.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.314+0000 7f1b1a800640 1 -- 172.21.15.102:0/4146690255 >> v2:172.21.15.193:3300/0 conn(0x7f1b1c15ea20 msgr2=0x7f1b1c06d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:00.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.314+0000 7f1b1a800640 1 --2- 172.21.15.102:0/4146690255 >> v2:172.21.15.193:3300/0 conn(0x7f1b1c15ea20 0x7f1b1c06d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.314+0000 7f1b1a800640 1 -- 172.21.15.102:0/4146690255 >> v1:172.21.15.193:6789/0 conn(0x7f1b1c15ee00 legacy=0x7f1b1c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.314+0000 7f1b1a800640 1 -- 172.21.15.102:0/4146690255 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c06d140 con 0x7f1b1c15f2b0 2024-04-20T21:17:00.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.315+0000 7f1b1a800640 1 --2- 172.21.15.102:0/4146690255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15f2b0 0x7f1b1c0710a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0cd8d0 tx=0x7f1b1402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5a80409fcc20ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:00.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.315+0000 7f1b18a00640 1 -- 172.21.15.102:0/4146690255 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14039070 con 0x7f1b1c15f2b0 2024-04-20T21:17:00.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.315+0000 7f1b18a00640 1 -- 172.21.15.102:0/4146690255 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1402cd00 con 0x7f1b1c15f2b0 2024-04-20T21:17:00.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.315+0000 7f1b18a00640 1 -- 172.21.15.102:0/4146690255 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14034640 con 0x7f1b1c15f2b0 2024-04-20T21:17:00.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.316+0000 7f1b21200640 1 -- 172.21.15.102:0/4146690255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15f2b0 msgr2=0x7f1b1c0710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.316+0000 7f1b21200640 1 --2- 172.21.15.102:0/4146690255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15f2b0 0x7f1b1c0710a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0cd8d0 tx=0x7f1b1402bf50 comp rx=0 tx=0).stop 2024-04-20T21:17:00.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.316+0000 7f1b21200640 1 -- 172.21.15.102:0/4146690255 shutdown_connections 2024-04-20T21:17:00.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.316+0000 7f1b21200640 1 --2- 172.21.15.102:0/4146690255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15f2b0 0x7f1b1c0710a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.316+0000 7f1b21200640 1 --2- 172.21.15.102:0/4146690255 >> v2:172.21.15.193:3300/0 conn(0x7f1b1c15ea20 0x7f1b1c06d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.316+0000 7f1b21200640 1 -- 172.21.15.102:0/4146690255 >> 172.21.15.102:0/4146690255 conn(0x7f1b1c0c9390 msgr2=0x7f1b1c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:00.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.316+0000 7f1b21200640 1 -- 172.21.15.102:0/4146690255 shutdown_connections 2024-04-20T21:17:00.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.316+0000 7f1b21200640 1 -- 172.21.15.102:0/4146690255 wait complete. 2024-04-20T21:17:00.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.318+0000 7f1b21200640 1 Processor -- start 2024-04-20T21:17:00.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.318+0000 7f1b21200640 1 -- start start 2024-04-20T21:17:00.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b21200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b1c15ea20 0x7f1b1c0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b21200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15ee00 0x7f1b1c14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b21200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c15f2b0 0x7f1b1c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b21200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c0c8c80 con 0x7f1b1c15ee00 2024-04-20T21:17:00.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b21200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c0c8b00 con 0x7f1b1c15f2b0 2024-04-20T21:17:00.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b21200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1c154e80 con 0x7f1b1c15ea20 2024-04-20T21:17:00.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15ee00 0x7f1b1c14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15ee00 0x7f1b1c14f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32980/0 (socket says 172.21.15.102:32980) 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19400640 1 -- 172.21.15.102:0/1848169427 learned_addr learned my addr 172.21.15.102:0/1848169427 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19e00640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b1c15ea20 0x7f1b1c0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b1a800640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c15f2b0 0x7f1b1c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19400640 1 -- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b1c15ea20 msgr2=0x7f1b1c0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19400640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b1c15ea20 0x7f1b1c0c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19400640 1 -- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c15f2b0 msgr2=0x7f1b1c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19400640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c15f2b0 0x7f1b1c14f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.319+0000 7f1b19400640 1 -- 172.21.15.102:0/1848169427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c15e120 con 0x7f1b1c15ee00 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.320+0000 7f1b19400640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15ee00 0x7f1b1c14f390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c4d0 tx=0x7f1b0800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.320+0000 7f1b12000640 1 -- 172.21.15.102:0/1848169427 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08017070 con 0x7f1b1c15ee00 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.320+0000 7f1b12000640 1 -- 172.21.15.102:0/1848169427 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b08005ce0 con 0x7f1b1c15ee00 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.320+0000 7f1b12000640 1 -- 172.21.15.102:0/1848169427 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08007270 con 0x7f1b1c15ee00 2024-04-20T21:17:00.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.320+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c15d5e0 con 0x7f1b1c15ee00 2024-04-20T21:17:00.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.320+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c15db60 con 0x7f1b1c15ee00 2024-04-20T21:17:00.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.321+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c074340 con 0x7f1b1c15ee00 2024-04-20T21:17:00.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.322+0000 7f1b12000640 1 -- 172.21.15.102:0/1848169427 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b0801a070 con 0x7f1b1c15ee00 2024-04-20T21:17:00.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.323+0000 7f1b12000640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1afc06c920 0x7f1afc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.324+0000 7f1b12000640 1 -- 172.21.15.102:0/1848169427 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1b0808b010 con 0x7f1b1c15ee00 2024-04-20T21:17:00.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.324+0000 7f1b19e00640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1afc06c920 0x7f1afc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.325+0000 7f1b19e00640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1afc06c920 0x7f1afc06ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009b40 tx=0x7f1b0c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.328+0000 7f1b12000640 1 -- 172.21.15.102:0/1848169427 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b08058b40 con 0x7f1b1c15ee00 2024-04-20T21:17:00.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.532+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b1c15c160 con 0x7f1b1c15ee00 2024-04-20T21:17:00.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.535+0000 7f1b12000640 1 -- 172.21.15.102:0/1848169427 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f1b0805ce40 con 0x7f1b1c15ee00 2024-04-20T21:17:00.536 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:00.536 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:00.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1afc06c920 msgr2=0x7f1afc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1afc06c920 0x7f1afc06ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009b40 tx=0x7f1b0c009190 comp rx=0 tx=0).stop 2024-04-20T21:17:00.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15ee00 msgr2=0x7f1b1c14f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15ee00 0x7f1b1c14f390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c4d0 tx=0x7f1b0800c990 comp rx=0 tx=0).stop 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 shutdown_connections 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1afc06c920 0x7f1afc06ede0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c15f2b0 0x7f1b1c14f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b1c15ee00 0x7f1b1c14f390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 --2- 172.21.15.102:0/1848169427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b1c15ea20 0x7f1b1c0c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 >> 172.21.15.102:0/1848169427 conn(0x7f1b1c0c9390 msgr2=0x7f1b1c06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 shutdown_connections 2024-04-20T21:17:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.539+0000 7f1b21200640 1 -- 172.21.15.102:0/1848169427 wait complete. 2024-04-20T21:17:00.559 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:00.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.759+0000 7f6c95800640 1 Processor -- start 2024-04-20T21:17:00.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.759+0000 7f6c95800640 1 -- start start 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.760+0000 7f6c95800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.760+0000 7f6c95800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c90073a10 0x7f6c90158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.760+0000 7f6c95800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c901536d0 con 0x7f6c900c9eb0 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.760+0000 7f6c95800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6c90159500 con 0x7f6c90073a10 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.760+0000 7f6c95800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6c900c76c0 con 0x7f6c9014fa50 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c90073a10 0x7f6c90158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35660/0 (socket says 172.21.15.102:35660) 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c90073a10 0x7f6c90158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:38746/0 (socket says 172.21.15.102:38746) 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8ea00640 1 -- 172.21.15.102:0/4056388253 learned_addr learned my addr 172.21.15.102:0/4056388253 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:00.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8ea00640 1 -- 172.21.15.102:0/4056388253 >> v2:172.21.15.193:3300/0 conn(0x7f6c90073a10 msgr2=0x7f6c90158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:00.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8ea00640 1 --2- 172.21.15.102:0/4056388253 >> v2:172.21.15.193:3300/0 conn(0x7f6c90073a10 0x7f6c90158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8ea00640 1 -- 172.21.15.102:0/4056388253 >> v1:172.21.15.193:6789/0 conn(0x7f6c900c9eb0 legacy=0x7f6c900c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.761+0000 7f6c8ea00640 1 -- 172.21.15.102:0/4056388253 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c900c7960 con 0x7f6c9014fa50 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.762+0000 7f6c8ea00640 1 --2- 172.21.15.102:0/4056388253 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900c4830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009e50 tx=0x7f6c8002bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82f9c9602e42a7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.762+0000 7f6c8e000640 1 -- 172.21.15.102:0/4056388253 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8002cb90 con 0x7f6c9014fa50 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c8e000640 1 -- 172.21.15.102:0/4056388253 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8002cd30 con 0x7f6c9014fa50 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c8e000640 1 -- 172.21.15.102:0/4056388253 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80034610 con 0x7f6c9014fa50 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c95800640 1 -- 172.21.15.102:0/4056388253 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 msgr2=0x7f6c900c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c95800640 1 --2- 172.21.15.102:0/4056388253 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900c4830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009e50 tx=0x7f6c8002bff0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c95800640 1 -- 172.21.15.102:0/4056388253 shutdown_connections 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c95800640 1 --2- 172.21.15.102:0/4056388253 >> v2:172.21.15.193:3300/0 conn(0x7f6c90073a10 0x7f6c90158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c95800640 1 --2- 172.21.15.102:0/4056388253 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900c4830 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c95800640 1 -- 172.21.15.102:0/4056388253 >> 172.21.15.102:0/4056388253 conn(0x7f6c900c9390 msgr2=0x7f6c900c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c95800640 1 -- 172.21.15.102:0/4056388253 shutdown_connections 2024-04-20T21:17:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.763+0000 7f6c95800640 1 -- 172.21.15.102:0/4056388253 wait complete. 2024-04-20T21:17:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.765+0000 7f6c95800640 1 Processor -- start 2024-04-20T21:17:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.765+0000 7f6c95800640 1 -- start start 2024-04-20T21:17:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c95800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c90073a10 0x7f6c9014afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c95800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c900c9eb0 0x7f6c90071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c95800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c95800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6c900c7c00 con 0x7f6c900c9eb0 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c95800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c900c3d90 con 0x7f6c90073a10 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c95800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6c900c4030 con 0x7f6c9014fa50 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c8ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c90073a10 0x7f6c9014afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c8ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c90073a10 0x7f6c9014afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:38750/0 (socket says 172.21.15.102:38750) 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c8ea00640 1 -- 172.21.15.102:0/1217571055 learned_addr learned my addr 172.21.15.102:0/1217571055 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c86000640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c900c9eb0 0x7f6c90071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.766+0000 7f6c8f400640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c8ea00640 1 -- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 msgr2=0x7f6c900cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c8ea00640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c8ea00640 1 -- 172.21.15.102:0/1217571055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c900c9eb0 msgr2=0x7f6c90071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c8ea00640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c900c9eb0 0x7f6c90071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c8ea00640 1 -- 172.21.15.102:0/1217571055 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c90070f20 con 0x7f6c90073a10 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c8f400640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c8ea00640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c90073a10 0x7f6c9014afe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009f70 tx=0x7f6c80004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c87400640 1 -- 172.21.15.102:0/1217571055 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80044020 con 0x7f6c90073a10 2024-04-20T21:17:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c87400640 1 -- 172.21.15.102:0/1217571055 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c800406b0 con 0x7f6c90073a10 2024-04-20T21:17:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c90071200 con 0x7f6c90073a10 2024-04-20T21:17:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c87400640 1 -- 172.21.15.102:0/1217571055 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80009510 con 0x7f6c90073a10 2024-04-20T21:17:00.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.767+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c900717c0 con 0x7f6c90073a10 2024-04-20T21:17:00.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.768+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c90074340 con 0x7f6c90073a10 2024-04-20T21:17:00.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.770+0000 7f6c87400640 1 -- 172.21.15.102:0/1217571055 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c80047020 con 0x7f6c90073a10 2024-04-20T21:17:00.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.770+0000 7f6c87400640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:00.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.771+0000 7f6c87400640 1 -- 172.21.15.102:0/1217571055 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6c800ae710 con 0x7f6c90073a10 2024-04-20T21:17:00.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.771+0000 7f6c86000640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:00.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.771+0000 7f6c86000640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c9006ec90 tx=0x7f6c78009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.776+0000 7f6c87400640 1 -- 172.21.15.102:0/1217571055 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c8004e030 con 0x7f6c90073a10 2024-04-20T21:17:00.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.981+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c901568e0 con 0x7f6c90073a10 2024-04-20T21:17:00.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.982+0000 7f6c87400640 1 -- 172.21.15.102:0/1217571055 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f6c8007c110 con 0x7f6c90073a10 2024-04-20T21:17:00.983 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:00.983 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c6c06c8f0 msgr2=0x7f6c6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c9006ec90 tx=0x7f6c78009040 comp rx=0 tx=0).stop 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c90073a10 msgr2=0x7f6c9014afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c90073a10 0x7f6c9014afe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009f70 tx=0x7f6c80004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 shutdown_connections 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c9014fa50 0x7f6c900cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c900c9eb0 0x7f6c90071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 --2- 172.21.15.102:0/1217571055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c90073a10 0x7f6c9014afe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:00.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 >> 172.21.15.102:0/1217571055 conn(0x7f6c900c9390 msgr2=0x7f6c9006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:00.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 shutdown_connections 2024-04-20T21:17:00.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:00.986+0000 7f6c95800640 1 -- 172.21.15.102:0/1217571055 wait complete. 2024-04-20T21:17:01.019 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.218+0000 7f8ad9800640 1 Processor -- start 2024-04-20T21:17:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.218+0000 7f8ad9800640 1 -- start start 2024-04-20T21:17:01.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad9800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ad4155690 0x7f8ad415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 0x7f8ad40751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad9800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8ad4075710 0x7f8ad40cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad9800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad40cc2f0 con 0x7f8ad4155690 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad9800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad40d0b30 con 0x7f8ad4075710 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad9800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad40d0e10 con 0x7f8ad40cb3e0 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad2a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ad4155690 0x7f8ad415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad3400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8ad4075710 0x7f8ad40cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad2a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ad4155690 0x7f8ad415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32992/0 (socket says 172.21.15.102:32992) 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8ad2a00640 1 -- 172.21.15.102:0/3823033104 learned_addr learned my addr 172.21.15.102:0/3823033104 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.219+0000 7f8aca000640 1 --2- 172.21.15.102:0/3823033104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 0x7f8ad40751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.220+0000 7f8ad3400640 1 -- 172.21.15.102:0/3823033104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 msgr2=0x7f8ad40751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.220+0000 7f8ad3400640 1 --2- 172.21.15.102:0/3823033104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 0x7f8ad40751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.220+0000 7f8ad3400640 1 -- 172.21.15.102:0/3823033104 >> v2:172.21.15.102:3300/0 conn(0x7f8ad4155690 msgr2=0x7f8ad415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.220+0000 7f8ad3400640 1 --2- 172.21.15.102:0/3823033104 >> v2:172.21.15.102:3300/0 conn(0x7f8ad4155690 0x7f8ad415bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.220+0000 7f8ad3400640 1 -- 172.21.15.102:0/3823033104 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40d10f0 con 0x7f8ad4075710 2024-04-20T21:17:01.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.221+0000 7f8ad3400640 1 --2- 172.21.15.102:0/3823033104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 0x7f8ad40cc780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4072920 tx=0x7f8acc02bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed406c0f0beeaed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:01.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.221+0000 7f8ad2000640 1 -- 172.21.15.102:0/3823033104 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc039070 con 0x7f8ad4075710 2024-04-20T21:17:01.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.221+0000 7f8ad2000640 1 -- 172.21.15.102:0/3823033104 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8acc02cc70 con 0x7f8ad4075710 2024-04-20T21:17:01.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.221+0000 7f8ad2000640 1 -- 172.21.15.102:0/3823033104 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc034660 con 0x7f8ad4075710 2024-04-20T21:17:01.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 -- 172.21.15.102:0/3823033104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 msgr2=0x7f8ad40cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 --2- 172.21.15.102:0/3823033104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 0x7f8ad40cc780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4072920 tx=0x7f8acc02bec0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 -- 172.21.15.102:0/3823033104 shutdown_connections 2024-04-20T21:17:01.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 --2- 172.21.15.102:0/3823033104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 0x7f8ad40cc780 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 --2- 172.21.15.102:0/3823033104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 0x7f8ad40751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 --2- 172.21.15.102:0/3823033104 >> v2:172.21.15.102:3300/0 conn(0x7f8ad4155690 0x7f8ad415bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 -- 172.21.15.102:0/3823033104 >> 172.21.15.102:0/3823033104 conn(0x7f8ad40685f0 msgr2=0x7f8ad406ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:01.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 -- 172.21.15.102:0/3823033104 shutdown_connections 2024-04-20T21:17:01.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.222+0000 7f8ad9800640 1 -- 172.21.15.102:0/3823033104 wait complete. 2024-04-20T21:17:01.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.224+0000 7f8ad9800640 1 Processor -- start 2024-04-20T21:17:01.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.225+0000 7f8ad9800640 1 -- start start 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 0x7f8ad414adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 0x7f8ad415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ad4155690 0x7f8ad415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40cb820 con 0x7f8ad4155690 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad9800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40d16b0 con 0x7f8ad4075710 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad9800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad40cb9a0 con 0x7f8ad40cb3e0 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ad4155690 0x7f8ad415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ad4155690 0x7f8ad415b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33002/0 (socket says 172.21.15.102:33002) 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad3400640 1 -- 172.21.15.102:0/4243681403 learned_addr learned my addr 172.21.15.102:0/4243681403 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8ad2a00640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 0x7f8ad414adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.226+0000 7f8aca000640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 0x7f8ad415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad3400640 1 -- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 msgr2=0x7f8ad415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad3400640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 0x7f8ad415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad3400640 1 -- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 msgr2=0x7f8ad414adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad3400640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 0x7f8ad414adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad3400640 1 -- 172.21.15.102:0/4243681403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad41f18f0 con 0x7f8ad4155690 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad2a00640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 0x7f8ad414adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad3400640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ad4155690 0x7f8ad415b4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8acc02c4d0 tx=0x7f8acc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8acb400640 1 -- 172.21.15.102:0/4243681403 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc044020 con 0x7f8ad4155690 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8acb400640 1 -- 172.21.15.102:0/4243681403 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8acc03f470 con 0x7f8ad4155690 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8acb400640 1 -- 172.21.15.102:0/4243681403 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc039040 con 0x7f8ad4155690 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad41f1bd0 con 0x7f8ad4155690 2024-04-20T21:17:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.227+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad41e4fe0 con 0x7f8ad4155690 2024-04-20T21:17:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.229+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad4074340 con 0x7f8ad4155690 2024-04-20T21:17:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.229+0000 7f8acb400640 1 -- 172.21.15.102:0/4243681403 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8acc047020 con 0x7f8ad4155690 2024-04-20T21:17:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.230+0000 7f8acb400640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ab006c8f0 0x7f8ab006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.230+0000 7f8acb400640 1 -- 172.21.15.102:0/4243681403 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8acc0aec60 con 0x7f8ad4155690 2024-04-20T21:17:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.231+0000 7f8ad2a00640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ab006c8f0 0x7f8ab006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.231+0000 7f8ad2a00640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ab006c8f0 0x7f8ab006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009ea0 tx=0x7f8ac4009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:01.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.236+0000 7f8acb400640 1 -- 172.21.15.102:0/4243681403 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8acc04e020 con 0x7f8ad4155690 2024-04-20T21:17:01.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.476+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ad40cae90 con 0x7f8ad4155690 2024-04-20T21:17:01.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.476+0000 7f8acb400640 1 -- 172.21.15.102:0/4243681403 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+6855 (secure 0 0 0) 0x7f8acc02ce20 con 0x7f8ad4155690 2024-04-20T21:17:01.475 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:01.475 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:00.427363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:17:01.476 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 6 2024-04-20T21:17:01.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.478+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ab006c8f0 msgr2=0x7f8ab006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.478+0000 7f8ad9800640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ab006c8f0 0x7f8ab006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009ea0 tx=0x7f8ac4009240 comp rx=0 tx=0).stop 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.478+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ad4155690 msgr2=0x7f8ad415b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.478+0000 7f8ad9800640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ad4155690 0x7f8ad415b4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8acc02c4d0 tx=0x7f8acc004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.479+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 shutdown_connections 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.479+0000 7f8ad9800640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ad4155690 0x7f8ad415b4b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.479+0000 7f8ad9800640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad40cb3e0 0x7f8ad415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.479+0000 7f8ad9800640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ab006c8f0 0x7f8ab006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.479+0000 7f8ad9800640 1 --2- 172.21.15.102:0/4243681403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad4075710 0x7f8ad414adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.479+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 >> 172.21.15.102:0/4243681403 conn(0x7f8ad40685f0 msgr2=0x7f8ad4150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.479+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 shutdown_connections 2024-04-20T21:17:01.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.479+0000 7f8ad9800640 1 -- 172.21.15.102:0/4243681403 wait complete. 2024-04-20T21:17:01.497 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:01.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.690+0000 7f7e44a00640 1 Processor -- start 2024-04-20T21:17:01.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.690+0000 7f7e44a00640 1 -- start start 2024-04-20T21:17:01.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e44a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e4015be30 0x7f7e400ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e44a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7e40073230 0x7f7e400736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e44a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e4015e7b0 con 0x7f7e40155560 2024-04-20T21:17:01.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e44a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f7e4015f0c0 con 0x7f7e40073230 2024-04-20T21:17:01.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e44a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7e40155940 con 0x7f7e4015be30 2024-04-20T21:17:01.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e3de00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7e40155560 0x7f7e400c5cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43334/0 (socket says 172.21.15.102:43334) 2024-04-20T21:17:01.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e3de00640 1 -- 172.21.15.102:0/2309279886 learned_addr learned my addr 172.21.15.102:0/2309279886 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e3e800640 1 --2- 172.21.15.102:0/2309279886 >> v2:172.21.15.102:3300/0 conn(0x7f7e40073230 0x7f7e400736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e3d400640 1 --2- 172.21.15.102:0/2309279886 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e4015be30 0x7f7e400ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e3ca00640 1 -- 172.21.15.102:0/2309279886 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 435127870 0 0) 0x7f7e4015e7b0 con 0x7f7e40155560 2024-04-20T21:17:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.692+0000 7f7e3ca00640 1 -- 172.21.15.102:0/2309279886 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e24003520 con 0x7f7e40155560 2024-04-20T21:17:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.693+0000 7f7e3e800640 1 -- 172.21.15.102:0/2309279886 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e4015be30 msgr2=0x7f7e400ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.693+0000 7f7e3e800640 1 --2- 172.21.15.102:0/2309279886 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e4015be30 0x7f7e400ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.693+0000 7f7e3e800640 1 -- 172.21.15.102:0/2309279886 >> v1:172.21.15.102:6789/0 conn(0x7f7e40155560 legacy=0x7f7e400c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.693+0000 7f7e3e800640 1 -- 172.21.15.102:0/2309279886 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e400cbea0 con 0x7f7e40073230 2024-04-20T21:17:01.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.694+0000 7f7e3e800640 1 --2- 172.21.15.102:0/2309279886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e40073230 0x7f7e400736a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e380097a0 tx=0x7f7e3802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ad63f3b4451765 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:01.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.694+0000 7f7e3ca00640 1 -- 172.21.15.102:0/2309279886 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38034070 con 0x7f7e40073230 2024-04-20T21:17:01.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.694+0000 7f7e3ca00640 1 -- 172.21.15.102:0/2309279886 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e38030800 con 0x7f7e40073230 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.694+0000 7f7e3ca00640 1 -- 172.21.15.102:0/2309279886 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38030bd0 con 0x7f7e40073230 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.694+0000 7f7e44a00640 1 -- 172.21.15.102:0/2309279886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e40073230 msgr2=0x7f7e400736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.694+0000 7f7e44a00640 1 --2- 172.21.15.102:0/2309279886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e40073230 0x7f7e400736a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e380097a0 tx=0x7f7e3802b940 comp rx=0 tx=0).stop 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.695+0000 7f7e44a00640 1 -- 172.21.15.102:0/2309279886 shutdown_connections 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.695+0000 7f7e44a00640 1 --2- 172.21.15.102:0/2309279886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e40073230 0x7f7e400736a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.695+0000 7f7e44a00640 1 --2- 172.21.15.102:0/2309279886 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e4015be30 0x7f7e400ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.695+0000 7f7e44a00640 1 -- 172.21.15.102:0/2309279886 >> 172.21.15.102:0/2309279886 conn(0x7f7e4006cd40 msgr2=0x7f7e4006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.695+0000 7f7e44a00640 1 -- 172.21.15.102:0/2309279886 shutdown_connections 2024-04-20T21:17:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.695+0000 7f7e44a00640 1 -- 172.21.15.102:0/2309279886 wait complete. 2024-04-20T21:17:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.696+0000 7f7e44a00640 1 Processor -- start 2024-04-20T21:17:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.696+0000 7f7e44a00640 1 -- start start 2024-04-20T21:17:01.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.697+0000 7f7e44a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e40073230 0x7f7e4014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.697+0000 7f7e44a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e40155560 0x7f7e4014b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.697+0000 7f7e44a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e4015be30 0x7f7e4015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.697+0000 7f7e44a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4015ed50 con 0x7f7e4015be30 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.697+0000 7f7e44a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4015ea70 con 0x7f7e40155560 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.697+0000 7f7e44a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7e400cbb60 con 0x7f7e40073230 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e40073230 0x7f7e4014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e40155560 0x7f7e4014b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e4015be30 0x7f7e4015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e40073230 0x7f7e4014b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35712/0 (socket says 172.21.15.102:35712) 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 -- 172.21.15.102:0/595107049 learned_addr learned my addr 172.21.15.102:0/595107049 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 -- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e40155560 msgr2=0x7f7e4014b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e40155560 0x7f7e4014b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 -- 172.21.15.102:0/595107049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e4015be30 msgr2=0x7f7e4015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e4015be30 0x7f7e4015cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 -- 172.21.15.102:0/595107049 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e400b4ca0 con 0x7f7e40073230 2024-04-20T21:17:01.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3e800640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e4015be30 0x7f7e4015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:01.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e3de00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e40073230 0x7f7e4014b400 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000cb40 tx=0x7f7e3000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:01.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.698+0000 7f7e36000640 1 -- 172.21.15.102:0/595107049 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30017070 con 0x7f7e40073230 2024-04-20T21:17:01.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.699+0000 7f7e36000640 1 -- 172.21.15.102:0/595107049 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e3000aca0 con 0x7f7e40073230 2024-04-20T21:17:01.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.699+0000 7f7e36000640 1 -- 172.21.15.102:0/595107049 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30012620 con 0x7f7e40073230 2024-04-20T21:17:01.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.699+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e401f1830 con 0x7f7e40073230 2024-04-20T21:17:01.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.699+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e401f1b10 con 0x7f7e40073230 2024-04-20T21:17:01.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.700+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e400b46e0 con 0x7f7e40073230 2024-04-20T21:17:01.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.701+0000 7f7e36000640 1 -- 172.21.15.102:0/595107049 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e3001a070 con 0x7f7e40073230 2024-04-20T21:17:01.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.701+0000 7f7e36000640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e2006c8f0 0x7f7e2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:01.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.702+0000 7f7e36000640 1 -- 172.21.15.102:0/595107049 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7e3008bc20 con 0x7f7e40073230 2024-04-20T21:17:01.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.702+0000 7f7e3d400640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e2006c8f0 0x7f7e2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:01.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.702+0000 7f7e3d400640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e2006c8f0 0x7f7e2006edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e400c5b10 tx=0x7f7e2c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:01.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.707+0000 7f7e36000640 1 -- 172.21.15.102:0/595107049 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7e400b46e0 con 0x7f7e40073230 2024-04-20T21:17:01.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.893+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e40074340 con 0x7f7e40073230 2024-04-20T21:17:01.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.894+0000 7f7e36000640 1 -- 172.21.15.102:0/595107049 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f7e300596b0 con 0x7f7e40073230 2024-04-20T21:17:01.894 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:01.894 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e2006c8f0 msgr2=0x7f7e2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e2006c8f0 0x7f7e2006edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e400c5b10 tx=0x7f7e2c009190 comp rx=0 tx=0).stop 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e40073230 msgr2=0x7f7e4014b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e40073230 0x7f7e4014b400 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000cb40 tx=0x7f7e3000a060 comp rx=0 tx=0).stop 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 shutdown_connections 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e4015be30 0x7f7e4015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e2006c8f0 0x7f7e2006edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e40155560 0x7f7e4014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 --2- 172.21.15.102:0/595107049 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e40073230 0x7f7e4014b400 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 >> 172.21.15.102:0/595107049 conn(0x7f7e4006cd40 msgr2=0x7f7e400cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:01.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 shutdown_connections 2024-04-20T21:17:01.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:01.897+0000 7f7e44a00640 1 -- 172.21.15.102:0/595107049 wait complete. 2024-04-20T21:17:02.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:16:54.017441+0000', 'modified': '2024-04-20T21:17:00.427363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.102:6835/2873333699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 2873333699}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 2873333699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4300, 'qdb_cluster': [4300]} 2024-04-20T21:17:02.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:17:02.518 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T21:17:02.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.733+0000 7f31d7000640 1 Processor -- start 2024-04-20T21:17:02.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.733+0000 7f31d7000640 1 -- start start 2024-04-20T21:17:02.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.734+0000 7f31d7000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31d00c3700 0x7f31d00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.734+0000 7f31d7000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d015c220 con 0x7f31d00c8330 2024-04-20T21:17:02.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.734+0000 7f31d7000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d00c4560 con 0x7f31d0152290 2024-04-20T21:17:02.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.734+0000 7f31d7000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f31d00c9180 con 0x7f31d00c3700 2024-04-20T21:17:02.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.735+0000 7f31d5200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31d00c3700 0x7f31d00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.735+0000 7f31d6600640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f31d00c8330 0x7f31d0162650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43336/0 (socket says 172.21.15.102:43336) 2024-04-20T21:17:02.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.735+0000 7f31d6600640 1 -- 172.21.15.102:0/3893979111 learned_addr learned my addr 172.21.15.102:0/3893979111 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:02.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.735+0000 7f31d5200640 1 -- 172.21.15.102:0/3893979111 >> v1:172.21.15.193:6789/0 conn(0x7f31d0152290 legacy=0x7f31d006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:02.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.735+0000 7f31d5200640 1 -- 172.21.15.102:0/3893979111 >> v1:172.21.15.102:6789/0 conn(0x7f31d00c8330 legacy=0x7f31d0162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:02.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.735+0000 7f31d5200640 1 -- 172.21.15.102:0/3893979111 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d015b460 con 0x7f31d00c3700 2024-04-20T21:17:02.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.735+0000 7f31cfe00640 1 -- 172.21.15.102:0/3893979111 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3017323653 0 0) 0x7f31d00c4560 con 0x7f31d0152290 2024-04-20T21:17:02.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.736+0000 7f31d5200640 1 --2- 172.21.15.102:0/3893979111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d00c3700 0x7f31d00cd310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f31bc0097a0 tx=0x7f31bc02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8889fd0a149a46d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:02.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.736+0000 7f31cfe00640 1 -- 172.21.15.102:0/3893979111 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc038070 con 0x7f31d00c3700 2024-04-20T21:17:02.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31cfe00640 1 -- 172.21.15.102:0/3893979111 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31bc031700 con 0x7f31d00c3700 2024-04-20T21:17:02.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31cfe00640 1 -- 172.21.15.102:0/3893979111 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc031ad0 con 0x7f31d00c3700 2024-04-20T21:17:02.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31d7000640 1 -- 172.21.15.102:0/3893979111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d00c3700 msgr2=0x7f31d00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:02.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31d7000640 1 --2- 172.21.15.102:0/3893979111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d00c3700 0x7f31d00cd310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f31bc0097a0 tx=0x7f31bc02b820 comp rx=0 tx=0).stop 2024-04-20T21:17:02.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31d7000640 1 -- 172.21.15.102:0/3893979111 shutdown_connections 2024-04-20T21:17:02.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31d7000640 1 --2- 172.21.15.102:0/3893979111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d00c3700 0x7f31d00cd310 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31d7000640 1 -- 172.21.15.102:0/3893979111 >> 172.21.15.102:0/3893979111 conn(0x7f31d00cc820 msgr2=0x7f31d00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:02.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31d7000640 1 -- 172.21.15.102:0/3893979111 shutdown_connections 2024-04-20T21:17:02.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.737+0000 7f31d7000640 1 -- 172.21.15.102:0/3893979111 wait complete. 2024-04-20T21:17:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.739+0000 7f31d7000640 1 Processor -- start 2024-04-20T21:17:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.739+0000 7f31d7000640 1 -- start start 2024-04-20T21:17:02.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d7000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31d00c3700 0x7f31d01eee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d7000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d00c8330 0x7f31d00cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d7000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d0152290 0x7f31d0072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d7000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31d015ba20 con 0x7f31d0152290 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d7000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31d00c4840 con 0x7f31d00c8330 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d7000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f31d015bd00 con 0x7f31d00c3700 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d0152290 0x7f31d0072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d0152290 0x7f31d0072ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33040/0 (socket says 172.21.15.102:33040) 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d6600640 1 -- 172.21.15.102:0/2675822057 learned_addr learned my addr 172.21.15.102:0/2675822057 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d5200640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d00c8330 0x7f31d00cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.740+0000 7f31d5c00640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31d00c3700 0x7f31d01eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d6600640 1 -- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31d00c3700 msgr2=0x7f31d01eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d6600640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31d00c3700 0x7f31d01eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d6600640 1 -- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d00c8330 msgr2=0x7f31d00cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d6600640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d00c8330 0x7f31d00cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d6600640 1 -- 172.21.15.102:0/2675822057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d01ef5a0 con 0x7f31d0152290 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d5c00640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31d00c3700 0x7f31d01eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:02.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d6600640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d0152290 0x7f31d0072ef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31c800caf0 tx=0x7f31c80020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:02.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31cd600640 1 -- 172.21.15.102:0/2675822057 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8018070 con 0x7f31d0152290 2024-04-20T21:17:02.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31cd600640 1 -- 172.21.15.102:0/2675822057 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c800a760 con 0x7f31d0152290 2024-04-20T21:17:02.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31cd600640 1 -- 172.21.15.102:0/2675822057 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c800ab90 con 0x7f31d0152290 2024-04-20T21:17:02.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d7000640 1 -- 172.21.15.102:0/2675822057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d01ef880 con 0x7f31d0152290 2024-04-20T21:17:02.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.741+0000 7f31d7000640 1 -- 172.21.15.102:0/2675822057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d00726f0 con 0x7f31d0152290 2024-04-20T21:17:02.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.743+0000 7f31cd600640 1 -- 172.21.15.102:0/2675822057 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31c80070d0 con 0x7f31d0152290 2024-04-20T21:17:02.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.744+0000 7f31cd600640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31b006c920 0x7f31b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.744+0000 7f31cd600640 1 -- 172.21.15.102:0/2675822057 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f31c808b650 con 0x7f31d0152290 2024-04-20T21:17:02.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.745+0000 7f31d5c00640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31b006c920 0x7f31b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.745+0000 7f31db401740 1 Processor -- start 2024-04-20T21:17:02.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.745+0000 7f31d5c00640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31b006c920 0x7f31b006ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31c4000f60 tx=0x7f31c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:02.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.745+0000 7f31db401740 1 -- start start 2024-04-20T21:17:02.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31db401740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1abd95f0 0x55ec1abdbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31db401740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 0x55ec1abd99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31db401740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac07270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31db401740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ec1ab449d0 con 0x55ec1ac29d70 2024-04-20T21:17:02.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31db401740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec1aaeb270 con 0x55ec1abd95f0 2024-04-20T21:17:02.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31db401740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x55ec1ab40990 con 0x55ec1ac2c880 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac07270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31d5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1abd95f0 0x55ec1abdbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.746+0000 7f31d5200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 0x55ec1abd99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1abd95f0 0x55ec1abdbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:38782/0 (socket says 172.21.15.102:38782) 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d5c00640 1 -- 172.21.15.102:0/378068320 learned_addr learned my addr 172.21.15.102:0/378068320 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac07270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33048/0 (socket says 172.21.15.102:33048) 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d5c00640 1 -- 172.21.15.102:0/378068320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 msgr2=0x55ec1abd99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d5c00640 1 --2- 172.21.15.102:0/378068320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 0x55ec1abd99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d5c00640 1 -- 172.21.15.102:0/378068320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 msgr2=0x55ec1ac07270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d5c00640 1 --2- 172.21.15.102:0/378068320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac07270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d5c00640 1 -- 172.21.15.102:0/378068320 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1a8572d0 con 0x55ec1abd95f0 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d6600640 1 --2- 172.21.15.102:0/378068320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31d5c00640 1 --2- 172.21.15.102:0/378068320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1abd95f0 0x55ec1abdbee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f31c40051d0 tx=0x7f31c4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ce4ab3b1de7dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:02.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31c2000640 1 -- 172.21.15.102:0/378068320 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c400ee60 con 0x55ec1abd95f0 2024-04-20T21:17:02.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31c2000640 1 -- 172.21.15.102:0/378068320 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c4003800 con 0x55ec1abd95f0 2024-04-20T21:17:02.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.747+0000 7f31c2000640 1 -- 172.21.15.102:0/378068320 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4003bb0 con 0x55ec1abd95f0 2024-04-20T21:17:02.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 -- 172.21.15.102:0/378068320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1abd95f0 msgr2=0x55ec1abdbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:02.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 --2- 172.21.15.102:0/378068320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1abd95f0 0x55ec1abdbee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f31c40051d0 tx=0x7f31c4011000 comp rx=0 tx=0).stop 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 -- 172.21.15.102:0/378068320 shutdown_connections 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 --2- 172.21.15.102:0/378068320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 --2- 172.21.15.102:0/378068320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 0x55ec1abd99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 --2- 172.21.15.102:0/378068320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1abd95f0 0x55ec1abdbee0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 -- 172.21.15.102:0/378068320 >> 172.21.15.102:0/378068320 conn(0x55ec1abdad60 msgr2=0x55ec1abdd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 -- 172.21.15.102:0/378068320 shutdown_connections 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.748+0000 7f31db401740 1 -- 172.21.15.102:0/378068320 wait complete. 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.749+0000 7f31db401740 1 -- start start 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.749+0000 7f31db401740 1 Processor -- start 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31db401740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 0x55ec1ac22a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31db401740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 0x55ec1ac35650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31db401740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac28e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31db401740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ec1aafe5e0 con 0x55ec1ac37c60 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31db401740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec1ac2cd50 con 0x55ec1ac2c880 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31db401740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x55ec1ac37050 con 0x55ec1ac29d70 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31db401740 10 client.0 _ll_register_callbacks cb 0x55ec1a856ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac28e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac28e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33052/0 (socket says 172.21.15.102:33052) 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31d6600640 1 -- 172.21.15.102:0/358330015 learned_addr learned my addr 172.21.15.102:0/358330015 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31d5200640 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 0x55ec1ac35650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.750+0000 7f31d5c00640 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 0x55ec1ac22a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31d6600640 1 -- 172.21.15.102:0/358330015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 msgr2=0x55ec1ac22a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31d6600640 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 0x55ec1ac22a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31d6600640 1 -- 172.21.15.102:0/358330015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 msgr2=0x55ec1ac35650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31d6600640 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 0x55ec1ac35650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31d6600640 1 -- 172.21.15.102:0/358330015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec1a9216b0 con 0x55ec1ac37c60 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31d6600640 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac28e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31c8058f50 tx=0x7f31c805d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9339c0d11c5f11f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31b6a00640 1 -- 172.21.15.102:0/358330015 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8058610 con 0x55ec1ac37c60 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31b6a00640 1 -- 172.21.15.102:0/358330015 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f31c8054d10 con 0x55ec1ac37c60 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31db401740 1 -- 172.21.15.102:0/358330015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4003800 con 0x55ec1ac37c60 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31b6a00640 1 -- 172.21.15.102:0/358330015 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f31c8054f10 con 0x55ec1ac37c60 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31db401740 10 client.4350 fetch_fsmap learned FSMap version 6 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31db401740 1 -- 172.21.15.102:0/358330015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55ec1ac35b90 con 0x55ec1ac37c60 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.751+0000 7f31b6a00640 1 -- 172.21.15.102:0/358330015 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c4003800 con 0x55ec1ac37c60 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31b6a00640 1 -- 172.21.15.102:0/358330015 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 6) ==== 4713+0+0 (secure 0 0 0) 0x7f31c8057e80 con 0x55ec1ac37c60 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31db401740 10 client.4350 fetch_fsmap finished waiting for FSMap version 6 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31db401740 10 client.4350 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:17:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31db401740 20 client.4350 populate_metadata read hostname 'smithi102' 2024-04-20T21:17:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31db401740 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x55ec1ac23dc0 0x55ec1ac43170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31db401740 4 client.4350 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:17:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31db401740 1 -- 172.21.15.102:0/358330015 --> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec1ab523f0 con 0x55ec1ac23dc0 2024-04-20T21:17:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31d5c00640 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x55ec1ac23dc0 0x55ec1ac43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.752+0000 7f31d5c00640 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x55ec1ac23dc0 0x55ec1ac43170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5a07e9ee15642d server_cookie=f1b2c83c5a2bb4e in_seq=0 out_seq=0 2024-04-20T21:17:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:02.753+0000 7f31b6a00640 10 client.4350 ms_handle_connect on v2:172.21.15.102:6834/2873333699 2024-04-20T21:17:05.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.434+0000 7f31b6a00640 1 -- 172.21.15.102:0/358330015 <== mds.0 v2:172.21.15.102:6834/2873333699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec1ab523f0 con 0x55ec1ac23dc0 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.434+0000 7f31b6a00640 10 client.4350 handle_command_reply: tid=0 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 client.4350 shutdown 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 -- 172.21.15.102:0/358330015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 msgr2=0x55ec1ac28e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac28e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31c8058f50 tx=0x7f31c805d3d0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 -- 172.21.15.102:0/358330015 shutdown_connections 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x55ec1ac23dc0 0x55ec1ac43170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac28e90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 0x55ec1ac35650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 --2- 172.21.15.102:0/358330015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 0x55ec1ac22a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31db401740 1 -- 172.21.15.102:0/358330015 >> 172.21.15.102:0/358330015 conn(0x55ec1abd95f0 msgr2=0x55ec1abd8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.435+0000 7f31b6a00640 0 client.4350 ms_handle_reset on v2:172.21.15.102:6834/2873333699 2024-04-20T21:17:05.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.436+0000 7f31db401740 1 -- 172.21.15.102:0/358330015 shutdown_connections 2024-04-20T21:17:05.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.436+0000 7f31db401740 1 -- 172.21.15.102:0/358330015 wait complete. 2024-04-20T21:17:05.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.436+0000 7f31db401740 20 client.4350 trim_cache size 0 max 16384 2024-04-20T21:17:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.457+0000 7f31db401740 1 Processor -- start 2024-04-20T21:17:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.457+0000 7f31db401740 1 -- start start 2024-04-20T21:17:05.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.458+0000 7f31db401740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac294b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.458+0000 7f31db401740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 0x55ec1abdbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.458+0000 7f31db401740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac37c60 0x55ec1abdc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.458+0000 7f31db401740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31d015b0c0 con 0x55ec1ac29d70 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.458+0000 7f31db401740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec1ac24e50 con 0x55ec1ac37c60 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.458+0000 7f31db401740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x55ec1ac254b0 con 0x55ec1ac2c880 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac294b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac294b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33056/0 (socket says 172.21.15.102:33056) 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 -- 172.21.15.102:0/1094303926 learned_addr learned my addr 172.21.15.102:0/1094303926 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5200640 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 0x55ec1abdbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d6600640 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac37c60 0x55ec1abdc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 -- 172.21.15.102:0/1094303926 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 msgr2=0x55ec1abdbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 0x55ec1abdbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 -- 172.21.15.102:0/1094303926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac37c60 msgr2=0x55ec1abdc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac37c60 0x55ec1abdc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 -- 172.21.15.102:0/1094303926 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1abdcea0 con 0x55ec1ac29d70 2024-04-20T21:17:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.459+0000 7f31d5c00640 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac294b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f31c4007e60 tx=0x7f31c4017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219ab419e0e1c1a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.460+0000 7f31c1600640 1 -- 172.21.15.102:0/1094303926 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4032070 con 0x55ec1ac29d70 2024-04-20T21:17:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.460+0000 7f31c1600640 1 -- 172.21.15.102:0/1094303926 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c4003800 con 0x55ec1ac29d70 2024-04-20T21:17:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.460+0000 7f31c1600640 1 -- 172.21.15.102:0/1094303926 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c402d5f0 con 0x55ec1ac29d70 2024-04-20T21:17:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 -- 172.21.15.102:0/1094303926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 msgr2=0x55ec1ac294b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac294b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f31c4007e60 tx=0x7f31c4017070 comp rx=0 tx=0).stop 2024-04-20T21:17:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 -- 172.21.15.102:0/1094303926 shutdown_connections 2024-04-20T21:17:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac37c60 0x55ec1abdc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac2c880 0x55ec1abdbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 --2- 172.21.15.102:0/1094303926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac29d70 0x55ec1ac294b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 -- 172.21.15.102:0/1094303926 >> 172.21.15.102:0/1094303926 conn(0x55ec1abd95f0 msgr2=0x55ec1ac22860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 -- 172.21.15.102:0/1094303926 shutdown_connections 2024-04-20T21:17:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.461+0000 7f31db401740 1 -- 172.21.15.102:0/1094303926 wait complete. 2024-04-20T21:17:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.462+0000 7f31db401740 1 -- start start 2024-04-20T21:17:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.462+0000 7f31db401740 1 Processor -- start 2024-04-20T21:17:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31db401740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 0x55ec1abdc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31db401740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 0x55ec1ac3a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31db401740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac3bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31db401740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ec1ac23c40 con 0x55ec1ac37c60 2024-04-20T21:17:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31db401740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec1ac35df0 con 0x55ec1ac2c880 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31db401740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x55ec1ac24290 con 0x55ec1ac29d70 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31db401740 10 client.0 _ll_register_callbacks cb 0x7f31c8058e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac3bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac3bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33058/0 (socket says 172.21.15.102:33058) 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31d6600640 1 -- 172.21.15.102:0/1683781586 learned_addr learned my addr 172.21.15.102:0/1683781586 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31d5200640 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 0x55ec1ac3a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.463+0000 7f31d5c00640 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 0x55ec1abdc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31d6600640 1 -- 172.21.15.102:0/1683781586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 msgr2=0x55ec1abdc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31d6600640 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 0x55ec1abdc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31d6600640 1 -- 172.21.15.102:0/1683781586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 msgr2=0x55ec1ac3a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31d6600640 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 0x55ec1ac3a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31d6600640 1 -- 172.21.15.102:0/1683781586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec1abdf200 con 0x55ec1ac37c60 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31d6600640 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac3bfb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31c8061650 tx=0x7f31c8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c01c36d1f83f4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31b6a00640 1 -- 172.21.15.102:0/1683781586 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8058fc0 con 0x55ec1ac37c60 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31b6a00640 1 -- 172.21.15.102:0/1683781586 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f31c8057e80 con 0x55ec1ac37c60 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.464+0000 7f31db401740 1 -- 172.21.15.102:0/1683781586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4003800 con 0x55ec1ac37c60 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31b6a00640 1 -- 172.21.15.102:0/1683781586 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f31c80782d0 con 0x55ec1ac37c60 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31db401740 10 client.4356 fetch_fsmap learned FSMap version 6 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31db401740 1 -- 172.21.15.102:0/1683781586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55ec1abde450 con 0x55ec1ac37c60 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31b6a00640 1 -- 172.21.15.102:0/1683781586 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c4003800 con 0x55ec1ac37c60 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31b6a00640 1 -- 172.21.15.102:0/1683781586 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 6) ==== 4713+0+0 (secure 0 0 0) 0x7f31c8078470 con 0x55ec1ac37c60 2024-04-20T21:17:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31db401740 10 client.4356 fetch_fsmap finished waiting for FSMap version 6 2024-04-20T21:17:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31db401740 10 client.4356 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:17:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31db401740 20 client.4356 populate_metadata read hostname 'smithi102' 2024-04-20T21:17:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31db401740 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x55ec1aca2f50 0x55ec1acaba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31db401740 4 client.4356 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:17:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31db401740 1 -- 172.21.15.102:0/1683781586 --> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] -- command(tid 0: {"prefix": "status"}) -- 0x55ec1ab4c260 con 0x55ec1aca2f50 2024-04-20T21:17:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.465+0000 7f31d5c00640 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x55ec1aca2f50 0x55ec1acaba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.466+0000 7f31d5c00640 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x55ec1aca2f50 0x55ec1acaba10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6158332fc5b1c653 server_cookie=9e1350ee94021fc7 in_seq=0 out_seq=0 2024-04-20T21:17:05.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.466+0000 7f31b6a00640 10 client.4356 ms_handle_connect on v2:172.21.15.102:6834/2873333699 2024-04-20T21:17:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.466+0000 7f31b6a00640 1 -- 172.21.15.102:0/1683781586 <== mds.0 v2:172.21.15.102:6834/2873333699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55ec1ab4c260 con 0x55ec1aca2f50 2024-04-20T21:17:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.466+0000 7f31b6a00640 10 client.4356 handle_command_reply: tid=0 2024-04-20T21:17:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.466+0000 7f31db401740 1 client.4356 shutdown 2024-04-20T21:17:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.466+0000 7f31db401740 1 -- 172.21.15.102:0/1683781586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 msgr2=0x55ec1ac3bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.466+0000 7f31db401740 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac3bfb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31c8061650 tx=0x7f31c8015040 comp rx=0 tx=0).stop 2024-04-20T21:17:05.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 1 -- 172.21.15.102:0/1683781586 shutdown_connections 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x55ec1aca2f50 0x55ec1acaba10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec1ac37c60 0x55ec1ac3bfb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec1ac2c880 0x55ec1ac3a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 1 --2- 172.21.15.102:0/1683781586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55ec1ac29d70 0x55ec1abdc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31b6a00640 0 client.4356 ms_handle_reset on v2:172.21.15.102:6834/2873333699 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 1 -- 172.21.15.102:0/1683781586 >> 172.21.15.102:0/1683781586 conn(0x55ec1abd95f0 msgr2=0x55ec1ac371e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 1 -- 172.21.15.102:0/1683781586 shutdown_connections 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 1 -- 172.21.15.102:0/1683781586 wait complete. 2024-04-20T21:17:05.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.467+0000 7f31db401740 20 client.4356 trim_cache size 0 max 16384 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "1c72cacb-b320-4a89-8b72-ff890245905f", 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 0, 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 4300, 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 10.907237417999999, 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 6, 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 24, 2024-04-20T21:17:05.472 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 24, 2024-04-20T21:17:05.473 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 16.643277869999999 2024-04-20T21:17:05.473 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-20T21:17:05.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 -- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31b006c920 msgr2=0x7f31b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31b006c920 0x7f31b006ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31c4000f60 tx=0x7f31c4009040 comp rx=0 tx=0).stop 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 -- 172.21.15.102:0/2675822057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d0152290 msgr2=0x7f31d0072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d0152290 0x7f31d0072ef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31c800caf0 tx=0x7f31c80020f0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 -- 172.21.15.102:0/2675822057 shutdown_connections 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d0152290 0x7f31d0072ef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f31b006c920 0x7f31b006ede0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d00c8330 0x7f31d00cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 --2- 172.21.15.102:0/2675822057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31d00c3700 0x7f31d01eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 -- 172.21.15.102:0/2675822057 >> 172.21.15.102:0/2675822057 conn(0x7f31d00cc820 msgr2=0x7f31d006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 -- 172.21.15.102:0/2675822057 shutdown_connections 2024-04-20T21:17:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.468+0000 7f31d7000640 1 -- 172.21.15.102:0/2675822057 wait complete. 2024-04-20T21:17:05.487 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T21:17:05.487 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:05.487 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:17:05.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.670+0000 7f6cfd800640 1 Processor -- start 2024-04-20T21:17:05.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.670+0000 7f6cfd800640 1 -- start start 2024-04-20T21:17:05.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.672+0000 7f6cfd800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6cf8155530 0x7f6cf815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.672+0000 7f6cfd800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6cf80cb6a0 0x7f6cf80cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.672+0000 7f6cfd800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf815ef60 con 0x7f6cf80cc470 2024-04-20T21:17:05.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.672+0000 7f6cfd800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf80c5ca0 con 0x7f6cf8155530 2024-04-20T21:17:05.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.672+0000 7f6cfd800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf80cd5f0 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.673+0000 7f6cf7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6cf8155530 0x7f6cf815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.673+0000 7f6cf7400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6cf80cc470 0x7f6cf80cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43342/0 (socket says 172.21.15.102:43342) 2024-04-20T21:17:05.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.673+0000 7f6cf7400640 1 -- 172.21.15.102:0/1822284789 learned_addr learned my addr 172.21.15.102:0/1822284789 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:05.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.673+0000 7f6cfce00640 1 --2- 172.21.15.102:0/1822284789 >> v2:172.21.15.193:3300/0 conn(0x7f6cf80cb6a0 0x7f6cf80cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.673+0000 7f6cf7e00640 1 -- 172.21.15.102:0/1822284789 >> v2:172.21.15.193:3300/0 conn(0x7f6cf80cb6a0 msgr2=0x7f6cf80cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:05.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.673+0000 7f6cf7e00640 1 --2- 172.21.15.102:0/1822284789 >> v2:172.21.15.193:3300/0 conn(0x7f6cf80cb6a0 0x7f6cf80cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.673+0000 7f6cf7e00640 1 -- 172.21.15.102:0/1822284789 >> v1:172.21.15.102:6789/0 conn(0x7f6cf80cc470 legacy=0x7f6cf80cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.673+0000 7f6cf7e00640 1 -- 172.21.15.102:0/1822284789 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf815f240 con 0x7f6cf8155530 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.674+0000 7f6cf7e00640 1 --2- 172.21.15.102:0/1822284789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf8155530 0x7f6cf815e640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6cf808e290 tx=0x7f6ce802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3683b7c92949c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.674+0000 7f6cf6a00640 1 -- 172.21.15.102:0/1822284789 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8035020 con 0x7f6cf8155530 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.674+0000 7f6cf6a00640 1 -- 172.21.15.102:0/1822284789 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce8031830 con 0x7f6cf8155530 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.674+0000 7f6cf6a00640 1 -- 172.21.15.102:0/1822284789 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8031c00 con 0x7f6cf8155530 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.675+0000 7f6cfd800640 1 -- 172.21.15.102:0/1822284789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf8155530 msgr2=0x7f6cf815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.675+0000 7f6cfd800640 1 --2- 172.21.15.102:0/1822284789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf8155530 0x7f6cf815e640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6cf808e290 tx=0x7f6ce802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.675+0000 7f6cfd800640 1 -- 172.21.15.102:0/1822284789 shutdown_connections 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.675+0000 7f6cfd800640 1 --2- 172.21.15.102:0/1822284789 >> v2:172.21.15.193:3300/0 conn(0x7f6cf80cb6a0 0x7f6cf80cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.675+0000 7f6cfd800640 1 --2- 172.21.15.102:0/1822284789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf8155530 0x7f6cf815e640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.675+0000 7f6cfd800640 1 -- 172.21.15.102:0/1822284789 >> 172.21.15.102:0/1822284789 conn(0x7f6cf8075080 msgr2=0x7f6cf8075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:05.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.675+0000 7f6cfd800640 1 -- 172.21.15.102:0/1822284789 shutdown_connections 2024-04-20T21:17:05.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.675+0000 7f6cfd800640 1 -- 172.21.15.102:0/1822284789 wait complete. 2024-04-20T21:17:05.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.677+0000 7f6cfd800640 1 Processor -- start 2024-04-20T21:17:05.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.677+0000 7f6cfd800640 1 -- start start 2024-04-20T21:17:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.678+0000 7f6cfd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf80cb6a0 0x7f6cf8149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.678+0000 7f6cfd800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf80cc470 0x7f6cf815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.678+0000 7f6cfd800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf8155530 0x7f6cf81e6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.678+0000 7f6cfd800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf80cd280 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.678+0000 7f6cfd800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf80ccfa0 con 0x7f6cf8155530 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.678+0000 7f6cfd800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf80ccbe0 con 0x7f6cf80cc470 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf80cb6a0 0x7f6cf8149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf80cc470 0x7f6cf815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf80cb6a0 0x7f6cf8149b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33088/0 (socket says 172.21.15.102:33088) 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cfce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf8155530 0x7f6cf81e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 -- 172.21.15.102:0/3842649932 learned_addr learned my addr 172.21.15.102:0/3842649932 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 -- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf80cc470 msgr2=0x7f6cf815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf80cc470 0x7f6cf815e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 -- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf8155530 msgr2=0x7f6cf81e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf8155530 0x7f6cf81e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 -- 172.21.15.102:0/3842649932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf815d860 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cfce00640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf8155530 0x7f6cf81e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7400640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf80cc470 0x7f6cf815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.679+0000 7f6cf7e00640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf80cb6a0 0x7f6cf8149b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8000c00 tx=0x7f6ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.680+0000 7f6cefe00640 1 -- 172.21.15.102:0/3842649932 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8040070 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.680+0000 7f6cefe00640 1 -- 172.21.15.102:0/3842649932 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce803a910 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.680+0000 7f6cefe00640 1 -- 172.21.15.102:0/3842649932 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8035040 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.680+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf81e6c80 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.680+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf815dad0 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.681+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf804fd60 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.682+0000 7f6cefe00640 1 -- 172.21.15.102:0/3842649932 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ce804b060 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.683+0000 7f6cefe00640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6cd806c8f0 0x7f6cd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:05.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.683+0000 7f6cefe00640 1 -- 172.21.15.102:0/3842649932 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6ce80ad520 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.684+0000 7f6cf7400640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6cd806c8f0 0x7f6cd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:05.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.684+0000 7f6cf7400640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6cd806c8f0 0x7f6cd806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6cf814a430 tx=0x7f6ce4007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:05.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.689+0000 7f6cefe00640 1 -- 172.21.15.102:0/3842649932 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6ce8031800 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.921+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6cf80cc9c0 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.921+0000 7f6cefe00640 1 -- 172.21.15.102:0/3842649932 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7f6ce807b050 con 0x7f6cf80cb6a0 2024-04-20T21:17:05.921 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.923+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6cd806c8f0 msgr2=0x7f6cd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.923+0000 7f6cfd800640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6cd806c8f0 0x7f6cd806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6cf814a430 tx=0x7f6ce4007e80 comp rx=0 tx=0).stop 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf80cb6a0 msgr2=0x7f6cf8149b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf80cb6a0 0x7f6cf8149b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8000c00 tx=0x7f6ce80046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 shutdown_connections 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6cd806c8f0 0x7f6cd806edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf8155530 0x7f6cf81e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf80cc470 0x7f6cf815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 --2- 172.21.15.102:0/3842649932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cf80cb6a0 0x7f6cf8149b20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 >> 172.21.15.102:0/3842649932 conn(0x7f6cf8075080 msgr2=0x7f6cf80ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 shutdown_connections 2024-04-20T21:17:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:05.924+0000 7f6cfd800640 1 -- 172.21.15.102:0/3842649932 wait complete. 2024-04-20T21:17:05.940 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-20T21:17:05.941 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:17:05.941 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi102.front.sepia.ceph.com 2024-04-20T21:17:05.941 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-20T21:17:05.941 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:05.941 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-20T21:17:05.941 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:17:05.941 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:17:05.941 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-20T21:17:05.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:05.942 DEBUG:teuthology.orchestra.run.smithi102:> ip addr 2024-04-20T21:17:05.988 INFO:teuthology.orchestra.run.smithi102.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:17:05.988 INFO:teuthology.orchestra.run.smithi102.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:17:05.988 INFO:teuthology.orchestra.run.smithi102.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:17:05.988 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:17:05.988 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 ::1/128 scope host 2024-04-20T21:17:05.988 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dc brd ff:ff:ff:ff:ff:ff 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f0 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dd brd ff:ff:ff:ff:ff:ff 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f1 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:68 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:69 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout: inet 172.21.15.102/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:17:05.989 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft 42810sec preferred_lft 42810sec 2024-04-20T21:17:05.990 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::ec4:7aff:fe8f:d769/64 scope link 2024-04-20T21:17:05.990 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:17:05.990 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T21:17:05.990 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:05.990 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:05.991 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add name ceph-brx type bridge 2024-04-20T21:17:05.991 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr flush dev ceph-brx 2024-04-20T21:17:05.991 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set ceph-brx up 2024-04-20T21:17:05.991 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T21:17:05.991 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:06.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:06.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:06.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:06.499 DEBUG:teuthology.orchestra.run.smithi102:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:17:06.564 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-04-20T21:17:06.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:06.567 DEBUG:teuthology.orchestra.run.smithi102:> route 2024-04-20T21:17:06.645 INFO:teuthology.orchestra.run.smithi102.stdout:Kernel IP routing table 2024-04-20T21:17:06.645 INFO:teuthology.orchestra.run.smithi102.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:17:06.645 INFO:teuthology.orchestra.run.smithi102.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:17:06.645 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:17:06.645 INFO:teuthology.orchestra.run.smithi102.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:17:06.646 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:06.646 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:06.646 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:17:06.646 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:17:06.646 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:17:06.646 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:06.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:08.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:08.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:08.417 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-04-20T21:17:08.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:08.468 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list-id 2024-04-20T21:17:08.515 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:08.515 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:08.515 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T21:17:08.515 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-20T21:17:08.515 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:08.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:08.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:08.703 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-20T21:17:08.703 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:08.703 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:08.703 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-20T21:17:08.703 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T21:17:08.703 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-20T21:17:08.704 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-20T21:17:08.704 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-20T21:17:08.704 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:08.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:08.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:08.987 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:08.988 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:08.988 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set brx.0 up 2024-04-20T21:17:08.988 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T21:17:08.988 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:09.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:09.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:09.186 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-20T21:17:09.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:17:09.187 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-20T21:17:09.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:17:09.233 DEBUG:teuthology.orchestra.run.smithi102:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:09.283 DEBUG:teuthology.orchestra.run.smithi102:> sudo modprobe fuse 2024-04-20T21:17:09.367 DEBUG:teuthology.orchestra.run.smithi102:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:17:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:/proc 2024-04-20T21:17:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:/sys 2024-04-20T21:17:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:/dev 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/security 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/shm 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/pts 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/run 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/cgroup 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/pstore 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/bpf 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/ 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/selinux 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/hugepages 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/mqueue 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/debug 2024-04-20T21:17:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/tracing 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/fuse/connections 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/config 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/1000 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:17:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns 2024-04-20T21:17:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T21:17:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T21:17:09.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:09.424 DEBUG:teuthology.orchestra.run.smithi102:> ls /sys/fs/fuse/connections 2024-04-20T21:17:09.470 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:17:09.470 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-20T21:17:09.515 DEBUG:teuthology.orchestra.run.smithi102:> sudo modprobe fuse 2024-04-20T21:17:09.584 DEBUG:teuthology.orchestra.run.smithi102:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:17:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:/proc 2024-04-20T21:17:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:/sys 2024-04-20T21:17:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:/dev 2024-04-20T21:17:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/security 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/shm 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/pts 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/run 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/cgroup 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/pstore 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/bpf 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/ 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/selinux 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/hugepages 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/mqueue 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/debug 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/tracing 2024-04-20T21:17:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/fuse/connections 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/config 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/1000 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T21:17:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T21:17:09.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:09.626 DEBUG:teuthology.orchestra.run.smithi102:> ls /sys/fs/fuse/connections 2024-04-20T21:17:09.701 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi102.stderr:2024-04-20T21:17:09.701+0000 7fa3c6c38480 -1 init, newargv = 0x55f7e07534f0 newargc=15 2024-04-20T21:17:09.701 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi102.stderr:ceph-fuse[29433]: starting ceph client 2024-04-20T21:17:09.781 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:17:09.781+0000 7fd01f200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:17:09.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi102.stderr:ceph-fuse[29433]: starting fuse 2024-04-20T21:17:10.811 DEBUG:teuthology.orchestra.run.smithi102:> sudo modprobe fuse 2024-04-20T21:17:10.902 DEBUG:teuthology.orchestra.run.smithi102:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:17:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:/proc 2024-04-20T21:17:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:/sys 2024-04-20T21:17:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:/dev 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/security 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/shm 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/pts 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/run 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/cgroup 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/pstore 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/bpf 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/ 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/selinux 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/hugepages 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/mqueue 2024-04-20T21:17:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/debug 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/tracing 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/fuse/connections 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/config 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/1000 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns 2024-04-20T21:17:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T21:17:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T21:17:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:10.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:10.961 DEBUG:teuthology.orchestra.run.smithi102:> ls /sys/fs/fuse/connections 2024-04-20T21:17:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:40 2024-04-20T21:17:11.011 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> import glob 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> import re 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> import os 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> import subprocess 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> def _find_admin_socket(client_name): 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> files = glob.glob(asok_path) 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> # Given a non-glob path, it better be there 2024-04-20T21:17:11.012 DEBUG:teuthology.orchestra.run.smithi102:> if "*" not in asok_path: 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> assert(len(files) == 1) 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> return files[0] 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> for f in files: 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> contents = proc_f.read() 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> if mountpoint in contents: 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> return f 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> print(_find_admin_socket("client.admin")) 2024-04-20T21:17:11.013 DEBUG:teuthology.orchestra.run.smithi102:> ' 2024-04-20T21:17:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:/var/run/ceph/ceph-client.admin.29433.asok 2024-04-20T21:17:11.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:11.197 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.29433.asok 2024-04-20T21:17:11.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:11.197 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.29433.asok status 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "metadata": { 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "entity_id": "admin", 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "hostname": "smithi102", 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "pid": "29433", 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "root": "/" 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "dentry_count": 0, 2024-04-20T21:17:11.423 INFO:teuthology.orchestra.run.smithi102.stdout: "dentry_pinned_count": 0, 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 4374, 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "inst": { 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "name": { 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "client", 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "num": 4374 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": { 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 4028892217 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": { 2024-04-20T21:17:11.424 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 4028892217 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "inst_str": "client.4374 192.168.0.1:0/4028892217", 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "addr_str": "192.168.0.1:0/4028892217", 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "inode_count": 1, 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_epoch": 6, 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_epoch": 24, 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "blocklisted": false, 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs" 2024-04-20T21:17:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-20T21:17:11.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:11.437 DEBUG:teuthology.orchestra.run.smithi102:> stat --file-system '--printf=%T 2024-04-20T21:17:11.437 DEBUG:teuthology.orchestra.run.smithi102:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:fuseblk 2024-04-20T21:17:11.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:11.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:11.489 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:11.571 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-20T21:17:11.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:17:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:# file: . 2024-04-20T21:17:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-20T21:17:11.709 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:11.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:11.719 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:17:11.719 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:17:11.769 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:11.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.965+0000 7f6ddfa00640 1 Processor -- start 2024-04-20T21:17:11.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.965+0000 7f6ddfa00640 1 -- start start 2024-04-20T21:17:11.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddfa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd814fa50 0x7f6dd80c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddfa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6dd8073a10 0x7f6dd8158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddfa00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd81536d0 con 0x7f6dd80c9eb0 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddfa00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd8159500 con 0x7f6dd8073a10 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddfa00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd80c76c0 con 0x7f6dd814fa50 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddd000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd814fa50 0x7f6dd80c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6dd7e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6dd80c9eb0 0x7f6dd80c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55692/0 (socket says 172.21.15.102:55692) 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6dd7e00640 1 -- 172.21.15.102:0/1395579772 learned_addr learned my addr 172.21.15.102:0/1395579772 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddda00640 1 --2- 172.21.15.102:0/1395579772 >> v2:172.21.15.102:3300/0 conn(0x7f6dd8073a10 0x7f6dd8158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:11.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddd000640 1 -- 172.21.15.102:0/1395579772 >> v2:172.21.15.102:3300/0 conn(0x7f6dd8073a10 msgr2=0x7f6dd8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:11.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddd000640 1 --2- 172.21.15.102:0/1395579772 >> v2:172.21.15.102:3300/0 conn(0x7f6dd8073a10 0x7f6dd8158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:11.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6ddd000640 1 -- 172.21.15.102:0/1395579772 >> v1:172.21.15.193:6789/0 conn(0x7f6dd80c9eb0 legacy=0x7f6dd80c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:11.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.967+0000 7f6dd7400640 1 -- 172.21.15.102:0/1395579772 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357479732 0 0) 0x7f6dc4001110 con 0x7f6dd80c9eb0 2024-04-20T21:17:11.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.968+0000 7f6ddd000640 1 -- 172.21.15.102:0/1395579772 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd80c7960 con 0x7f6dd814fa50 2024-04-20T21:17:11.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.969+0000 7f6ddd000640 1 --2- 172.21.15.102:0/1395579772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd814fa50 0x7f6dd80c4830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0097a0 tx=0x7f6dcc02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37408e250166c1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:11.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.969+0000 7f6dd7400640 1 -- 172.21.15.102:0/1395579772 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc034070 con 0x7f6dd814fa50 2024-04-20T21:17:11.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.969+0000 7f6dd7400640 1 -- 172.21.15.102:0/1395579772 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc030800 con 0x7f6dd814fa50 2024-04-20T21:17:11.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.969+0000 7f6dd7400640 1 -- 172.21.15.102:0/1395579772 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc030bd0 con 0x7f6dd814fa50 2024-04-20T21:17:11.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.970+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1395579772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd814fa50 msgr2=0x7f6dd80c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:11.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.970+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1395579772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd814fa50 0x7f6dd80c4830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0097a0 tx=0x7f6dcc02b940 comp rx=0 tx=0).stop 2024-04-20T21:17:11.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.970+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1395579772 shutdown_connections 2024-04-20T21:17:11.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.970+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1395579772 >> v2:172.21.15.102:3300/0 conn(0x7f6dd8073a10 0x7f6dd8158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:11.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.970+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1395579772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd814fa50 0x7f6dd80c4830 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:11.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.970+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1395579772 >> 172.21.15.102:0/1395579772 conn(0x7f6dd80c9390 msgr2=0x7f6dd80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:11.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.970+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1395579772 shutdown_connections 2024-04-20T21:17:11.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.970+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1395579772 wait complete. 2024-04-20T21:17:11.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.972+0000 7f6ddfa00640 1 Processor -- start 2024-04-20T21:17:11.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.972+0000 7f6ddfa00640 1 -- start start 2024-04-20T21:17:11.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.973+0000 7f6ddfa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dd8073a10 0x7f6dd814a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddfa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd80c9eb0 0x7f6dd8156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddfa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dd814fa50 0x7f6dd814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddfa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd80c7c00 con 0x7f6dd8073a10 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddfa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd80c3d90 con 0x7f6dd814fa50 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddfa00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd80c4030 con 0x7f6dd80c9eb0 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dd8073a10 0x7f6dd814a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dd8073a10 0x7f6dd814a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49122/0 (socket says 172.21.15.102:49122) 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddd000640 1 -- 172.21.15.102:0/1583337718 learned_addr learned my addr 172.21.15.102:0/1583337718 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddda00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dd814fa50 0x7f6dd814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:11.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6dd7e00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd80c9eb0 0x7f6dd8156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:11.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddd000640 1 -- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd80c9eb0 msgr2=0x7f6dd8156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:11.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddd000640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd80c9eb0 0x7f6dd8156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:11.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddd000640 1 -- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dd814fa50 msgr2=0x7f6dd814e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:11.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddd000640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dd814fa50 0x7f6dd814e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:11.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.974+0000 7f6ddd000640 1 -- 172.21.15.102:0/1583337718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd81541b0 con 0x7f6dd8073a10 2024-04-20T21:17:11.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.975+0000 7f6ddd000640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dd8073a10 0x7f6dd814a9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0098c0 tx=0x7f6dcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:11.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.975+0000 7f6dd4c00640 1 -- 172.21.15.102:0/1583337718 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc034040 con 0x7f6dd8073a10 2024-04-20T21:17:11.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.975+0000 7f6dd4c00640 1 -- 172.21.15.102:0/1583337718 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc03ada0 con 0x7f6dd8073a10 2024-04-20T21:17:11.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.975+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd8154490 con 0x7f6dd8073a10 2024-04-20T21:17:11.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.975+0000 7f6dd4c00640 1 -- 172.21.15.102:0/1583337718 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc006e10 con 0x7f6dd8073a10 2024-04-20T21:17:11.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.975+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd8154a50 con 0x7f6dd8073a10 2024-04-20T21:17:11.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.976+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd8074340 con 0x7f6dd8073a10 2024-04-20T21:17:11.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.978+0000 7f6dd4c00640 1 -- 172.21.15.102:0/1583337718 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dcc037070 con 0x7f6dd8073a10 2024-04-20T21:17:11.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.979+0000 7f6dd4c00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6db806c8f0 0x7f6db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:11.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.979+0000 7f6dd4c00640 1 -- 172.21.15.102:0/1583337718 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6dcc0ae6f0 con 0x7f6dd8073a10 2024-04-20T21:17:11.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.979+0000 7f6dd7e00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6db806c8f0 0x7f6db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:11.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.979+0000 7f6dd7e00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6db806c8f0 0x7f6db806edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6dd814af30 tx=0x7f6dc4007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:11.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:11.984+0000 7f6dd4c00640 1 -- 172.21.15.102:0/1583337718 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6dcc03f070 con 0x7f6dd8073a10 2024-04-20T21:17:12.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.187+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6dd806d600 con 0x7f6dd8073a10 2024-04-20T21:17:12.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.190+0000 7f6dd4c00640 1 -- 172.21.15.102:0/1583337718 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f6dcc07c060 con 0x7f6dd8073a10 2024-04-20T21:17:12.190 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:12.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6db806c8f0 msgr2=0x7f6db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6db806c8f0 0x7f6db806edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6dd814af30 tx=0x7f6dc4007b50 comp rx=0 tx=0).stop 2024-04-20T21:17:12.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dd8073a10 msgr2=0x7f6dd814a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dd8073a10 0x7f6dd814a9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0098c0 tx=0x7f6dcc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 shutdown_connections 2024-04-20T21:17:12.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6db806c8f0 0x7f6db806edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dd814fa50 0x7f6dd814e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6dd80c9eb0 0x7f6dd8156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 --2- 172.21.15.102:0/1583337718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dd8073a10 0x7f6dd814a9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 >> 172.21.15.102:0/1583337718 conn(0x7f6dd80c9390 msgr2=0x7f6dd806f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:12.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 shutdown_connections 2024-04-20T21:17:12.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.193+0000 7f6ddfa00640 1 -- 172.21.15.102:0/1583337718 wait complete. 2024-04-20T21:17:12.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:16:51.795761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:16:52.983434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T21:16:56.582297+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T21:17:12.211 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.405+0000 7fec67e00640 1 Processor -- start 2024-04-20T21:17:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.405+0000 7fec67e00640 1 -- start start 2024-04-20T21:17:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.407+0000 7fec67e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec6015b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.407+0000 7fec67e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fec600ccaa0 0x7fec6015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.407+0000 7fec67e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec600cc5f0 con 0x7fec600ccff0 2024-04-20T21:17:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.407+0000 7fec67e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fec600cb450 con 0x7fec600ccaa0 2024-04-20T21:17:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.407+0000 7fec67e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fec60073b60 con 0x7fec60155690 2024-04-20T21:17:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.407+0000 7fec65400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec6015b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.407+0000 7fec65e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fec600ccff0 0x7fec600cd480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45526/0 (socket says 172.21.15.102:45526) 2024-04-20T21:17:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.408+0000 7fec65e00640 1 -- 172.21.15.102:0/657280077 learned_addr learned my addr 172.21.15.102:0/657280077 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:12.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.408+0000 7fec64a00640 1 --2- 172.21.15.102:0/657280077 >> v2:172.21.15.193:3300/0 conn(0x7fec600ccaa0 0x7fec6015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.408+0000 7fec65400640 1 -- 172.21.15.102:0/657280077 >> v2:172.21.15.193:3300/0 conn(0x7fec600ccaa0 msgr2=0x7fec6015bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:12.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.408+0000 7fec65400640 1 --2- 172.21.15.102:0/657280077 >> v2:172.21.15.193:3300/0 conn(0x7fec600ccaa0 0x7fec6015bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.408+0000 7fec65400640 1 -- 172.21.15.102:0/657280077 >> v1:172.21.15.102:6789/0 conn(0x7fec600ccff0 legacy=0x7fec600cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.408+0000 7fec65400640 1 -- 172.21.15.102:0/657280077 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6015f2f0 con 0x7fec60155690 2024-04-20T21:17:12.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.408+0000 7fec5fe00640 1 -- 172.21.15.102:0/657280077 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 672766884 0 0) 0x7fec600cc5f0 con 0x7fec600ccff0 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.409+0000 7fec65400640 1 --2- 172.21.15.102:0/657280077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec6015b5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec540097a0 tx=0x7fec5402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede286c22e740039 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.409+0000 7fec5fe00640 1 -- 172.21.15.102:0/657280077 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54035020 con 0x7fec60155690 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.409+0000 7fec5fe00640 1 -- 172.21.15.102:0/657280077 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec54031830 con 0x7fec60155690 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.409+0000 7fec5fe00640 1 -- 172.21.15.102:0/657280077 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54031c00 con 0x7fec60155690 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.410+0000 7fec67e00640 1 -- 172.21.15.102:0/657280077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 msgr2=0x7fec6015b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.410+0000 7fec67e00640 1 --2- 172.21.15.102:0/657280077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec6015b5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec540097a0 tx=0x7fec5402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.410+0000 7fec67e00640 1 -- 172.21.15.102:0/657280077 shutdown_connections 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.410+0000 7fec67e00640 1 --2- 172.21.15.102:0/657280077 >> v2:172.21.15.193:3300/0 conn(0x7fec600ccaa0 0x7fec6015bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.410+0000 7fec67e00640 1 --2- 172.21.15.102:0/657280077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec6015b5a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.410+0000 7fec67e00640 1 -- 172.21.15.102:0/657280077 >> 172.21.15.102:0/657280077 conn(0x7fec600754d0 msgr2=0x7fec60072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:12.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.410+0000 7fec67e00640 1 -- 172.21.15.102:0/657280077 shutdown_connections 2024-04-20T21:17:12.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.410+0000 7fec67e00640 1 -- 172.21.15.102:0/657280077 wait complete. 2024-04-20T21:17:12.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.412+0000 7fec67e00640 1 Processor -- start 2024-04-20T21:17:12.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.412+0000 7fec67e00640 1 -- start start 2024-04-20T21:17:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec600ccaa0 0x7fec600b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec67e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ccff0 0x7fec600c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec67e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec601f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec67e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec600cb980 con 0x7fec600ccaa0 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec67e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec600cb6a0 con 0x7fec600ccff0 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec67e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fec60073740 con 0x7fec60155690 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec65400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec600ccaa0 0x7fec600b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec65e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec601f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec64a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ccff0 0x7fec600c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec65400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec600ccaa0 0x7fec600b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49128/0 (socket says 172.21.15.102:49128) 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.413+0000 7fec65400640 1 -- 172.21.15.102:0/73965211 learned_addr learned my addr 172.21.15.102:0/73965211 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec64a00640 1 -- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 msgr2=0x7fec601f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec64a00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec601f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec64a00640 1 -- 172.21.15.102:0/73965211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec600ccaa0 msgr2=0x7fec600b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec64a00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec600ccaa0 0x7fec600b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec64a00640 1 -- 172.21.15.102:0/73965211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec600c9880 con 0x7fec600ccff0 2024-04-20T21:17:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec64a00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ccff0 0x7fec600c8ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00e850 tx=0x7fec4c00ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec5d600640 1 -- 172.21.15.102:0/73965211 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c017070 con 0x7fec600ccff0 2024-04-20T21:17:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec5d600640 1 -- 172.21.15.102:0/73965211 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4c00c940 con 0x7fec600ccff0 2024-04-20T21:17:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec5d600640 1 -- 172.21.15.102:0/73965211 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00cd10 con 0x7fec600ccff0 2024-04-20T21:17:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec600cab40 con 0x7fec600ccff0 2024-04-20T21:17:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.414+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec600ca0b0 con 0x7fec600ccff0 2024-04-20T21:17:12.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.415+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec600c9c00 con 0x7fec600ccff0 2024-04-20T21:17:12.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.416+0000 7fec5d600640 1 -- 172.21.15.102:0/73965211 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec4c01a070 con 0x7fec600ccff0 2024-04-20T21:17:12.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.417+0000 7fec5d600640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fec4006c8f0 0x7fec4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.417+0000 7fec5d600640 1 -- 172.21.15.102:0/73965211 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fec4c08be10 con 0x7fec600ccff0 2024-04-20T21:17:12.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.418+0000 7fec65400640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fec4006c8f0 0x7fec4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.418+0000 7fec65400640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fec4006c8f0 0x7fec4006edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec5402b910 tx=0x7fec5403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:12.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.422+0000 7fec5d600640 1 -- 172.21.15.102:0/73965211 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fec4c08f030 con 0x7fec600ccff0 2024-04-20T21:17:12.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.619+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec6004fd60 con 0x7fec600ccff0 2024-04-20T21:17:12.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.620+0000 7fec5d600640 1 -- 172.21.15.102:0/73965211 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fec4c059810 con 0x7fec600ccff0 2024-04-20T21:17:12.620 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fec4006c8f0 msgr2=0x7fec4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fec4006c8f0 0x7fec4006edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec5402b910 tx=0x7fec5403c040 comp rx=0 tx=0).stop 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ccff0 msgr2=0x7fec600c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ccff0 0x7fec600c8ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00e850 tx=0x7fec4c00ed10 comp rx=0 tx=0).stop 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 shutdown_connections 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60155690 0x7fec601f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fec4006c8f0 0x7fec4006edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ccff0 0x7fec600c8ce0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 --2- 172.21.15.102:0/73965211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec600ccaa0 0x7fec600b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 >> 172.21.15.102:0/73965211 conn(0x7fec600754d0 msgr2=0x7fec60153150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 shutdown_connections 2024-04-20T21:17:12.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.623+0000 7fec67e00640 1 -- 172.21.15.102:0/73965211 wait complete. 2024-04-20T21:17:12.643 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:17:12.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.849+0000 7f95a0e00640 1 Processor -- start 2024-04-20T21:17:12.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.849+0000 7f95a0e00640 1 -- start start 2024-04-20T21:17:12.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c151860 0x7f959c0c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f95a0e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f959c0c9220 0x7f959c06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f95a0e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959c073d20 con 0x7f959c073770 2024-04-20T21:17:12.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f95a0e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f959c073fe0 con 0x7f959c0c9220 2024-04-20T21:17:12.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f95a0e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f959c0c7250 con 0x7f959c151860 2024-04-20T21:17:12.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f9599e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f959c073770 0x7f959c1580c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45532/0 (socket says 172.21.15.102:45532) 2024-04-20T21:17:12.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f9599e00640 1 -- 172.21.15.102:0/3458113690 learned_addr learned my addr 172.21.15.102:0/3458113690 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:12.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f959a800640 1 --2- 172.21.15.102:0/3458113690 >> v2:172.21.15.193:3300/0 conn(0x7f959c0c9220 0x7f959c06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.851+0000 7f9599400640 1 --2- 172.21.15.102:0/3458113690 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c151860 0x7f959c0c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.852+0000 7f9598a00640 1 -- 172.21.15.102:0/3458113690 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 183175433 0 0) 0x7f959c073d20 con 0x7f959c073770 2024-04-20T21:17:12.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.852+0000 7f9598a00640 1 -- 172.21.15.102:0/3458113690 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9580003520 con 0x7f959c073770 2024-04-20T21:17:12.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.852+0000 7f959a800640 1 -- 172.21.15.102:0/3458113690 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c151860 msgr2=0x7f959c0c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:12.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.852+0000 7f959a800640 1 --2- 172.21.15.102:0/3458113690 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c151860 0x7f959c0c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.852+0000 7f959a800640 1 -- 172.21.15.102:0/3458113690 >> v1:172.21.15.102:6789/0 conn(0x7f959c073770 legacy=0x7f959c1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.852+0000 7f959a800640 1 -- 172.21.15.102:0/3458113690 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c0c7530 con 0x7f959c0c9220 2024-04-20T21:17:12.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.853+0000 7f959a800640 1 --2- 172.21.15.102:0/3458113690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c0c9220 0x7f959c06e560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9594009d90 tx=0x7f959402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4759cb941322e491 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:12.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.853+0000 7f9598a00640 1 -- 172.21.15.102:0/3458113690 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594039070 con 0x7f959c0c9220 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.853+0000 7f9598a00640 1 -- 172.21.15.102:0/3458113690 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959402cc50 con 0x7f959c0c9220 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.853+0000 7f9598a00640 1 -- 172.21.15.102:0/3458113690 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594034440 con 0x7f959c0c9220 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.854+0000 7f95a0e00640 1 -- 172.21.15.102:0/3458113690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c0c9220 msgr2=0x7f959c06e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.854+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3458113690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c0c9220 0x7f959c06e560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9594009d90 tx=0x7f959402bea0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.854+0000 7f95a0e00640 1 -- 172.21.15.102:0/3458113690 shutdown_connections 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.854+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3458113690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c0c9220 0x7f959c06e560 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.854+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3458113690 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c151860 0x7f959c0c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.854+0000 7f95a0e00640 1 -- 172.21.15.102:0/3458113690 >> 172.21.15.102:0/3458113690 conn(0x7f959c074ea0 msgr2=0x7f959c075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.854+0000 7f95a0e00640 1 -- 172.21.15.102:0/3458113690 shutdown_connections 2024-04-20T21:17:12.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.854+0000 7f95a0e00640 1 -- 172.21.15.102:0/3458113690 wait complete. 2024-04-20T21:17:12.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.856+0000 7f95a0e00640 1 Processor -- start 2024-04-20T21:17:12.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.856+0000 7f95a0e00640 1 -- start start 2024-04-20T21:17:12.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c073770 0x7f959c1ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c0c9220 0x7f959c1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f95a0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c151860 0x7f959c14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f95a0e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f959c151c80 con 0x7f959c151860 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f95a0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f959c0c7af0 con 0x7f959c073770 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f95a0e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f959c0c8550 con 0x7f959c0c9220 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f959a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c151860 0x7f959c14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f959a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c151860 0x7f959c14bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49136/0 (socket says 172.21.15.102:49136) 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f959a800640 1 -- 172.21.15.102:0/3404675173 learned_addr learned my addr 172.21.15.102:0/3404675173 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f9599e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c073770 0x7f959c1ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f9599400640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c0c9220 0x7f959c1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f959a800640 1 -- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c0c9220 msgr2=0x7f959c1567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f959a800640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c0c9220 0x7f959c1567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f959a800640 1 -- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c073770 msgr2=0x7f959c1ef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.857+0000 7f959a800640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c073770 0x7f959c1ef5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:12.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.858+0000 7f959a800640 1 -- 172.21.15.102:0/3404675173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c14a3b0 con 0x7f959c151860 2024-04-20T21:17:12.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.858+0000 7f959a800640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c151860 0x7f959c14bdb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9594009d90 tx=0x7f9594004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:12.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.858+0000 7f9592000640 1 -- 172.21.15.102:0/3404675173 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594039050 con 0x7f959c151860 2024-04-20T21:17:12.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.858+0000 7f9592000640 1 -- 172.21.15.102:0/3404675173 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594045070 con 0x7f959c151860 2024-04-20T21:17:12.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.858+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c149870 con 0x7f959c151860 2024-04-20T21:17:12.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.858+0000 7f9592000640 1 -- 172.21.15.102:0/3404675173 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594009450 con 0x7f959c151860 2024-04-20T21:17:12.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.858+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c149b10 con 0x7f959c151860 2024-04-20T21:17:12.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.860+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959c14bb90 con 0x7f959c151860 2024-04-20T21:17:12.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.861+0000 7f9592000640 1 -- 172.21.15.102:0/3404675173 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f959403e610 con 0x7f959c151860 2024-04-20T21:17:12.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.861+0000 7f9592000640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957c06c8f0 0x7f957c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:12.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.862+0000 7f9592000640 1 -- 172.21.15.102:0/3404675173 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f95940ae7b0 con 0x7f959c151860 2024-04-20T21:17:12.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.862+0000 7f9599e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957c06c8f0 0x7f957c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:12.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.863+0000 7f9599e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957c06c8f0 0x7f957c06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f958c005880 tx=0x7f958c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:12.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:12.866+0000 7f9592000640 1 -- 172.21.15.102:0/3404675173 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f959407c1b0 con 0x7f959c151860 2024-04-20T21:17:13.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.066+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f959c1e3bb0 con 0x7f959c151860 2024-04-20T21:17:13.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.067+0000 7f9592000640 1 -- 172.21.15.102:0/3404675173 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f95940804b0 con 0x7f959c151860 2024-04-20T21:17:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.069+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957c06c8f0 msgr2=0x7f957c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957c06c8f0 0x7f957c06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f958c005880 tx=0x7f958c009040 comp rx=0 tx=0).stop 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c151860 msgr2=0x7f959c14bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c151860 0x7f959c14bdb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9594009d90 tx=0x7f9594004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 shutdown_connections 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c151860 0x7f959c14bdb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c0c9220 0x7f959c1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957c06c8f0 0x7f957c06edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 --2- 172.21.15.102:0/3404675173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c073770 0x7f959c1ef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 >> 172.21.15.102:0/3404675173 conn(0x7f959c074ea0 msgr2=0x7f959c14a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:13.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 shutdown_connections 2024-04-20T21:17:13.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.070+0000 7f95a0e00640 1 -- 172.21.15.102:0/3404675173 wait complete. 2024-04-20T21:17:13.089 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:17:13.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.288+0000 7fdc6ee00640 1 Processor -- start 2024-04-20T21:17:13.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.288+0000 7fdc6ee00640 1 -- start start 2024-04-20T21:17:13.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.289+0000 7fdc6ee00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc680c6ba0 0x7fdc6815c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ee00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdc680c9eb0 0x7fdc68159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ee00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc680c44d0 con 0x7fdc68153210 2024-04-20T21:17:13.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ee00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fdc681596f0 con 0x7fdc680c9eb0 2024-04-20T21:17:13.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ee00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdc680c47b0 con 0x7fdc680c6ba0 2024-04-20T21:17:13.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc67e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdc68153210 0x7fdc68072fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45538/0 (socket says 172.21.15.102:45538) 2024-04-20T21:17:13.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc67e00640 1 -- 172.21.15.102:0/2008353680 learned_addr learned my addr 172.21.15.102:0/2008353680 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:13.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ce00640 1 --2- 172.21.15.102:0/2008353680 >> v2:172.21.15.193:3300/0 conn(0x7fdc680c9eb0 0x7fdc68159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc5f400640 1 --2- 172.21.15.102:0/2008353680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc680c6ba0 0x7fdc6815c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ce00640 1 -- 172.21.15.102:0/2008353680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc680c6ba0 msgr2=0x7fdc6815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:13.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ce00640 1 --2- 172.21.15.102:0/2008353680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc680c6ba0 0x7fdc6815c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ce00640 1 -- 172.21.15.102:0/2008353680 >> v1:172.21.15.102:6789/0 conn(0x7fdc68153210 legacy=0x7fdc68072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc6ce00640 1 -- 172.21.15.102:0/2008353680 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc680c4a90 con 0x7fdc680c9eb0 2024-04-20T21:17:13.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.290+0000 7fdc67400640 1 -- 172.21.15.102:0/2008353680 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222269578 0 0) 0x7fdc680c44d0 con 0x7fdc68153210 2024-04-20T21:17:13.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.292+0000 7fdc6ce00640 1 --2- 172.21.15.102:0/2008353680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c9eb0 0x7fdc68159b80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009d90 tx=0x7fdc6002bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c2f325c4c6bccc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:13.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.292+0000 7fdc67400640 1 -- 172.21.15.102:0/2008353680 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60039070 con 0x7fdc680c9eb0 2024-04-20T21:17:13.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.292+0000 7fdc67400640 1 -- 172.21.15.102:0/2008353680 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6002cce0 con 0x7fdc680c9eb0 2024-04-20T21:17:13.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.292+0000 7fdc67400640 1 -- 172.21.15.102:0/2008353680 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60034660 con 0x7fdc680c9eb0 2024-04-20T21:17:13.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.292+0000 7fdc6ee00640 1 -- 172.21.15.102:0/2008353680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c9eb0 msgr2=0x7fdc68159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.293+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/2008353680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c9eb0 0x7fdc68159b80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009d90 tx=0x7fdc6002bf30 comp rx=0 tx=0).stop 2024-04-20T21:17:13.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.293+0000 7fdc6ee00640 1 -- 172.21.15.102:0/2008353680 shutdown_connections 2024-04-20T21:17:13.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.293+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/2008353680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c9eb0 0x7fdc68159b80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.293+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/2008353680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc680c6ba0 0x7fdc6815c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.293+0000 7fdc6ee00640 1 -- 172.21.15.102:0/2008353680 >> 172.21.15.102:0/2008353680 conn(0x7fdc680c9390 msgr2=0x7fdc680c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:13.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.293+0000 7fdc6ee00640 1 -- 172.21.15.102:0/2008353680 shutdown_connections 2024-04-20T21:17:13.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.293+0000 7fdc6ee00640 1 -- 172.21.15.102:0/2008353680 wait complete. 2024-04-20T21:17:13.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.295+0000 7fdc6ee00640 1 Processor -- start 2024-04-20T21:17:13.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.295+0000 7fdc6ee00640 1 -- start start 2024-04-20T21:17:13.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.296+0000 7fdc6ee00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c6ba0 0x7fdc681f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.296+0000 7fdc6ee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc680c9eb0 0x7fdc681e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.296+0000 7fdc6ee00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc68153210 0x7fdc68071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.296+0000 7fdc6ee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdc680736d0 con 0x7fdc680c9eb0 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.296+0000 7fdc6ee00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68159030 con 0x7fdc680c6ba0 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.296+0000 7fdc6ee00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdc68073850 con 0x7fdc68153210 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.296+0000 7fdc5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc680c9eb0 0x7fdc681e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc67e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c6ba0 0x7fdc681f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc6ce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc68153210 0x7fdc68071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc680c9eb0 0x7fdc681e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49140/0 (socket says 172.21.15.102:49140) 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc5f400640 1 -- 172.21.15.102:0/9581317 learned_addr learned my addr 172.21.15.102:0/9581317 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc67e00640 1 -- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc68153210 msgr2=0x7fdc68071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc67e00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc68153210 0x7fdc68071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc67e00640 1 -- 172.21.15.102:0/9581317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc680c9eb0 msgr2=0x7fdc681e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc67e00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc680c9eb0 0x7fdc681e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc67e00640 1 -- 172.21.15.102:0/9581317 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc681f1c60 con 0x7fdc680c6ba0 2024-04-20T21:17:13.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc67e00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c6ba0 0x7fdc681f1720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c410 tx=0x7fdc5800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:13.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.297+0000 7fdc64c00640 1 -- 172.21.15.102:0/9581317 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580074e0 con 0x7fdc680c6ba0 2024-04-20T21:17:13.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.298+0000 7fdc64c00640 1 -- 172.21.15.102:0/9581317 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc580047e0 con 0x7fdc680c6ba0 2024-04-20T21:17:13.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.298+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc68072c40 con 0x7fdc680c6ba0 2024-04-20T21:17:13.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.298+0000 7fdc64c00640 1 -- 172.21.15.102:0/9581317 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004bd0 con 0x7fdc680c6ba0 2024-04-20T21:17:13.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.298+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc681e4580 con 0x7fdc680c6ba0 2024-04-20T21:17:13.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.299+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6804fd60 con 0x7fdc680c6ba0 2024-04-20T21:17:13.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.300+0000 7fdc64c00640 1 -- 172.21.15.102:0/9581317 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc58011070 con 0x7fdc680c6ba0 2024-04-20T21:17:13.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.301+0000 7fdc64c00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdc4406c8f0 0x7fdc4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.301+0000 7fdc64c00640 1 -- 172.21.15.102:0/9581317 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdc5808b9a0 con 0x7fdc680c6ba0 2024-04-20T21:17:13.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.301+0000 7fdc5f400640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdc4406c8f0 0x7fdc4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.302+0000 7fdc5f400640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdc4406c8f0 0x7fdc4406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdc681e5690 tx=0x7fdc50009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:13.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.305+0000 7fdc64c00640 1 -- 172.21.15.102:0/9581317 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdc580593d0 con 0x7fdc680c6ba0 2024-04-20T21:17:13.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.507+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc681e51b0 con 0x7fdc680c6ba0 2024-04-20T21:17:13.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.508+0000 7fdc64c00640 1 -- 172.21.15.102:0/9581317 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fdc5805d6d0 con 0x7fdc680c6ba0 2024-04-20T21:17:13.507 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-20T21:17:13.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdc4406c8f0 msgr2=0x7fdc4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdc4406c8f0 0x7fdc4406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdc681e5690 tx=0x7fdc50009190 comp rx=0 tx=0).stop 2024-04-20T21:17:13.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c6ba0 msgr2=0x7fdc681f1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c6ba0 0x7fdc681f1720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c410 tx=0x7fdc5800c8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 shutdown_connections 2024-04-20T21:17:13.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc68153210 0x7fdc68071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc680c9eb0 0x7fdc681e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdc4406c8f0 0x7fdc4406edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 --2- 172.21.15.102:0/9581317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc680c6ba0 0x7fdc681f1720 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.510+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 >> 172.21.15.102:0/9581317 conn(0x7fdc680c9390 msgr2=0x7fdc681588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:13.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.511+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 shutdown_connections 2024-04-20T21:17:13.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.511+0000 7fdc6ee00640 1 -- 172.21.15.102:0/9581317 wait complete. 2024-04-20T21:17:13.529 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T21:17:13.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.730+0000 7f86e8c00640 1 Processor -- start 2024-04-20T21:17:13.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.730+0000 7f86e8c00640 1 -- start start 2024-04-20T21:17:13.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.732+0000 7f86e8c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f86e4155660 0x7f86e415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.732+0000 7f86e8c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e415b790 con 0x7f86e40c9340 2024-04-20T21:17:13.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.732+0000 7f86e8c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e415ba50 con 0x7f86e40c9720 2024-04-20T21:17:13.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.732+0000 7f86e8c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f86e415bd30 con 0x7f86e4155660 2024-04-20T21:17:13.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.732+0000 7f86e2800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f86e4155660 0x7f86e415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.732+0000 7f86e1e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f86e40c9340 0x7f86e4154f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45544/0 (socket says 172.21.15.102:45544) 2024-04-20T21:17:13.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.732+0000 7f86e1e00640 1 -- 172.21.15.102:0/2291077193 learned_addr learned my addr 172.21.15.102:0/2291077193 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:13.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.732+0000 7f86e1400640 1 --1- 172.21.15.102:0/2291077193 >> v1:172.21.15.193:6789/0 conn(0x7f86e40c9720 0x7f86e4163ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55694/0 (socket says 172.21.15.102:55694) 2024-04-20T21:17:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.733+0000 7f86e2800640 1 -- 172.21.15.102:0/2291077193 >> v1:172.21.15.193:6789/0 conn(0x7f86e40c9720 legacy=0x7f86e4163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.733+0000 7f86e2800640 1 -- 172.21.15.102:0/2291077193 >> v1:172.21.15.102:6789/0 conn(0x7f86e40c9340 legacy=0x7f86e4154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.733+0000 7f86e0a00640 1 -- 172.21.15.102:0/2291077193 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335595854 0 0) 0x7f86e415ba50 con 0x7f86e40c9720 2024-04-20T21:17:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.733+0000 7f86e2800640 1 -- 172.21.15.102:0/2291077193 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e415c010 con 0x7f86e4155660 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.734+0000 7f86e2800640 1 --2- 172.21.15.102:0/2291077193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 0x7f86e415b1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009d90 tx=0x7f86dc02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69acbbe770177c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.734+0000 7f86e0a00640 1 -- 172.21.15.102:0/2291077193 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc039070 con 0x7f86e4155660 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.734+0000 7f86e0a00640 1 -- 172.21.15.102:0/2291077193 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86dc02cd00 con 0x7f86e4155660 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.734+0000 7f86e0a00640 1 -- 172.21.15.102:0/2291077193 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc034660 con 0x7f86e4155660 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.735+0000 7f86e8c00640 1 -- 172.21.15.102:0/2291077193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 msgr2=0x7f86e415b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.735+0000 7f86e8c00640 1 --2- 172.21.15.102:0/2291077193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 0x7f86e415b1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009d90 tx=0x7f86dc02bf50 comp rx=0 tx=0).stop 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.735+0000 7f86e8c00640 1 -- 172.21.15.102:0/2291077193 shutdown_connections 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.735+0000 7f86e8c00640 1 --2- 172.21.15.102:0/2291077193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 0x7f86e415b1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.735+0000 7f86e8c00640 1 -- 172.21.15.102:0/2291077193 >> 172.21.15.102:0/2291077193 conn(0x7f86e4074ea0 msgr2=0x7f86e4075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.735+0000 7f86e8c00640 1 -- 172.21.15.102:0/2291077193 shutdown_connections 2024-04-20T21:17:13.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.735+0000 7f86e8c00640 1 -- 172.21.15.102:0/2291077193 wait complete. 2024-04-20T21:17:13.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.737+0000 7f86e8c00640 1 Processor -- start 2024-04-20T21:17:13.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.737+0000 7f86e8c00640 1 -- start start 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86e40c9340 0x7f86e41581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e8c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86e40c9720 0x7f86e4150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e8c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 0x7f86e4148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e8c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f86e415ecf0 con 0x7f86e40c9340 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e8c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86e415ea10 con 0x7f86e4155660 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e8c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f86e40c3ff0 con 0x7f86e40c9720 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86e40c9340 0x7f86e41581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86e40c9340 0x7f86e41581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49152/0 (socket says 172.21.15.102:49152) 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1e00640 1 -- 172.21.15.102:0/73882257 learned_addr learned my addr 172.21.15.102:0/73882257 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:13.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e2800640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 0x7f86e4148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1400640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86e40c9720 0x7f86e4150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1e00640 1 -- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86e40c9720 msgr2=0x7f86e4150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1e00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86e40c9720 0x7f86e4150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1e00640 1 -- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 msgr2=0x7f86e4148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1e00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 0x7f86e4148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.738+0000 7f86e1e00640 1 -- 172.21.15.102:0/73882257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e4072a10 con 0x7f86e40c9340 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.739+0000 7f86e1e00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86e40c9340 0x7f86e41581f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86d400c4b0 tx=0x7f86d400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.739+0000 7f86da000640 1 -- 172.21.15.102:0/73882257 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4017070 con 0x7f86e40c9340 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.739+0000 7f86da000640 1 -- 172.21.15.102:0/73882257 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d4005ce0 con 0x7f86e40c9340 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.739+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e415a420 con 0x7f86e40c9340 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.739+0000 7f86da000640 1 -- 172.21.15.102:0/73882257 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4007270 con 0x7f86e40c9340 2024-04-20T21:17:13.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.739+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e4071280 con 0x7f86e40c9340 2024-04-20T21:17:13.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.740+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e40c5b50 con 0x7f86e40c9340 2024-04-20T21:17:13.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.741+0000 7f86da000640 1 -- 172.21.15.102:0/73882257 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86d401a070 con 0x7f86e40c9340 2024-04-20T21:17:13.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.742+0000 7f86da000640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:13.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.742+0000 7f86da000640 1 -- 172.21.15.102:0/73882257 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f86d408b0d0 con 0x7f86e40c9340 2024-04-20T21:17:13.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.743+0000 7f86e1400640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:13.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.743+0000 7f86e1400640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f86c006c920 0x7f86c006ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86cc007f70 tx=0x7f86cc007f00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:13.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.747+0000 7f86da000640 1 -- 172.21.15.102:0/73882257 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f86d4058b60 con 0x7f86e40c9340 2024-04-20T21:17:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.948+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f86e404fdd0 con 0x7f86e40c9340 2024-04-20T21:17:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.949+0000 7f86da000640 1 -- 172.21.15.102:0/73882257 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f86d405ce60 con 0x7f86e40c9340 2024-04-20T21:17:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f86c006c920 msgr2=0x7f86c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f86c006c920 0x7f86c006ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86cc007f70 tx=0x7f86cc007f00 comp rx=0 tx=0).stop 2024-04-20T21:17:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86e40c9340 msgr2=0x7f86e41581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86e40c9340 0x7f86e41581f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86d400c4b0 tx=0x7f86d400c970 comp rx=0 tx=0).stop 2024-04-20T21:17:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 shutdown_connections 2024-04-20T21:17:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86e4155660 0x7f86e4148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86e40c9720 0x7f86e4150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 --2- 172.21.15.102:0/73882257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86e40c9340 0x7f86e41581f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:13.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 >> 172.21.15.102:0/73882257 conn(0x7f86e4074ea0 msgr2=0x7f86e414b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:13.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.951+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 shutdown_connections 2024-04-20T21:17:13.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:13.952+0000 7f86e8c00640 1 -- 172.21.15.102:0/73882257 wait complete. 2024-04-20T21:17:13.969 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:17:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.172+0000 7f2b31000640 1 Processor -- start 2024-04-20T21:17:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.172+0000 7f2b31000640 1 -- start start 2024-04-20T21:17:14.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.174+0000 7f2b31000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2b2c155530 0x7f2b2c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.174+0000 7f2b31000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2b2c0cc470 0x7f2b2c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.174+0000 7f2b31000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cdfc0 0x7f2b2c0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.174+0000 7f2b31000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f2b2c0c5ca0 con 0x7f2b2c0cc470 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.174+0000 7f2b31000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f2b2c0cd5f0 con 0x7f2b2c155530 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.174+0000 7f2b31000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2c15ef60 con 0x7f2b2c0cdfc0 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2b400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2b2c155530 0x7f2b2c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2b400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2b2c155530 0x7f2b2c15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:42004/0 (socket says 172.21.15.102:42004) 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2b400640 1 -- 172.21.15.102:0/3471400788 learned_addr learned my addr 172.21.15.102:0/3471400788 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2be00640 1 --2- 172.21.15.102:0/3471400788 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cdfc0 0x7f2b2c0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2aa00640 1 --2- 172.21.15.102:0/3471400788 >> v2:172.21.15.102:3300/0 conn(0x7f2b2c0cc470 0x7f2b2c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2b400640 1 -- 172.21.15.102:0/3471400788 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cdfc0 msgr2=0x7f2b2c0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2b400640 1 --2- 172.21.15.102:0/3471400788 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cdfc0 0x7f2b2c0cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2b400640 1 -- 172.21.15.102:0/3471400788 >> v2:172.21.15.102:3300/0 conn(0x7f2b2c0cc470 msgr2=0x7f2b2c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:14.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2b400640 1 --2- 172.21.15.102:0/3471400788 >> v2:172.21.15.102:3300/0 conn(0x7f2b2c0cc470 0x7f2b2c0cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2b400640 1 -- 172.21.15.102:0/3471400788 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c15f240 con 0x7f2b2c155530 2024-04-20T21:17:14.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.175+0000 7f2b2aa00640 1 --2- 172.21.15.102:0/3471400788 >> v2:172.21.15.102:3300/0 conn(0x7f2b2c0cc470 0x7f2b2c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.176+0000 7f2b2b400640 1 --2- 172.21.15.102:0/3471400788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c155530 0x7f2b2c15e640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c06a050 tx=0x7f2b1c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5380d2468ed83821 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.176+0000 7f2b2a000640 1 -- 172.21.15.102:0/3471400788 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c035020 con 0x7f2b2c155530 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b2a000640 1 -- 172.21.15.102:0/3471400788 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c031870 con 0x7f2b2c155530 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b2a000640 1 -- 172.21.15.102:0/3471400788 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c031c40 con 0x7f2b2c155530 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 -- 172.21.15.102:0/3471400788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c155530 msgr2=0x7f2b2c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 --2- 172.21.15.102:0/3471400788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c155530 0x7f2b2c15e640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c06a050 tx=0x7f2b1c02b910 comp rx=0 tx=0).stop 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 -- 172.21.15.102:0/3471400788 shutdown_connections 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 --2- 172.21.15.102:0/3471400788 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cdfc0 0x7f2b2c0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 --2- 172.21.15.102:0/3471400788 >> v2:172.21.15.102:3300/0 conn(0x7f2b2c0cc470 0x7f2b2c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 --2- 172.21.15.102:0/3471400788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c155530 0x7f2b2c15e640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 -- 172.21.15.102:0/3471400788 >> 172.21.15.102:0/3471400788 conn(0x7f2b2c075080 msgr2=0x7f2b2c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 -- 172.21.15.102:0/3471400788 shutdown_connections 2024-04-20T21:17:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.177+0000 7f2b31000640 1 -- 172.21.15.102:0/3471400788 wait complete. 2024-04-20T21:17:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.179+0000 7f2b31000640 1 Processor -- start 2024-04-20T21:17:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.179+0000 7f2b31000640 1 -- start start 2024-04-20T21:17:14.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b31000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cc470 0x7f2b2c1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b31000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c0cdfc0 0x7f2b2c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b31000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b2c155530 0x7f2b2c1e5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b31000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c0cd280 con 0x7f2b2c155530 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b31000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c0ccfa0 con 0x7f2b2c0cdfc0 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b31000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2c0ccbe0 con 0x7f2b2c0cc470 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b2c155530 0x7f2b2c1e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b2c155530 0x7f2b2c1e5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49166/0 (socket says 172.21.15.102:49166) 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b2be00640 1 -- 172.21.15.102:0/816001846 learned_addr learned my addr 172.21.15.102:0/816001846 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b2b400640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cc470 0x7f2b2c1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.180+0000 7f2b2aa00640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c0cdfc0 0x7f2b2c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b2be00640 1 -- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cc470 msgr2=0x7f2b2c1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b2be00640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cc470 0x7f2b2c1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b2be00640 1 -- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c0cdfc0 msgr2=0x7f2b2c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b2be00640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c0cdfc0 0x7f2b2c15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b2be00640 1 -- 172.21.15.102:0/816001846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c15ddc0 con 0x7f2b2c155530 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b2be00640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b2c155530 0x7f2b2c1e5540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400e960 tx=0x7f2b2400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b23400640 1 -- 172.21.15.102:0/816001846 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24017070 con 0x7f2b2c155530 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b23400640 1 -- 172.21.15.102:0/816001846 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b2400c980 con 0x7f2b2c155530 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b23400640 1 -- 172.21.15.102:0/816001846 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400cd70 con 0x7f2b2c155530 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.181+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c15d1a0 con 0x7f2b2c155530 2024-04-20T21:17:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.182+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c1f4630 con 0x7f2b2c155530 2024-04-20T21:17:14.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.183+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2c04fd60 con 0x7f2b2c155530 2024-04-20T21:17:14.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.184+0000 7f2b23400640 1 -- 172.21.15.102:0/816001846 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b2401a070 con 0x7f2b2c155530 2024-04-20T21:17:14.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.184+0000 7f2b23400640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2b0806c8f0 0x7f2b0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.185+0000 7f2b23400640 1 -- 172.21.15.102:0/816001846 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2b2408be90 con 0x7f2b2c155530 2024-04-20T21:17:14.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.185+0000 7f2b2b400640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2b0806c8f0 0x7f2b0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.186+0000 7f2b2b400640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2b0806c8f0 0x7f2b0806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c036000 tx=0x7f2b1c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:14.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.189+0000 7f2b23400640 1 -- 172.21.15.102:0/816001846 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2b2c15ddc0 con 0x7f2b2c155530 2024-04-20T21:17:14.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.399+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2b2c0cc9c0 con 0x7f2b2c155530 2024-04-20T21:17:14.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.401+0000 7f2b23400640 1 -- 172.21.15.102:0/816001846 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f2b2408d080 con 0x7f2b2c155530 2024-04-20T21:17:14.400 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-04-20T21:17:14.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2b0806c8f0 msgr2=0x7f2b0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2b0806c8f0 0x7f2b0806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c036000 tx=0x7f2b1c03c040 comp rx=0 tx=0).stop 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b2c155530 msgr2=0x7f2b2c1e5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b2c155530 0x7f2b2c1e5540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400e960 tx=0x7f2b2400ee20 comp rx=0 tx=0).stop 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 shutdown_connections 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b2c155530 0x7f2b2c1e5540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2b0806c8f0 0x7f2b0806edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b2c0cdfc0 0x7f2b2c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 --2- 172.21.15.102:0/816001846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b2c0cc470 0x7f2b2c1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 >> 172.21.15.102:0/816001846 conn(0x7f2b2c075080 msgr2=0x7f2b2c0d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.403+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 shutdown_connections 2024-04-20T21:17:14.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.404+0000 7f2b31000640 1 -- 172.21.15.102:0/816001846 wait complete. 2024-04-20T21:17:14.422 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:17:14.423 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:17:14.476 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:14.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.681+0000 7f6294a00640 1 Processor -- start 2024-04-20T21:17:14.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.681+0000 7f6294a00640 1 -- start start 2024-04-20T21:17:14.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f6294a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f62900c9eb0 0x7f62900c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f6294a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6290073a10 0x7f6290158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f6294a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62901536d0 con 0x7f629014fa50 2024-04-20T21:17:14.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f6294a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6290159500 con 0x7f62900c9eb0 2024-04-20T21:17:14.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f6294a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f62900c76c0 con 0x7f6290073a10 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f6285400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f62900c9eb0 0x7f62900c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f6285400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f62900c9eb0 0x7f62900c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49170/0 (socket says 172.21.15.102:49170) 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f6285400640 1 -- 172.21.15.102:0/1507673694 learned_addr learned my addr 172.21.15.102:0/1507673694 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.683+0000 7f628e800640 1 --2- 172.21.15.102:0/1507673694 >> v2:172.21.15.193:3300/0 conn(0x7f6290073a10 0x7f6290158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.684+0000 7f6285400640 1 -- 172.21.15.102:0/1507673694 >> v2:172.21.15.193:3300/0 conn(0x7f6290073a10 msgr2=0x7f6290158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.684+0000 7f6285400640 1 --2- 172.21.15.102:0/1507673694 >> v2:172.21.15.193:3300/0 conn(0x7f6290073a10 0x7f6290158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.684+0000 7f6285400640 1 -- 172.21.15.102:0/1507673694 >> v1:172.21.15.193:6789/0 conn(0x7f629014fa50 legacy=0x7f62900c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.684+0000 7f6285400640 1 -- 172.21.15.102:0/1507673694 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62900c7960 con 0x7f62900c9eb0 2024-04-20T21:17:14.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.684+0000 7f628e800640 1 --2- 172.21.15.102:0/1507673694 >> v2:172.21.15.193:3300/0 conn(0x7f6290073a10 0x7f6290158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:14.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.685+0000 7f6285400640 1 --2- 172.21.15.102:0/1507673694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62900c4830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6290156b00 tx=0x7f627802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a188f8b5c8e9fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:14.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.685+0000 7f628d400640 1 -- 172.21.15.102:0/1507673694 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627802f850 con 0x7f62900c9eb0 2024-04-20T21:17:14.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.685+0000 7f628d400640 1 -- 172.21.15.102:0/1507673694 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627802f9f0 con 0x7f62900c9eb0 2024-04-20T21:17:14.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.685+0000 7f628d400640 1 -- 172.21.15.102:0/1507673694 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627802fdc0 con 0x7f62900c9eb0 2024-04-20T21:17:14.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.686+0000 7f6294a00640 1 -- 172.21.15.102:0/1507673694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 msgr2=0x7f62900c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.686+0000 7f6294a00640 1 --2- 172.21.15.102:0/1507673694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62900c4830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6290156b00 tx=0x7f627802ba30 comp rx=0 tx=0).stop 2024-04-20T21:17:14.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.686+0000 7f6294a00640 1 -- 172.21.15.102:0/1507673694 shutdown_connections 2024-04-20T21:17:14.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.686+0000 7f6294a00640 1 --2- 172.21.15.102:0/1507673694 >> v2:172.21.15.193:3300/0 conn(0x7f6290073a10 0x7f6290158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.686+0000 7f6294a00640 1 --2- 172.21.15.102:0/1507673694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62900c4830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.686+0000 7f6294a00640 1 -- 172.21.15.102:0/1507673694 >> 172.21.15.102:0/1507673694 conn(0x7f62900c9390 msgr2=0x7f62900c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:14.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.686+0000 7f6294a00640 1 -- 172.21.15.102:0/1507673694 shutdown_connections 2024-04-20T21:17:14.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.686+0000 7f6294a00640 1 -- 172.21.15.102:0/1507673694 wait complete. 2024-04-20T21:17:14.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.688+0000 7f6294a00640 1 Processor -- start 2024-04-20T21:17:14.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.688+0000 7f6294a00640 1 -- start start 2024-04-20T21:17:14.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6294a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6290073a10 0x7f6290142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6294a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62901e2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6294a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f629014fa50 0x7f62901484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6294a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62900c7c00 con 0x7f62900c9eb0 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6294a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62900c3d90 con 0x7f629014fa50 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6294a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f62900c4030 con 0x7f6290073a10 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6285400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62901e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f628e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f629014fa50 0x7f62901484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f628de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6290073a10 0x7f6290142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6285400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62901e2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49174/0 (socket says 172.21.15.102:49174) 2024-04-20T21:17:14.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6285400640 1 -- 172.21.15.102:0/1592083022 learned_addr learned my addr 172.21.15.102:0/1592083022 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:14.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6285400640 1 -- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6290073a10 msgr2=0x7f6290142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6285400640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6290073a10 0x7f6290142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6285400640 1 -- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f629014fa50 msgr2=0x7f62901484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6285400640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f629014fa50 0x7f62901484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.689+0000 7f6285400640 1 -- 172.21.15.102:0/1592083022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6290148a10 con 0x7f62900c9eb0 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.690+0000 7f6285400640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62901e2410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f627802ba70 tx=0x7f6278004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.690+0000 7f6286a00640 1 -- 172.21.15.102:0/1592083022 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278037c90 con 0x7f62900c9eb0 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.690+0000 7f6286a00640 1 -- 172.21.15.102:0/1592083022 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6278046070 con 0x7f62900c9eb0 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.690+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62901e2b50 con 0x7f62900c9eb0 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.690+0000 7f6286a00640 1 -- 172.21.15.102:0/1592083022 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278041640 con 0x7f62900c9eb0 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.690+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62901ef130 con 0x7f62900c9eb0 2024-04-20T21:17:14.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.691+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629004fd90 con 0x7f62900c9eb0 2024-04-20T21:17:14.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.692+0000 7f6286a00640 1 -- 172.21.15.102:0/1592083022 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6278040070 con 0x7f62900c9eb0 2024-04-20T21:17:14.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.693+0000 7f6286a00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f626c06c8f0 0x7f626c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:14.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.693+0000 7f6286a00640 1 -- 172.21.15.102:0/1592083022 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f62780ae590 con 0x7f62900c9eb0 2024-04-20T21:17:14.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.693+0000 7f628de00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f626c06c8f0 0x7f626c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:14.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.694+0000 7f628de00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f626c06c8f0 0x7f626c06edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f62800016e0 tx=0x7f6280009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:14.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.697+0000 7f6286a00640 1 -- 172.21.15.102:0/1592083022 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f629004fd90 con 0x7f62900c9eb0 2024-04-20T21:17:14.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.905+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6290074340 con 0x7f62900c9eb0 2024-04-20T21:17:14.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.908+0000 7f6286a00640 1 -- 172.21.15.102:0/1592083022 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f6278033030 con 0x7f62900c9eb0 2024-04-20T21:17:14.909 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:14.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f626c06c8f0 msgr2=0x7f626c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f626c06c8f0 0x7f626c06edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f62800016e0 tx=0x7f6280009040 comp rx=0 tx=0).stop 2024-04-20T21:17:14.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 msgr2=0x7f62901e2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:14.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62901e2410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f627802ba70 tx=0x7f6278004730 comp rx=0 tx=0).stop 2024-04-20T21:17:14.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 shutdown_connections 2024-04-20T21:17:14.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f626c06c8f0 0x7f626c06edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f629014fa50 0x7f62901484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62900c9eb0 0x7f62901e2410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 --2- 172.21.15.102:0/1592083022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6290073a10 0x7f6290142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:14.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 >> 172.21.15.102:0/1592083022 conn(0x7f62900c9390 msgr2=0x7f62901441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:14.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 shutdown_connections 2024-04-20T21:17:14.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:14.911+0000 7f6294a00640 1 -- 172.21.15.102:0/1592083022 wait complete. 2024-04-20T21:17:14.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:16:51.795761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:16:52.983434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T21:16:56.582297+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T21:17:14.931 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:15.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.133+0000 7fb347800640 1 Processor -- start 2024-04-20T21:17:15.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.133+0000 7fb347800640 1 -- start start 2024-04-20T21:17:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb347800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb340153210 0x7fb34015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb347800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb3400c6ba0 0x7fb340159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb347800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3400c44d0 con 0x7fb3400c9eb0 2024-04-20T21:17:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb347800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb3401596f0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb347800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb3400c47b0 con 0x7fb340153210 2024-04-20T21:17:15.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb33fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb3400c6ba0 0x7fb340159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb344e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb340153210 0x7fb34015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb33fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb3400c6ba0 0x7fb340159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49176/0 (socket says 172.21.15.102:49176) 2024-04-20T21:17:15.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.135+0000 7fb33fe00640 1 -- 172.21.15.102:0/788325606 learned_addr learned my addr 172.21.15.102:0/788325606 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:15.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.136+0000 7fb33fe00640 1 -- 172.21.15.102:0/788325606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb340153210 msgr2=0x7fb34015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:15.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.136+0000 7fb33fe00640 1 --2- 172.21.15.102:0/788325606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb340153210 0x7fb34015c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.136+0000 7fb33fe00640 1 -- 172.21.15.102:0/788325606 >> v1:172.21.15.102:6789/0 conn(0x7fb3400c9eb0 legacy=0x7fb340072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.136+0000 7fb33fe00640 1 -- 172.21.15.102:0/788325606 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3400c4a90 con 0x7fb3400c6ba0 2024-04-20T21:17:15.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.136+0000 7fb33f400640 1 -- 172.21.15.102:0/788325606 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525954703 0 0) 0x7fb3400c44d0 con 0x7fb3400c9eb0 2024-04-20T21:17:15.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.137+0000 7fb33fe00640 1 --2- 172.21.15.102:0/788325606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb340159b80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009870 tx=0x7fb32c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6298e99d3fc08bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:15.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.137+0000 7fb33f400640 1 -- 172.21.15.102:0/788325606 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c02f840 con 0x7fb3400c6ba0 2024-04-20T21:17:15.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.137+0000 7fb33f400640 1 -- 172.21.15.102:0/788325606 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32c02f9e0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.137+0000 7fb33f400640 1 -- 172.21.15.102:0/788325606 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c02fdb0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.138+0000 7fb347800640 1 -- 172.21.15.102:0/788325606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 msgr2=0x7fb340159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.138+0000 7fb347800640 1 --2- 172.21.15.102:0/788325606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb340159b80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009870 tx=0x7fb32c02ba20 comp rx=0 tx=0).stop 2024-04-20T21:17:15.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.138+0000 7fb347800640 1 -- 172.21.15.102:0/788325606 shutdown_connections 2024-04-20T21:17:15.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.138+0000 7fb347800640 1 --2- 172.21.15.102:0/788325606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb340159b80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.138+0000 7fb347800640 1 --2- 172.21.15.102:0/788325606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb340153210 0x7fb34015c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.138+0000 7fb347800640 1 -- 172.21.15.102:0/788325606 >> 172.21.15.102:0/788325606 conn(0x7fb3400c9390 msgr2=0x7fb3400c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:15.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.138+0000 7fb347800640 1 -- 172.21.15.102:0/788325606 shutdown_connections 2024-04-20T21:17:15.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.138+0000 7fb347800640 1 -- 172.21.15.102:0/788325606 wait complete. 2024-04-20T21:17:15.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.140+0000 7fb347800640 1 Processor -- start 2024-04-20T21:17:15.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.140+0000 7fb347800640 1 -- start start 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb347800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb3400cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb347800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3400c9eb0 0x7fb3401e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb347800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340153210 0x7fb340149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb347800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb3400736d0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb347800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb340159030 con 0x7fb340153210 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb347800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb340073850 con 0x7fb3400c9eb0 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb344e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb3400cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb344e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb3400cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49184/0 (socket says 172.21.15.102:49184) 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb344e00640 1 -- 172.21.15.102:0/4161287106 learned_addr learned my addr 172.21.15.102:0/4161287106 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb33fe00640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3400c9eb0 0x7fb3401e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb345800640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340153210 0x7fb340149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb344e00640 1 -- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3400c9eb0 msgr2=0x7fb3401e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb344e00640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3400c9eb0 0x7fb3401e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb344e00640 1 -- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340153210 msgr2=0x7fb340149b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.141+0000 7fb344e00640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340153210 0x7fb340149b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.142+0000 7fb344e00640 1 -- 172.21.15.102:0/4161287106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34014f1f0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.142+0000 7fb344e00640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb3400cb0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e8a0 tx=0x7fb33400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.142+0000 7fb33cc00640 1 -- 172.21.15.102:0/4161287106 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334017070 con 0x7fb3400c6ba0 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.142+0000 7fb33cc00640 1 -- 172.21.15.102:0/4161287106 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33400c8e0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.142+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34014f490 con 0x7fb3400c6ba0 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.142+0000 7fb33cc00640 1 -- 172.21.15.102:0/4161287106 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33400ccd0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.142+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3400ca8e0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.144+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3400c44d0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.144+0000 7fb33cc00640 1 -- 172.21.15.102:0/4161287106 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb33401a070 con 0x7fb3400c6ba0 2024-04-20T21:17:15.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.145+0000 7fb33cc00640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c920 0x7fb32006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.145+0000 7fb33cc00640 1 -- 172.21.15.102:0/4161287106 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb33408bc80 con 0x7fb3400c6ba0 2024-04-20T21:17:15.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.146+0000 7fb33fe00640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c920 0x7fb32006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.146+0000 7fb33fe00640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c920 0x7fb32006ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb32c036000 tx=0x7fb32c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:15.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.150+0000 7fb33cc00640 1 -- 172.21.15.102:0/4161287106 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb3400c44d0 con 0x7fb3400c6ba0 2024-04-20T21:17:15.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.350+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb34004fd60 con 0x7fb3400c6ba0 2024-04-20T21:17:15.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.351+0000 7fb33cc00640 1 -- 172.21.15.102:0/4161287106 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fb334059690 con 0x7fb3400c6ba0 2024-04-20T21:17:15.351 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:17:15.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c920 msgr2=0x7fb32006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c920 0x7fb32006ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb32c036000 tx=0x7fb32c008030 comp rx=0 tx=0).stop 2024-04-20T21:17:15.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 msgr2=0x7fb3400cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb3400cb0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e8a0 tx=0x7fb33400ed60 comp rx=0 tx=0).stop 2024-04-20T21:17:15.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 shutdown_connections 2024-04-20T21:17:15.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c920 0x7fb32006ede0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340153210 0x7fb340149b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3400c9eb0 0x7fb3401e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 --2- 172.21.15.102:0/4161287106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3400c6ba0 0x7fb3400cb0c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.353+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 >> 172.21.15.102:0/4161287106 conn(0x7fb3400c9390 msgr2=0x7fb34006e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:15.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.354+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 shutdown_connections 2024-04-20T21:17:15.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.354+0000 7fb347800640 1 -- 172.21.15.102:0/4161287106 wait complete. 2024-04-20T21:17:15.372 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:17:15.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.570+0000 7fd9f8a00640 1 Processor -- start 2024-04-20T21:17:15.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.570+0000 7fd9f8a00640 1 -- start start 2024-04-20T21:17:15.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.572+0000 7fd9f8a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f40ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.572+0000 7fd9f8a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f40cd3b0 con 0x7fd9f415e660 2024-04-20T21:17:15.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.572+0000 7fd9f8a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f415b180 con 0x7fd9f40cccf0 2024-04-20T21:17:15.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.572+0000 7fd9f8a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f40cb770 con 0x7fd9f4155250 2024-04-20T21:17:15.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.572+0000 7fd9f3400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f40ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.572+0000 7fd9eaa00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fd9f40cccf0 0x7fd9f40cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55722/0 (socket says 172.21.15.102:55722) 2024-04-20T21:17:15.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.572+0000 7fd9eaa00640 1 -- 172.21.15.102:0/2958242071 learned_addr learned my addr 172.21.15.102:0/2958242071 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:15.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.573+0000 7fd9f3400640 1 -- 172.21.15.102:0/2958242071 >> v1:172.21.15.193:6789/0 conn(0x7fd9f40cccf0 legacy=0x7fd9f40cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.573+0000 7fd9f3400640 1 -- 172.21.15.102:0/2958242071 >> v1:172.21.15.102:6789/0 conn(0x7fd9f415e660 legacy=0x7fd9f415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.573+0000 7fd9f3400640 1 -- 172.21.15.102:0/2958242071 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f415b420 con 0x7fd9f4155250 2024-04-20T21:17:15.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.573+0000 7fd9f2a00640 1 -- 172.21.15.102:0/2958242071 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 638544263 0 0) 0x7fd9f415b180 con 0x7fd9f40cccf0 2024-04-20T21:17:15.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.574+0000 7fd9f3400640 1 --2- 172.21.15.102:0/2958242071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f40ce030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097d0 tx=0x7fd9e402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4e38f2b3c3905 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:15.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.574+0000 7fd9f2a00640 1 -- 172.21.15.102:0/2958242071 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4034070 con 0x7fd9f4155250 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.574+0000 7fd9f2a00640 1 -- 172.21.15.102:0/2958242071 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e4030800 con 0x7fd9f4155250 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.574+0000 7fd9f2a00640 1 -- 172.21.15.102:0/2958242071 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4030bd0 con 0x7fd9f4155250 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.575+0000 7fd9f8a00640 1 -- 172.21.15.102:0/2958242071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 msgr2=0x7fd9f40ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.575+0000 7fd9f8a00640 1 --2- 172.21.15.102:0/2958242071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f40ce030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097d0 tx=0x7fd9e402b940 comp rx=0 tx=0).stop 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.575+0000 7fd9f8a00640 1 -- 172.21.15.102:0/2958242071 shutdown_connections 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.575+0000 7fd9f8a00640 1 --2- 172.21.15.102:0/2958242071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f40ce030 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.575+0000 7fd9f8a00640 1 -- 172.21.15.102:0/2958242071 >> 172.21.15.102:0/2958242071 conn(0x7fd9f406cb40 msgr2=0x7fd9f406cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.575+0000 7fd9f8a00640 1 -- 172.21.15.102:0/2958242071 shutdown_connections 2024-04-20T21:17:15.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.575+0000 7fd9f8a00640 1 -- 172.21.15.102:0/2958242071 wait complete. 2024-04-20T21:17:15.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.577+0000 7fd9f8a00640 1 Processor -- start 2024-04-20T21:17:15.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.577+0000 7fd9f8a00640 1 -- start start 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f40cccf0 0x7fd9f415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f8a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f4158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9f415e660 0x7fd9f40d0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f415bcc0 con 0x7fd9f415e660 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f415b9e0 con 0x7fd9f40cccf0 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f8a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f4155630 con 0x7fd9f4155250 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9f415e660 0x7fd9f40d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9eaa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f4158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f40cccf0 0x7fd9f415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9f415e660 0x7fd9f40d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49200/0 (socket says 172.21.15.102:49200) 2024-04-20T21:17:15.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.578+0000 7fd9f3e00640 1 -- 172.21.15.102:0/261120048 learned_addr learned my addr 172.21.15.102:0/261120048 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:15.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f3e00640 1 -- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 msgr2=0x7fd9f4158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f3e00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f4158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f3e00640 1 -- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f40cccf0 msgr2=0x7fd9f415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f3e00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f40cccf0 0x7fd9f415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f3e00640 1 -- 172.21.15.102:0/261120048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f41509f0 con 0x7fd9f415e660 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f3400640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f40cccf0 0x7fd9f415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f3e00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9f415e660 0x7fd9f40d0940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c680 tx=0x7fd9ec00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9ebe00640 1 -- 172.21.15.102:0/261120048 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec003b70 con 0x7fd9f415e660 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9ebe00640 1 -- 172.21.15.102:0/261120048 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec003d10 con 0x7fd9f415e660 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f41f0980 con 0x7fd9f415e660 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9ebe00640 1 -- 172.21.15.102:0/261120048 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec005310 con 0x7fd9f415e660 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.579+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41f4c80 con 0x7fd9f415e660 2024-04-20T21:17:15.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.581+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f404fcf0 con 0x7fd9f415e660 2024-04-20T21:17:15.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.581+0000 7fd9ebe00640 1 -- 172.21.15.102:0/261120048 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9ec0054b0 con 0x7fd9f415e660 2024-04-20T21:17:15.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.582+0000 7fd9ebe00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd9d006c8f0 0x7fd9d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:15.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.583+0000 7fd9ebe00640 1 -- 172.21.15.102:0/261120048 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd9ec08bbd0 con 0x7fd9f415e660 2024-04-20T21:17:15.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.583+0000 7fd9f3400640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd9d006c8f0 0x7fd9d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:15.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.583+0000 7fd9f3400640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd9d006c8f0 0x7fd9d006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4036000 tx=0x7fd9e403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:15.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.587+0000 7fd9ebe00640 1 -- 172.21.15.102:0/261120048 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd9ec0595d0 con 0x7fd9f415e660 2024-04-20T21:17:15.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.787+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd9f406b940 con 0x7fd9f415e660 2024-04-20T21:17:15.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.787+0000 7fd9ebe00640 1 -- 172.21.15.102:0/261120048 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fd9ec05d8d0 con 0x7fd9f415e660 2024-04-20T21:17:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:15.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd9d006c8f0 msgr2=0x7fd9d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd9d006c8f0 0x7fd9d006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4036000 tx=0x7fd9e403b040 comp rx=0 tx=0).stop 2024-04-20T21:17:15.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9f415e660 msgr2=0x7fd9f40d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:15.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9f415e660 0x7fd9f40d0940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c680 tx=0x7fd9ec00cb40 comp rx=0 tx=0).stop 2024-04-20T21:17:15.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 shutdown_connections 2024-04-20T21:17:15.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9f415e660 0x7fd9f40d0940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9f4155250 0x7fd9f4158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd9d006c8f0 0x7fd9d006edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 --2- 172.21.15.102:0/261120048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9f40cccf0 0x7fd9f415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:15.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 >> 172.21.15.102:0/261120048 conn(0x7fd9f406cb40 msgr2=0x7fd9f415eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:15.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 shutdown_connections 2024-04-20T21:17:15.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:15.790+0000 7fd9f8a00640 1 -- 172.21.15.102:0/261120048 wait complete. 2024-04-20T21:17:15.809 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:17:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.007+0000 7fae8ca00640 1 Processor -- start 2024-04-20T21:17:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.008+0000 7fae8ca00640 1 -- start start 2024-04-20T21:17:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.009+0000 7fae8ca00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae881554f0 0x7fae8815b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.009+0000 7fae8ca00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fae880c5de0 0x7fae881659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.009+0000 7fae8ca00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae880cce30 con 0x7fae880c59c0 2024-04-20T21:17:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.009+0000 7fae8ca00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fae881558d0 con 0x7fae880c5de0 2024-04-20T21:17:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.009+0000 7fae8ca00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fae880cd110 con 0x7fae881554f0 2024-04-20T21:17:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.009+0000 7fae87400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae881554f0 0x7fae8815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae87400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae881554f0 0x7fae8815b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:53578/0 (socket says 172.21.15.102:53578) 2024-04-20T21:17:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae87400640 1 -- 172.21.15.102:0/3861384078 learned_addr learned my addr 172.21.15.102:0/3861384078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae86a00640 1 --1- 172.21.15.102:0/3861384078 >> v1:172.21.15.193:6789/0 conn(0x7fae880c59c0 0x7fae880d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55728/0 (socket says 172.21.15.102:55728) 2024-04-20T21:17:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae87e00640 1 --2- 172.21.15.102:0/3861384078 >> v2:172.21.15.102:3300/0 conn(0x7fae880c5de0 0x7fae881659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae87400640 1 -- 172.21.15.102:0/3861384078 >> v2:172.21.15.102:3300/0 conn(0x7fae880c5de0 msgr2=0x7fae881659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae87400640 1 --2- 172.21.15.102:0/3861384078 >> v2:172.21.15.102:3300/0 conn(0x7fae880c5de0 0x7fae881659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae87400640 1 -- 172.21.15.102:0/3861384078 >> v1:172.21.15.193:6789/0 conn(0x7fae880c59c0 legacy=0x7fae880d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae87400640 1 -- 172.21.15.102:0/3861384078 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae880cd3f0 con 0x7fae881554f0 2024-04-20T21:17:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.010+0000 7fae86000640 1 -- 172.21.15.102:0/3861384078 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3925789422 0 0) 0x7fae880cce30 con 0x7fae880c59c0 2024-04-20T21:17:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.011+0000 7fae87400640 1 --2- 172.21.15.102:0/3861384078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae881554f0 0x7fae8815b120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fae78009e50 tx=0x7fae7802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b898198b312da41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.011+0000 7fae86000640 1 -- 172.21.15.102:0/3861384078 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7802cb90 con 0x7fae881554f0 2024-04-20T21:17:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae86000640 1 -- 172.21.15.102:0/3861384078 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7802cd30 con 0x7fae881554f0 2024-04-20T21:17:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae86000640 1 -- 172.21.15.102:0/3861384078 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78034610 con 0x7fae881554f0 2024-04-20T21:17:16.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae8ca00640 1 -- 172.21.15.102:0/3861384078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae881554f0 msgr2=0x7fae8815b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae8ca00640 1 --2- 172.21.15.102:0/3861384078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae881554f0 0x7fae8815b120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fae78009e50 tx=0x7fae7802bff0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae8ca00640 1 -- 172.21.15.102:0/3861384078 shutdown_connections 2024-04-20T21:17:16.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae8ca00640 1 --2- 172.21.15.102:0/3861384078 >> v2:172.21.15.102:3300/0 conn(0x7fae880c5de0 0x7fae881659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae8ca00640 1 --2- 172.21.15.102:0/3861384078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae881554f0 0x7fae8815b120 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae8ca00640 1 -- 172.21.15.102:0/3861384078 >> 172.21.15.102:0/3861384078 conn(0x7fae8806d150 msgr2=0x7fae88074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:16.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae8ca00640 1 -- 172.21.15.102:0/3861384078 shutdown_connections 2024-04-20T21:17:16.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.012+0000 7fae8ca00640 1 -- 172.21.15.102:0/3861384078 wait complete. 2024-04-20T21:17:16.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.014+0000 7fae8ca00640 1 Processor -- start 2024-04-20T21:17:16.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.014+0000 7fae8ca00640 1 -- start start 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae8ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae880c59c0 0x7fae880b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae8ca00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae880c5de0 0x7fae88145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae8ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae881554f0 0x7fae880b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae8ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae880cdc90 con 0x7fae880c59c0 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae8ca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae880cd9b0 con 0x7fae881554f0 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae8ca00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fae880ccad0 con 0x7fae880c5de0 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae880c59c0 0x7fae880b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae880c59c0 0x7fae880b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49222/0 (socket says 172.21.15.102:49222) 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.015+0000 7fae87400640 1 -- 172.21.15.102:0/2349401855 learned_addr learned my addr 172.21.15.102:0/2349401855 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae86a00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae880c5de0 0x7fae88145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae87e00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae881554f0 0x7fae880b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae87400640 1 -- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae880c5de0 msgr2=0x7fae88145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae87400640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae880c5de0 0x7fae88145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae87400640 1 -- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae881554f0 msgr2=0x7fae880b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae87400640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae881554f0 0x7fae880b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae87400640 1 -- 172.21.15.102:0/2349401855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae881460c0 con 0x7fae880c59c0 2024-04-20T21:17:16.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae87e00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae881554f0 0x7fae880b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:16.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.016+0000 7fae87400640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae880c59c0 0x7fae880b6310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fae7802c600 tx=0x7fae78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.017+0000 7fae7f400640 1 -- 172.21.15.102:0/2349401855 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78044020 con 0x7fae880c59c0 2024-04-20T21:17:16.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.017+0000 7fae7f400640 1 -- 172.21.15.102:0/2349401855 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae78040690 con 0x7fae880c59c0 2024-04-20T21:17:16.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.017+0000 7fae7f400640 1 -- 172.21.15.102:0/2349401855 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78009510 con 0x7fae880c59c0 2024-04-20T21:17:16.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.017+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae881f1b00 con 0x7fae880c59c0 2024-04-20T21:17:16.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.017+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8814ca60 con 0x7fae880c59c0 2024-04-20T21:17:16.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.018+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae88073be0 con 0x7fae880c59c0 2024-04-20T21:17:16.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.019+0000 7fae7f400640 1 -- 172.21.15.102:0/2349401855 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae78005d10 con 0x7fae880c59c0 2024-04-20T21:17:16.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.020+0000 7fae7f400640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.020+0000 7fae7f400640 1 -- 172.21.15.102:0/2349401855 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fae780ae710 con 0x7fae880c59c0 2024-04-20T21:17:16.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.020+0000 7fae86a00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.021+0000 7fae86a00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae6406c8f0 0x7fae6406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae880ce2d0 tx=0x7fae70007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.025+0000 7fae7f400640 1 -- 172.21.15.102:0/2349401855 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fae88073be0 con 0x7fae880c59c0 2024-04-20T21:17:16.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.228+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fae880cce30 con 0x7fae880c59c0 2024-04-20T21:17:16.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.229+0000 7fae7f400640 1 -- 172.21.15.102:0/2349401855 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fae880cce30 con 0x7fae880c59c0 2024-04-20T21:17:16.229 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-20T21:17:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae6406c8f0 msgr2=0x7fae6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae6406c8f0 0x7fae6406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae880ce2d0 tx=0x7fae70007d50 comp rx=0 tx=0).stop 2024-04-20T21:17:16.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae880c59c0 msgr2=0x7fae880b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae880c59c0 0x7fae880b6310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fae7802c600 tx=0x7fae78004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 shutdown_connections 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae881554f0 0x7fae880b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae880c5de0 0x7fae88145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 --2- 172.21.15.102:0/2349401855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae880c59c0 0x7fae880b6310 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.232+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 >> 172.21.15.102:0/2349401855 conn(0x7fae8806d150 msgr2=0x7fae88153790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.233+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 shutdown_connections 2024-04-20T21:17:16.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.233+0000 7fae8ca00640 1 -- 172.21.15.102:0/2349401855 wait complete. 2024-04-20T21:17:16.252 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T21:17:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.456+0000 7f1d9b200640 1 Processor -- start 2024-04-20T21:17:16.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.456+0000 7f1d9b200640 1 -- start start 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d9b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c0c7250 0x7f1d9c1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d9b200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1d9c06ea30 0x7f1d9c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d9b200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d9c073480 con 0x7f1d9c1422c0 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d9b200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1d9c073230 con 0x7f1d9c06ea30 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d9b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9c06e5a0 con 0x7f1d9c0c7250 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d99400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c0c7250 0x7f1d9c1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d9a800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1d9c06ea30 0x7f1d9c06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d99e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1d9c1422c0 0x7f1d9c0c5d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45576/0 (socket says 172.21.15.102:45576) 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d99e00640 1 -- 172.21.15.102:0/72486991 learned_addr learned my addr 172.21.15.102:0/72486991 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d99400640 1 --2- 172.21.15.102:0/72486991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c0c7250 0x7f1d9c1597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:53600/0 (socket says 172.21.15.102:53600) 2024-04-20T21:17:16.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.458+0000 7f1d9a800640 1 --2- 172.21.15.102:0/72486991 >> v2:172.21.15.193:3300/0 conn(0x7f1d9c06ea30 0x7f1d9c06eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:42062/0 (socket says 172.21.15.102:42062) 2024-04-20T21:17:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.459+0000 7f1d99400640 1 -- 172.21.15.102:0/72486991 >> v2:172.21.15.193:3300/0 conn(0x7f1d9c06ea30 msgr2=0x7f1d9c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.459+0000 7f1d99400640 1 --2- 172.21.15.102:0/72486991 >> v2:172.21.15.193:3300/0 conn(0x7f1d9c06ea30 0x7f1d9c06eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.459+0000 7f1d99400640 1 -- 172.21.15.102:0/72486991 >> v1:172.21.15.102:6789/0 conn(0x7f1d9c1422c0 legacy=0x7f1d9c0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.459+0000 7f1d99400640 1 -- 172.21.15.102:0/72486991 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c073760 con 0x7f1d9c0c7250 2024-04-20T21:17:16.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.460+0000 7f1d99400640 1 --2- 172.21.15.102:0/72486991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c0c7250 0x7f1d9c1597d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1d880097a0 tx=0x7f1d8802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b720e24cc356380 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.460+0000 7f1d98a00640 1 -- 172.21.15.102:0/72486991 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88035020 con 0x7f1d9c0c7250 2024-04-20T21:17:16.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.460+0000 7f1d98a00640 1 -- 172.21.15.102:0/72486991 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d88031810 con 0x7f1d9c0c7250 2024-04-20T21:17:16.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.460+0000 7f1d98a00640 1 -- 172.21.15.102:0/72486991 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88031be0 con 0x7f1d9c0c7250 2024-04-20T21:17:16.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.461+0000 7f1d9b200640 1 -- 172.21.15.102:0/72486991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c0c7250 msgr2=0x7f1d9c1597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.461+0000 7f1d9b200640 1 --2- 172.21.15.102:0/72486991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c0c7250 0x7f1d9c1597d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1d880097a0 tx=0x7f1d8802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.461+0000 7f1d9b200640 1 -- 172.21.15.102:0/72486991 shutdown_connections 2024-04-20T21:17:16.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.461+0000 7f1d9b200640 1 --2- 172.21.15.102:0/72486991 >> v2:172.21.15.193:3300/0 conn(0x7f1d9c06ea30 0x7f1d9c06eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.461+0000 7f1d9b200640 1 --2- 172.21.15.102:0/72486991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c0c7250 0x7f1d9c1597d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.461+0000 7f1d9b200640 1 -- 172.21.15.102:0/72486991 >> 172.21.15.102:0/72486991 conn(0x7f1d9c0cc620 msgr2=0x7f1d9c0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:16.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.461+0000 7f1d9b200640 1 -- 172.21.15.102:0/72486991 shutdown_connections 2024-04-20T21:17:16.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.461+0000 7f1d9b200640 1 -- 172.21.15.102:0/72486991 wait complete. 2024-04-20T21:17:16.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.463+0000 7f1d9b200640 1 Processor -- start 2024-04-20T21:17:16.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.463+0000 7f1d9b200640 1 -- start start 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d9b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c06ea30 0x7f1d9c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d9b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9c0c7250 0x7f1d9c13d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d9b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c1422c0 0x7f1d9c13de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d9b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c073d20 con 0x7f1d9c0c7250 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d9b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c159d10 con 0x7f1d9c06ea30 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d9b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9c1426a0 con 0x7f1d9c1422c0 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d99400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9c0c7250 0x7f1d9c13d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d99400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9c0c7250 0x7f1d9c13d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49238/0 (socket says 172.21.15.102:49238) 2024-04-20T21:17:16.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d99400640 1 -- 172.21.15.102:0/835490493 learned_addr learned my addr 172.21.15.102:0/835490493 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:16.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d9a800640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c1422c0 0x7f1d9c13de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.464+0000 7f1d99e00640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c06ea30 0x7f1d9c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d9a800640 1 -- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c06ea30 msgr2=0x7f1d9c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d9a800640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c06ea30 0x7f1d9c153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d9a800640 1 -- 172.21.15.102:0/835490493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9c0c7250 msgr2=0x7f1d9c13d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d9a800640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9c0c7250 0x7f1d9c13d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d9a800640 1 -- 172.21.15.102:0/835490493 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c0c4780 con 0x7f1d9c1422c0 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d99400640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9c0c7250 0x7f1d9c13d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d9a800640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c1422c0 0x7f1d9c13de40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009d90 tx=0x7f1d9400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d92000640 1 -- 172.21.15.102:0/835490493 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94018020 con 0x7f1d9c1422c0 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d92000640 1 -- 172.21.15.102:0/835490493 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d94014480 con 0x7f1d9c1422c0 2024-04-20T21:17:16.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c0c4a60 con 0x7f1d9c1422c0 2024-04-20T21:17:16.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.465+0000 7f1d92000640 1 -- 172.21.15.102:0/835490493 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94014870 con 0x7f1d9c1422c0 2024-04-20T21:17:16.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.466+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c0c5020 con 0x7f1d9c1422c0 2024-04-20T21:17:16.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.467+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c04fcf0 con 0x7f1d9c1422c0 2024-04-20T21:17:16.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.467+0000 7f1d92000640 1 -- 172.21.15.102:0/835490493 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d94005d10 con 0x7f1d9c1422c0 2024-04-20T21:17:16.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.468+0000 7f1d92000640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7c06c8f0 0x7f1d7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.469+0000 7f1d92000640 1 -- 172.21.15.102:0/835490493 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1d9408cbc0 con 0x7f1d9c1422c0 2024-04-20T21:17:16.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.469+0000 7f1d99e00640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7c06c8f0 0x7f1d7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.469+0000 7f1d99e00640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7c06c8f0 0x7f1d7c06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c007fd0 tx=0x7f1d8c007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.473+0000 7f1d92000640 1 -- 172.21.15.102:0/835490493 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d94022030 con 0x7f1d9c1422c0 2024-04-20T21:17:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.674+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1d9c156610 con 0x7f1d9c1422c0 2024-04-20T21:17:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.675+0000 7f1d92000640 1 -- 172.21.15.102:0/835490493 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f1d94010040 con 0x7f1d9c1422c0 2024-04-20T21:17:16.674 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:16.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.677+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7c06c8f0 msgr2=0x7f1d7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.677+0000 7f1d9b200640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7c06c8f0 0x7f1d7c06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c007fd0 tx=0x7f1d8c007e80 comp rx=0 tx=0).stop 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.677+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c1422c0 msgr2=0x7f1d9c13de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.677+0000 7f1d9b200640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c1422c0 0x7f1d9c13de40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009d90 tx=0x7f1d9400c490 comp rx=0 tx=0).stop 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.678+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 shutdown_connections 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.678+0000 7f1d9b200640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9c1422c0 0x7f1d9c13de40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.678+0000 7f1d9b200640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9c0c7250 0x7f1d9c13d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.678+0000 7f1d9b200640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7c06c8f0 0x7f1d7c06edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.678+0000 7f1d9b200640 1 --2- 172.21.15.102:0/835490493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c06ea30 0x7f1d9c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.678+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 >> 172.21.15.102:0/835490493 conn(0x7f1d9c0cc620 msgr2=0x7f1d9c0cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.678+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 shutdown_connections 2024-04-20T21:17:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.678+0000 7f1d9b200640 1 -- 172.21.15.102:0/835490493 wait complete. 2024-04-20T21:17:16.698 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:17:16.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.899+0000 7f1c22400640 1 Processor -- start 2024-04-20T21:17:16.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.899+0000 7f1c22400640 1 -- start start 2024-04-20T21:17:16.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.900+0000 7f1c22400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c152170 0x7f1c1c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.900+0000 7f1c22400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1c071f70 con 0x7f1c1c158b50 2024-04-20T21:17:16.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.900+0000 7f1c22400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1c072210 con 0x7f1c1c158770 2024-04-20T21:17:16.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.900+0000 7f1c22400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c0c6dc0 con 0x7f1c1c152170 2024-04-20T21:17:16.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1be00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1c1c158b50 0x7f1c1c1628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45586/0 (socket says 172.21.15.102:45586) 2024-04-20T21:17:16.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1be00640 1 -- 172.21.15.102:0/2475149819 learned_addr learned my addr 172.21.15.102:0/2475149819 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:16.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1b400640 1 --2- 172.21.15.102:0/2475149819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c152170 0x7f1c1c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1a000640 1 -- 172.21.15.102:0/2475149819 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658837305 0 0) 0x7f1c1c071f70 con 0x7f1c1c158b50 2024-04-20T21:17:16.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1a000640 1 -- 172.21.15.102:0/2475149819 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c00003520 con 0x7f1c1c158b50 2024-04-20T21:17:16.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1a000640 1 -- 172.21.15.102:0/2475149819 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444460569 0 0) 0x7f1c1c072210 con 0x7f1c1c158770 2024-04-20T21:17:16.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1b400640 1 -- 172.21.15.102:0/2475149819 >> v1:172.21.15.193:6789/0 conn(0x7f1c1c158770 legacy=0x7f1c1c070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1b400640 1 -- 172.21.15.102:0/2475149819 >> v1:172.21.15.102:6789/0 conn(0x7f1c1c158b50 legacy=0x7f1c1c1628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.901+0000 7f1c1b400640 1 -- 172.21.15.102:0/2475149819 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c0724b0 con 0x7f1c1c152170 2024-04-20T21:17:16.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.903+0000 7f1c1b400640 1 --2- 172.21.15.102:0/2475149819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c152170 0x7f1c1c0c8db0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0723e0 tx=0x7f1c0c02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a13bf9ac4c2209e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.903+0000 7f1c1a000640 1 -- 172.21.15.102:0/2475149819 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c034070 con 0x7f1c1c152170 2024-04-20T21:17:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.903+0000 7f1c1a000640 1 -- 172.21.15.102:0/2475149819 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0c030880 con 0x7f1c1c152170 2024-04-20T21:17:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.903+0000 7f1c1a000640 1 -- 172.21.15.102:0/2475149819 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c030c50 con 0x7f1c1c152170 2024-04-20T21:17:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.903+0000 7f1c22400640 1 -- 172.21.15.102:0/2475149819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c152170 msgr2=0x7f1c1c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.903+0000 7f1c22400640 1 --2- 172.21.15.102:0/2475149819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c152170 0x7f1c1c0c8db0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0723e0 tx=0x7f1c0c02b9c0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.904+0000 7f1c22400640 1 -- 172.21.15.102:0/2475149819 shutdown_connections 2024-04-20T21:17:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.904+0000 7f1c22400640 1 --2- 172.21.15.102:0/2475149819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c152170 0x7f1c1c0c8db0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.904+0000 7f1c22400640 1 -- 172.21.15.102:0/2475149819 >> 172.21.15.102:0/2475149819 conn(0x7f1c1c0c9390 msgr2=0x7f1c1c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.904+0000 7f1c22400640 1 -- 172.21.15.102:0/2475149819 shutdown_connections 2024-04-20T21:17:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.904+0000 7f1c22400640 1 -- 172.21.15.102:0/2475149819 wait complete. 2024-04-20T21:17:16.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.905+0000 7f1c22400640 1 Processor -- start 2024-04-20T21:17:16.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.906+0000 7f1c22400640 1 -- start start 2024-04-20T21:17:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c22400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c152170 0x7f1c1c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c22400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c158770 0x7f1c1c156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c22400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c1c158b50 0x7f1c1c14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c22400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c0c9f20 con 0x7f1c1c158b50 2024-04-20T21:17:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c22400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c158250 con 0x7f1c1c152170 2024-04-20T21:17:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c22400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c157f20 con 0x7f1c1c158770 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c1c158b50 0x7f1c1c14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c152170 0x7f1c1c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c1c158b50 0x7f1c1c14dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49244/0 (socket says 172.21.15.102:49244) 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1be00640 1 -- 172.21.15.102:0/3846399527 learned_addr learned my addr 172.21.15.102:0/3846399527 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1aa00640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c158770 0x7f1c1c156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1b400640 1 -- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c158770 msgr2=0x7f1c1c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1b400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c158770 0x7f1c1c156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1b400640 1 -- 172.21.15.102:0/3846399527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c1c158b50 msgr2=0x7f1c1c14dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1b400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c1c158b50 0x7f1c1c14dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1b400640 1 -- 172.21.15.102:0/3846399527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c150450 con 0x7f1c1c152170 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.907+0000 7f1c1be00640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c1c158b50 0x7f1c1c14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.908+0000 7f1c1b400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c152170 0x7f1c1c1567e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c072140 tx=0x7f1c0c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.908+0000 7f1c13400640 1 -- 172.21.15.102:0/3846399527 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c049070 con 0x7f1c1c152170 2024-04-20T21:17:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.908+0000 7f1c13400640 1 -- 172.21.15.102:0/3846399527 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0c030680 con 0x7f1c1c152170 2024-04-20T21:17:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.908+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c150730 con 0x7f1c1c152170 2024-04-20T21:17:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.908+0000 7f1c13400640 1 -- 172.21.15.102:0/3846399527 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c034040 con 0x7f1c1c152170 2024-04-20T21:17:16.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.908+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c155fe0 con 0x7f1c1c152170 2024-04-20T21:17:16.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.909+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c074340 con 0x7f1c1c152170 2024-04-20T21:17:16.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.910+0000 7f1c13400640 1 -- 172.21.15.102:0/3846399527 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c0c04c070 con 0x7f1c1c152170 2024-04-20T21:17:16.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.911+0000 7f1c13400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:16.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.911+0000 7f1c13400640 1 -- 172.21.15.102:0/3846399527 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1c0c0adda0 con 0x7f1c1c152170 2024-04-20T21:17:16.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.911+0000 7f1c1aa00640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:16.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.912+0000 7f1c1aa00640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c06d690 tx=0x7f1c08009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:16.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:16.916+0000 7f1c13400640 1 -- 172.21.15.102:0/3846399527 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1c0c030850 con 0x7f1c1c152170 2024-04-20T21:17:17.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.129+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1c1c0c3460 con 0x7f1c1c152170 2024-04-20T21:17:17.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.130+0000 7f1c13400640 1 -- 172.21.15.102:0/3846399527 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f1c0c07b7a0 con 0x7f1c1c152170 2024-04-20T21:17:17.129 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-04-20T21:17:17.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1bfc06c8f0 msgr2=0x7f1bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c06d690 tx=0x7f1c08009040 comp rx=0 tx=0).stop 2024-04-20T21:17:17.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c152170 msgr2=0x7f1c1c1567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c152170 0x7f1c1c1567e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c072140 tx=0x7f1c0c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 shutdown_connections 2024-04-20T21:17:17.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c1c158b50 0x7f1c1c14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c1c158770 0x7f1c1c156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1bfc06c8f0 0x7f1bfc06edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 --2- 172.21.15.102:0/3846399527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c152170 0x7f1c1c1567e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 >> 172.21.15.102:0/3846399527 conn(0x7f1c1c0c9390 msgr2=0x7f1c1c06e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:17.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.132+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 shutdown_connections 2024-04-20T21:17:17.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.133+0000 7f1c22400640 1 -- 172.21.15.102:0/3846399527 wait complete. 2024-04-20T21:17:17.151 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi102.front.sepia.ceph.com... 2024-04-20T21:17:17.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:17.151 DEBUG:teuthology.orchestra.run.smithi102:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-20T21:17:17.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi102.stderr:ceph-fuse[29433]: fuse finished with error 0 and tester_r 0 2024-04-20T21:17:17.228 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:17:17.301 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:17:17.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:17.302 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-20T21:17:17.367 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:17.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.571+0000 7fc5fbe00640 1 Processor -- start 2024-04-20T21:17:17.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.571+0000 7fc5fbe00640 1 -- start start 2024-04-20T21:17:17.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.573+0000 7fc5fbe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc5fc073c70 0x7fc5fc1647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:17.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.573+0000 7fc5fbe00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5fc153560 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.573+0000 7fc5fbe00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5fc0c7a70 con 0x7fc5fc153180 2024-04-20T21:17:17.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.573+0000 7fc5fbe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc5fc0c7d30 con 0x7fc5fc073c70 2024-04-20T21:17:17.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.573+0000 7fc5f2000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc5fc0c75a0 0x7fc5fc0c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45594/0 (socket says 172.21.15.102:45594) 2024-04-20T21:17:17.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.573+0000 7fc5f2000640 1 -- 172.21.15.102:0/3046832907 learned_addr learned my addr 172.21.15.102:0/3046832907 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:17.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.574+0000 7fc5fb400640 1 --2- 172.21.15.102:0/3046832907 >> v2:172.21.15.102:3300/0 conn(0x7fc5fc073c70 0x7fc5fc1647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:17.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.574+0000 7fc5fa000640 1 -- 172.21.15.102:0/3046832907 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3022746428 0 0) 0x7fc5fc0c7a70 con 0x7fc5fc153180 2024-04-20T21:17:17.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.574+0000 7fc5fa000640 1 -- 172.21.15.102:0/3046832907 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5e0003520 con 0x7fc5fc153180 2024-04-20T21:17:17.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.574+0000 7fc5fa000640 1 -- 172.21.15.102:0/3046832907 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384014392 0 0) 0x7fc5fc153560 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.574+0000 7fc5fa000640 1 -- 172.21.15.102:0/3046832907 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5fc0c7a70 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.574+0000 7fc5fb400640 1 -- 172.21.15.102:0/3046832907 >> v1:172.21.15.193:6789/0 conn(0x7fc5fc153180 legacy=0x7fc5fc158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.574+0000 7fc5fb400640 1 -- 172.21.15.102:0/3046832907 >> v1:172.21.15.102:6789/0 conn(0x7fc5fc0c75a0 legacy=0x7fc5fc0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.574+0000 7fc5fb400640 1 -- 172.21.15.102:0/3046832907 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc15c4d0 con 0x7fc5fc073c70 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.576+0000 7fc5fb400640 1 --2- 172.21.15.102:0/3046832907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc073c70 0x7fc5fc1647d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40097a0 tx=0x7fc5f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f57f0522f9a304 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.576+0000 7fc5fa000640 1 -- 172.21.15.102:0/3046832907 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4034070 con 0x7fc5fc073c70 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.576+0000 7fc5fa000640 1 -- 172.21.15.102:0/3046832907 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f4030610 con 0x7fc5fc073c70 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.576+0000 7fc5fa000640 1 -- 172.21.15.102:0/3046832907 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40309c0 con 0x7fc5fc073c70 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.577+0000 7fc5fbe00640 1 -- 172.21.15.102:0/3046832907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc073c70 msgr2=0x7fc5fc1647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.577+0000 7fc5fbe00640 1 --2- 172.21.15.102:0/3046832907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc073c70 0x7fc5fc1647d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40097a0 tx=0x7fc5f402b960 comp rx=0 tx=0).stop 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.577+0000 7fc5fbe00640 1 -- 172.21.15.102:0/3046832907 shutdown_connections 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.577+0000 7fc5fbe00640 1 --2- 172.21.15.102:0/3046832907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc073c70 0x7fc5fc1647d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.577+0000 7fc5fbe00640 1 -- 172.21.15.102:0/3046832907 >> 172.21.15.102:0/3046832907 conn(0x7fc5fc0cc820 msgr2=0x7fc5fc0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:17.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.577+0000 7fc5fbe00640 1 -- 172.21.15.102:0/3046832907 shutdown_connections 2024-04-20T21:17:17.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.577+0000 7fc5fbe00640 1 -- 172.21.15.102:0/3046832907 wait complete. 2024-04-20T21:17:17.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.578+0000 7fc5fbe00640 1 Processor -- start 2024-04-20T21:17:17.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.579+0000 7fc5fbe00640 1 -- start start 2024-04-20T21:17:17.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.579+0000 7fc5fbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5fc073c70 0x7fc5fc151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:17.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5fbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc0c75a0 0x7fc5fc1f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:17.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5fbe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5fc153180 0x7fc5fc1f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:17.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5fbe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc159550 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5fbe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc15ca90 con 0x7fc5fc073c70 2024-04-20T21:17:17.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5fbe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc15d290 con 0x7fc5fc153180 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5f2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc0c75a0 0x7fc5fc1f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5f2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc0c75a0 0x7fc5fc1f44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49256/0 (socket says 172.21.15.102:49256) 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5f2000640 1 -- 172.21.15.102:0/2365128427 learned_addr learned my addr 172.21.15.102:0/2365128427 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5faa00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5fc073c70 0x7fc5fc151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5fb400640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5fc153180 0x7fc5fc1f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5f2000640 1 -- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5fc153180 msgr2=0x7fc5fc1f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5f2000640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5fc153180 0x7fc5fc1f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5f2000640 1 -- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5fc073c70 msgr2=0x7fc5fc151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5f2000640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5fc073c70 0x7fc5fc151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.580+0000 7fc5f2000640 1 -- 172.21.15.102:0/2365128427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc1480a0 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.581+0000 7fc5f2000640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc0c75a0 0x7fc5fc1f44c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c3c0 tx=0x7fc5e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.581+0000 7fc5f3400640 1 -- 172.21.15.102:0/2365128427 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4017070 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.581+0000 7fc5f3400640 1 -- 172.21.15.102:0/2365128427 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e4005ce0 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.581+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc1e40e0 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.581+0000 7fc5f3400640 1 -- 172.21.15.102:0/2365128427 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40072f0 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.581+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc1e46a0 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.582+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc04fd60 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.583+0000 7fc5f3400640 1 -- 172.21.15.102:0/2365128427 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5e401a070 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.584+0000 7fc5f3400640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc5d806c920 0x7fc5d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:17.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.585+0000 7fc5f3400640 1 -- 172.21.15.102:0/2365128427 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc5e408b210 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.585+0000 7fc5faa00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc5d806c920 0x7fc5d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:17.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.585+0000 7fc5faa00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc5d806c920 0x7fc5d806ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001390 tx=0x7fc5ec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:17.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.589+0000 7fc5f3400640 1 -- 172.21.15.102:0/2365128427 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc5e4058cd0 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.789+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5fc15c230 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.792+0000 7fc5f3400640 1 -- 172.21.15.102:0/2365128427 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7fc5e405cfd0 con 0x7fc5fc0c75a0 2024-04-20T21:17:17.792 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:17.793 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc5d806c920 msgr2=0x7fc5d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc5d806c920 0x7fc5d806ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001390 tx=0x7fc5ec009040 comp rx=0 tx=0).stop 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc0c75a0 msgr2=0x7fc5fc1f44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc0c75a0 0x7fc5fc1f44c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c3c0 tx=0x7fc5e400c880 comp rx=0 tx=0).stop 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 shutdown_connections 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5fc153180 0x7fc5fc1f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5fc0c75a0 0x7fc5fc1f44c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc5d806c920 0x7fc5d806ede0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 --2- 172.21.15.102:0/2365128427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5fc073c70 0x7fc5fc151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:17.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.795+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 >> 172.21.15.102:0/2365128427 conn(0x7fc5fc0cc820 msgr2=0x7fc5fc0c2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:17.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.796+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 shutdown_connections 2024-04-20T21:17:17.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:17.796+0000 7fc5fbe00640 1 -- 172.21.15.102:0/2365128427 wait complete. 2024-04-20T21:17:17.815 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:18.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.015+0000 7f08d9800640 1 Processor -- start 2024-04-20T21:17:18.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.015+0000 7f08d9800640 1 -- start start 2024-04-20T21:17:18.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d9800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08d40c9eb0 0x7f08d40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d9800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d41536d0 con 0x7f08d414fa50 2024-04-20T21:17:18.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d9800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d40c76c0 con 0x7f08d4073a10 2024-04-20T21:17:18.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d9800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f08d40c7960 con 0x7f08d40c9eb0 2024-04-20T21:17:18.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d2a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f08d414fa50 0x7f08d40c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38612/0 (socket says 172.21.15.102:38612) 2024-04-20T21:17:18.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d2a00640 1 -- 172.21.15.102:0/3430087214 learned_addr learned my addr 172.21.15.102:0/3430087214 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:18.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d2000640 1 --2- 172.21.15.102:0/3430087214 >> v2:172.21.15.193:3300/0 conn(0x7f08d40c9eb0 0x7f08d40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d1600640 1 -- 172.21.15.102:0/3430087214 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 115275330 0 0) 0x7f08c4001110 con 0x7f08d414fa50 2024-04-20T21:17:18.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.017+0000 7f08d1600640 1 -- 172.21.15.102:0/3430087214 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08b8003520 con 0x7f08d414fa50 2024-04-20T21:17:18.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.018+0000 7f08d2000640 1 -- 172.21.15.102:0/3430087214 >> v1:172.21.15.193:6789/0 conn(0x7f08d4073a10 legacy=0x7f08d4163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.018+0000 7f08d1600640 1 -- 172.21.15.102:0/3430087214 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 189869168 0 0) 0x7f08d40c76c0 con 0x7f08d4073a10 2024-04-20T21:17:18.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.018+0000 7f08d2000640 1 -- 172.21.15.102:0/3430087214 >> v1:172.21.15.102:6789/0 conn(0x7f08d414fa50 legacy=0x7f08d40c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.018+0000 7f08d2000640 1 -- 172.21.15.102:0/3430087214 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d40c3d90 con 0x7f08d40c9eb0 2024-04-20T21:17:18.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.019+0000 7f08d2000640 1 --2- 172.21.15.102:0/3430087214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d40c9eb0 0x7f08d40c4830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08d40c7890 tx=0x7f08c002b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc16dc50115cb57d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:18.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.019+0000 7f08d1600640 1 -- 172.21.15.102:0/3430087214 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0034070 con 0x7f08d40c9eb0 2024-04-20T21:17:18.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.019+0000 7f08d1600640 1 -- 172.21.15.102:0/3430087214 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c0030890 con 0x7f08d40c9eb0 2024-04-20T21:17:18.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.019+0000 7f08d1600640 1 -- 172.21.15.102:0/3430087214 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0030c40 con 0x7f08d40c9eb0 2024-04-20T21:17:18.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.020+0000 7f08d9800640 1 -- 172.21.15.102:0/3430087214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d40c9eb0 msgr2=0x7f08d40c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.020+0000 7f08d9800640 1 --2- 172.21.15.102:0/3430087214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d40c9eb0 0x7f08d40c4830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08d40c7890 tx=0x7f08c002b9d0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.020+0000 7f08d9800640 1 -- 172.21.15.102:0/3430087214 shutdown_connections 2024-04-20T21:17:18.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.020+0000 7f08d9800640 1 --2- 172.21.15.102:0/3430087214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d40c9eb0 0x7f08d40c4830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.020+0000 7f08d9800640 1 -- 172.21.15.102:0/3430087214 >> 172.21.15.102:0/3430087214 conn(0x7f08d40c9390 msgr2=0x7f08d40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:18.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.020+0000 7f08d9800640 1 -- 172.21.15.102:0/3430087214 shutdown_connections 2024-04-20T21:17:18.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.020+0000 7f08d9800640 1 -- 172.21.15.102:0/3430087214 wait complete. 2024-04-20T21:17:18.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.022+0000 7f08d9800640 1 Processor -- start 2024-04-20T21:17:18.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.022+0000 7f08d9800640 1 -- start start 2024-04-20T21:17:18.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d4073a10 0x7f08d41f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d9800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d40c9eb0 0x7f08d41f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08d414fa50 0x7f08d41e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f08d40c4030 con 0x7f08d414fa50 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d9800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08d40c7c00 con 0x7f08d4073a10 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d9800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f08d40c4310 con 0x7f08d40c9eb0 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d4073a10 0x7f08d41f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d4073a10 0x7f08d41f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56554/0 (socket says 172.21.15.102:56554) 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d2a00640 1 -- 172.21.15.102:0/2349509325 learned_addr learned my addr 172.21.15.102:0/2349509325 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d2000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d40c9eb0 0x7f08d41f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.023+0000 7f08d3400640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08d414fa50 0x7f08d41e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d2a00640 1 -- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d40c9eb0 msgr2=0x7f08d41f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d2a00640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d40c9eb0 0x7f08d41f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d2a00640 1 -- 172.21.15.102:0/2349509325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08d414fa50 msgr2=0x7f08d41e5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d2a00640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08d414fa50 0x7f08d41e5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d2a00640 1 -- 172.21.15.102:0/2349509325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d41e4a20 con 0x7f08d4073a10 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d2000640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d40c9eb0 0x7f08d41f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:18.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d2a00640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d4073a10 0x7f08d41f4c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c3d0 tx=0x7f08c400c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:18.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08caa00640 1 -- 172.21.15.102:0/2349509325 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4017070 con 0x7f08d4073a10 2024-04-20T21:17:18.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08caa00640 1 -- 172.21.15.102:0/2349509325 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c4005ce0 con 0x7f08d4073a10 2024-04-20T21:17:18.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08caa00640 1 -- 172.21.15.102:0/2349509325 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4007310 con 0x7f08d4073a10 2024-04-20T21:17:18.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d41e4740 con 0x7f08d4073a10 2024-04-20T21:17:18.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.024+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d41f4480 con 0x7f08d4073a10 2024-04-20T21:17:18.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.025+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d4074340 con 0x7f08d4073a10 2024-04-20T21:17:18.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.026+0000 7f08caa00640 1 -- 172.21.15.102:0/2349509325 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08c401a070 con 0x7f08d4073a10 2024-04-20T21:17:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.027+0000 7f08caa00640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f08b406c920 0x7f08b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.027+0000 7f08caa00640 1 -- 172.21.15.102:0/2349509325 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f08c408b1e0 con 0x7f08d4073a10 2024-04-20T21:17:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.027+0000 7f08d2000640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f08b406c920 0x7f08b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.028+0000 7f08d2000640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f08b406c920 0x7f08b406ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f08c0036000 tx=0x7f08c0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.032+0000 7f08caa00640 1 -- 172.21.15.102:0/2349509325 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f08d41536d0 con 0x7f08d4073a10 2024-04-20T21:17:18.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.231+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08d40c7320 con 0x7f08d4073a10 2024-04-20T21:17:18.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.233+0000 7f08caa00640 1 -- 172.21.15.102:0/2349509325 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f08c4058c70 con 0x7f08d4073a10 2024-04-20T21:17:18.234 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:18.234 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:18.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.236+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f08b406c920 msgr2=0x7f08b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.236+0000 7f08d9800640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f08b406c920 0x7f08b406ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f08c0036000 tx=0x7f08c0008030 comp rx=0 tx=0).stop 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.236+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d4073a10 msgr2=0x7f08d41f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.236+0000 7f08d9800640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d4073a10 0x7f08d41f4c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c3d0 tx=0x7f08c400c890 comp rx=0 tx=0).stop 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.237+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 shutdown_connections 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.237+0000 7f08d9800640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08d414fa50 0x7f08d41e5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.237+0000 7f08d9800640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d40c9eb0 0x7f08d41f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.237+0000 7f08d9800640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f08b406c920 0x7f08b406ede0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.237+0000 7f08d9800640 1 --2- 172.21.15.102:0/2349509325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d4073a10 0x7f08d41f4c80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.237+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 >> 172.21.15.102:0/2349509325 conn(0x7f08d40c9390 msgr2=0x7f08d414d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.237+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 shutdown_connections 2024-04-20T21:17:18.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.237+0000 7f08d9800640 1 -- 172.21.15.102:0/2349509325 wait complete. 2024-04-20T21:17:18.256 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:17:18.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.462+0000 7f9c69600640 1 Processor -- start 2024-04-20T21:17:18.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.462+0000 7f9c69600640 1 -- start start 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c69600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9c64153110 0x7f9c640c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c69600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c69600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c640c89f0 con 0x7f9c641534f0 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c69600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9c640c3920 con 0x7f9c64153110 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c69600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9c640c91b0 con 0x7f9c64158f70 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c62000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9c641534f0 0x7f9c6406f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38628/0 (socket says 172.21.15.102:38628) 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c62000640 1 -- 172.21.15.102:0/1565917159 learned_addr learned my addr 172.21.15.102:0/1565917159 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c63400640 1 --2- 172.21.15.102:0/1565917159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.464+0000 7f9c62a00640 1 --2- 172.21.15.102:0/1565917159 >> v2:172.21.15.193:3300/0 conn(0x7f9c64153110 0x7f9c640c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.465+0000 7f9c63400640 1 -- 172.21.15.102:0/1565917159 >> v2:172.21.15.193:3300/0 conn(0x7f9c64153110 msgr2=0x7f9c640c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:18.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.465+0000 7f9c63400640 1 --2- 172.21.15.102:0/1565917159 >> v2:172.21.15.193:3300/0 conn(0x7f9c64153110 0x7f9c640c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.465+0000 7f9c63400640 1 -- 172.21.15.102:0/1565917159 >> v1:172.21.15.102:6789/0 conn(0x7f9c641534f0 legacy=0x7f9c6406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.465+0000 7f9c63400640 1 -- 172.21.15.102:0/1565917159 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c640c8cd0 con 0x7f9c64158f70 2024-04-20T21:17:18.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.465+0000 7f9c61600640 1 -- 172.21.15.102:0/1565917159 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 467872209 0 0) 0x7f9c640c89f0 con 0x7f9c641534f0 2024-04-20T21:17:18.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.466+0000 7f9c63400640 1 --2- 172.21.15.102:0/1565917159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640c5e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009d90 tx=0x7f9c5c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff038b480750342d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:18.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.466+0000 7f9c61600640 1 -- 172.21.15.102:0/1565917159 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c039070 con 0x7f9c64158f70 2024-04-20T21:17:18.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.466+0000 7f9c61600640 1 -- 172.21.15.102:0/1565917159 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c5c02cbc0 con 0x7f9c64158f70 2024-04-20T21:17:18.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.466+0000 7f9c61600640 1 -- 172.21.15.102:0/1565917159 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c034440 con 0x7f9c64158f70 2024-04-20T21:17:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.467+0000 7f9c69600640 1 -- 172.21.15.102:0/1565917159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 msgr2=0x7f9c640c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.467+0000 7f9c69600640 1 --2- 172.21.15.102:0/1565917159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640c5e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009d90 tx=0x7f9c5c02be10 comp rx=0 tx=0).stop 2024-04-20T21:17:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.467+0000 7f9c69600640 1 -- 172.21.15.102:0/1565917159 shutdown_connections 2024-04-20T21:17:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.467+0000 7f9c69600640 1 --2- 172.21.15.102:0/1565917159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640c5e30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.467+0000 7f9c69600640 1 --2- 172.21.15.102:0/1565917159 >> v2:172.21.15.193:3300/0 conn(0x7f9c64153110 0x7f9c640c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.467+0000 7f9c69600640 1 -- 172.21.15.102:0/1565917159 >> 172.21.15.102:0/1565917159 conn(0x7f9c64074ea0 msgr2=0x7f9c64075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.467+0000 7f9c69600640 1 -- 172.21.15.102:0/1565917159 shutdown_connections 2024-04-20T21:17:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.467+0000 7f9c69600640 1 -- 172.21.15.102:0/1565917159 wait complete. 2024-04-20T21:17:18.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.469+0000 7f9c69600640 1 Processor -- start 2024-04-20T21:17:18.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.469+0000 7f9c69600640 1 -- start start 2024-04-20T21:17:18.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c69600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c64153110 0x7f9c641538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c69600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c69600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c64144840 0x7f9c6413f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c69600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9c640c99c0 con 0x7f9c64144840 2024-04-20T21:17:18.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c69600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9c640c96e0 con 0x7f9c64153110 2024-04-20T21:17:18.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c69600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9c640c9ee0 con 0x7f9c64158f70 2024-04-20T21:17:18.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c63400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c64144840 0x7f9c6413f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c63400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c64144840 0x7f9c6413f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51222/0 (socket says 172.21.15.102:51222) 2024-04-20T21:17:18.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c63400640 1 -- 172.21.15.102:0/2030908437 learned_addr learned my addr 172.21.15.102:0/2030908437 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:18.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c62a00640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c64153110 0x7f9c641538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c62000640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c63400640 1 -- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 msgr2=0x7f9c640ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c63400640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c63400640 1 -- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c64153110 msgr2=0x7f9c641538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c63400640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c64153110 0x7f9c641538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.470+0000 7f9c63400640 1 -- 172.21.15.102:0/2030908437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c64158670 con 0x7f9c64144840 2024-04-20T21:17:18.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.471+0000 7f9c63400640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c64144840 0x7f9c6413f1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009d90 tx=0x7f9c5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:18.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.471+0000 7f9c5aa00640 1 -- 172.21.15.102:0/2030908437 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c044020 con 0x7f9c64144840 2024-04-20T21:17:18.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.471+0000 7f9c5aa00640 1 -- 172.21.15.102:0/2030908437 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c5c03a040 con 0x7f9c64144840 2024-04-20T21:17:18.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.471+0000 7f9c5aa00640 1 -- 172.21.15.102:0/2030908437 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c039050 con 0x7f9c64144840 2024-04-20T21:17:18.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.471+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c64158910 con 0x7f9c64144840 2024-04-20T21:17:18.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.471+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c64158bd0 con 0x7f9c64144840 2024-04-20T21:17:18.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.473+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6406eb10 con 0x7f9c64144840 2024-04-20T21:17:18.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.474+0000 7f9c5aa00640 1 -- 172.21.15.102:0/2030908437 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c5c003bb0 con 0x7f9c64144840 2024-04-20T21:17:18.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.474+0000 7f9c5aa00640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c4406c920 0x7f9c4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:18.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.475+0000 7f9c5aa00640 1 -- 172.21.15.102:0/2030908437 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9c5c0ae330 con 0x7f9c64144840 2024-04-20T21:17:18.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.475+0000 7f9c62a00640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c4406c920 0x7f9c4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:18.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.475+0000 7f9c62a00640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c4406c920 0x7f9c4406ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009aa0 tx=0x7f9c54009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:18.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.479+0000 7f9c5aa00640 1 -- 172.21.15.102:0/2030908437 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9c6406eb10 con 0x7f9c64144840 2024-04-20T21:17:18.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.722+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9c640c7d40 con 0x7f9c64144840 2024-04-20T21:17:18.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.781+0000 7f9c5aa00640 1 -- 172.21.15.102:0/2030908437 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f9c5c04e060 con 0x7f9c64144840 2024-04-20T21:17:18.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.784+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c4406c920 msgr2=0x7f9c4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.784+0000 7f9c69600640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c4406c920 0x7f9c4406ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c54009aa0 tx=0x7f9c54009190 comp rx=0 tx=0).stop 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.784+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c64144840 msgr2=0x7f9c6413f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.784+0000 7f9c69600640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c64144840 0x7f9c6413f1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009d90 tx=0x7f9c5c004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.785+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 shutdown_connections 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.785+0000 7f9c69600640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c64144840 0x7f9c6413f1b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.785+0000 7f9c69600640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c64158f70 0x7f9c640ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.785+0000 7f9c69600640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c4406c920 0x7f9c4406ede0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.785+0000 7f9c69600640 1 --2- 172.21.15.102:0/2030908437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c64153110 0x7f9c641538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.785+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 >> 172.21.15.102:0/2030908437 conn(0x7f9c64074ea0 msgr2=0x7f9c640c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.785+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 shutdown_connections 2024-04-20T21:17:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:18.785+0000 7f9c69600640 1 -- 172.21.15.102:0/2030908437 wait complete. 2024-04-20T21:17:18.803 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300} 2024-04-20T21:17:18.803 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:17:19.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.001+0000 7f5bbb200640 1 Processor -- start 2024-04-20T21:17:19.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.001+0000 7f5bbb200640 1 -- start start 2024-04-20T21:17:19.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bbb200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5bbc0c5ea0 0x7f5bbc073580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:19.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bbb200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5bbc06ec00 0x7f5bbc06f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:19.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bbb200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bbc0c53b0 con 0x7f5bbc157ef0 2024-04-20T21:17:19.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bbb200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5bbc073e80 con 0x7f5bbc06ec00 2024-04-20T21:17:19.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bbb200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f5bbc073ba0 con 0x7f5bbc0c5ea0 2024-04-20T21:17:19.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bba800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5bbc06ec00 0x7f5bbc06f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:19.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bb1400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5bbc157ef0 0x7f5bbc0c4ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38636/0 (socket says 172.21.15.102:38636) 2024-04-20T21:17:19.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bb1400640 1 -- 172.21.15.102:0/2175229259 learned_addr learned my addr 172.21.15.102:0/2175229259 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:19.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.003+0000 7f5bb9e00640 1 --2- 172.21.15.102:0/2175229259 >> v2:172.21.15.193:3300/0 conn(0x7f5bbc0c5ea0 0x7f5bbc073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:19.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.004+0000 7f5bb9e00640 1 -- 172.21.15.102:0/2175229259 >> v2:172.21.15.102:3300/0 conn(0x7f5bbc06ec00 msgr2=0x7f5bbc06f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:19.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.004+0000 7f5bb9e00640 1 --2- 172.21.15.102:0/2175229259 >> v2:172.21.15.102:3300/0 conn(0x7f5bbc06ec00 0x7f5bbc06f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.004+0000 7f5bb9e00640 1 -- 172.21.15.102:0/2175229259 >> v1:172.21.15.102:6789/0 conn(0x7f5bbc157ef0 legacy=0x7f5bbc0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:19.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.004+0000 7f5bb9e00640 1 -- 172.21.15.102:0/2175229259 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc0c3c50 con 0x7f5bbc0c5ea0 2024-04-20T21:17:19.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.005+0000 7f5bb9e00640 1 --2- 172.21.15.102:0/2175229259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc0c5ea0 0x7f5bbc073580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc069d70 tx=0x7f5bac02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9485d0ac6878fb41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:19.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.005+0000 7f5bb9400640 1 -- 172.21.15.102:0/2175229259 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac035020 con 0x7f5bbc0c5ea0 2024-04-20T21:17:19.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.005+0000 7f5bb9400640 1 -- 172.21.15.102:0/2175229259 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bac031830 con 0x7f5bbc0c5ea0 2024-04-20T21:17:19.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.005+0000 7f5bb9400640 1 -- 172.21.15.102:0/2175229259 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac031c00 con 0x7f5bbc0c5ea0 2024-04-20T21:17:19.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.006+0000 7f5bbb200640 1 -- 172.21.15.102:0/2175229259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc0c5ea0 msgr2=0x7f5bbc073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:19.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.006+0000 7f5bbb200640 1 --2- 172.21.15.102:0/2175229259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc0c5ea0 0x7f5bbc073580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc069d70 tx=0x7f5bac02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.006+0000 7f5bbb200640 1 -- 172.21.15.102:0/2175229259 shutdown_connections 2024-04-20T21:17:19.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.006+0000 7f5bbb200640 1 --2- 172.21.15.102:0/2175229259 >> v2:172.21.15.102:3300/0 conn(0x7f5bbc06ec00 0x7f5bbc06f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.006+0000 7f5bbb200640 1 --2- 172.21.15.102:0/2175229259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc0c5ea0 0x7f5bbc073580 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.006+0000 7f5bbb200640 1 -- 172.21.15.102:0/2175229259 >> 172.21.15.102:0/2175229259 conn(0x7f5bbc0cc620 msgr2=0x7f5bbc0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:19.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.006+0000 7f5bbb200640 1 -- 172.21.15.102:0/2175229259 shutdown_connections 2024-04-20T21:17:19.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.006+0000 7f5bbb200640 1 -- 172.21.15.102:0/2175229259 wait complete. 2024-04-20T21:17:19.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.008+0000 7f5bbb200640 1 Processor -- start 2024-04-20T21:17:19.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.008+0000 7f5bbb200640 1 -- start start 2024-04-20T21:17:19.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bbc06ec00 0x7f5bbc0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bbc0c5ea0 0x7f5bbc1f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc157ef0 0x7f5bbc14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bbb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc0c44f0 con 0x7f5bbc06ec00 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bbb200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc0c4210 con 0x7f5bbc157ef0 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bbb200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc158360 con 0x7f5bbc0c5ea0 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bb9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bbc06ec00 0x7f5bbc0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bb9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bbc06ec00 0x7f5bbc0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51254/0 (socket says 172.21.15.102:51254) 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bb9e00640 1 -- 172.21.15.102:0/3925476448 learned_addr learned my addr 172.21.15.102:0/3925476448 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bba800640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc157ef0 0x7f5bbc14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.009+0000 7f5bb1400640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bbc0c5ea0 0x7f5bbc1f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb9e00640 1 -- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bbc0c5ea0 msgr2=0x7f5bbc1f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb9e00640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bbc0c5ea0 0x7f5bbc1f0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb9e00640 1 -- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc157ef0 msgr2=0x7f5bbc14e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:19.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb9e00640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc157ef0 0x7f5bbc14e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb9e00640 1 -- 172.21.15.102:0/3925476448 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc0be930 con 0x7f5bbc06ec00 2024-04-20T21:17:19.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb1400640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bbc0c5ea0 0x7f5bbc1f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:19.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb9e00640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bbc06ec00 0x7f5bbc0cba60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5bac02bee0 tx=0x7f5bac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:19.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb2a00640 1 -- 172.21.15.102:0/3925476448 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac041070 con 0x7f5bbc06ec00 2024-04-20T21:17:19.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb2a00640 1 -- 172.21.15.102:0/3925476448 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bac03f070 con 0x7f5bbc06ec00 2024-04-20T21:17:19.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bb2a00640 1 -- 172.21.15.102:0/3925476448 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac035040 con 0x7f5bbc06ec00 2024-04-20T21:17:19.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.010+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc0c20c0 con 0x7f5bbc06ec00 2024-04-20T21:17:19.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.011+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc1f4780 con 0x7f5bbc06ec00 2024-04-20T21:17:19.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.012+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc0c55b0 con 0x7f5bbc06ec00 2024-04-20T21:17:19.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.013+0000 7f5bb2a00640 1 -- 172.21.15.102:0/3925476448 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bac038020 con 0x7f5bbc06ec00 2024-04-20T21:17:19.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.014+0000 7f5bb2a00640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5b9806c8f0 0x7f5b9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:19.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.014+0000 7f5bb2a00640 1 -- 172.21.15.102:0/3925476448 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5bac0ad370 con 0x7f5bbc06ec00 2024-04-20T21:17:19.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.014+0000 7f5bb1400640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5b9806c8f0 0x7f5b9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:19.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.015+0000 7f5bb1400640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5b9806c8f0 0x7f5b9806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc15aae0 tx=0x7f5ba4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.019+0000 7f5bb2a00640 1 -- 172.21.15.102:0/3925476448 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5bbc0c55b0 con 0x7f5bbc06ec00 2024-04-20T21:17:19.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.257+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5bbc06c740 con 0x7f5bbc06ec00 2024-04-20T21:17:19.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.784+0000 7f5bb2a00640 1 -- 172.21.15.102:0/3925476448 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f5bbc06c740 con 0x7f5bbc06ec00 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5b9806c8f0 msgr2=0x7f5b9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5b9806c8f0 0x7f5b9806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc15aae0 tx=0x7f5ba4009040 comp rx=0 tx=0).stop 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bbc06ec00 msgr2=0x7f5bbc0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bbc06ec00 0x7f5bbc0cba60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5bac02bee0 tx=0x7f5bac0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 shutdown_connections 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5b9806c8f0 0x7f5b9806edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bbc157ef0 0x7f5bbc14e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bbc0c5ea0 0x7f5bbc1f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 --2- 172.21.15.102:0/3925476448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bbc06ec00 0x7f5bbc0cba60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 >> 172.21.15.102:0/3925476448 conn(0x7f5bbc0cc620 msgr2=0x7f5bbc06e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.788+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 shutdown_connections 2024-04-20T21:17:19.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:19.789+0000 7f5bbb200640 1 -- 172.21.15.102:0/3925476448 wait complete. 2024-04-20T21:17:19.810 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-20T21:17:20.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.010+0000 7f43b0400640 1 Processor -- start 2024-04-20T21:17:20.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.010+0000 7f43b0400640 1 -- start start 2024-04-20T21:17:20.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43b0400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f43a80cc6a0 0x7f43a80c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:20.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43b0400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f43a8073740 0x7f43a80cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:20.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43b0400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a815bd00 con 0x7f43a8155620 2024-04-20T21:17:20.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43b0400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f43a80cb9b0 con 0x7f43a8073740 2024-04-20T21:17:20.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43b0400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f43a80c59b0 con 0x7f43a80cc6a0 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43ae400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f43a8073740 0x7f43a80cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43ad000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f43a80cc6a0 0x7f43a80c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43ada00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f43a8155620 0x7f43a806c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49008/0 (socket says 172.21.15.102:49008) 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.012+0000 7f43ada00640 1 -- 172.21.15.102:0/3517317924 learned_addr learned my addr 172.21.15.102:0/3517317924 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.013+0000 7f43ad000640 1 -- 172.21.15.102:0/3517317924 >> v2:172.21.15.102:3300/0 conn(0x7f43a8073740 msgr2=0x7f43a80cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.013+0000 7f43ad000640 1 --2- 172.21.15.102:0/3517317924 >> v2:172.21.15.102:3300/0 conn(0x7f43a8073740 0x7f43a80cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.013+0000 7f43ad000640 1 -- 172.21.15.102:0/3517317924 >> v1:172.21.15.193:6789/0 conn(0x7f43a8155620 legacy=0x7f43a806c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.013+0000 7f43ad000640 1 -- 172.21.15.102:0/3517317924 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a815bfe0 con 0x7f43a80cc6a0 2024-04-20T21:17:20.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.013+0000 7f43a7e00640 1 -- 172.21.15.102:0/3517317924 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 284442991 0 0) 0x7f43a815bd00 con 0x7f43a8155620 2024-04-20T21:17:20.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.014+0000 7f43ad000640 1 --2- 172.21.15.102:0/3517317924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 0x7f43a80c66e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43a80ce240 tx=0x7f439802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cae87cb59f124de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:20.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.014+0000 7f43a7e00640 1 -- 172.21.15.102:0/3517317924 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398039070 con 0x7f43a80cc6a0 2024-04-20T21:17:20.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.014+0000 7f43a7e00640 1 -- 172.21.15.102:0/3517317924 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439802cc80 con 0x7f43a80cc6a0 2024-04-20T21:17:20.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.014+0000 7f43a7e00640 1 -- 172.21.15.102:0/3517317924 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398034660 con 0x7f43a80cc6a0 2024-04-20T21:17:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.015+0000 7f43b0400640 1 -- 172.21.15.102:0/3517317924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 msgr2=0x7f43a80c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.015+0000 7f43b0400640 1 --2- 172.21.15.102:0/3517317924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 0x7f43a80c66e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43a80ce240 tx=0x7f439802bed0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.015+0000 7f43b0400640 1 -- 172.21.15.102:0/3517317924 shutdown_connections 2024-04-20T21:17:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.015+0000 7f43b0400640 1 --2- 172.21.15.102:0/3517317924 >> v2:172.21.15.102:3300/0 conn(0x7f43a8073740 0x7f43a80cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.015+0000 7f43b0400640 1 --2- 172.21.15.102:0/3517317924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 0x7f43a80c66e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.015+0000 7f43b0400640 1 -- 172.21.15.102:0/3517317924 >> 172.21.15.102:0/3517317924 conn(0x7f43a80754d0 msgr2=0x7f43a8072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.015+0000 7f43b0400640 1 -- 172.21.15.102:0/3517317924 shutdown_connections 2024-04-20T21:17:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.015+0000 7f43b0400640 1 -- 172.21.15.102:0/3517317924 wait complete. 2024-04-20T21:17:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.018+0000 7f43b0400640 1 Processor -- start 2024-04-20T21:17:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.018+0000 7f43b0400640 1 -- start start 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43b0400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43a8073740 0x7f43a80b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43b0400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 0x7f43a8150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43b0400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43a8155620 0x7f43a81f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43b0400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f43a80c5ee0 con 0x7f43a8073740 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43b0400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8073bb0 con 0x7f43a80cc6a0 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43b0400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f43a80c61c0 con 0x7f43a8155620 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43ada00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43a8073740 0x7f43a80b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43ada00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43a8073740 0x7f43a80b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51268/0 (socket says 172.21.15.102:51268) 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.019+0000 7f43ada00640 1 -- 172.21.15.102:0/1964360957 learned_addr learned my addr 172.21.15.102:0/1964360957 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43ad000640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 0x7f43a8150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43ae400640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43a8155620 0x7f43a81f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43ada00640 1 -- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43a8155620 msgr2=0x7f43a81f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:20.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43ada00640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43a8155620 0x7f43a81f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43ada00640 1 -- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 msgr2=0x7f43a8150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:20.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43ada00640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 0x7f43a8150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43ada00640 1 -- 172.21.15.102:0/1964360957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a80c9650 con 0x7f43a8073740 2024-04-20T21:17:20.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43ada00640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43a8073740 0x7f43a80b5a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c3f0 tx=0x7f439c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:20.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.020+0000 7f43a5600640 1 -- 172.21.15.102:0/1964360957 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c017070 con 0x7f43a8073740 2024-04-20T21:17:20.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.021+0000 7f43a5600640 1 -- 172.21.15.102:0/1964360957 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c005ce0 con 0x7f43a8073740 2024-04-20T21:17:20.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.021+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a80c7df0 con 0x7f43a8073740 2024-04-20T21:17:20.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.021+0000 7f43a5600640 1 -- 172.21.15.102:0/1964360957 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c007310 con 0x7f43a8073740 2024-04-20T21:17:20.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.021+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a815a7b0 con 0x7f43a8073740 2024-04-20T21:17:20.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.022+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a80ce100 con 0x7f43a8073740 2024-04-20T21:17:20.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.023+0000 7f43a5600640 1 -- 172.21.15.102:0/1964360957 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f439c01a070 con 0x7f43a8073740 2024-04-20T21:17:20.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.024+0000 7f43a5600640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f438c06c8f0 0x7f438c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:20.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.024+0000 7f43a5600640 1 -- 172.21.15.102:0/1964360957 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f439c08b190 con 0x7f43a8073740 2024-04-20T21:17:20.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.025+0000 7f43ad000640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f438c06c8f0 0x7f438c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:20.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.025+0000 7f43ad000640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f438c06c8f0 0x7f438c06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4398036000 tx=0x7f4398030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.028+0000 7f43a5600640 1 -- 172.21.15.102:0/1964360957 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f43a80ce100 con 0x7f43a8073740 2024-04-20T21:17:20.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.268+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f43a804fdd0 con 0x7f43a8073740 2024-04-20T21:17:20.782 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:20.790 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:20.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.792+0000 7f43a5600640 1 -- 172.21.15.102:0/1964360957 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v9) ==== 115+0+0 (secure 0 0 0) 0x7f43a80c9650 con 0x7f43a8073740 2024-04-20T21:17:20.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f438c06c8f0 msgr2=0x7f438c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:20.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f438c06c8f0 0x7f438c06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4398036000 tx=0x7f4398030000 comp rx=0 tx=0).stop 2024-04-20T21:17:20.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43a8073740 msgr2=0x7f43a80b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:20.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43a8073740 0x7f43a80b5a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c3f0 tx=0x7f439c00c8b0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 shutdown_connections 2024-04-20T21:17:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43a8155620 0x7f43a81f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f438c06c8f0 0x7f438c06edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43a80cc6a0 0x7f43a8150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 --2- 172.21.15.102:0/1964360957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43a8073740 0x7f43a80b5a70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.796+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 >> 172.21.15.102:0/1964360957 conn(0x7f43a80754d0 msgr2=0x7f43a81544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.797+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 shutdown_connections 2024-04-20T21:17:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:20.797+0000 7f43b0400640 1 -- 172.21.15.102:0/1964360957 wait complete. 2024-04-20T21:17:20.818 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:17:20.819 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:21.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.053+0000 7f90dbe00640 1 Processor -- start 2024-04-20T21:17:21.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.053+0000 7f90dbe00640 1 -- start start 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90dbe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 0x7f90dc158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90dbe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f90dc0c75a0 0x7f90dc1637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90dbe00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90dc153560 con 0x7f90dc159380 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90dbe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f90dc0c7a70 con 0x7f90dc0c75a0 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90dbe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0c7d30 con 0x7f90dc153180 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90db400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f90dc0c75a0 0x7f90dc1637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90db400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f90dc0c75a0 0x7f90dc1637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51272/0 (socket says 172.21.15.102:51272) 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90db400640 1 -- 172.21.15.102:0/2009282903 learned_addr learned my addr 172.21.15.102:0/2009282903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90da000640 1 --1- 172.21.15.102:0/2009282903 >> v1:172.21.15.193:6789/0 conn(0x7f90dc159380 0x7f90dc0c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49020/0 (socket says 172.21.15.102:49020) 2024-04-20T21:17:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.055+0000 7f90daa00640 1 --2- 172.21.15.102:0/2009282903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 0x7f90dc158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.056+0000 7f90d9600640 1 -- 172.21.15.102:0/2009282903 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 552385111 0 0) 0x7f90dc153560 con 0x7f90dc159380 2024-04-20T21:17:21.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.056+0000 7f90db400640 1 -- 172.21.15.102:0/2009282903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 msgr2=0x7f90dc158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:21.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.056+0000 7f90db400640 1 --2- 172.21.15.102:0/2009282903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 0x7f90dc158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.056+0000 7f90db400640 1 -- 172.21.15.102:0/2009282903 >> v1:172.21.15.193:6789/0 conn(0x7f90dc159380 legacy=0x7f90dc0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.056+0000 7f90db400640 1 -- 172.21.15.102:0/2009282903 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc15c4d0 con 0x7f90dc0c75a0 2024-04-20T21:17:21.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.057+0000 7f90db400640 1 --2- 172.21.15.102:0/2009282903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc0c75a0 0x7f90dc1637c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009e50 tx=0x7f90d402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b61cdceb6abf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.057+0000 7f90d9600640 1 -- 172.21.15.102:0/2009282903 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d402cb00 con 0x7f90dc0c75a0 2024-04-20T21:17:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.057+0000 7f90d9600640 1 -- 172.21.15.102:0/2009282903 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d402cca0 con 0x7f90dc0c75a0 2024-04-20T21:17:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.057+0000 7f90d9600640 1 -- 172.21.15.102:0/2009282903 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4034610 con 0x7f90dc0c75a0 2024-04-20T21:17:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.058+0000 7f90dbe00640 1 -- 172.21.15.102:0/2009282903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc0c75a0 msgr2=0x7f90dc1637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.058+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2009282903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc0c75a0 0x7f90dc1637c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009e50 tx=0x7f90d402bf60 comp rx=0 tx=0).stop 2024-04-20T21:17:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.058+0000 7f90dbe00640 1 -- 172.21.15.102:0/2009282903 shutdown_connections 2024-04-20T21:17:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.058+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2009282903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc0c75a0 0x7f90dc1637c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.058+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2009282903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 0x7f90dc158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.058+0000 7f90dbe00640 1 -- 172.21.15.102:0/2009282903 >> 172.21.15.102:0/2009282903 conn(0x7f90dc0cc820 msgr2=0x7f90dc0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.058+0000 7f90dbe00640 1 -- 172.21.15.102:0/2009282903 shutdown_connections 2024-04-20T21:17:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.058+0000 7f90dbe00640 1 -- 172.21.15.102:0/2009282903 wait complete. 2024-04-20T21:17:21.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.060+0000 7f90dbe00640 1 Processor -- start 2024-04-20T21:17:21.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.060+0000 7f90dbe00640 1 -- start start 2024-04-20T21:17:21.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90dbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90dc0c75a0 0x7f90dc15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90dbe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 0x7f90dc144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc159380 0x7f90dc145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90dbe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc15d290 con 0x7f90dc159380 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90dbe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc15ca90 con 0x7f90dc0c75a0 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90dbe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0c6be0 con 0x7f90dc153180 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90db400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc159380 0x7f90dc145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90db400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc159380 0x7f90dc145490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51280/0 (socket says 172.21.15.102:51280) 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90db400640 1 -- 172.21.15.102:0/2044559601 learned_addr learned my addr 172.21.15.102:0/2044559601 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90daa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90dc0c75a0 0x7f90dc15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90da000640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 0x7f90dc144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.061+0000 7f90db400640 1 -- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 msgr2=0x7f90dc144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90db400640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 0x7f90dc144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90db400640 1 -- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90dc0c75a0 msgr2=0x7f90dc15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90db400640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90dc0c75a0 0x7f90dc15aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90db400640 1 -- 172.21.15.102:0/2044559601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc1579b0 con 0x7f90dc159380 2024-04-20T21:17:21.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90daa00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90dc0c75a0 0x7f90dc15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:21.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90db400640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc159380 0x7f90dc145490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009e50 tx=0x7f90d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90d2a00640 1 -- 172.21.15.102:0/2044559601 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4043020 con 0x7f90dc159380 2024-04-20T21:17:21.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90d2a00640 1 -- 172.21.15.102:0/2044559601 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d402ce50 con 0x7f90dc159380 2024-04-20T21:17:21.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90d2a00640 1 -- 172.21.15.102:0/2044559601 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d403e690 con 0x7f90dc159380 2024-04-20T21:17:21.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc1459d0 con 0x7f90dc159380 2024-04-20T21:17:21.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.062+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc0c5bb0 con 0x7f90dc159380 2024-04-20T21:17:21.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.063+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc04fd60 con 0x7f90dc159380 2024-04-20T21:17:21.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.064+0000 7f90d2a00640 1 -- 172.21.15.102:0/2044559601 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90d4046020 con 0x7f90dc159380 2024-04-20T21:17:21.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.065+0000 7f90d2a00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90bc06c8f0 0x7f90bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.065+0000 7f90d2a00640 1 -- 172.21.15.102:0/2044559601 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f90d40ae360 con 0x7f90dc159380 2024-04-20T21:17:21.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.066+0000 7f90daa00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90bc06c8f0 0x7f90bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.066+0000 7f90daa00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90bc06c8f0 0x7f90bc06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009e90 tx=0x7f90cc009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.070+0000 7f90d2a00640 1 -- 172.21.15.102:0/2044559601 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f90d404d030 con 0x7f90dc159380 2024-04-20T21:17:21.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.308+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90dc1529a0 con 0x7f90dc159380 2024-04-20T21:17:21.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.309+0000 7f90d2a00640 1 -- 172.21.15.102:0/2044559601 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f90d407bd60 con 0x7f90dc159380 2024-04-20T21:17:21.309 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:21.309 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:17:21.309 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:17:21.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.311+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90bc06c8f0 msgr2=0x7f90bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.311+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90bc06c8f0 0x7f90bc06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009e90 tx=0x7f90cc009190 comp rx=0 tx=0).stop 2024-04-20T21:17:21.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.311+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc159380 msgr2=0x7f90dc145490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.311+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc159380 0x7f90dc145490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009e50 tx=0x7f90d4004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:21.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.312+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 shutdown_connections 2024-04-20T21:17:21.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.312+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90dc159380 0x7f90dc145490 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.312+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90dc153180 0x7f90dc144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.312+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90bc06c8f0 0x7f90bc06edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.312+0000 7f90dbe00640 1 --2- 172.21.15.102:0/2044559601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90dc0c75a0 0x7f90dc15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.312+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 >> 172.21.15.102:0/2044559601 conn(0x7f90dc0cc820 msgr2=0x7f90dc148440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:21.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.312+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 shutdown_connections 2024-04-20T21:17:21.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.312+0000 7f90dbe00640 1 -- 172.21.15.102:0/2044559601 wait complete. 2024-04-20T21:17:21.331 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:21.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.519+0000 7ff7b2a00640 1 Processor -- start 2024-04-20T21:17:21.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.519+0000 7ff7b2a00640 1 -- start start 2024-04-20T21:17:21.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff7ac1516d0 0x7ff7ac0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac151ab0 0x7ff7ac157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff7ac0c3d20 0x7ff7ac0c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac0c4dd0 con 0x7ff7ac1516d0 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac06ef60 con 0x7ff7ac0c3d20 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac06f240 con 0x7ff7ac151ab0 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b0c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac151ab0 0x7ff7ac157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff7ac0c3d20 0x7ff7ac0c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff7ac0c3d20 0x7ff7ac0c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56626/0 (socket says 172.21.15.102:56626) 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b0c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac151ab0 0x7ff7ac157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55124/0 (socket says 172.21.15.102:55124) 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b2000640 1 -- 172.21.15.102:0/3687050146 learned_addr learned my addr 172.21.15.102:0/3687050146 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.520+0000 7ff7b1600640 1 --2- 172.21.15.102:0/3687050146 >> v2:172.21.15.102:3300/0 conn(0x7ff7ac1516d0 0x7ff7ac0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.521+0000 7ff7b2000640 1 -- 172.21.15.102:0/3687050146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac151ab0 msgr2=0x7ff7ac157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.521+0000 7ff7b2000640 1 --2- 172.21.15.102:0/3687050146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac151ab0 0x7ff7ac157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.521+0000 7ff7b2000640 1 -- 172.21.15.102:0/3687050146 >> v2:172.21.15.102:3300/0 conn(0x7ff7ac1516d0 msgr2=0x7ff7ac0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:21.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.521+0000 7ff7b2000640 1 --2- 172.21.15.102:0/3687050146 >> v2:172.21.15.102:3300/0 conn(0x7ff7ac1516d0 0x7ff7ac0cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.521+0000 7ff7b2000640 1 -- 172.21.15.102:0/3687050146 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac06f520 con 0x7ff7ac0c3d20 2024-04-20T21:17:21.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.521+0000 7ff7b0c00640 1 --2- 172.21.15.102:0/3687050146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac151ab0 0x7ff7ac157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:21.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.522+0000 7ff7b2000640 1 --2- 172.21.15.102:0/3687050146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 0x7ff7ac0c41b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac074030 tx=0x7ff7a402b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8764e9c7c6cf0097 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.522+0000 7ff7abe00640 1 -- 172.21.15.102:0/3687050146 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4034070 con 0x7ff7ac0c3d20 2024-04-20T21:17:21.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.522+0000 7ff7abe00640 1 -- 172.21.15.102:0/3687050146 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a4030850 con 0x7ff7ac0c3d20 2024-04-20T21:17:21.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.522+0000 7ff7abe00640 1 -- 172.21.15.102:0/3687050146 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4030c20 con 0x7ff7ac0c3d20 2024-04-20T21:17:21.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 -- 172.21.15.102:0/3687050146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 msgr2=0x7ff7ac0c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/3687050146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 0x7ff7ac0c41b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac074030 tx=0x7ff7a402b990 comp rx=0 tx=0).stop 2024-04-20T21:17:21.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 -- 172.21.15.102:0/3687050146 shutdown_connections 2024-04-20T21:17:21.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/3687050146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 0x7ff7ac0c41b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/3687050146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac151ab0 0x7ff7ac157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/3687050146 >> v2:172.21.15.102:3300/0 conn(0x7ff7ac1516d0 0x7ff7ac0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 -- 172.21.15.102:0/3687050146 >> 172.21.15.102:0/3687050146 conn(0x7ff7ac0cc820 msgr2=0x7ff7ac0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:21.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 -- 172.21.15.102:0/3687050146 shutdown_connections 2024-04-20T21:17:21.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.523+0000 7ff7b2a00640 1 -- 172.21.15.102:0/3687050146 wait complete. 2024-04-20T21:17:21.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.524+0000 7ff7b2a00640 1 Processor -- start 2024-04-20T21:17:21.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.525+0000 7ff7b2a00640 1 -- start start 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.525+0000 7ff7b2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 0x7ff7ac1efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.525+0000 7ff7b2a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac1516d0 0x7ff7ac1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.525+0000 7ff7b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7ac151ab0 0x7ff7ac14d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.525+0000 7ff7b2a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0c4a40 con 0x7ff7ac151ab0 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.525+0000 7ff7b2a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0c5300 con 0x7ff7ac0c3d20 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.525+0000 7ff7b2a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac0c4bc0 con 0x7ff7ac1516d0 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7ac151ab0 0x7ff7ac14d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 0x7ff7ac1efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b0c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac1516d0 0x7ff7ac1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7ac151ab0 0x7ff7ac14d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51300/0 (socket says 172.21.15.102:51300) 2024-04-20T21:17:21.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 -- 172.21.15.102:0/2064850796 learned_addr learned my addr 172.21.15.102:0/2064850796 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:21.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 -- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac1516d0 msgr2=0x7ff7ac1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac1516d0 0x7ff7ac1546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 -- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 msgr2=0x7ff7ac1efcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 0x7ff7ac1efcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 -- 172.21.15.102:0/2064850796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac14e4d0 con 0x7ff7ac151ab0 2024-04-20T21:17:21.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b1600640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 0x7ff7ac1efcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:21.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.526+0000 7ff7b2000640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7ac151ab0 0x7ff7ac14d5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40090b0 tx=0x7ff7a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.527+0000 7ff7a9600640 1 -- 172.21.15.102:0/2064850796 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4048070 con 0x7ff7ac151ab0 2024-04-20T21:17:21.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.527+0000 7ff7a9600640 1 -- 172.21.15.102:0/2064850796 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a4030650 con 0x7ff7ac151ab0 2024-04-20T21:17:21.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.527+0000 7ff7a9600640 1 -- 172.21.15.102:0/2064850796 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4034040 con 0x7ff7ac151ab0 2024-04-20T21:17:21.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.527+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac14e7b0 con 0x7ff7ac151ab0 2024-04-20T21:17:21.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.527+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac14ed70 con 0x7ff7ac151ab0 2024-04-20T21:17:21.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.528+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ac04fd60 con 0x7ff7ac151ab0 2024-04-20T21:17:21.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.529+0000 7ff7a9600640 1 -- 172.21.15.102:0/2064850796 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7a404b070 con 0x7ff7ac151ab0 2024-04-20T21:17:21.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.530+0000 7ff7a9600640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff78c06c8f0 0x7ff78c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.530+0000 7ff7a9600640 1 -- 172.21.15.102:0/2064850796 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff7a40adcf0 con 0x7ff7ac151ab0 2024-04-20T21:17:21.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.530+0000 7ff7b1600640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff78c06c8f0 0x7ff78c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.531+0000 7ff7b1600640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff78c06c8f0 0x7ff78c06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e6c0 tx=0x7ff7a000d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.534+0000 7ff7a9600640 1 -- 172.21.15.102:0/2064850796 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff7a40b2070 con 0x7ff7ac151ab0 2024-04-20T21:17:21.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.737+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7ac0c44a0 con 0x7ff7ac151ab0 2024-04-20T21:17:21.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.740+0000 7ff7a9600640 1 -- 172.21.15.102:0/2064850796 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7ff7a407b790 con 0x7ff7ac151ab0 2024-04-20T21:17:21.740 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:21.740 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:21.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff78c06c8f0 msgr2=0x7ff78c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff78c06c8f0 0x7ff78c06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e6c0 tx=0x7ff7a000d040 comp rx=0 tx=0).stop 2024-04-20T21:17:21.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7ac151ab0 msgr2=0x7ff7ac14d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7ac151ab0 0x7ff7ac14d5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40090b0 tx=0x7ff7a4004730 comp rx=0 tx=0).stop 2024-04-20T21:17:21.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 shutdown_connections 2024-04-20T21:17:21.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7ac151ab0 0x7ff7ac14d5a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7ac1516d0 0x7ff7ac1546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff78c06c8f0 0x7ff78c06edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 --2- 172.21.15.102:0/2064850796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ac0c3d20 0x7ff7ac1efcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.742+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 >> 172.21.15.102:0/2064850796 conn(0x7ff7ac0cc820 msgr2=0x7ff7ac0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:21.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.743+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 shutdown_connections 2024-04-20T21:17:21.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.743+0000 7ff7b2a00640 1 -- 172.21.15.102:0/2064850796 wait complete. 2024-04-20T21:17:21.761 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:17:21.761 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:21.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.960+0000 7f31a0a00640 1 Processor -- start 2024-04-20T21:17:21.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.960+0000 7f31a0a00640 1 -- start start 2024-04-20T21:17:21.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f31a0a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f319c14fa50 0x7f319c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f31a0a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f319c1536d0 con 0x7f319c073a10 2024-04-20T21:17:21.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f31a0a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f319c0c76c0 con 0x7f319c0c9eb0 2024-04-20T21:17:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f31a0a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f319c0c7960 con 0x7f319c14fa50 2024-04-20T21:17:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f3199e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f319c14fa50 0x7f319c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f319a800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f319c073a10 0x7f319c163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38642/0 (socket says 172.21.15.102:38642) 2024-04-20T21:17:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f319a800640 1 -- 172.21.15.102:0/973702025 learned_addr learned my addr 172.21.15.102:0/973702025 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f3199e00640 1 -- 172.21.15.102:0/973702025 >> v1:172.21.15.193:6789/0 conn(0x7f319c0c9eb0 legacy=0x7f319c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f3199e00640 1 -- 172.21.15.102:0/973702025 >> v1:172.21.15.102:6789/0 conn(0x7f319c073a10 legacy=0x7f319c163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.962+0000 7f3199400640 1 -- 172.21.15.102:0/973702025 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743454857 0 0) 0x7f319c0c76c0 con 0x7f319c0c9eb0 2024-04-20T21:17:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.963+0000 7f3199e00640 1 -- 172.21.15.102:0/973702025 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c0c3d90 con 0x7f319c14fa50 2024-04-20T21:17:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.964+0000 7f3199e00640 1 --2- 172.21.15.102:0/973702025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c14fa50 0x7f319c0c4830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f319c0c7890 tx=0x7f318c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b25916e725e73a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.964+0000 7f3199400640 1 -- 172.21.15.102:0/973702025 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c039070 con 0x7f319c14fa50 2024-04-20T21:17:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.964+0000 7f3199400640 1 -- 172.21.15.102:0/973702025 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c0316f0 con 0x7f319c14fa50 2024-04-20T21:17:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.964+0000 7f3199400640 1 -- 172.21.15.102:0/973702025 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c031ac0 con 0x7f319c14fa50 2024-04-20T21:17:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.964+0000 7f31a0a00640 1 -- 172.21.15.102:0/973702025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c14fa50 msgr2=0x7f319c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.964+0000 7f31a0a00640 1 --2- 172.21.15.102:0/973702025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c14fa50 0x7f319c0c4830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f319c0c7890 tx=0x7f318c02b810 comp rx=0 tx=0).stop 2024-04-20T21:17:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.965+0000 7f31a0a00640 1 -- 172.21.15.102:0/973702025 shutdown_connections 2024-04-20T21:17:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.965+0000 7f31a0a00640 1 --2- 172.21.15.102:0/973702025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c14fa50 0x7f319c0c4830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.965+0000 7f31a0a00640 1 -- 172.21.15.102:0/973702025 >> 172.21.15.102:0/973702025 conn(0x7f319c0c9390 msgr2=0x7f319c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.965+0000 7f31a0a00640 1 -- 172.21.15.102:0/973702025 shutdown_connections 2024-04-20T21:17:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.965+0000 7f31a0a00640 1 -- 172.21.15.102:0/973702025 wait complete. 2024-04-20T21:17:21.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.966+0000 7f31a0a00640 1 Processor -- start 2024-04-20T21:17:21.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.966+0000 7f31a0a00640 1 -- start start 2024-04-20T21:17:21.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c073a10 0x7f319c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319c0c9eb0 0x7f319c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f319c14fa50 0x7f319c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f31a0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f319c0c4030 con 0x7f319c073a10 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f31a0a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f319c0c7c00 con 0x7f319c0c9eb0 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f31a0a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f319c0c4310 con 0x7f319c14fa50 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f3199e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c073a10 0x7f319c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f3199e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c073a10 0x7f319c0cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51314/0 (socket says 172.21.15.102:51314) 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f3199e00640 1 -- 172.21.15.102:0/1741331475 learned_addr learned my addr 172.21.15.102:0/1741331475 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.967+0000 7f3191400640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319c0c9eb0 0x7f319c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f319a800640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f319c14fa50 0x7f319c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f3199e00640 1 -- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f319c14fa50 msgr2=0x7f319c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f3199e00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f319c14fa50 0x7f319c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f3199e00640 1 -- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319c0c9eb0 msgr2=0x7f319c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f3199e00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319c0c9eb0 0x7f319c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f3199e00640 1 -- 172.21.15.102:0/1741331475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c06f000 con 0x7f319c073a10 2024-04-20T21:17:21.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f3199e00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c073a10 0x7f319c0cba50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f3192a00640 1 -- 172.21.15.102:0/1741331475 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c044070 con 0x7f319c073a10 2024-04-20T21:17:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.968+0000 7f3192a00640 1 -- 172.21.15.102:0/1741331475 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c034680 con 0x7f319c073a10 2024-04-20T21:17:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.969+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c1e64d0 con 0x7f319c073a10 2024-04-20T21:17:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.969+0000 7f3192a00640 1 -- 172.21.15.102:0/1741331475 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c039040 con 0x7f319c073a10 2024-04-20T21:17:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.969+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c157ce0 con 0x7f319c073a10 2024-04-20T21:17:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.970+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319c074340 con 0x7f319c073a10 2024-04-20T21:17:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.971+0000 7f3192a00640 1 -- 172.21.15.102:0/1741331475 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f318c047070 con 0x7f319c073a10 2024-04-20T21:17:21.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.972+0000 7f3192a00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:21.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.972+0000 7f3192a00640 1 -- 172.21.15.102:0/1741331475 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f318c0ad3c0 con 0x7f319c073a10 2024-04-20T21:17:21.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.972+0000 7f3191400640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:21.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.973+0000 7f3191400640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f317806c920 0x7f317806ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f319c0c9d00 tx=0x7f3184009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:21.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:21.976+0000 7f3192a00640 1 -- 172.21.15.102:0/1741331475 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f318c04e020 con 0x7f319c073a10 2024-04-20T21:17:22.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.174+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 --> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f319c1e6280 con 0x7f317806c920 2024-04-20T21:17:22.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.185+0000 7f3192a00640 1 -- 172.21.15.102:0/1741331475 <== mgr.4112 v2:172.21.15.193:6832/2803621564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332562 (secure 0 0 0) 0x7f319c1e6280 con 0x7f317806c920 2024-04-20T21:17:22.190 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:22.192 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-20T21:17:22.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f317806c920 msgr2=0x7f317806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f317806c920 0x7f317806ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f319c0c9d00 tx=0x7f3184009040 comp rx=0 tx=0).stop 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c073a10 msgr2=0x7f319c0cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c073a10 0x7f319c0cba50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 shutdown_connections 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f319c14fa50 0x7f319c1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f317806c920 0x7f317806ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319c0c9eb0 0x7f319c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 --2- 172.21.15.102:0/1741331475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f319c073a10 0x7f319c0cba50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 >> 172.21.15.102:0/1741331475 conn(0x7f319c0c9390 msgr2=0x7f319c06d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 shutdown_connections 2024-04-20T21:17:22.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.192+0000 7f31a0a00640 1 -- 172.21.15.102:0/1741331475 wait complete. 2024-04-20T21:17:22.209 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-20T21:17:20.667363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1991923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003319"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918498+0000","last_change":"2024-04-20T21:16:53.564922+0000","last_active":"2024-04-20T21:16:59.918498+0000","last_peered":"2024-04-20T21:16:59.918498+0000","last_clean":"2024-04-20T21:16:59.918498+0000","last_became_active":"2024-04-20T21:16:53.564589+0000","last_became_peered":"2024-04-20T21:16:53.564589+0000","last_unstale":"2024-04-20T21:16:59.918498+0000","last_undegraded":"2024-04-20T21:16:59.918498+0000","last_fullsized":"2024-04-20T21:16:59.918498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:47.284664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916138+0000","last_change":"2024-04-20T21:16:54.580132+0000","last_active":"2024-04-20T21:16:59.916138+0000","last_peered":"2024-04-20T21:16:59.916138+0000","last_clean":"2024-04-20T21:16:59.916138+0000","last_became_active":"2024-04-20T21:16:54.579769+0000","last_became_peered":"2024-04-20T21:16:54.579769+0000","last_unstale":"2024-04-20T21:16:59.916138+0000","last_undegraded":"2024-04-20T21:16:59.916138+0000","last_fullsized":"2024-04-20T21:16:59.916138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:00.705947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913935+0000","last_change":"2024-04-20T21:16:58.781707+0000","last_active":"2024-04-20T21:16:59.913935+0000","last_peered":"2024-04-20T21:16:59.913935+0000","last_clean":"2024-04-20T21:16:59.913935+0000","last_became_active":"2024-04-20T21:16:58.780685+0000","last_became_peered":"2024-04-20T21:16:58.780685+0000","last_unstale":"2024-04-20T21:16:59.913935+0000","last_undegraded":"2024-04-20T21:16:59.913935+0000","last_fullsized":"2024-04-20T21:16:59.913935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:50.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916578+0000","last_change":"2024-04-20T21:16:53.572137+0000","last_active":"2024-04-20T21:16:59.916578+0000","last_peered":"2024-04-20T21:16:59.916578+0000","last_clean":"2024-04-20T21:16:59.916578+0000","last_became_active":"2024-04-20T21:16:53.567907+0000","last_became_peered":"2024-04-20T21:16:53.567907+0000","last_unstale":"2024-04-20T21:16:59.916578+0000","last_undegraded":"2024-04-20T21:16:59.916578+0000","last_fullsized":"2024-04-20T21:16:59.916578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:23:28.367174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918240+0000","last_change":"2024-04-20T21:16:54.567096+0000","last_active":"2024-04-20T21:16:59.918240+0000","last_peered":"2024-04-20T21:16:59.918240+0000","last_clean":"2024-04-20T21:16:59.918240+0000","last_became_active":"2024-04-20T21:16:54.566803+0000","last_became_peered":"2024-04-20T21:16:54.566803+0000","last_unstale":"2024-04-20T21:16:59.918240+0000","last_undegraded":"2024-04-20T21:16:59.918240+0000","last_fullsized":"2024-04-20T21:16:59.918240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:13:44.730056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604352+0000","last_change":"2024-04-20T21:16:58.619283+0000","last_active":"2024-04-20T21:16:59.604352+0000","last_peered":"2024-04-20T21:16:59.604352+0000","last_clean":"2024-04-20T21:16:59.604352+0000","last_became_active":"2024-04-20T21:16:58.618881+0000","last_became_peered":"2024-04-20T21:16:58.618881+0000","last_unstale":"2024-04-20T21:16:59.604352+0000","last_undegraded":"2024-04-20T21:16:59.604352+0000","last_fullsized":"2024-04-20T21:16:59.604352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:01:40.143389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617802+0000","last_change":"2024-04-20T21:16:53.554847+0000","last_active":"2024-04-20T21:16:59.617802+0000","last_peered":"2024-04-20T21:16:59.617802+0000","last_clean":"2024-04-20T21:16:59.617802+0000","last_became_active":"2024-04-20T21:16:53.554333+0000","last_became_peered":"2024-04-20T21:16:53.554333+0000","last_unstale":"2024-04-20T21:16:59.617802+0000","last_undegraded":"2024-04-20T21:16:59.617802+0000","last_fullsized":"2024-04-20T21:16:59.617802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:33.255945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912613+0000","last_change":"2024-04-20T21:16:54.556407+0000","last_active":"2024-04-20T21:16:59.912613+0000","last_peered":"2024-04-20T21:16:59.912613+0000","last_clean":"2024-04-20T21:16:59.912613+0000","last_became_active":"2024-04-20T21:16:54.556161+0000","last_became_peered":"2024-04-20T21:16:54.556161+0000","last_unstale":"2024-04-20T21:16:59.912613+0000","last_undegraded":"2024-04-20T21:16:59.912613+0000","last_fullsized":"2024-04-20T21:16:59.912613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:44:53.586714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604519+0000","last_change":"2024-04-20T21:16:58.612063+0000","last_active":"2024-04-20T21:16:59.604519+0000","last_peered":"2024-04-20T21:16:59.604519+0000","last_clean":"2024-04-20T21:16:59.604519+0000","last_became_active":"2024-04-20T21:16:58.611487+0000","last_became_peered":"2024-04-20T21:16:58.611487+0000","last_unstale":"2024-04-20T21:16:59.604519+0000","last_undegraded":"2024-04-20T21:16:59.604519+0000","last_fullsized":"2024-04-20T21:16:59.604519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:15.097731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617458+0000","last_change":"2024-04-20T21:16:53.551212+0000","last_active":"2024-04-20T21:16:59.617458+0000","last_peered":"2024-04-20T21:16:59.617458+0000","last_clean":"2024-04-20T21:16:59.617458+0000","last_became_active":"2024-04-20T21:16:53.550899+0000","last_became_peered":"2024-04-20T21:16:53.550899+0000","last_unstale":"2024-04-20T21:16:59.617458+0000","last_undegraded":"2024-04-20T21:16:59.617458+0000","last_fullsized":"2024-04-20T21:16:59.617458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:07:18.283243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912476+0000","last_change":"2024-04-20T21:16:54.557285+0000","last_active":"2024-04-20T21:16:59.912476+0000","last_peered":"2024-04-20T21:16:59.912476+0000","last_clean":"2024-04-20T21:16:59.912476+0000","last_became_active":"2024-04-20T21:16:54.556921+0000","last_became_peered":"2024-04-20T21:16:54.556921+0000","last_unstale":"2024-04-20T21:16:59.912476+0000","last_undegraded":"2024-04-20T21:16:59.912476+0000","last_fullsized":"2024-04-20T21:16:59.912476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:33.989679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604784+0000","last_change":"2024-04-20T21:16:58.617360+0000","last_active":"2024-04-20T21:16:59.604784+0000","last_peered":"2024-04-20T21:16:59.604784+0000","last_clean":"2024-04-20T21:16:59.604784+0000","last_became_active":"2024-04-20T21:16:58.616947+0000","last_became_peered":"2024-04-20T21:16:58.616947+0000","last_unstale":"2024-04-20T21:16:59.604784+0000","last_undegraded":"2024-04-20T21:16:59.604784+0000","last_fullsized":"2024-04-20T21:16:59.604784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.202955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911434+0000","last_change":"2024-04-20T21:16:54.558067+0000","last_active":"2024-04-20T21:16:59.911434+0000","last_peered":"2024-04-20T21:16:59.911434+0000","last_clean":"2024-04-20T21:16:59.911434+0000","last_became_active":"2024-04-20T21:16:54.557781+0000","last_became_peered":"2024-04-20T21:16:54.557781+0000","last_unstale":"2024-04-20T21:16:59.911434+0000","last_undegraded":"2024-04-20T21:16:59.911434+0000","last_fullsized":"2024-04-20T21:16:59.911434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:56.576081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617756+0000","last_change":"2024-04-20T21:16:53.551615+0000","last_active":"2024-04-20T21:16:59.617756+0000","last_peered":"2024-04-20T21:16:59.617756+0000","last_clean":"2024-04-20T21:16:59.617756+0000","last_became_active":"2024-04-20T21:16:53.551073+0000","last_became_peered":"2024-04-20T21:16:53.551073+0000","last_unstale":"2024-04-20T21:16:59.617756+0000","last_undegraded":"2024-04-20T21:16:59.617756+0000","last_fullsized":"2024-04-20T21:16:59.617756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:26:29.994289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605362+0000","last_change":"2024-04-20T21:16:58.617434+0000","last_active":"2024-04-20T21:16:59.605362+0000","last_peered":"2024-04-20T21:16:59.605362+0000","last_clean":"2024-04-20T21:16:59.605362+0000","last_became_active":"2024-04-20T21:16:58.617027+0000","last_became_peered":"2024-04-20T21:16:58.617027+0000","last_unstale":"2024-04-20T21:16:59.605362+0000","last_undegraded":"2024-04-20T21:16:59.605362+0000","last_fullsized":"2024-04-20T21:16:59.605362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:44:20.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913331+0000","last_change":"2024-04-20T21:16:54.585333+0000","last_active":"2024-04-20T21:16:59.913331+0000","last_peered":"2024-04-20T21:16:59.913331+0000","last_clean":"2024-04-20T21:16:59.913331+0000","last_became_active":"2024-04-20T21:16:54.584978+0000","last_became_peered":"2024-04-20T21:16:54.584978+0000","last_unstale":"2024-04-20T21:16:59.913331+0000","last_undegraded":"2024-04-20T21:16:59.913331+0000","last_fullsized":"2024-04-20T21:16:59.913331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:23.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608374+0000","last_change":"2024-04-20T21:16:53.567775+0000","last_active":"2024-04-20T21:16:59.608374+0000","last_peered":"2024-04-20T21:16:59.608374+0000","last_clean":"2024-04-20T21:16:59.608374+0000","last_became_active":"2024-04-20T21:16:53.567498+0000","last_became_peered":"2024-04-20T21:16:53.567498+0000","last_unstale":"2024-04-20T21:16:59.608374+0000","last_undegraded":"2024-04-20T21:16:59.608374+0000","last_fullsized":"2024-04-20T21:16:59.608374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:10.695659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604469+0000","last_change":"2024-04-20T21:16:58.619302+0000","last_active":"2024-04-20T21:16:59.604469+0000","last_peered":"2024-04-20T21:16:59.604469+0000","last_clean":"2024-04-20T21:16:59.604469+0000","last_became_active":"2024-04-20T21:16:58.618843+0000","last_became_peered":"2024-04-20T21:16:58.618843+0000","last_unstale":"2024-04-20T21:16:59.604469+0000","last_undegraded":"2024-04-20T21:16:59.604469+0000","last_fullsized":"2024-04-20T21:16:59.604469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:14.927199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914247+0000","last_change":"2024-04-20T21:16:54.575862+0000","last_active":"2024-04-20T21:16:59.914247+0000","last_peered":"2024-04-20T21:16:59.914247+0000","last_clean":"2024-04-20T21:16:59.914247+0000","last_became_active":"2024-04-20T21:16:54.575523+0000","last_became_peered":"2024-04-20T21:16:54.575523+0000","last_unstale":"2024-04-20T21:16:59.914247+0000","last_undegraded":"2024-04-20T21:16:59.914247+0000","last_fullsized":"2024-04-20T21:16:59.914247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:51.041414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606833+0000","last_change":"2024-04-20T21:16:53.551995+0000","last_active":"2024-04-20T21:16:59.606833+0000","last_peered":"2024-04-20T21:16:59.606833+0000","last_clean":"2024-04-20T21:16:59.606833+0000","last_became_active":"2024-04-20T21:16:53.551603+0000","last_became_peered":"2024-04-20T21:16:53.551603+0000","last_unstale":"2024-04-20T21:16:59.606833+0000","last_undegraded":"2024-04-20T21:16:59.606833+0000","last_fullsized":"2024-04-20T21:16:59.606833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:56:21.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608819+0000","last_change":"2024-04-20T21:16:58.776463+0000","last_active":"2024-04-20T21:16:59.608819+0000","last_peered":"2024-04-20T21:16:59.608819+0000","last_clean":"2024-04-20T21:16:59.608819+0000","last_became_active":"2024-04-20T21:16:58.775746+0000","last_became_peered":"2024-04-20T21:16:58.775746+0000","last_unstale":"2024-04-20T21:16:59.608819+0000","last_undegraded":"2024-04-20T21:16:59.608819+0000","last_fullsized":"2024-04-20T21:16:59.608819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:28.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605047+0000","last_change":"2024-04-20T21:16:54.566449+0000","last_active":"2024-04-20T21:16:59.605047+0000","last_peered":"2024-04-20T21:16:59.605047+0000","last_clean":"2024-04-20T21:16:59.605047+0000","last_became_active":"2024-04-20T21:16:54.566251+0000","last_became_peered":"2024-04-20T21:16:54.566251+0000","last_unstale":"2024-04-20T21:16:59.605047+0000","last_undegraded":"2024-04-20T21:16:59.605047+0000","last_fullsized":"2024-04-20T21:16:59.605047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:53.403420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617047+0000","last_change":"2024-04-20T21:16:53.564029+0000","last_active":"2024-04-20T21:16:59.617047+0000","last_peered":"2024-04-20T21:16:59.617047+0000","last_clean":"2024-04-20T21:16:59.617047+0000","last_became_active":"2024-04-20T21:16:53.563711+0000","last_became_peered":"2024-04-20T21:16:53.563711+0000","last_unstale":"2024-04-20T21:16:59.617047+0000","last_undegraded":"2024-04-20T21:16:59.617047+0000","last_fullsized":"2024-04-20T21:16:59.617047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.634644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604995+0000","last_change":"2024-04-20T21:16:58.615080+0000","last_active":"2024-04-20T21:16:59.604995+0000","last_peered":"2024-04-20T21:16:59.604995+0000","last_clean":"2024-04-20T21:16:59.604995+0000","last_became_active":"2024-04-20T21:16:58.614611+0000","last_became_peered":"2024-04-20T21:16:58.614611+0000","last_unstale":"2024-04-20T21:16:59.604995+0000","last_undegraded":"2024-04-20T21:16:59.604995+0000","last_fullsized":"2024-04-20T21:16:59.604995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:08.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917639+0000","last_change":"2024-04-20T21:16:54.578272+0000","last_active":"2024-04-20T21:16:59.917639+0000","last_peered":"2024-04-20T21:16:59.917639+0000","last_clean":"2024-04-20T21:16:59.917639+0000","last_became_active":"2024-04-20T21:16:54.577949+0000","last_became_peered":"2024-04-20T21:16:54.577949+0000","last_unstale":"2024-04-20T21:16:59.917639+0000","last_undegraded":"2024-04-20T21:16:59.917639+0000","last_fullsized":"2024-04-20T21:16:59.917639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:35:33.807091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912813+0000","last_change":"2024-04-20T21:16:53.554600+0000","last_active":"2024-04-20T21:16:59.912813+0000","last_peered":"2024-04-20T21:16:59.912813+0000","last_clean":"2024-04-20T21:16:59.912813+0000","last_became_active":"2024-04-20T21:16:53.554347+0000","last_became_peered":"2024-04-20T21:16:53.554347+0000","last_unstale":"2024-04-20T21:16:59.912813+0000","last_undegraded":"2024-04-20T21:16:59.912813+0000","last_fullsized":"2024-04-20T21:16:59.912813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:40:58.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604315+0000","last_change":"2024-04-20T21:16:58.774474+0000","last_active":"2024-04-20T21:16:59.604315+0000","last_peered":"2024-04-20T21:16:59.604315+0000","last_clean":"2024-04-20T21:16:59.604315+0000","last_became_active":"2024-04-20T21:16:58.774037+0000","last_became_peered":"2024-04-20T21:16:58.774037+0000","last_unstale":"2024-04-20T21:16:59.604315+0000","last_undegraded":"2024-04-20T21:16:59.604315+0000","last_fullsized":"2024-04-20T21:16:59.604315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:46.992667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916707+0000","last_change":"2024-04-20T21:16:54.579435+0000","last_active":"2024-04-20T21:16:59.916707+0000","last_peered":"2024-04-20T21:16:59.916707+0000","last_clean":"2024-04-20T21:16:59.916707+0000","last_became_active":"2024-04-20T21:16:54.574617+0000","last_became_peered":"2024-04-20T21:16:54.574617+0000","last_unstale":"2024-04-20T21:16:59.916707+0000","last_undegraded":"2024-04-20T21:16:59.916707+0000","last_fullsized":"2024-04-20T21:16:59.916707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:51.500741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608822+0000","last_change":"2024-04-20T21:16:53.561144+0000","last_active":"2024-04-20T21:16:59.608822+0000","last_peered":"2024-04-20T21:16:59.608822+0000","last_clean":"2024-04-20T21:16:59.608822+0000","last_became_active":"2024-04-20T21:16:53.560742+0000","last_became_peered":"2024-04-20T21:16:53.560742+0000","last_unstale":"2024-04-20T21:16:59.608822+0000","last_undegraded":"2024-04-20T21:16:59.608822+0000","last_fullsized":"2024-04-20T21:16:59.608822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:39.212951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913482+0000","last_change":"2024-04-20T21:16:58.780124+0000","last_active":"2024-04-20T21:16:59.913482+0000","last_peered":"2024-04-20T21:16:59.913482+0000","last_clean":"2024-04-20T21:16:59.913482+0000","last_became_active":"2024-04-20T21:16:58.777449+0000","last_became_peered":"2024-04-20T21:16:58.777449+0000","last_unstale":"2024-04-20T21:16:59.913482+0000","last_undegraded":"2024-04-20T21:16:59.913482+0000","last_fullsized":"2024-04-20T21:16:59.913482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:07.622146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608726+0000","last_change":"2024-04-20T21:16:54.575331+0000","last_active":"2024-04-20T21:16:59.608726+0000","last_peered":"2024-04-20T21:16:59.608726+0000","last_clean":"2024-04-20T21:16:59.608726+0000","last_became_active":"2024-04-20T21:16:54.575061+0000","last_became_peered":"2024-04-20T21:16:54.575061+0000","last_unstale":"2024-04-20T21:16:59.608726+0000","last_undegraded":"2024-04-20T21:16:59.608726+0000","last_fullsized":"2024-04-20T21:16:59.608726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:47:20.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617871+0000","last_change":"2024-04-20T21:16:53.565561+0000","last_active":"2024-04-20T21:16:59.617871+0000","last_peered":"2024-04-20T21:16:59.617871+0000","last_clean":"2024-04-20T21:16:59.617871+0000","last_became_active":"2024-04-20T21:16:53.565207+0000","last_became_peered":"2024-04-20T21:16:53.565207+0000","last_unstale":"2024-04-20T21:16:59.617871+0000","last_undegraded":"2024-04-20T21:16:59.617871+0000","last_fullsized":"2024-04-20T21:16:59.617871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:07.512132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915133+0000","last_change":"2024-04-20T21:16:58.776337+0000","last_active":"2024-04-20T21:16:59.915133+0000","last_peered":"2024-04-20T21:16:59.915133+0000","last_clean":"2024-04-20T21:16:59.915133+0000","last_became_active":"2024-04-20T21:16:58.772404+0000","last_became_peered":"2024-04-20T21:16:58.772404+0000","last_unstale":"2024-04-20T21:16:59.915133+0000","last_undegraded":"2024-04-20T21:16:59.915133+0000","last_fullsized":"2024-04-20T21:16:59.915133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:11.554173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912522+0000","last_change":"2024-04-20T21:16:54.571199+0000","last_active":"2024-04-20T21:16:59.912522+0000","last_peered":"2024-04-20T21:16:59.912522+0000","last_clean":"2024-04-20T21:16:59.912522+0000","last_became_active":"2024-04-20T21:16:54.570882+0000","last_became_peered":"2024-04-20T21:16:54.570882+0000","last_unstale":"2024-04-20T21:16:59.912522+0000","last_undegraded":"2024-04-20T21:16:59.912522+0000","last_fullsized":"2024-04-20T21:16:59.912522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:52.046176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608915+0000","last_change":"2024-04-20T21:16:53.566253+0000","last_active":"2024-04-20T21:16:59.608915+0000","last_peered":"2024-04-20T21:16:59.608915+0000","last_clean":"2024-04-20T21:16:59.608915+0000","last_became_active":"2024-04-20T21:16:53.565893+0000","last_became_peered":"2024-04-20T21:16:53.565893+0000","last_unstale":"2024-04-20T21:16:59.608915+0000","last_undegraded":"2024-04-20T21:16:59.608915+0000","last_fullsized":"2024-04-20T21:16:59.608915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:16.302801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.607352+0000","last_change":"2024-04-20T21:16:58.611470+0000","last_active":"2024-04-20T21:16:59.607352+0000","last_peered":"2024-04-20T21:16:59.607352+0000","last_clean":"2024-04-20T21:16:59.607352+0000","last_became_active":"2024-04-20T21:16:58.611038+0000","last_became_peered":"2024-04-20T21:16:58.611038+0000","last_unstale":"2024-04-20T21:16:59.607352+0000","last_undegraded":"2024-04-20T21:16:59.607352+0000","last_fullsized":"2024-04-20T21:16:59.607352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:58:51.267480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604848+0000","last_change":"2024-04-20T21:16:54.577608+0000","last_active":"2024-04-20T21:16:59.604848+0000","last_peered":"2024-04-20T21:16:59.604848+0000","last_clean":"2024-04-20T21:16:59.604848+0000","last_became_active":"2024-04-20T21:16:54.577339+0000","last_became_peered":"2024-04-20T21:16:54.577339+0000","last_unstale":"2024-04-20T21:16:59.604848+0000","last_undegraded":"2024-04-20T21:16:59.604848+0000","last_fullsized":"2024-04-20T21:16:59.604848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:03.028585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912331+0000","last_change":"2024-04-20T21:16:53.547751+0000","last_active":"2024-04-20T21:16:59.912331+0000","last_peered":"2024-04-20T21:16:59.912331+0000","last_clean":"2024-04-20T21:16:59.912331+0000","last_became_active":"2024-04-20T21:16:53.547371+0000","last_became_peered":"2024-04-20T21:16:53.547371+0000","last_unstale":"2024-04-20T21:16:59.912331+0000","last_undegraded":"2024-04-20T21:16:59.912331+0000","last_fullsized":"2024-04-20T21:16:59.912331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:50.728500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608358+0000","last_change":"2024-04-20T21:16:58.622369+0000","last_active":"2024-04-20T21:16:59.608358+0000","last_peered":"2024-04-20T21:16:59.608358+0000","last_clean":"2024-04-20T21:16:59.608358+0000","last_became_active":"2024-04-20T21:16:58.621911+0000","last_became_peered":"2024-04-20T21:16:58.621911+0000","last_unstale":"2024-04-20T21:16:59.608358+0000","last_undegraded":"2024-04-20T21:16:59.608358+0000","last_fullsized":"2024-04-20T21:16:59.608358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:41.519583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606736+0000","last_change":"2024-04-20T21:16:54.579577+0000","last_active":"2024-04-20T21:16:59.606736+0000","last_peered":"2024-04-20T21:16:59.606736+0000","last_clean":"2024-04-20T21:16:59.606736+0000","last_became_active":"2024-04-20T21:16:54.579156+0000","last_became_peered":"2024-04-20T21:16:54.579156+0000","last_unstale":"2024-04-20T21:16:59.606736+0000","last_undegraded":"2024-04-20T21:16:59.606736+0000","last_fullsized":"2024-04-20T21:16:59.606736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:28:14.743667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605543+0000","last_change":"2024-04-20T21:16:53.567995+0000","last_active":"2024-04-20T21:16:59.605543+0000","last_peered":"2024-04-20T21:16:59.605543+0000","last_clean":"2024-04-20T21:16:59.605543+0000","last_became_active":"2024-04-20T21:16:53.567601+0000","last_became_peered":"2024-04-20T21:16:53.567601+0000","last_unstale":"2024-04-20T21:16:59.605543+0000","last_undegraded":"2024-04-20T21:16:59.605543+0000","last_fullsized":"2024-04-20T21:16:59.605543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:30.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609728+0000","last_change":"2024-04-20T21:16:58.626438+0000","last_active":"2024-04-20T21:16:59.609728+0000","last_peered":"2024-04-20T21:16:59.609728+0000","last_clean":"2024-04-20T21:16:59.609728+0000","last_became_active":"2024-04-20T21:16:58.625326+0000","last_became_peered":"2024-04-20T21:16:58.625326+0000","last_unstale":"2024-04-20T21:16:59.609728+0000","last_undegraded":"2024-04-20T21:16:59.609728+0000","last_fullsized":"2024-04-20T21:16:59.609728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:06.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914171+0000","last_change":"2024-04-20T21:16:54.560925+0000","last_active":"2024-04-20T21:16:59.914171+0000","last_peered":"2024-04-20T21:16:59.914171+0000","last_clean":"2024-04-20T21:16:59.914171+0000","last_became_active":"2024-04-20T21:16:54.560615+0000","last_became_peered":"2024-04-20T21:16:54.560615+0000","last_unstale":"2024-04-20T21:16:59.914171+0000","last_undegraded":"2024-04-20T21:16:59.914171+0000","last_fullsized":"2024-04-20T21:16:59.914171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:12.815692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617155+0000","last_change":"2024-04-20T21:16:53.559359+0000","last_active":"2024-04-20T21:16:59.617155+0000","last_peered":"2024-04-20T21:16:59.617155+0000","last_clean":"2024-04-20T21:16:59.617155+0000","last_became_active":"2024-04-20T21:16:53.558955+0000","last_became_peered":"2024-04-20T21:16:53.558955+0000","last_unstale":"2024-04-20T21:16:59.617155+0000","last_undegraded":"2024-04-20T21:16:59.617155+0000","last_fullsized":"2024-04-20T21:16:59.617155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:28.664404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917377+0000","last_change":"2024-04-20T21:16:58.772175+0000","last_active":"2024-04-20T21:16:59.917377+0000","last_peered":"2024-04-20T21:16:59.917377+0000","last_clean":"2024-04-20T21:16:59.917377+0000","last_became_active":"2024-04-20T21:16:58.771732+0000","last_became_peered":"2024-04-20T21:16:58.771732+0000","last_unstale":"2024-04-20T21:16:59.917377+0000","last_undegraded":"2024-04-20T21:16:59.917377+0000","last_fullsized":"2024-04-20T21:16:59.917377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:13.155547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912641+0000","last_change":"2024-04-20T21:16:54.575375+0000","last_active":"2024-04-20T21:16:59.912641+0000","last_peered":"2024-04-20T21:16:59.912641+0000","last_clean":"2024-04-20T21:16:59.912641+0000","last_became_active":"2024-04-20T21:16:54.574194+0000","last_became_peered":"2024-04-20T21:16:54.574194+0000","last_unstale":"2024-04-20T21:16:59.912641+0000","last_undegraded":"2024-04-20T21:16:59.912641+0000","last_fullsized":"2024-04-20T21:16:59.912641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:07.948431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911980+0000","last_change":"2024-04-20T21:16:53.559374+0000","last_active":"2024-04-20T21:16:59.911980+0000","last_peered":"2024-04-20T21:16:59.911980+0000","last_clean":"2024-04-20T21:16:59.911980+0000","last_became_active":"2024-04-20T21:16:53.559133+0000","last_became_peered":"2024-04-20T21:16:53.559133+0000","last_unstale":"2024-04-20T21:16:59.911980+0000","last_undegraded":"2024-04-20T21:16:59.911980+0000","last_fullsized":"2024-04-20T21:16:59.911980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:38.576103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617268+0000","last_change":"2024-04-20T21:16:44.100529+0000","last_active":"2024-04-20T21:16:59.617268+0000","last_peered":"2024-04-20T21:16:59.617268+0000","last_clean":"2024-04-20T21:16:59.617268+0000","last_became_active":"2024-04-20T21:16:44.099590+0000","last_became_peered":"2024-04-20T21:16:44.099590+0000","last_unstale":"2024-04-20T21:16:59.617268+0000","last_undegraded":"2024-04-20T21:16:59.617268+0000","last_fullsized":"2024-04-20T21:16:59.617268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_clean_scrub_stamp":"2024-04-20T21:16:42.782844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:08.054837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911874+0000","last_change":"2024-04-20T21:16:58.772894+0000","last_active":"2024-04-20T21:16:59.911874+0000","last_peered":"2024-04-20T21:16:59.911874+0000","last_clean":"2024-04-20T21:16:59.911874+0000","last_became_active":"2024-04-20T21:16:58.772451+0000","last_became_peered":"2024-04-20T21:16:58.772451+0000","last_unstale":"2024-04-20T21:16:59.911874+0000","last_undegraded":"2024-04-20T21:16:59.911874+0000","last_fullsized":"2024-04-20T21:16:59.911874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:29.914096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617716+0000","last_change":"2024-04-20T21:16:54.562592+0000","last_active":"2024-04-20T21:16:59.617716+0000","last_peered":"2024-04-20T21:16:59.617716+0000","last_clean":"2024-04-20T21:16:59.617716+0000","last_became_active":"2024-04-20T21:16:54.562243+0000","last_became_peered":"2024-04-20T21:16:54.562243+0000","last_unstale":"2024-04-20T21:16:59.617716+0000","last_undegraded":"2024-04-20T21:16:59.617716+0000","last_fullsized":"2024-04-20T21:16:59.617716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:52.832159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606010+0000","last_change":"2024-04-20T21:16:53.561132+0000","last_active":"2024-04-20T21:16:59.606010+0000","last_peered":"2024-04-20T21:16:59.606010+0000","last_clean":"2024-04-20T21:16:59.606010+0000","last_became_active":"2024-04-20T21:16:53.559901+0000","last_became_peered":"2024-04-20T21:16:53.559901+0000","last_unstale":"2024-04-20T21:16:59.606010+0000","last_undegraded":"2024-04-20T21:16:59.606010+0000","last_fullsized":"2024-04-20T21:16:59.606010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:20.229573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911503+0000","last_change":"2024-04-20T21:16:58.771204+0000","last_active":"2024-04-20T21:16:59.911503+0000","last_peered":"2024-04-20T21:16:59.911503+0000","last_clean":"2024-04-20T21:16:59.911503+0000","last_became_active":"2024-04-20T21:16:58.770627+0000","last_became_peered":"2024-04-20T21:16:58.770627+0000","last_unstale":"2024-04-20T21:16:59.911503+0000","last_undegraded":"2024-04-20T21:16:59.911503+0000","last_fullsized":"2024-04-20T21:16:59.911503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:39.472931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912685+0000","last_change":"2024-04-20T21:16:54.571417+0000","last_active":"2024-04-20T21:16:59.912685+0000","last_peered":"2024-04-20T21:16:59.912685+0000","last_clean":"2024-04-20T21:16:59.912685+0000","last_became_active":"2024-04-20T21:16:54.570434+0000","last_became_peered":"2024-04-20T21:16:54.570434+0000","last_unstale":"2024-04-20T21:16:59.912685+0000","last_undegraded":"2024-04-20T21:16:59.912685+0000","last_fullsized":"2024-04-20T21:16:59.912685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:24:41.026138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609282+0000","last_change":"2024-04-20T21:16:53.558554+0000","last_active":"2024-04-20T21:16:59.609282+0000","last_peered":"2024-04-20T21:16:59.609282+0000","last_clean":"2024-04-20T21:16:59.609282+0000","last_became_active":"2024-04-20T21:16:53.556934+0000","last_became_peered":"2024-04-20T21:16:53.556934+0000","last_unstale":"2024-04-20T21:16:59.609282+0000","last_undegraded":"2024-04-20T21:16:59.609282+0000","last_fullsized":"2024-04-20T21:16:59.609282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:17:52.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915951+0000","last_change":"2024-04-20T21:16:58.624410+0000","last_active":"2024-04-20T21:16:59.915951+0000","last_peered":"2024-04-20T21:16:59.915951+0000","last_clean":"2024-04-20T21:16:59.915951+0000","last_became_active":"2024-04-20T21:16:58.623677+0000","last_became_peered":"2024-04-20T21:16:58.623677+0000","last_unstale":"2024-04-20T21:16:59.915951+0000","last_undegraded":"2024-04-20T21:16:59.915951+0000","last_fullsized":"2024-04-20T21:16:59.915951+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:34.465594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617870+0000","last_change":"2024-04-20T21:16:54.560217+0000","last_active":"2024-04-20T21:16:59.617870+0000","last_peered":"2024-04-20T21:16:59.617870+0000","last_clean":"2024-04-20T21:16:59.617870+0000","last_became_active":"2024-04-20T21:16:54.559947+0000","last_became_peered":"2024-04-20T21:16:54.559947+0000","last_unstale":"2024-04-20T21:16:59.617870+0000","last_undegraded":"2024-04-20T21:16:59.617870+0000","last_fullsized":"2024-04-20T21:16:59.617870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:54.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.607164+0000","last_change":"2024-04-20T21:16:53.564779+0000","last_active":"2024-04-20T21:16:59.607164+0000","last_peered":"2024-04-20T21:16:59.607164+0000","last_clean":"2024-04-20T21:16:59.607164+0000","last_became_active":"2024-04-20T21:16:53.564500+0000","last_became_peered":"2024-04-20T21:16:53.564500+0000","last_unstale":"2024-04-20T21:16:59.607164+0000","last_undegraded":"2024-04-20T21:16:59.607164+0000","last_fullsized":"2024-04-20T21:16:59.607164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:00:58.673145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912319+0000","last_change":"2024-04-20T21:16:58.771524+0000","last_active":"2024-04-20T21:16:59.912319+0000","last_peered":"2024-04-20T21:16:59.912319+0000","last_clean":"2024-04-20T21:16:59.912319+0000","last_became_active":"2024-04-20T21:16:58.770672+0000","last_became_peered":"2024-04-20T21:16:58.770672+0000","last_unstale":"2024-04-20T21:16:59.912319+0000","last_undegraded":"2024-04-20T21:16:59.912319+0000","last_fullsized":"2024-04-20T21:16:59.912319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:52:27.037256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617704+0000","last_change":"2024-04-20T21:16:54.559570+0000","last_active":"2024-04-20T21:16:59.617704+0000","last_peered":"2024-04-20T21:16:59.617704+0000","last_clean":"2024-04-20T21:16:59.617704+0000","last_became_active":"2024-04-20T21:16:54.559307+0000","last_became_peered":"2024-04-20T21:16:54.559307+0000","last_unstale":"2024-04-20T21:16:59.617704+0000","last_undegraded":"2024-04-20T21:16:59.617704+0000","last_fullsized":"2024-04-20T21:16:59.617704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:04.417348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605512+0000","last_change":"2024-04-20T21:16:53.565289+0000","last_active":"2024-04-20T21:16:59.605512+0000","last_peered":"2024-04-20T21:16:59.605512+0000","last_clean":"2024-04-20T21:16:59.605512+0000","last_became_active":"2024-04-20T21:16:53.565014+0000","last_became_peered":"2024-04-20T21:16:53.565014+0000","last_unstale":"2024-04-20T21:16:59.605512+0000","last_undegraded":"2024-04-20T21:16:59.605512+0000","last_fullsized":"2024-04-20T21:16:59.605512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:22.125301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604480+0000","last_change":"2024-04-20T21:16:58.615103+0000","last_active":"2024-04-20T21:16:59.604480+0000","last_peered":"2024-04-20T21:16:59.604480+0000","last_clean":"2024-04-20T21:16:59.604480+0000","last_became_active":"2024-04-20T21:16:58.614068+0000","last_became_peered":"2024-04-20T21:16:58.614068+0000","last_unstale":"2024-04-20T21:16:59.604480+0000","last_undegraded":"2024-04-20T21:16:59.604480+0000","last_fullsized":"2024-04-20T21:16:59.604480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:09.555911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617489+0000","last_change":"2024-04-20T21:16:54.557439+0000","last_active":"2024-04-20T21:16:59.617489+0000","last_peered":"2024-04-20T21:16:59.617489+0000","last_clean":"2024-04-20T21:16:59.617489+0000","last_became_active":"2024-04-20T21:16:54.557178+0000","last_became_peered":"2024-04-20T21:16:54.557178+0000","last_unstale":"2024-04-20T21:16:59.617489+0000","last_undegraded":"2024-04-20T21:16:59.617489+0000","last_fullsized":"2024-04-20T21:16:59.617489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:55:33.964948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605750+0000","last_change":"2024-04-20T21:16:53.570182+0000","last_active":"2024-04-20T21:16:59.605750+0000","last_peered":"2024-04-20T21:16:59.605750+0000","last_clean":"2024-04-20T21:16:59.605750+0000","last_became_active":"2024-04-20T21:16:53.569632+0000","last_became_peered":"2024-04-20T21:16:53.569632+0000","last_unstale":"2024-04-20T21:16:59.605750+0000","last_undegraded":"2024-04-20T21:16:59.605750+0000","last_fullsized":"2024-04-20T21:16:59.605750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:12.140497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917381+0000","last_change":"2024-04-20T21:16:58.625602+0000","last_active":"2024-04-20T21:16:59.917381+0000","last_peered":"2024-04-20T21:16:59.917381+0000","last_clean":"2024-04-20T21:16:59.917381+0000","last_became_active":"2024-04-20T21:16:58.625158+0000","last_became_peered":"2024-04-20T21:16:58.625158+0000","last_unstale":"2024-04-20T21:16:59.917381+0000","last_undegraded":"2024-04-20T21:16:59.917381+0000","last_fullsized":"2024-04-20T21:16:59.917381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:09.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911816+0000","last_change":"2024-04-20T21:16:54.578603+0000","last_active":"2024-04-20T21:16:59.911816+0000","last_peered":"2024-04-20T21:16:59.911816+0000","last_clean":"2024-04-20T21:16:59.911816+0000","last_became_active":"2024-04-20T21:16:54.578338+0000","last_became_peered":"2024-04-20T21:16:54.578338+0000","last_unstale":"2024-04-20T21:16:59.911816+0000","last_undegraded":"2024-04-20T21:16:59.911816+0000","last_fullsized":"2024-04-20T21:16:59.911816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:14:56.355343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911553+0000","last_change":"2024-04-20T21:16:53.551847+0000","last_active":"2024-04-20T21:16:59.911553+0000","last_peered":"2024-04-20T21:16:59.911553+0000","last_clean":"2024-04-20T21:16:59.911553+0000","last_became_active":"2024-04-20T21:16:53.551427+0000","last_became_peered":"2024-04-20T21:16:53.551427+0000","last_unstale":"2024-04-20T21:16:59.911553+0000","last_undegraded":"2024-04-20T21:16:59.911553+0000","last_fullsized":"2024-04-20T21:16:59.911553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:55:45.930305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916262+0000","last_change":"2024-04-20T21:16:58.777159+0000","last_active":"2024-04-20T21:16:59.916262+0000","last_peered":"2024-04-20T21:16:59.916262+0000","last_clean":"2024-04-20T21:16:59.916262+0000","last_became_active":"2024-04-20T21:16:58.776233+0000","last_became_peered":"2024-04-20T21:16:58.776233+0000","last_unstale":"2024-04-20T21:16:59.916262+0000","last_undegraded":"2024-04-20T21:16:59.916262+0000","last_fullsized":"2024-04-20T21:16:59.916262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:05.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617642+0000","last_change":"2024-04-20T21:16:54.570687+0000","last_active":"2024-04-20T21:16:59.617642+0000","last_peered":"2024-04-20T21:16:59.617642+0000","last_clean":"2024-04-20T21:16:59.617642+0000","last_became_active":"2024-04-20T21:16:54.570360+0000","last_became_peered":"2024-04-20T21:16:54.570360+0000","last_unstale":"2024-04-20T21:16:59.617642+0000","last_undegraded":"2024-04-20T21:16:59.617642+0000","last_fullsized":"2024-04-20T21:16:59.617642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:05.606229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912222+0000","last_change":"2024-04-20T21:16:53.567599+0000","last_active":"2024-04-20T21:16:59.912222+0000","last_peered":"2024-04-20T21:16:59.912222+0000","last_clean":"2024-04-20T21:16:59.912222+0000","last_became_active":"2024-04-20T21:16:53.567378+0000","last_became_peered":"2024-04-20T21:16:53.567378+0000","last_unstale":"2024-04-20T21:16:59.912222+0000","last_undegraded":"2024-04-20T21:16:59.912222+0000","last_fullsized":"2024-04-20T21:16:59.912222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:36.611887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914503+0000","last_change":"2024-04-20T21:16:58.774226+0000","last_active":"2024-04-20T21:16:59.914503+0000","last_peered":"2024-04-20T21:16:59.914503+0000","last_clean":"2024-04-20T21:16:59.914503+0000","last_became_active":"2024-04-20T21:16:58.771584+0000","last_became_peered":"2024-04-20T21:16:58.771584+0000","last_unstale":"2024-04-20T21:16:59.914503+0000","last_undegraded":"2024-04-20T21:16:59.914503+0000","last_fullsized":"2024-04-20T21:16:59.914503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:31.432077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915175+0000","last_change":"2024-04-20T21:16:54.560001+0000","last_active":"2024-04-20T21:16:59.915175+0000","last_peered":"2024-04-20T21:16:59.915175+0000","last_clean":"2024-04-20T21:16:59.915175+0000","last_became_active":"2024-04-20T21:16:54.559643+0000","last_became_peered":"2024-04-20T21:16:54.559643+0000","last_unstale":"2024-04-20T21:16:59.915175+0000","last_undegraded":"2024-04-20T21:16:59.915175+0000","last_fullsized":"2024-04-20T21:16:59.915175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:52:37.325946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916695+0000","last_change":"2024-04-20T21:16:53.570498+0000","last_active":"2024-04-20T21:16:59.916695+0000","last_peered":"2024-04-20T21:16:59.916695+0000","last_clean":"2024-04-20T21:16:59.916695+0000","last_became_active":"2024-04-20T21:16:53.570095+0000","last_became_peered":"2024-04-20T21:16:53.570095+0000","last_unstale":"2024-04-20T21:16:59.916695+0000","last_undegraded":"2024-04-20T21:16:59.916695+0000","last_fullsized":"2024-04-20T21:16:59.916695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:58:28.677814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912403+0000","last_change":"2024-04-20T21:16:58.773349+0000","last_active":"2024-04-20T21:16:59.912403+0000","last_peered":"2024-04-20T21:16:59.912403+0000","last_clean":"2024-04-20T21:16:59.912403+0000","last_became_active":"2024-04-20T21:16:58.772781+0000","last_became_peered":"2024-04-20T21:16:58.772781+0000","last_unstale":"2024-04-20T21:16:59.912403+0000","last_undegraded":"2024-04-20T21:16:59.912403+0000","last_fullsized":"2024-04-20T21:16:59.912403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:53:02.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617377+0000","last_change":"2024-04-20T21:16:54.572749+0000","last_active":"2024-04-20T21:16:59.617377+0000","last_peered":"2024-04-20T21:16:59.617377+0000","last_clean":"2024-04-20T21:16:59.617377+0000","last_became_active":"2024-04-20T21:16:54.572456+0000","last_became_peered":"2024-04-20T21:16:54.572456+0000","last_unstale":"2024-04-20T21:16:59.617377+0000","last_undegraded":"2024-04-20T21:16:59.617377+0000","last_fullsized":"2024-04-20T21:16:59.617377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.787612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:17:17.226094+0000","last_change":"2024-04-20T21:16:53.568688+0000","last_active":"2024-04-20T21:17:17.226094+0000","last_peered":"2024-04-20T21:17:17.226094+0000","last_clean":"2024-04-20T21:17:17.226094+0000","last_became_active":"2024-04-20T21:16:53.568495+0000","last_became_peered":"2024-04-20T21:16:53.568495+0000","last_unstale":"2024-04-20T21:17:17.226094+0000","last_undegraded":"2024-04-20T21:17:17.226094+0000","last_fullsized":"2024-04-20T21:17:17.226094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:00.086004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608039+0000","last_change":"2024-04-20T21:16:58.609575+0000","last_active":"2024-04-20T21:16:59.608039+0000","last_peered":"2024-04-20T21:16:59.608039+0000","last_clean":"2024-04-20T21:16:59.608039+0000","last_became_active":"2024-04-20T21:16:58.609127+0000","last_became_peered":"2024-04-20T21:16:58.609127+0000","last_unstale":"2024-04-20T21:16:59.608039+0000","last_undegraded":"2024-04-20T21:16:59.608039+0000","last_fullsized":"2024-04-20T21:16:59.608039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:02.645274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605015+0000","last_change":"2024-04-20T21:16:54.563213+0000","last_active":"2024-04-20T21:16:59.605015+0000","last_peered":"2024-04-20T21:16:59.605015+0000","last_clean":"2024-04-20T21:16:59.605015+0000","last_became_active":"2024-04-20T21:16:54.562917+0000","last_became_peered":"2024-04-20T21:16:54.562917+0000","last_unstale":"2024-04-20T21:16:59.605015+0000","last_undegraded":"2024-04-20T21:16:59.605015+0000","last_fullsized":"2024-04-20T21:16:59.605015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.155771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609082+0000","last_change":"2024-04-20T21:16:53.556988+0000","last_active":"2024-04-20T21:16:59.609082+0000","last_peered":"2024-04-20T21:16:59.609082+0000","last_clean":"2024-04-20T21:16:59.609082+0000","last_became_active":"2024-04-20T21:16:53.556672+0000","last_became_peered":"2024-04-20T21:16:53.556672+0000","last_unstale":"2024-04-20T21:16:59.609082+0000","last_undegraded":"2024-04-20T21:16:59.609082+0000","last_fullsized":"2024-04-20T21:16:59.609082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:25.219611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604713+0000","last_change":"2024-04-20T21:16:58.595827+0000","last_active":"2024-04-20T21:16:59.604713+0000","last_peered":"2024-04-20T21:16:59.604713+0000","last_clean":"2024-04-20T21:16:59.604713+0000","last_became_active":"2024-04-20T21:16:58.595403+0000","last_became_peered":"2024-04-20T21:16:58.595403+0000","last_unstale":"2024-04-20T21:16:59.604713+0000","last_undegraded":"2024-04-20T21:16:59.604713+0000","last_fullsized":"2024-04-20T21:16:59.604713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:53.227573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918376+0000","last_change":"2024-04-20T21:16:54.562128+0000","last_active":"2024-04-20T21:16:59.918376+0000","last_peered":"2024-04-20T21:16:59.918376+0000","last_clean":"2024-04-20T21:16:59.918376+0000","last_became_active":"2024-04-20T21:16:54.561495+0000","last_became_peered":"2024-04-20T21:16:54.561495+0000","last_unstale":"2024-04-20T21:16:59.918376+0000","last_undegraded":"2024-04-20T21:16:59.918376+0000","last_fullsized":"2024-04-20T21:16:59.918376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:19.582218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911795+0000","last_change":"2024-04-20T21:16:53.554881+0000","last_active":"2024-04-20T21:16:59.911795+0000","last_peered":"2024-04-20T21:16:59.911795+0000","last_clean":"2024-04-20T21:16:59.911795+0000","last_became_active":"2024-04-20T21:16:53.554590+0000","last_became_peered":"2024-04-20T21:16:53.554590+0000","last_unstale":"2024-04-20T21:16:59.911795+0000","last_undegraded":"2024-04-20T21:16:59.911795+0000","last_fullsized":"2024-04-20T21:16:59.911795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:36.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913934+0000","last_change":"2024-04-20T21:16:58.610704+0000","last_active":"2024-04-20T21:16:59.913934+0000","last_peered":"2024-04-20T21:16:59.913934+0000","last_clean":"2024-04-20T21:16:59.913934+0000","last_became_active":"2024-04-20T21:16:58.610309+0000","last_became_peered":"2024-04-20T21:16:58.610309+0000","last_unstale":"2024-04-20T21:16:59.913934+0000","last_undegraded":"2024-04-20T21:16:59.913934+0000","last_fullsized":"2024-04-20T21:16:59.913934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:04.215050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911685+0000","last_change":"2024-04-20T21:16:54.562351+0000","last_active":"2024-04-20T21:16:59.911685+0000","last_peered":"2024-04-20T21:16:59.911685+0000","last_clean":"2024-04-20T21:16:59.911685+0000","last_became_active":"2024-04-20T21:16:54.561947+0000","last_became_peered":"2024-04-20T21:16:54.561947+0000","last_unstale":"2024-04-20T21:16:59.911685+0000","last_undegraded":"2024-04-20T21:16:59.911685+0000","last_fullsized":"2024-04-20T21:16:59.911685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:52.802328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912701+0000","last_change":"2024-04-20T21:16:53.554081+0000","last_active":"2024-04-20T21:16:59.912701+0000","last_peered":"2024-04-20T21:16:59.912701+0000","last_clean":"2024-04-20T21:16:59.912701+0000","last_became_active":"2024-04-20T21:16:53.553800+0000","last_became_peered":"2024-04-20T21:16:53.553800+0000","last_unstale":"2024-04-20T21:16:59.912701+0000","last_undegraded":"2024-04-20T21:16:59.912701+0000","last_fullsized":"2024-04-20T21:16:59.912701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:52.865485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914034+0000","last_change":"2024-04-20T21:16:58.779184+0000","last_active":"2024-04-20T21:16:59.914034+0000","last_peered":"2024-04-20T21:16:59.914034+0000","last_clean":"2024-04-20T21:16:59.914034+0000","last_became_active":"2024-04-20T21:16:58.778627+0000","last_became_peered":"2024-04-20T21:16:58.778627+0000","last_unstale":"2024-04-20T21:16:59.914034+0000","last_undegraded":"2024-04-20T21:16:59.914034+0000","last_fullsized":"2024-04-20T21:16:59.914034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:50:07.247736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917097+0000","last_change":"2024-04-20T21:16:58.774438+0000","last_active":"2024-04-20T21:16:59.917097+0000","last_peered":"2024-04-20T21:16:59.917097+0000","last_clean":"2024-04-20T21:16:59.917097+0000","last_became_active":"2024-04-20T21:16:58.774030+0000","last_became_peered":"2024-04-20T21:16:58.774030+0000","last_unstale":"2024-04-20T21:16:59.917097+0000","last_undegraded":"2024-04-20T21:16:59.917097+0000","last_fullsized":"2024-04-20T21:16:59.917097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:09:22.583239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606553+0000","last_change":"2024-04-20T21:16:54.573779+0000","last_active":"2024-04-20T21:16:59.606553+0000","last_peered":"2024-04-20T21:16:59.606553+0000","last_clean":"2024-04-20T21:16:59.606553+0000","last_became_active":"2024-04-20T21:16:54.573444+0000","last_became_peered":"2024-04-20T21:16:54.573444+0000","last_unstale":"2024-04-20T21:16:59.606553+0000","last_undegraded":"2024-04-20T21:16:59.606553+0000","last_fullsized":"2024-04-20T21:16:59.606553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:27.950525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911680+0000","last_change":"2024-04-20T21:16:53.543678+0000","last_active":"2024-04-20T21:16:59.911680+0000","last_peered":"2024-04-20T21:16:59.911680+0000","last_clean":"2024-04-20T21:16:59.911680+0000","last_became_active":"2024-04-20T21:16:53.543381+0000","last_became_peered":"2024-04-20T21:16:53.543381+0000","last_unstale":"2024-04-20T21:16:59.911680+0000","last_undegraded":"2024-04-20T21:16:59.911680+0000","last_fullsized":"2024-04-20T21:16:59.911680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:02.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912679+0000","last_change":"2024-04-20T21:16:58.626882+0000","last_active":"2024-04-20T21:16:59.912679+0000","last_peered":"2024-04-20T21:16:59.912679+0000","last_clean":"2024-04-20T21:16:59.912679+0000","last_became_active":"2024-04-20T21:16:58.626489+0000","last_became_peered":"2024-04-20T21:16:58.626489+0000","last_unstale":"2024-04-20T21:16:59.912679+0000","last_undegraded":"2024-04-20T21:16:59.912679+0000","last_fullsized":"2024-04-20T21:16:59.912679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:33:22.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913977+0000","last_change":"2024-04-20T21:16:54.580893+0000","last_active":"2024-04-20T21:16:59.913977+0000","last_peered":"2024-04-20T21:16:59.913977+0000","last_clean":"2024-04-20T21:16:59.913977+0000","last_became_active":"2024-04-20T21:16:54.580528+0000","last_became_peered":"2024-04-20T21:16:54.580528+0000","last_unstale":"2024-04-20T21:16:59.913977+0000","last_undegraded":"2024-04-20T21:16:59.913977+0000","last_fullsized":"2024-04-20T21:16:59.913977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:14.699123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914639+0000","last_change":"2024-04-20T21:16:53.563930+0000","last_active":"2024-04-20T21:16:59.914639+0000","last_peered":"2024-04-20T21:16:59.914639+0000","last_clean":"2024-04-20T21:16:59.914639+0000","last_became_active":"2024-04-20T21:16:53.559991+0000","last_became_peered":"2024-04-20T21:16:53.559991+0000","last_unstale":"2024-04-20T21:16:59.914639+0000","last_undegraded":"2024-04-20T21:16:59.914639+0000","last_fullsized":"2024-04-20T21:16:59.914639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:22.529712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912173+0000","last_change":"2024-04-20T21:16:53.552509+0000","last_active":"2024-04-20T21:16:59.912173+0000","last_peered":"2024-04-20T21:16:59.912173+0000","last_clean":"2024-04-20T21:16:59.912173+0000","last_became_active":"2024-04-20T21:16:53.552173+0000","last_became_peered":"2024-04-20T21:16:53.552173+0000","last_unstale":"2024-04-20T21:16:59.912173+0000","last_undegraded":"2024-04-20T21:16:59.912173+0000","last_fullsized":"2024-04-20T21:16:59.912173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:14.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916470+0000","last_change":"2024-04-20T21:16:54.562849+0000","last_active":"2024-04-20T21:16:59.916470+0000","last_peered":"2024-04-20T21:16:59.916470+0000","last_clean":"2024-04-20T21:16:59.916470+0000","last_became_active":"2024-04-20T21:16:54.561283+0000","last_became_peered":"2024-04-20T21:16:54.561283+0000","last_unstale":"2024-04-20T21:16:59.916470+0000","last_undegraded":"2024-04-20T21:16:59.916470+0000","last_fullsized":"2024-04-20T21:16:59.916470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:52.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605952+0000","last_change":"2024-04-20T21:16:58.614752+0000","last_active":"2024-04-20T21:16:59.605952+0000","last_peered":"2024-04-20T21:16:59.605952+0000","last_clean":"2024-04-20T21:16:59.605952+0000","last_became_active":"2024-04-20T21:16:58.614201+0000","last_became_peered":"2024-04-20T21:16:58.614201+0000","last_unstale":"2024-04-20T21:16:59.605952+0000","last_undegraded":"2024-04-20T21:16:59.605952+0000","last_fullsized":"2024-04-20T21:16:59.605952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:46.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913587+0000","last_change":"2024-04-20T21:16:53.564303+0000","last_active":"2024-04-20T21:16:59.913587+0000","last_peered":"2024-04-20T21:16:59.913587+0000","last_clean":"2024-04-20T21:16:59.913587+0000","last_became_active":"2024-04-20T21:16:53.560203+0000","last_became_peered":"2024-04-20T21:16:53.560203+0000","last_unstale":"2024-04-20T21:16:59.913587+0000","last_undegraded":"2024-04-20T21:16:59.913587+0000","last_fullsized":"2024-04-20T21:16:59.913587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:42.691233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604601+0000","last_change":"2024-04-20T21:16:54.554499+0000","last_active":"2024-04-20T21:16:59.604601+0000","last_peered":"2024-04-20T21:16:59.604601+0000","last_clean":"2024-04-20T21:16:59.604601+0000","last_became_active":"2024-04-20T21:16:54.554230+0000","last_became_peered":"2024-04-20T21:16:54.554230+0000","last_unstale":"2024-04-20T21:16:59.604601+0000","last_undegraded":"2024-04-20T21:16:59.604601+0000","last_fullsized":"2024-04-20T21:16:59.604601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:24:31.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915386+0000","last_change":"2024-04-20T21:16:58.610424+0000","last_active":"2024-04-20T21:16:59.915386+0000","last_peered":"2024-04-20T21:16:59.915386+0000","last_clean":"2024-04-20T21:16:59.915386+0000","last_became_active":"2024-04-20T21:16:58.610025+0000","last_became_peered":"2024-04-20T21:16:58.610025+0000","last_unstale":"2024-04-20T21:16:59.915386+0000","last_undegraded":"2024-04-20T21:16:59.915386+0000","last_fullsized":"2024-04-20T21:16:59.915386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:19.290865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911710+0000","last_change":"2024-04-20T21:16:54.578801+0000","last_active":"2024-04-20T21:16:59.911710+0000","last_peered":"2024-04-20T21:16:59.911710+0000","last_clean":"2024-04-20T21:16:59.911710+0000","last_became_active":"2024-04-20T21:16:54.578543+0000","last_became_peered":"2024-04-20T21:16:54.578543+0000","last_unstale":"2024-04-20T21:16:59.911710+0000","last_undegraded":"2024-04-20T21:16:59.911710+0000","last_fullsized":"2024-04-20T21:16:59.911710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:15.042318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605448+0000","last_change":"2024-04-20T21:16:54.565658+0000","last_active":"2024-04-20T21:16:59.605448+0000","last_peered":"2024-04-20T21:16:59.605448+0000","last_clean":"2024-04-20T21:16:59.605448+0000","last_became_active":"2024-04-20T21:16:54.565371+0000","last_became_peered":"2024-04-20T21:16:54.565371+0000","last_unstale":"2024-04-20T21:16:59.605448+0000","last_undegraded":"2024-04-20T21:16:59.605448+0000","last_fullsized":"2024-04-20T21:16:59.605448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:21.781013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912115+0000","last_change":"2024-04-20T21:16:54.568472+0000","last_active":"2024-04-20T21:16:59.912115+0000","last_peered":"2024-04-20T21:16:59.912115+0000","last_clean":"2024-04-20T21:16:59.912115+0000","last_became_active":"2024-04-20T21:16:54.568223+0000","last_became_peered":"2024-04-20T21:16:54.568223+0000","last_unstale":"2024-04-20T21:16:59.912115+0000","last_undegraded":"2024-04-20T21:16:59.912115+0000","last_fullsized":"2024-04-20T21:16:59.912115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:42:53.921935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916243+0000","last_change":"2024-04-20T21:16:54.573747+0000","last_active":"2024-04-20T21:16:59.916243+0000","last_peered":"2024-04-20T21:16:59.916243+0000","last_clean":"2024-04-20T21:16:59.916243+0000","last_became_active":"2024-04-20T21:16:54.573476+0000","last_became_peered":"2024-04-20T21:16:54.573476+0000","last_unstale":"2024-04-20T21:16:59.916243+0000","last_undegraded":"2024-04-20T21:16:59.916243+0000","last_fullsized":"2024-04-20T21:16:59.916243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:02.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913473+0000","last_change":"2024-04-20T21:16:54.577630+0000","last_active":"2024-04-20T21:16:59.913473+0000","last_peered":"2024-04-20T21:16:59.913473+0000","last_clean":"2024-04-20T21:16:59.913473+0000","last_became_active":"2024-04-20T21:16:54.577318+0000","last_became_peered":"2024-04-20T21:16:54.577318+0000","last_unstale":"2024-04-20T21:16:59.913473+0000","last_undegraded":"2024-04-20T21:16:59.913473+0000","last_fullsized":"2024-04-20T21:16:59.913473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:45.584050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608267+0000","last_change":"2024-04-20T21:16:54.579850+0000","last_active":"2024-04-20T21:16:59.608267+0000","last_peered":"2024-04-20T21:16:59.608267+0000","last_clean":"2024-04-20T21:16:59.608267+0000","last_became_active":"2024-04-20T21:16:54.579546+0000","last_became_peered":"2024-04-20T21:16:54.579546+0000","last_unstale":"2024-04-20T21:16:59.608267+0000","last_undegraded":"2024-04-20T21:16:59.608267+0000","last_fullsized":"2024-04-20T21:16:59.608267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:49:53.577077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912501+0000","last_change":"2024-04-20T21:16:54.559174+0000","last_active":"2024-04-20T21:16:59.912501+0000","last_peered":"2024-04-20T21:16:59.912501+0000","last_clean":"2024-04-20T21:16:59.912501+0000","last_became_active":"2024-04-20T21:16:54.558904+0000","last_became_peered":"2024-04-20T21:16:54.558904+0000","last_unstale":"2024-04-20T21:16:59.912501+0000","last_undegraded":"2024-04-20T21:16:59.912501+0000","last_fullsized":"2024-04-20T21:16:59.912501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:09.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916492+0000","last_change":"2024-04-20T21:16:54.574943+0000","last_active":"2024-04-20T21:16:59.916492+0000","last_peered":"2024-04-20T21:16:59.916492+0000","last_clean":"2024-04-20T21:16:59.916492+0000","last_became_active":"2024-04-20T21:16:54.574471+0000","last_became_peered":"2024-04-20T21:16:54.574471+0000","last_unstale":"2024-04-20T21:16:59.916492+0000","last_undegraded":"2024-04-20T21:16:59.916492+0000","last_fullsized":"2024-04-20T21:16:59.916492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:51:44.741339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911545+0000","last_change":"2024-04-20T21:16:54.566222+0000","last_active":"2024-04-20T21:16:59.911545+0000","last_peered":"2024-04-20T21:16:59.911545+0000","last_clean":"2024-04-20T21:16:59.911545+0000","last_became_active":"2024-04-20T21:16:54.566011+0000","last_became_peered":"2024-04-20T21:16:54.566011+0000","last_unstale":"2024-04-20T21:16:59.911545+0000","last_undegraded":"2024-04-20T21:16:59.911545+0000","last_fullsized":"2024-04-20T21:16:59.911545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:53.369950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914398+0000","last_change":"2024-04-20T21:16:54.571179+0000","last_active":"2024-04-20T21:16:59.914398+0000","last_peered":"2024-04-20T21:16:59.914398+0000","last_clean":"2024-04-20T21:16:59.914398+0000","last_became_active":"2024-04-20T21:16:54.570855+0000","last_became_peered":"2024-04-20T21:16:54.570855+0000","last_unstale":"2024-04-20T21:16:59.914398+0000","last_undegraded":"2024-04-20T21:16:59.914398+0000","last_fullsized":"2024-04-20T21:16:59.914398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:59.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911999+0000","last_change":"2024-04-20T21:16:54.559092+0000","last_active":"2024-04-20T21:16:59.911999+0000","last_peered":"2024-04-20T21:16:59.911999+0000","last_clean":"2024-04-20T21:16:59.911999+0000","last_became_active":"2024-04-20T21:16:54.558847+0000","last_became_peered":"2024-04-20T21:16:54.558847+0000","last_unstale":"2024-04-20T21:16:59.911999+0000","last_undegraded":"2024-04-20T21:16:59.911999+0000","last_fullsized":"2024-04-20T21:16:59.911999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:02.752070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915426+0000","last_change":"2024-04-20T21:16:54.571815+0000","last_active":"2024-04-20T21:16:59.915426+0000","last_peered":"2024-04-20T21:16:59.915426+0000","last_clean":"2024-04-20T21:16:59.915426+0000","last_became_active":"2024-04-20T21:16:54.570704+0000","last_became_peered":"2024-04-20T21:16:54.570704+0000","last_unstale":"2024-04-20T21:16:59.915426+0000","last_undegraded":"2024-04-20T21:16:59.915426+0000","last_fullsized":"2024-04-20T21:16:59.915426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:56.511539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912064+0000","last_change":"2024-04-20T21:16:54.555591+0000","last_active":"2024-04-20T21:16:59.912064+0000","last_peered":"2024-04-20T21:16:59.912064+0000","last_clean":"2024-04-20T21:16:59.912064+0000","last_became_active":"2024-04-20T21:16:54.555389+0000","last_became_peered":"2024-04-20T21:16:54.555389+0000","last_unstale":"2024-04-20T21:16:59.912064+0000","last_undegraded":"2024-04-20T21:16:59.912064+0000","last_fullsized":"2024-04-20T21:16:59.912064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:16:25.976707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617803+0000","last_change":"2024-04-20T21:16:54.564088+0000","last_active":"2024-04-20T21:16:59.617803+0000","last_peered":"2024-04-20T21:16:59.617803+0000","last_clean":"2024-04-20T21:16:59.617803+0000","last_became_active":"2024-04-20T21:16:54.563796+0000","last_became_peered":"2024-04-20T21:16:54.563796+0000","last_unstale":"2024-04-20T21:16:59.617803+0000","last_undegraded":"2024-04-20T21:16:59.617803+0000","last_fullsized":"2024-04-20T21:16:59.617803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:07.986736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606878+0000","last_change":"2024-04-20T21:16:54.563578+0000","last_active":"2024-04-20T21:16:59.606878+0000","last_peered":"2024-04-20T21:16:59.606878+0000","last_clean":"2024-04-20T21:16:59.606878+0000","last_became_active":"2024-04-20T21:16:54.563317+0000","last_became_peered":"2024-04-20T21:16:54.563317+0000","last_unstale":"2024-04-20T21:16:59.606878+0000","last_undegraded":"2024-04-20T21:16:59.606878+0000","last_fullsized":"2024-04-20T21:16:59.606878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:06.093675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912217+0000","last_change":"2024-04-20T21:16:54.565168+0000","last_active":"2024-04-20T21:16:59.912217+0000","last_peered":"2024-04-20T21:16:59.912217+0000","last_clean":"2024-04-20T21:16:59.912217+0000","last_became_active":"2024-04-20T21:16:54.564712+0000","last_became_peered":"2024-04-20T21:16:54.564712+0000","last_unstale":"2024-04-20T21:16:59.912217+0000","last_undegraded":"2024-04-20T21:16:59.912217+0000","last_fullsized":"2024-04-20T21:16:59.912217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:07:30.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911433+0000","last_change":"2024-04-20T21:16:54.572211+0000","last_active":"2024-04-20T21:16:59.911433+0000","last_peered":"2024-04-20T21:16:59.911433+0000","last_clean":"2024-04-20T21:16:59.911433+0000","last_became_active":"2024-04-20T21:16:54.571894+0000","last_became_peered":"2024-04-20T21:16:54.571894+0000","last_unstale":"2024-04-20T21:16:59.911433+0000","last_undegraded":"2024-04-20T21:16:59.911433+0000","last_fullsized":"2024-04-20T21:16:59.911433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:59.376514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606682+0000","last_change":"2024-04-20T21:16:54.555195+0000","last_active":"2024-04-20T21:16:59.606682+0000","last_peered":"2024-04-20T21:16:59.606682+0000","last_clean":"2024-04-20T21:16:59.606682+0000","last_became_active":"2024-04-20T21:16:54.554284+0000","last_became_peered":"2024-04-20T21:16:54.554284+0000","last_unstale":"2024-04-20T21:16:59.606682+0000","last_undegraded":"2024-04-20T21:16:59.606682+0000","last_fullsized":"2024-04-20T21:16:59.606682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:42:21.500952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915085+0000","last_change":"2024-04-20T21:16:54.565527+0000","last_active":"2024-04-20T21:16:59.915085+0000","last_peered":"2024-04-20T21:16:59.915085+0000","last_clean":"2024-04-20T21:16:59.915085+0000","last_became_active":"2024-04-20T21:16:54.565211+0000","last_became_peered":"2024-04-20T21:16:54.565211+0000","last_unstale":"2024-04-20T21:16:59.915085+0000","last_undegraded":"2024-04-20T21:16:59.915085+0000","last_fullsized":"2024-04-20T21:16:59.915085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:17:40.462670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913206+0000","last_change":"2024-04-20T21:16:54.576464+0000","last_active":"2024-04-20T21:16:59.913206+0000","last_peered":"2024-04-20T21:16:59.913206+0000","last_clean":"2024-04-20T21:16:59.913206+0000","last_became_active":"2024-04-20T21:16:54.575947+0000","last_became_peered":"2024-04-20T21:16:54.575947+0000","last_unstale":"2024-04-20T21:16:59.913206+0000","last_undegraded":"2024-04-20T21:16:59.913206+0000","last_fullsized":"2024-04-20T21:16:59.913206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:50:29.581938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604732+0000","last_change":"2024-04-20T21:16:54.558162+0000","last_active":"2024-04-20T21:16:59.604732+0000","last_peered":"2024-04-20T21:16:59.604732+0000","last_clean":"2024-04-20T21:16:59.604732+0000","last_became_active":"2024-04-20T21:16:54.557875+0000","last_became_peered":"2024-04-20T21:16:54.557875+0000","last_unstale":"2024-04-20T21:16:59.604732+0000","last_undegraded":"2024-04-20T21:16:59.604732+0000","last_fullsized":"2024-04-20T21:16:59.604732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:42:46.886405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911994+0000","last_change":"2024-04-20T21:16:54.560566+0000","last_active":"2024-04-20T21:16:59.911994+0000","last_peered":"2024-04-20T21:16:59.911994+0000","last_clean":"2024-04-20T21:16:59.911994+0000","last_became_active":"2024-04-20T21:16:54.559409+0000","last_became_peered":"2024-04-20T21:16:54.559409+0000","last_unstale":"2024-04-20T21:16:59.911994+0000","last_undegraded":"2024-04-20T21:16:59.911994+0000","last_fullsized":"2024-04-20T21:16:59.911994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:00.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912405+0000","last_change":"2024-04-20T21:16:54.559491+0000","last_active":"2024-04-20T21:16:59.912405+0000","last_peered":"2024-04-20T21:16:59.912405+0000","last_clean":"2024-04-20T21:16:59.912405+0000","last_became_active":"2024-04-20T21:16:54.558187+0000","last_became_peered":"2024-04-20T21:16:54.558187+0000","last_unstale":"2024-04-20T21:16:59.912405+0000","last_undegraded":"2024-04-20T21:16:59.912405+0000","last_fullsized":"2024-04-20T21:16:59.912405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:33.686248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617814+0000","last_change":"2024-04-20T21:16:54.564258+0000","last_active":"2024-04-20T21:16:59.617814+0000","last_peered":"2024-04-20T21:16:59.617814+0000","last_clean":"2024-04-20T21:16:59.617814+0000","last_became_active":"2024-04-20T21:16:54.563966+0000","last_became_peered":"2024-04-20T21:16:54.563966+0000","last_unstale":"2024-04-20T21:16:59.617814+0000","last_undegraded":"2024-04-20T21:16:59.617814+0000","last_fullsized":"2024-04-20T21:16:59.617814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:13:34.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912591+0000","last_change":"2024-04-20T21:16:54.573554+0000","last_active":"2024-04-20T21:16:59.912591+0000","last_peered":"2024-04-20T21:16:59.912591+0000","last_clean":"2024-04-20T21:16:59.912591+0000","last_became_active":"2024-04-20T21:16:54.571849+0000","last_became_peered":"2024-04-20T21:16:54.571849+0000","last_unstale":"2024-04-20T21:16:59.912591+0000","last_undegraded":"2024-04-20T21:16:59.912591+0000","last_fullsized":"2024-04-20T21:16:59.912591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:09.481779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617599+0000","last_change":"2024-04-20T21:16:54.571471+0000","last_active":"2024-04-20T21:16:59.617599+0000","last_peered":"2024-04-20T21:16:59.617599+0000","last_clean":"2024-04-20T21:16:59.617599+0000","last_became_active":"2024-04-20T21:16:54.571209+0000","last_became_peered":"2024-04-20T21:16:54.571209+0000","last_unstale":"2024-04-20T21:16:59.617599+0000","last_undegraded":"2024-04-20T21:16:59.617599+0000","last_fullsized":"2024-04-20T21:16:59.617599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:35.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911811+0000","last_change":"2024-04-20T21:16:54.572168+0000","last_active":"2024-04-20T21:16:59.911811+0000","last_peered":"2024-04-20T21:16:59.911811+0000","last_clean":"2024-04-20T21:16:59.911811+0000","last_became_active":"2024-04-20T21:16:54.571740+0000","last_became_peered":"2024-04-20T21:16:54.571740+0000","last_unstale":"2024-04-20T21:16:59.911811+0000","last_undegraded":"2024-04-20T21:16:59.911811+0000","last_fullsized":"2024-04-20T21:16:59.911811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:11:35.728824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617393+0000","last_change":"2024-04-20T21:16:54.569081+0000","last_active":"2024-04-20T21:16:59.617393+0000","last_peered":"2024-04-20T21:16:59.617393+0000","last_clean":"2024-04-20T21:16:59.617393+0000","last_became_active":"2024-04-20T21:16:54.568847+0000","last_became_peered":"2024-04-20T21:16:54.568847+0000","last_unstale":"2024-04-20T21:16:59.617393+0000","last_undegraded":"2024-04-20T21:16:59.617393+0000","last_fullsized":"2024-04-20T21:16:59.617393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:35.737237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912610+0000","last_change":"2024-04-20T21:16:54.571525+0000","last_active":"2024-04-20T21:16:59.912610+0000","last_peered":"2024-04-20T21:16:59.912610+0000","last_clean":"2024-04-20T21:16:59.912610+0000","last_became_active":"2024-04-20T21:16:54.570776+0000","last_became_peered":"2024-04-20T21:16:54.570776+0000","last_unstale":"2024-04-20T21:16:59.912610+0000","last_undegraded":"2024-04-20T21:16:59.912610+0000","last_fullsized":"2024-04-20T21:16:59.912610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:03.693476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916192+0000","last_change":"2024-04-20T21:16:54.570895+0000","last_active":"2024-04-20T21:16:59.916192+0000","last_peered":"2024-04-20T21:16:59.916192+0000","last_clean":"2024-04-20T21:16:59.916192+0000","last_became_active":"2024-04-20T21:16:54.570577+0000","last_became_peered":"2024-04-20T21:16:54.570577+0000","last_unstale":"2024-04-20T21:16:59.916192+0000","last_undegraded":"2024-04-20T21:16:59.916192+0000","last_fullsized":"2024-04-20T21:16:59.916192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:26.787487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608644+0000","last_change":"2024-04-20T21:16:54.581648+0000","last_active":"2024-04-20T21:16:59.608644+0000","last_peered":"2024-04-20T21:16:59.608644+0000","last_clean":"2024-04-20T21:16:59.608644+0000","last_became_active":"2024-04-20T21:16:54.581382+0000","last_became_peered":"2024-04-20T21:16:54.581382+0000","last_unstale":"2024-04-20T21:16:59.608644+0000","last_undegraded":"2024-04-20T21:16:59.608644+0000","last_fullsized":"2024-04-20T21:16:59.608644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:21.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617750+0000","last_change":"2024-04-20T21:16:54.563949+0000","last_active":"2024-04-20T21:16:59.617750+0000","last_peered":"2024-04-20T21:16:59.617750+0000","last_clean":"2024-04-20T21:16:59.617750+0000","last_became_active":"2024-04-20T21:16:54.563606+0000","last_became_peered":"2024-04-20T21:16:54.563606+0000","last_unstale":"2024-04-20T21:16:59.617750+0000","last_undegraded":"2024-04-20T21:16:59.617750+0000","last_fullsized":"2024-04-20T21:16:59.617750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:26:05.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:22.215 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:22.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.452+0000 7f976be00640 1 Processor -- start 2024-04-20T21:17:22.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.452+0000 7f976be00640 1 -- start start 2024-04-20T21:17:22.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.454+0000 7f976be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f976414fa50 0x7f97640c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.454+0000 7f976be00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97641536d0 con 0x7f97640c9eb0 2024-04-20T21:17:22.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.454+0000 7f976be00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97640c76c0 con 0x7f9764073a10 2024-04-20T21:17:22.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.454+0000 7f976be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f97640c7960 con 0x7f976414fa50 2024-04-20T21:17:22.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.454+0000 7f9768a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f97640c9eb0 0x7f97640c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38644/0 (socket says 172.21.15.102:38644) 2024-04-20T21:17:22.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.454+0000 7f9768a00640 1 -- 172.21.15.102:0/3772205450 learned_addr learned my addr 172.21.15.102:0/3772205450 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:22.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.454+0000 7f9769400640 1 --2- 172.21.15.102:0/3772205450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f976414fa50 0x7f97640c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.454+0000 7f9763e00640 1 -- 172.21.15.102:0/3772205450 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052227454 0 0) 0x7f9750001110 con 0x7f97640c9eb0 2024-04-20T21:17:22.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.455+0000 7f9763e00640 1 -- 172.21.15.102:0/3772205450 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f974c003520 con 0x7f97640c9eb0 2024-04-20T21:17:22.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.455+0000 7f9763e00640 1 -- 172.21.15.102:0/3772205450 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599888014 0 0) 0x7f97640c76c0 con 0x7f9764073a10 2024-04-20T21:17:22.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.455+0000 7f9769400640 1 -- 172.21.15.102:0/3772205450 >> v1:172.21.15.193:6789/0 conn(0x7f9764073a10 legacy=0x7f9764163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.455+0000 7f9769400640 1 -- 172.21.15.102:0/3772205450 >> v1:172.21.15.102:6789/0 conn(0x7f97640c9eb0 legacy=0x7f97640c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.455+0000 7f9769400640 1 -- 172.21.15.102:0/3772205450 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97640c3d90 con 0x7f976414fa50 2024-04-20T21:17:22.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.456+0000 7f9769400640 1 --2- 172.21.15.102:0/3772205450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f976414fa50 0x7f97640c4830 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97640c3f60 tx=0x7f975802bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4f91933280a6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:22.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.456+0000 7f9763e00640 1 -- 172.21.15.102:0/3772205450 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758039070 con 0x7f976414fa50 2024-04-20T21:17:22.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.456+0000 7f9763e00640 1 -- 172.21.15.102:0/3772205450 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975802cd50 con 0x7f976414fa50 2024-04-20T21:17:22.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.456+0000 7f9763e00640 1 -- 172.21.15.102:0/3772205450 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758034660 con 0x7f976414fa50 2024-04-20T21:17:22.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.457+0000 7f976be00640 1 -- 172.21.15.102:0/3772205450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f976414fa50 msgr2=0x7f97640c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.457+0000 7f976be00640 1 --2- 172.21.15.102:0/3772205450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f976414fa50 0x7f97640c4830 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97640c3f60 tx=0x7f975802bfa0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.457+0000 7f976be00640 1 -- 172.21.15.102:0/3772205450 shutdown_connections 2024-04-20T21:17:22.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.457+0000 7f976be00640 1 --2- 172.21.15.102:0/3772205450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f976414fa50 0x7f97640c4830 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.457+0000 7f976be00640 1 -- 172.21.15.102:0/3772205450 >> 172.21.15.102:0/3772205450 conn(0x7f97640c9390 msgr2=0x7f97640c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:22.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.457+0000 7f976be00640 1 -- 172.21.15.102:0/3772205450 shutdown_connections 2024-04-20T21:17:22.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.457+0000 7f976be00640 1 -- 172.21.15.102:0/3772205450 wait complete. 2024-04-20T21:17:22.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.458+0000 7f976be00640 1 Processor -- start 2024-04-20T21:17:22.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.459+0000 7f976be00640 1 -- start start 2024-04-20T21:17:22.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.459+0000 7f976be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9764073a10 0x7f97641f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.459+0000 7f976be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97640c9eb0 0x7f9764146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.459+0000 7f976be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f976414fa50 0x7f97640cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f976be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f97640c4030 con 0x7f976414fa50 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f976be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97640c7c00 con 0x7f97640c9eb0 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f976be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f97640c4310 con 0x7f9764073a10 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f976414fa50 0x7f97640cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f976414fa50 0x7f97640cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51322/0 (socket says 172.21.15.102:51322) 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9768a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97640c9eb0 0x7f9764146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769e00640 1 -- 172.21.15.102:0/785649249 learned_addr learned my addr 172.21.15.102:0/785649249 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769400640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9764073a10 0x7f97641f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769e00640 1 -- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9764073a10 msgr2=0x7f97641f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769e00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9764073a10 0x7f97641f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769e00640 1 -- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97640c9eb0 msgr2=0x7f9764146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769e00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97640c9eb0 0x7f9764146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.460+0000 7f9769e00640 1 -- 172.21.15.102:0/785649249 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9764155eb0 con 0x7f976414fa50 2024-04-20T21:17:22.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.461+0000 7f9769e00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f976414fa50 0x7f97640cba80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c480 tx=0x7f975c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:22.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.461+0000 7f9761600640 1 -- 172.21.15.102:0/785649249 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c005480 con 0x7f976414fa50 2024-04-20T21:17:22.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.461+0000 7f9761600640 1 -- 172.21.15.102:0/785649249 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975c0035f0 con 0x7f976414fa50 2024-04-20T21:17:22.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.461+0000 7f9761600640 1 -- 172.21.15.102:0/785649249 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0039c0 con 0x7f976414fa50 2024-04-20T21:17:22.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.461+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9764156190 con 0x7f976414fa50 2024-04-20T21:17:22.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.461+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9764154b50 con 0x7f976414fa50 2024-04-20T21:17:22.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.462+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97640c56b0 con 0x7f976414fa50 2024-04-20T21:17:22.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.463+0000 7f9761600640 1 -- 172.21.15.102:0/785649249 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f975c003b60 con 0x7f976414fa50 2024-04-20T21:17:22.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.464+0000 7f9761600640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f974406ca90 0x7f974406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.464+0000 7f9761600640 1 -- 172.21.15.102:0/785649249 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f975c08b940 con 0x7f976414fa50 2024-04-20T21:17:22.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.464+0000 7f9769400640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f974406ca90 0x7f974406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.465+0000 7f9769400640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f974406ca90 0x7f974406ef50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9758036000 tx=0x7f9758030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:22.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.468+0000 7f9761600640 1 -- 172.21.15.102:0/785649249 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f97640c56b0 con 0x7f976414fa50 2024-04-20T21:17:22.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.654+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 --> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9764074340 con 0x7f974406ca90 2024-04-20T21:17:22.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.665+0000 7f9761600640 1 -- 172.21.15.102:0/785649249 <== mgr.4112 v2:172.21.15.193:6832/2803621564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332562 (secure 0 0 0) 0x7f9764074340 con 0x7f974406ca90 2024-04-20T21:17:22.667 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:22.671 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-20T21:17:22.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.671+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f974406ca90 msgr2=0x7f974406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.671+0000 7f976be00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f974406ca90 0x7f974406ef50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9758036000 tx=0x7f9758030000 comp rx=0 tx=0).stop 2024-04-20T21:17:22.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f976414fa50 msgr2=0x7f97640cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f976414fa50 0x7f97640cba80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c480 tx=0x7f975c00c940 comp rx=0 tx=0).stop 2024-04-20T21:17:22.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 shutdown_connections 2024-04-20T21:17:22.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f976414fa50 0x7f97640cba80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f974406ca90 0x7f974406ef50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97640c9eb0 0x7f9764146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 --2- 172.21.15.102:0/785649249 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9764073a10 0x7f97641f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 >> 172.21.15.102:0/785649249 conn(0x7f97640c9390 msgr2=0x7f976406d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:22.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 shutdown_connections 2024-04-20T21:17:22.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.672+0000 7f976be00640 1 -- 172.21.15.102:0/785649249 wait complete. 2024-04-20T21:17:22.692 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-20T21:17:20.667363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1991923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003319"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918498+0000","last_change":"2024-04-20T21:16:53.564922+0000","last_active":"2024-04-20T21:16:59.918498+0000","last_peered":"2024-04-20T21:16:59.918498+0000","last_clean":"2024-04-20T21:16:59.918498+0000","last_became_active":"2024-04-20T21:16:53.564589+0000","last_became_peered":"2024-04-20T21:16:53.564589+0000","last_unstale":"2024-04-20T21:16:59.918498+0000","last_undegraded":"2024-04-20T21:16:59.918498+0000","last_fullsized":"2024-04-20T21:16:59.918498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:47.284664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916138+0000","last_change":"2024-04-20T21:16:54.580132+0000","last_active":"2024-04-20T21:16:59.916138+0000","last_peered":"2024-04-20T21:16:59.916138+0000","last_clean":"2024-04-20T21:16:59.916138+0000","last_became_active":"2024-04-20T21:16:54.579769+0000","last_became_peered":"2024-04-20T21:16:54.579769+0000","last_unstale":"2024-04-20T21:16:59.916138+0000","last_undegraded":"2024-04-20T21:16:59.916138+0000","last_fullsized":"2024-04-20T21:16:59.916138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:00.705947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913935+0000","last_change":"2024-04-20T21:16:58.781707+0000","last_active":"2024-04-20T21:16:59.913935+0000","last_peered":"2024-04-20T21:16:59.913935+0000","last_clean":"2024-04-20T21:16:59.913935+0000","last_became_active":"2024-04-20T21:16:58.780685+0000","last_became_peered":"2024-04-20T21:16:58.780685+0000","last_unstale":"2024-04-20T21:16:59.913935+0000","last_undegraded":"2024-04-20T21:16:59.913935+0000","last_fullsized":"2024-04-20T21:16:59.913935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:50.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916578+0000","last_change":"2024-04-20T21:16:53.572137+0000","last_active":"2024-04-20T21:16:59.916578+0000","last_peered":"2024-04-20T21:16:59.916578+0000","last_clean":"2024-04-20T21:16:59.916578+0000","last_became_active":"2024-04-20T21:16:53.567907+0000","last_became_peered":"2024-04-20T21:16:53.567907+0000","last_unstale":"2024-04-20T21:16:59.916578+0000","last_undegraded":"2024-04-20T21:16:59.916578+0000","last_fullsized":"2024-04-20T21:16:59.916578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:23:28.367174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918240+0000","last_change":"2024-04-20T21:16:54.567096+0000","last_active":"2024-04-20T21:16:59.918240+0000","last_peered":"2024-04-20T21:16:59.918240+0000","last_clean":"2024-04-20T21:16:59.918240+0000","last_became_active":"2024-04-20T21:16:54.566803+0000","last_became_peered":"2024-04-20T21:16:54.566803+0000","last_unstale":"2024-04-20T21:16:59.918240+0000","last_undegraded":"2024-04-20T21:16:59.918240+0000","last_fullsized":"2024-04-20T21:16:59.918240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:13:44.730056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604352+0000","last_change":"2024-04-20T21:16:58.619283+0000","last_active":"2024-04-20T21:16:59.604352+0000","last_peered":"2024-04-20T21:16:59.604352+0000","last_clean":"2024-04-20T21:16:59.604352+0000","last_became_active":"2024-04-20T21:16:58.618881+0000","last_became_peered":"2024-04-20T21:16:58.618881+0000","last_unstale":"2024-04-20T21:16:59.604352+0000","last_undegraded":"2024-04-20T21:16:59.604352+0000","last_fullsized":"2024-04-20T21:16:59.604352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:01:40.143389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617802+0000","last_change":"2024-04-20T21:16:53.554847+0000","last_active":"2024-04-20T21:16:59.617802+0000","last_peered":"2024-04-20T21:16:59.617802+0000","last_clean":"2024-04-20T21:16:59.617802+0000","last_became_active":"2024-04-20T21:16:53.554333+0000","last_became_peered":"2024-04-20T21:16:53.554333+0000","last_unstale":"2024-04-20T21:16:59.617802+0000","last_undegraded":"2024-04-20T21:16:59.617802+0000","last_fullsized":"2024-04-20T21:16:59.617802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:33.255945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912613+0000","last_change":"2024-04-20T21:16:54.556407+0000","last_active":"2024-04-20T21:16:59.912613+0000","last_peered":"2024-04-20T21:16:59.912613+0000","last_clean":"2024-04-20T21:16:59.912613+0000","last_became_active":"2024-04-20T21:16:54.556161+0000","last_became_peered":"2024-04-20T21:16:54.556161+0000","last_unstale":"2024-04-20T21:16:59.912613+0000","last_undegraded":"2024-04-20T21:16:59.912613+0000","last_fullsized":"2024-04-20T21:16:59.912613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:44:53.586714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604519+0000","last_change":"2024-04-20T21:16:58.612063+0000","last_active":"2024-04-20T21:16:59.604519+0000","last_peered":"2024-04-20T21:16:59.604519+0000","last_clean":"2024-04-20T21:16:59.604519+0000","last_became_active":"2024-04-20T21:16:58.611487+0000","last_became_peered":"2024-04-20T21:16:58.611487+0000","last_unstale":"2024-04-20T21:16:59.604519+0000","last_undegraded":"2024-04-20T21:16:59.604519+0000","last_fullsized":"2024-04-20T21:16:59.604519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:15.097731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617458+0000","last_change":"2024-04-20T21:16:53.551212+0000","last_active":"2024-04-20T21:16:59.617458+0000","last_peered":"2024-04-20T21:16:59.617458+0000","last_clean":"2024-04-20T21:16:59.617458+0000","last_became_active":"2024-04-20T21:16:53.550899+0000","last_became_peered":"2024-04-20T21:16:53.550899+0000","last_unstale":"2024-04-20T21:16:59.617458+0000","last_undegraded":"2024-04-20T21:16:59.617458+0000","last_fullsized":"2024-04-20T21:16:59.617458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:07:18.283243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912476+0000","last_change":"2024-04-20T21:16:54.557285+0000","last_active":"2024-04-20T21:16:59.912476+0000","last_peered":"2024-04-20T21:16:59.912476+0000","last_clean":"2024-04-20T21:16:59.912476+0000","last_became_active":"2024-04-20T21:16:54.556921+0000","last_became_peered":"2024-04-20T21:16:54.556921+0000","last_unstale":"2024-04-20T21:16:59.912476+0000","last_undegraded":"2024-04-20T21:16:59.912476+0000","last_fullsized":"2024-04-20T21:16:59.912476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:33.989679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604784+0000","last_change":"2024-04-20T21:16:58.617360+0000","last_active":"2024-04-20T21:16:59.604784+0000","last_peered":"2024-04-20T21:16:59.604784+0000","last_clean":"2024-04-20T21:16:59.604784+0000","last_became_active":"2024-04-20T21:16:58.616947+0000","last_became_peered":"2024-04-20T21:16:58.616947+0000","last_unstale":"2024-04-20T21:16:59.604784+0000","last_undegraded":"2024-04-20T21:16:59.604784+0000","last_fullsized":"2024-04-20T21:16:59.604784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.202955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911434+0000","last_change":"2024-04-20T21:16:54.558067+0000","last_active":"2024-04-20T21:16:59.911434+0000","last_peered":"2024-04-20T21:16:59.911434+0000","last_clean":"2024-04-20T21:16:59.911434+0000","last_became_active":"2024-04-20T21:16:54.557781+0000","last_became_peered":"2024-04-20T21:16:54.557781+0000","last_unstale":"2024-04-20T21:16:59.911434+0000","last_undegraded":"2024-04-20T21:16:59.911434+0000","last_fullsized":"2024-04-20T21:16:59.911434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:56.576081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617756+0000","last_change":"2024-04-20T21:16:53.551615+0000","last_active":"2024-04-20T21:16:59.617756+0000","last_peered":"2024-04-20T21:16:59.617756+0000","last_clean":"2024-04-20T21:16:59.617756+0000","last_became_active":"2024-04-20T21:16:53.551073+0000","last_became_peered":"2024-04-20T21:16:53.551073+0000","last_unstale":"2024-04-20T21:16:59.617756+0000","last_undegraded":"2024-04-20T21:16:59.617756+0000","last_fullsized":"2024-04-20T21:16:59.617756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:26:29.994289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605362+0000","last_change":"2024-04-20T21:16:58.617434+0000","last_active":"2024-04-20T21:16:59.605362+0000","last_peered":"2024-04-20T21:16:59.605362+0000","last_clean":"2024-04-20T21:16:59.605362+0000","last_became_active":"2024-04-20T21:16:58.617027+0000","last_became_peered":"2024-04-20T21:16:58.617027+0000","last_unstale":"2024-04-20T21:16:59.605362+0000","last_undegraded":"2024-04-20T21:16:59.605362+0000","last_fullsized":"2024-04-20T21:16:59.605362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:44:20.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913331+0000","last_change":"2024-04-20T21:16:54.585333+0000","last_active":"2024-04-20T21:16:59.913331+0000","last_peered":"2024-04-20T21:16:59.913331+0000","last_clean":"2024-04-20T21:16:59.913331+0000","last_became_active":"2024-04-20T21:16:54.584978+0000","last_became_peered":"2024-04-20T21:16:54.584978+0000","last_unstale":"2024-04-20T21:16:59.913331+0000","last_undegraded":"2024-04-20T21:16:59.913331+0000","last_fullsized":"2024-04-20T21:16:59.913331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:23.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608374+0000","last_change":"2024-04-20T21:16:53.567775+0000","last_active":"2024-04-20T21:16:59.608374+0000","last_peered":"2024-04-20T21:16:59.608374+0000","last_clean":"2024-04-20T21:16:59.608374+0000","last_became_active":"2024-04-20T21:16:53.567498+0000","last_became_peered":"2024-04-20T21:16:53.567498+0000","last_unstale":"2024-04-20T21:16:59.608374+0000","last_undegraded":"2024-04-20T21:16:59.608374+0000","last_fullsized":"2024-04-20T21:16:59.608374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:10.695659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604469+0000","last_change":"2024-04-20T21:16:58.619302+0000","last_active":"2024-04-20T21:16:59.604469+0000","last_peered":"2024-04-20T21:16:59.604469+0000","last_clean":"2024-04-20T21:16:59.604469+0000","last_became_active":"2024-04-20T21:16:58.618843+0000","last_became_peered":"2024-04-20T21:16:58.618843+0000","last_unstale":"2024-04-20T21:16:59.604469+0000","last_undegraded":"2024-04-20T21:16:59.604469+0000","last_fullsized":"2024-04-20T21:16:59.604469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:14.927199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914247+0000","last_change":"2024-04-20T21:16:54.575862+0000","last_active":"2024-04-20T21:16:59.914247+0000","last_peered":"2024-04-20T21:16:59.914247+0000","last_clean":"2024-04-20T21:16:59.914247+0000","last_became_active":"2024-04-20T21:16:54.575523+0000","last_became_peered":"2024-04-20T21:16:54.575523+0000","last_unstale":"2024-04-20T21:16:59.914247+0000","last_undegraded":"2024-04-20T21:16:59.914247+0000","last_fullsized":"2024-04-20T21:16:59.914247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:51.041414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606833+0000","last_change":"2024-04-20T21:16:53.551995+0000","last_active":"2024-04-20T21:16:59.606833+0000","last_peered":"2024-04-20T21:16:59.606833+0000","last_clean":"2024-04-20T21:16:59.606833+0000","last_became_active":"2024-04-20T21:16:53.551603+0000","last_became_peered":"2024-04-20T21:16:53.551603+0000","last_unstale":"2024-04-20T21:16:59.606833+0000","last_undegraded":"2024-04-20T21:16:59.606833+0000","last_fullsized":"2024-04-20T21:16:59.606833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:56:21.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608819+0000","last_change":"2024-04-20T21:16:58.776463+0000","last_active":"2024-04-20T21:16:59.608819+0000","last_peered":"2024-04-20T21:16:59.608819+0000","last_clean":"2024-04-20T21:16:59.608819+0000","last_became_active":"2024-04-20T21:16:58.775746+0000","last_became_peered":"2024-04-20T21:16:58.775746+0000","last_unstale":"2024-04-20T21:16:59.608819+0000","last_undegraded":"2024-04-20T21:16:59.608819+0000","last_fullsized":"2024-04-20T21:16:59.608819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:28.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605047+0000","last_change":"2024-04-20T21:16:54.566449+0000","last_active":"2024-04-20T21:16:59.605047+0000","last_peered":"2024-04-20T21:16:59.605047+0000","last_clean":"2024-04-20T21:16:59.605047+0000","last_became_active":"2024-04-20T21:16:54.566251+0000","last_became_peered":"2024-04-20T21:16:54.566251+0000","last_unstale":"2024-04-20T21:16:59.605047+0000","last_undegraded":"2024-04-20T21:16:59.605047+0000","last_fullsized":"2024-04-20T21:16:59.605047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:53.403420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617047+0000","last_change":"2024-04-20T21:16:53.564029+0000","last_active":"2024-04-20T21:16:59.617047+0000","last_peered":"2024-04-20T21:16:59.617047+0000","last_clean":"2024-04-20T21:16:59.617047+0000","last_became_active":"2024-04-20T21:16:53.563711+0000","last_became_peered":"2024-04-20T21:16:53.563711+0000","last_unstale":"2024-04-20T21:16:59.617047+0000","last_undegraded":"2024-04-20T21:16:59.617047+0000","last_fullsized":"2024-04-20T21:16:59.617047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.634644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604995+0000","last_change":"2024-04-20T21:16:58.615080+0000","last_active":"2024-04-20T21:16:59.604995+0000","last_peered":"2024-04-20T21:16:59.604995+0000","last_clean":"2024-04-20T21:16:59.604995+0000","last_became_active":"2024-04-20T21:16:58.614611+0000","last_became_peered":"2024-04-20T21:16:58.614611+0000","last_unstale":"2024-04-20T21:16:59.604995+0000","last_undegraded":"2024-04-20T21:16:59.604995+0000","last_fullsized":"2024-04-20T21:16:59.604995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:08.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917639+0000","last_change":"2024-04-20T21:16:54.578272+0000","last_active":"2024-04-20T21:16:59.917639+0000","last_peered":"2024-04-20T21:16:59.917639+0000","last_clean":"2024-04-20T21:16:59.917639+0000","last_became_active":"2024-04-20T21:16:54.577949+0000","last_became_peered":"2024-04-20T21:16:54.577949+0000","last_unstale":"2024-04-20T21:16:59.917639+0000","last_undegraded":"2024-04-20T21:16:59.917639+0000","last_fullsized":"2024-04-20T21:16:59.917639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:35:33.807091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912813+0000","last_change":"2024-04-20T21:16:53.554600+0000","last_active":"2024-04-20T21:16:59.912813+0000","last_peered":"2024-04-20T21:16:59.912813+0000","last_clean":"2024-04-20T21:16:59.912813+0000","last_became_active":"2024-04-20T21:16:53.554347+0000","last_became_peered":"2024-04-20T21:16:53.554347+0000","last_unstale":"2024-04-20T21:16:59.912813+0000","last_undegraded":"2024-04-20T21:16:59.912813+0000","last_fullsized":"2024-04-20T21:16:59.912813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:40:58.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604315+0000","last_change":"2024-04-20T21:16:58.774474+0000","last_active":"2024-04-20T21:16:59.604315+0000","last_peered":"2024-04-20T21:16:59.604315+0000","last_clean":"2024-04-20T21:16:59.604315+0000","last_became_active":"2024-04-20T21:16:58.774037+0000","last_became_peered":"2024-04-20T21:16:58.774037+0000","last_unstale":"2024-04-20T21:16:59.604315+0000","last_undegraded":"2024-04-20T21:16:59.604315+0000","last_fullsized":"2024-04-20T21:16:59.604315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:46.992667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916707+0000","last_change":"2024-04-20T21:16:54.579435+0000","last_active":"2024-04-20T21:16:59.916707+0000","last_peered":"2024-04-20T21:16:59.916707+0000","last_clean":"2024-04-20T21:16:59.916707+0000","last_became_active":"2024-04-20T21:16:54.574617+0000","last_became_peered":"2024-04-20T21:16:54.574617+0000","last_unstale":"2024-04-20T21:16:59.916707+0000","last_undegraded":"2024-04-20T21:16:59.916707+0000","last_fullsized":"2024-04-20T21:16:59.916707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:51.500741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608822+0000","last_change":"2024-04-20T21:16:53.561144+0000","last_active":"2024-04-20T21:16:59.608822+0000","last_peered":"2024-04-20T21:16:59.608822+0000","last_clean":"2024-04-20T21:16:59.608822+0000","last_became_active":"2024-04-20T21:16:53.560742+0000","last_became_peered":"2024-04-20T21:16:53.560742+0000","last_unstale":"2024-04-20T21:16:59.608822+0000","last_undegraded":"2024-04-20T21:16:59.608822+0000","last_fullsized":"2024-04-20T21:16:59.608822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:39.212951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913482+0000","last_change":"2024-04-20T21:16:58.780124+0000","last_active":"2024-04-20T21:16:59.913482+0000","last_peered":"2024-04-20T21:16:59.913482+0000","last_clean":"2024-04-20T21:16:59.913482+0000","last_became_active":"2024-04-20T21:16:58.777449+0000","last_became_peered":"2024-04-20T21:16:58.777449+0000","last_unstale":"2024-04-20T21:16:59.913482+0000","last_undegraded":"2024-04-20T21:16:59.913482+0000","last_fullsized":"2024-04-20T21:16:59.913482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:07.622146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608726+0000","last_change":"2024-04-20T21:16:54.575331+0000","last_active":"2024-04-20T21:16:59.608726+0000","last_peered":"2024-04-20T21:16:59.608726+0000","last_clean":"2024-04-20T21:16:59.608726+0000","last_became_active":"2024-04-20T21:16:54.575061+0000","last_became_peered":"2024-04-20T21:16:54.575061+0000","last_unstale":"2024-04-20T21:16:59.608726+0000","last_undegraded":"2024-04-20T21:16:59.608726+0000","last_fullsized":"2024-04-20T21:16:59.608726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:47:20.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617871+0000","last_change":"2024-04-20T21:16:53.565561+0000","last_active":"2024-04-20T21:16:59.617871+0000","last_peered":"2024-04-20T21:16:59.617871+0000","last_clean":"2024-04-20T21:16:59.617871+0000","last_became_active":"2024-04-20T21:16:53.565207+0000","last_became_peered":"2024-04-20T21:16:53.565207+0000","last_unstale":"2024-04-20T21:16:59.617871+0000","last_undegraded":"2024-04-20T21:16:59.617871+0000","last_fullsized":"2024-04-20T21:16:59.617871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:07.512132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915133+0000","last_change":"2024-04-20T21:16:58.776337+0000","last_active":"2024-04-20T21:16:59.915133+0000","last_peered":"2024-04-20T21:16:59.915133+0000","last_clean":"2024-04-20T21:16:59.915133+0000","last_became_active":"2024-04-20T21:16:58.772404+0000","last_became_peered":"2024-04-20T21:16:58.772404+0000","last_unstale":"2024-04-20T21:16:59.915133+0000","last_undegraded":"2024-04-20T21:16:59.915133+0000","last_fullsized":"2024-04-20T21:16:59.915133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:11.554173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912522+0000","last_change":"2024-04-20T21:16:54.571199+0000","last_active":"2024-04-20T21:16:59.912522+0000","last_peered":"2024-04-20T21:16:59.912522+0000","last_clean":"2024-04-20T21:16:59.912522+0000","last_became_active":"2024-04-20T21:16:54.570882+0000","last_became_peered":"2024-04-20T21:16:54.570882+0000","last_unstale":"2024-04-20T21:16:59.912522+0000","last_undegraded":"2024-04-20T21:16:59.912522+0000","last_fullsized":"2024-04-20T21:16:59.912522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:52.046176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608915+0000","last_change":"2024-04-20T21:16:53.566253+0000","last_active":"2024-04-20T21:16:59.608915+0000","last_peered":"2024-04-20T21:16:59.608915+0000","last_clean":"2024-04-20T21:16:59.608915+0000","last_became_active":"2024-04-20T21:16:53.565893+0000","last_became_peered":"2024-04-20T21:16:53.565893+0000","last_unstale":"2024-04-20T21:16:59.608915+0000","last_undegraded":"2024-04-20T21:16:59.608915+0000","last_fullsized":"2024-04-20T21:16:59.608915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:16.302801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.607352+0000","last_change":"2024-04-20T21:16:58.611470+0000","last_active":"2024-04-20T21:16:59.607352+0000","last_peered":"2024-04-20T21:16:59.607352+0000","last_clean":"2024-04-20T21:16:59.607352+0000","last_became_active":"2024-04-20T21:16:58.611038+0000","last_became_peered":"2024-04-20T21:16:58.611038+0000","last_unstale":"2024-04-20T21:16:59.607352+0000","last_undegraded":"2024-04-20T21:16:59.607352+0000","last_fullsized":"2024-04-20T21:16:59.607352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:58:51.267480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604848+0000","last_change":"2024-04-20T21:16:54.577608+0000","last_active":"2024-04-20T21:16:59.604848+0000","last_peered":"2024-04-20T21:16:59.604848+0000","last_clean":"2024-04-20T21:16:59.604848+0000","last_became_active":"2024-04-20T21:16:54.577339+0000","last_became_peered":"2024-04-20T21:16:54.577339+0000","last_unstale":"2024-04-20T21:16:59.604848+0000","last_undegraded":"2024-04-20T21:16:59.604848+0000","last_fullsized":"2024-04-20T21:16:59.604848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:03.028585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912331+0000","last_change":"2024-04-20T21:16:53.547751+0000","last_active":"2024-04-20T21:16:59.912331+0000","last_peered":"2024-04-20T21:16:59.912331+0000","last_clean":"2024-04-20T21:16:59.912331+0000","last_became_active":"2024-04-20T21:16:53.547371+0000","last_became_peered":"2024-04-20T21:16:53.547371+0000","last_unstale":"2024-04-20T21:16:59.912331+0000","last_undegraded":"2024-04-20T21:16:59.912331+0000","last_fullsized":"2024-04-20T21:16:59.912331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:50.728500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608358+0000","last_change":"2024-04-20T21:16:58.622369+0000","last_active":"2024-04-20T21:16:59.608358+0000","last_peered":"2024-04-20T21:16:59.608358+0000","last_clean":"2024-04-20T21:16:59.608358+0000","last_became_active":"2024-04-20T21:16:58.621911+0000","last_became_peered":"2024-04-20T21:16:58.621911+0000","last_unstale":"2024-04-20T21:16:59.608358+0000","last_undegraded":"2024-04-20T21:16:59.608358+0000","last_fullsized":"2024-04-20T21:16:59.608358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:41.519583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606736+0000","last_change":"2024-04-20T21:16:54.579577+0000","last_active":"2024-04-20T21:16:59.606736+0000","last_peered":"2024-04-20T21:16:59.606736+0000","last_clean":"2024-04-20T21:16:59.606736+0000","last_became_active":"2024-04-20T21:16:54.579156+0000","last_became_peered":"2024-04-20T21:16:54.579156+0000","last_unstale":"2024-04-20T21:16:59.606736+0000","last_undegraded":"2024-04-20T21:16:59.606736+0000","last_fullsized":"2024-04-20T21:16:59.606736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:28:14.743667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605543+0000","last_change":"2024-04-20T21:16:53.567995+0000","last_active":"2024-04-20T21:16:59.605543+0000","last_peered":"2024-04-20T21:16:59.605543+0000","last_clean":"2024-04-20T21:16:59.605543+0000","last_became_active":"2024-04-20T21:16:53.567601+0000","last_became_peered":"2024-04-20T21:16:53.567601+0000","last_unstale":"2024-04-20T21:16:59.605543+0000","last_undegraded":"2024-04-20T21:16:59.605543+0000","last_fullsized":"2024-04-20T21:16:59.605543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:30.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609728+0000","last_change":"2024-04-20T21:16:58.626438+0000","last_active":"2024-04-20T21:16:59.609728+0000","last_peered":"2024-04-20T21:16:59.609728+0000","last_clean":"2024-04-20T21:16:59.609728+0000","last_became_active":"2024-04-20T21:16:58.625326+0000","last_became_peered":"2024-04-20T21:16:58.625326+0000","last_unstale":"2024-04-20T21:16:59.609728+0000","last_undegraded":"2024-04-20T21:16:59.609728+0000","last_fullsized":"2024-04-20T21:16:59.609728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:06.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914171+0000","last_change":"2024-04-20T21:16:54.560925+0000","last_active":"2024-04-20T21:16:59.914171+0000","last_peered":"2024-04-20T21:16:59.914171+0000","last_clean":"2024-04-20T21:16:59.914171+0000","last_became_active":"2024-04-20T21:16:54.560615+0000","last_became_peered":"2024-04-20T21:16:54.560615+0000","last_unstale":"2024-04-20T21:16:59.914171+0000","last_undegraded":"2024-04-20T21:16:59.914171+0000","last_fullsized":"2024-04-20T21:16:59.914171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:12.815692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617155+0000","last_change":"2024-04-20T21:16:53.559359+0000","last_active":"2024-04-20T21:16:59.617155+0000","last_peered":"2024-04-20T21:16:59.617155+0000","last_clean":"2024-04-20T21:16:59.617155+0000","last_became_active":"2024-04-20T21:16:53.558955+0000","last_became_peered":"2024-04-20T21:16:53.558955+0000","last_unstale":"2024-04-20T21:16:59.617155+0000","last_undegraded":"2024-04-20T21:16:59.617155+0000","last_fullsized":"2024-04-20T21:16:59.617155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:28.664404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917377+0000","last_change":"2024-04-20T21:16:58.772175+0000","last_active":"2024-04-20T21:16:59.917377+0000","last_peered":"2024-04-20T21:16:59.917377+0000","last_clean":"2024-04-20T21:16:59.917377+0000","last_became_active":"2024-04-20T21:16:58.771732+0000","last_became_peered":"2024-04-20T21:16:58.771732+0000","last_unstale":"2024-04-20T21:16:59.917377+0000","last_undegraded":"2024-04-20T21:16:59.917377+0000","last_fullsized":"2024-04-20T21:16:59.917377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:13.155547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912641+0000","last_change":"2024-04-20T21:16:54.575375+0000","last_active":"2024-04-20T21:16:59.912641+0000","last_peered":"2024-04-20T21:16:59.912641+0000","last_clean":"2024-04-20T21:16:59.912641+0000","last_became_active":"2024-04-20T21:16:54.574194+0000","last_became_peered":"2024-04-20T21:16:54.574194+0000","last_unstale":"2024-04-20T21:16:59.912641+0000","last_undegraded":"2024-04-20T21:16:59.912641+0000","last_fullsized":"2024-04-20T21:16:59.912641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:07.948431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911980+0000","last_change":"2024-04-20T21:16:53.559374+0000","last_active":"2024-04-20T21:16:59.911980+0000","last_peered":"2024-04-20T21:16:59.911980+0000","last_clean":"2024-04-20T21:16:59.911980+0000","last_became_active":"2024-04-20T21:16:53.559133+0000","last_became_peered":"2024-04-20T21:16:53.559133+0000","last_unstale":"2024-04-20T21:16:59.911980+0000","last_undegraded":"2024-04-20T21:16:59.911980+0000","last_fullsized":"2024-04-20T21:16:59.911980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:38.576103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617268+0000","last_change":"2024-04-20T21:16:44.100529+0000","last_active":"2024-04-20T21:16:59.617268+0000","last_peered":"2024-04-20T21:16:59.617268+0000","last_clean":"2024-04-20T21:16:59.617268+0000","last_became_active":"2024-04-20T21:16:44.099590+0000","last_became_peered":"2024-04-20T21:16:44.099590+0000","last_unstale":"2024-04-20T21:16:59.617268+0000","last_undegraded":"2024-04-20T21:16:59.617268+0000","last_fullsized":"2024-04-20T21:16:59.617268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_clean_scrub_stamp":"2024-04-20T21:16:42.782844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:08.054837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911874+0000","last_change":"2024-04-20T21:16:58.772894+0000","last_active":"2024-04-20T21:16:59.911874+0000","last_peered":"2024-04-20T21:16:59.911874+0000","last_clean":"2024-04-20T21:16:59.911874+0000","last_became_active":"2024-04-20T21:16:58.772451+0000","last_became_peered":"2024-04-20T21:16:58.772451+0000","last_unstale":"2024-04-20T21:16:59.911874+0000","last_undegraded":"2024-04-20T21:16:59.911874+0000","last_fullsized":"2024-04-20T21:16:59.911874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:29.914096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617716+0000","last_change":"2024-04-20T21:16:54.562592+0000","last_active":"2024-04-20T21:16:59.617716+0000","last_peered":"2024-04-20T21:16:59.617716+0000","last_clean":"2024-04-20T21:16:59.617716+0000","last_became_active":"2024-04-20T21:16:54.562243+0000","last_became_peered":"2024-04-20T21:16:54.562243+0000","last_unstale":"2024-04-20T21:16:59.617716+0000","last_undegraded":"2024-04-20T21:16:59.617716+0000","last_fullsized":"2024-04-20T21:16:59.617716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:52.832159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606010+0000","last_change":"2024-04-20T21:16:53.561132+0000","last_active":"2024-04-20T21:16:59.606010+0000","last_peered":"2024-04-20T21:16:59.606010+0000","last_clean":"2024-04-20T21:16:59.606010+0000","last_became_active":"2024-04-20T21:16:53.559901+0000","last_became_peered":"2024-04-20T21:16:53.559901+0000","last_unstale":"2024-04-20T21:16:59.606010+0000","last_undegraded":"2024-04-20T21:16:59.606010+0000","last_fullsized":"2024-04-20T21:16:59.606010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:20.229573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911503+0000","last_change":"2024-04-20T21:16:58.771204+0000","last_active":"2024-04-20T21:16:59.911503+0000","last_peered":"2024-04-20T21:16:59.911503+0000","last_clean":"2024-04-20T21:16:59.911503+0000","last_became_active":"2024-04-20T21:16:58.770627+0000","last_became_peered":"2024-04-20T21:16:58.770627+0000","last_unstale":"2024-04-20T21:16:59.911503+0000","last_undegraded":"2024-04-20T21:16:59.911503+0000","last_fullsized":"2024-04-20T21:16:59.911503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:39.472931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912685+0000","last_change":"2024-04-20T21:16:54.571417+0000","last_active":"2024-04-20T21:16:59.912685+0000","last_peered":"2024-04-20T21:16:59.912685+0000","last_clean":"2024-04-20T21:16:59.912685+0000","last_became_active":"2024-04-20T21:16:54.570434+0000","last_became_peered":"2024-04-20T21:16:54.570434+0000","last_unstale":"2024-04-20T21:16:59.912685+0000","last_undegraded":"2024-04-20T21:16:59.912685+0000","last_fullsized":"2024-04-20T21:16:59.912685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:24:41.026138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609282+0000","last_change":"2024-04-20T21:16:53.558554+0000","last_active":"2024-04-20T21:16:59.609282+0000","last_peered":"2024-04-20T21:16:59.609282+0000","last_clean":"2024-04-20T21:16:59.609282+0000","last_became_active":"2024-04-20T21:16:53.556934+0000","last_became_peered":"2024-04-20T21:16:53.556934+0000","last_unstale":"2024-04-20T21:16:59.609282+0000","last_undegraded":"2024-04-20T21:16:59.609282+0000","last_fullsized":"2024-04-20T21:16:59.609282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:17:52.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915951+0000","last_change":"2024-04-20T21:16:58.624410+0000","last_active":"2024-04-20T21:16:59.915951+0000","last_peered":"2024-04-20T21:16:59.915951+0000","last_clean":"2024-04-20T21:16:59.915951+0000","last_became_active":"2024-04-20T21:16:58.623677+0000","last_became_peered":"2024-04-20T21:16:58.623677+0000","last_unstale":"2024-04-20T21:16:59.915951+0000","last_undegraded":"2024-04-20T21:16:59.915951+0000","last_fullsized":"2024-04-20T21:16:59.915951+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:34.465594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617870+0000","last_change":"2024-04-20T21:16:54.560217+0000","last_active":"2024-04-20T21:16:59.617870+0000","last_peered":"2024-04-20T21:16:59.617870+0000","last_clean":"2024-04-20T21:16:59.617870+0000","last_became_active":"2024-04-20T21:16:54.559947+0000","last_became_peered":"2024-04-20T21:16:54.559947+0000","last_unstale":"2024-04-20T21:16:59.617870+0000","last_undegraded":"2024-04-20T21:16:59.617870+0000","last_fullsized":"2024-04-20T21:16:59.617870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:54.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.607164+0000","last_change":"2024-04-20T21:16:53.564779+0000","last_active":"2024-04-20T21:16:59.607164+0000","last_peered":"2024-04-20T21:16:59.607164+0000","last_clean":"2024-04-20T21:16:59.607164+0000","last_became_active":"2024-04-20T21:16:53.564500+0000","last_became_peered":"2024-04-20T21:16:53.564500+0000","last_unstale":"2024-04-20T21:16:59.607164+0000","last_undegraded":"2024-04-20T21:16:59.607164+0000","last_fullsized":"2024-04-20T21:16:59.607164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:00:58.673145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912319+0000","last_change":"2024-04-20T21:16:58.771524+0000","last_active":"2024-04-20T21:16:59.912319+0000","last_peered":"2024-04-20T21:16:59.912319+0000","last_clean":"2024-04-20T21:16:59.912319+0000","last_became_active":"2024-04-20T21:16:58.770672+0000","last_became_peered":"2024-04-20T21:16:58.770672+0000","last_unstale":"2024-04-20T21:16:59.912319+0000","last_undegraded":"2024-04-20T21:16:59.912319+0000","last_fullsized":"2024-04-20T21:16:59.912319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:52:27.037256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617704+0000","last_change":"2024-04-20T21:16:54.559570+0000","last_active":"2024-04-20T21:16:59.617704+0000","last_peered":"2024-04-20T21:16:59.617704+0000","last_clean":"2024-04-20T21:16:59.617704+0000","last_became_active":"2024-04-20T21:16:54.559307+0000","last_became_peered":"2024-04-20T21:16:54.559307+0000","last_unstale":"2024-04-20T21:16:59.617704+0000","last_undegraded":"2024-04-20T21:16:59.617704+0000","last_fullsized":"2024-04-20T21:16:59.617704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:04.417348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605512+0000","last_change":"2024-04-20T21:16:53.565289+0000","last_active":"2024-04-20T21:16:59.605512+0000","last_peered":"2024-04-20T21:16:59.605512+0000","last_clean":"2024-04-20T21:16:59.605512+0000","last_became_active":"2024-04-20T21:16:53.565014+0000","last_became_peered":"2024-04-20T21:16:53.565014+0000","last_unstale":"2024-04-20T21:16:59.605512+0000","last_undegraded":"2024-04-20T21:16:59.605512+0000","last_fullsized":"2024-04-20T21:16:59.605512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:22.125301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604480+0000","last_change":"2024-04-20T21:16:58.615103+0000","last_active":"2024-04-20T21:16:59.604480+0000","last_peered":"2024-04-20T21:16:59.604480+0000","last_clean":"2024-04-20T21:16:59.604480+0000","last_became_active":"2024-04-20T21:16:58.614068+0000","last_became_peered":"2024-04-20T21:16:58.614068+0000","last_unstale":"2024-04-20T21:16:59.604480+0000","last_undegraded":"2024-04-20T21:16:59.604480+0000","last_fullsized":"2024-04-20T21:16:59.604480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:09.555911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617489+0000","last_change":"2024-04-20T21:16:54.557439+0000","last_active":"2024-04-20T21:16:59.617489+0000","last_peered":"2024-04-20T21:16:59.617489+0000","last_clean":"2024-04-20T21:16:59.617489+0000","last_became_active":"2024-04-20T21:16:54.557178+0000","last_became_peered":"2024-04-20T21:16:54.557178+0000","last_unstale":"2024-04-20T21:16:59.617489+0000","last_undegraded":"2024-04-20T21:16:59.617489+0000","last_fullsized":"2024-04-20T21:16:59.617489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:55:33.964948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605750+0000","last_change":"2024-04-20T21:16:53.570182+0000","last_active":"2024-04-20T21:16:59.605750+0000","last_peered":"2024-04-20T21:16:59.605750+0000","last_clean":"2024-04-20T21:16:59.605750+0000","last_became_active":"2024-04-20T21:16:53.569632+0000","last_became_peered":"2024-04-20T21:16:53.569632+0000","last_unstale":"2024-04-20T21:16:59.605750+0000","last_undegraded":"2024-04-20T21:16:59.605750+0000","last_fullsized":"2024-04-20T21:16:59.605750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:12.140497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917381+0000","last_change":"2024-04-20T21:16:58.625602+0000","last_active":"2024-04-20T21:16:59.917381+0000","last_peered":"2024-04-20T21:16:59.917381+0000","last_clean":"2024-04-20T21:16:59.917381+0000","last_became_active":"2024-04-20T21:16:58.625158+0000","last_became_peered":"2024-04-20T21:16:58.625158+0000","last_unstale":"2024-04-20T21:16:59.917381+0000","last_undegraded":"2024-04-20T21:16:59.917381+0000","last_fullsized":"2024-04-20T21:16:59.917381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:09.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911816+0000","last_change":"2024-04-20T21:16:54.578603+0000","last_active":"2024-04-20T21:16:59.911816+0000","last_peered":"2024-04-20T21:16:59.911816+0000","last_clean":"2024-04-20T21:16:59.911816+0000","last_became_active":"2024-04-20T21:16:54.578338+0000","last_became_peered":"2024-04-20T21:16:54.578338+0000","last_unstale":"2024-04-20T21:16:59.911816+0000","last_undegraded":"2024-04-20T21:16:59.911816+0000","last_fullsized":"2024-04-20T21:16:59.911816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:14:56.355343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911553+0000","last_change":"2024-04-20T21:16:53.551847+0000","last_active":"2024-04-20T21:16:59.911553+0000","last_peered":"2024-04-20T21:16:59.911553+0000","last_clean":"2024-04-20T21:16:59.911553+0000","last_became_active":"2024-04-20T21:16:53.551427+0000","last_became_peered":"2024-04-20T21:16:53.551427+0000","last_unstale":"2024-04-20T21:16:59.911553+0000","last_undegraded":"2024-04-20T21:16:59.911553+0000","last_fullsized":"2024-04-20T21:16:59.911553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:55:45.930305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916262+0000","last_change":"2024-04-20T21:16:58.777159+0000","last_active":"2024-04-20T21:16:59.916262+0000","last_peered":"2024-04-20T21:16:59.916262+0000","last_clean":"2024-04-20T21:16:59.916262+0000","last_became_active":"2024-04-20T21:16:58.776233+0000","last_became_peered":"2024-04-20T21:16:58.776233+0000","last_unstale":"2024-04-20T21:16:59.916262+0000","last_undegraded":"2024-04-20T21:16:59.916262+0000","last_fullsized":"2024-04-20T21:16:59.916262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:05.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617642+0000","last_change":"2024-04-20T21:16:54.570687+0000","last_active":"2024-04-20T21:16:59.617642+0000","last_peered":"2024-04-20T21:16:59.617642+0000","last_clean":"2024-04-20T21:16:59.617642+0000","last_became_active":"2024-04-20T21:16:54.570360+0000","last_became_peered":"2024-04-20T21:16:54.570360+0000","last_unstale":"2024-04-20T21:16:59.617642+0000","last_undegraded":"2024-04-20T21:16:59.617642+0000","last_fullsized":"2024-04-20T21:16:59.617642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:05.606229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912222+0000","last_change":"2024-04-20T21:16:53.567599+0000","last_active":"2024-04-20T21:16:59.912222+0000","last_peered":"2024-04-20T21:16:59.912222+0000","last_clean":"2024-04-20T21:16:59.912222+0000","last_became_active":"2024-04-20T21:16:53.567378+0000","last_became_peered":"2024-04-20T21:16:53.567378+0000","last_unstale":"2024-04-20T21:16:59.912222+0000","last_undegraded":"2024-04-20T21:16:59.912222+0000","last_fullsized":"2024-04-20T21:16:59.912222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:36.611887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914503+0000","last_change":"2024-04-20T21:16:58.774226+0000","last_active":"2024-04-20T21:16:59.914503+0000","last_peered":"2024-04-20T21:16:59.914503+0000","last_clean":"2024-04-20T21:16:59.914503+0000","last_became_active":"2024-04-20T21:16:58.771584+0000","last_became_peered":"2024-04-20T21:16:58.771584+0000","last_unstale":"2024-04-20T21:16:59.914503+0000","last_undegraded":"2024-04-20T21:16:59.914503+0000","last_fullsized":"2024-04-20T21:16:59.914503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:31.432077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915175+0000","last_change":"2024-04-20T21:16:54.560001+0000","last_active":"2024-04-20T21:16:59.915175+0000","last_peered":"2024-04-20T21:16:59.915175+0000","last_clean":"2024-04-20T21:16:59.915175+0000","last_became_active":"2024-04-20T21:16:54.559643+0000","last_became_peered":"2024-04-20T21:16:54.559643+0000","last_unstale":"2024-04-20T21:16:59.915175+0000","last_undegraded":"2024-04-20T21:16:59.915175+0000","last_fullsized":"2024-04-20T21:16:59.915175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:52:37.325946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916695+0000","last_change":"2024-04-20T21:16:53.570498+0000","last_active":"2024-04-20T21:16:59.916695+0000","last_peered":"2024-04-20T21:16:59.916695+0000","last_clean":"2024-04-20T21:16:59.916695+0000","last_became_active":"2024-04-20T21:16:53.570095+0000","last_became_peered":"2024-04-20T21:16:53.570095+0000","last_unstale":"2024-04-20T21:16:59.916695+0000","last_undegraded":"2024-04-20T21:16:59.916695+0000","last_fullsized":"2024-04-20T21:16:59.916695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:58:28.677814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912403+0000","last_change":"2024-04-20T21:16:58.773349+0000","last_active":"2024-04-20T21:16:59.912403+0000","last_peered":"2024-04-20T21:16:59.912403+0000","last_clean":"2024-04-20T21:16:59.912403+0000","last_became_active":"2024-04-20T21:16:58.772781+0000","last_became_peered":"2024-04-20T21:16:58.772781+0000","last_unstale":"2024-04-20T21:16:59.912403+0000","last_undegraded":"2024-04-20T21:16:59.912403+0000","last_fullsized":"2024-04-20T21:16:59.912403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:53:02.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617377+0000","last_change":"2024-04-20T21:16:54.572749+0000","last_active":"2024-04-20T21:16:59.617377+0000","last_peered":"2024-04-20T21:16:59.617377+0000","last_clean":"2024-04-20T21:16:59.617377+0000","last_became_active":"2024-04-20T21:16:54.572456+0000","last_became_peered":"2024-04-20T21:16:54.572456+0000","last_unstale":"2024-04-20T21:16:59.617377+0000","last_undegraded":"2024-04-20T21:16:59.617377+0000","last_fullsized":"2024-04-20T21:16:59.617377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.787612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:17:17.226094+0000","last_change":"2024-04-20T21:16:53.568688+0000","last_active":"2024-04-20T21:17:17.226094+0000","last_peered":"2024-04-20T21:17:17.226094+0000","last_clean":"2024-04-20T21:17:17.226094+0000","last_became_active":"2024-04-20T21:16:53.568495+0000","last_became_peered":"2024-04-20T21:16:53.568495+0000","last_unstale":"2024-04-20T21:17:17.226094+0000","last_undegraded":"2024-04-20T21:17:17.226094+0000","last_fullsized":"2024-04-20T21:17:17.226094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:00.086004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608039+0000","last_change":"2024-04-20T21:16:58.609575+0000","last_active":"2024-04-20T21:16:59.608039+0000","last_peered":"2024-04-20T21:16:59.608039+0000","last_clean":"2024-04-20T21:16:59.608039+0000","last_became_active":"2024-04-20T21:16:58.609127+0000","last_became_peered":"2024-04-20T21:16:58.609127+0000","last_unstale":"2024-04-20T21:16:59.608039+0000","last_undegraded":"2024-04-20T21:16:59.608039+0000","last_fullsized":"2024-04-20T21:16:59.608039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:02.645274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605015+0000","last_change":"2024-04-20T21:16:54.563213+0000","last_active":"2024-04-20T21:16:59.605015+0000","last_peered":"2024-04-20T21:16:59.605015+0000","last_clean":"2024-04-20T21:16:59.605015+0000","last_became_active":"2024-04-20T21:16:54.562917+0000","last_became_peered":"2024-04-20T21:16:54.562917+0000","last_unstale":"2024-04-20T21:16:59.605015+0000","last_undegraded":"2024-04-20T21:16:59.605015+0000","last_fullsized":"2024-04-20T21:16:59.605015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.155771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609082+0000","last_change":"2024-04-20T21:16:53.556988+0000","last_active":"2024-04-20T21:16:59.609082+0000","last_peered":"2024-04-20T21:16:59.609082+0000","last_clean":"2024-04-20T21:16:59.609082+0000","last_became_active":"2024-04-20T21:16:53.556672+0000","last_became_peered":"2024-04-20T21:16:53.556672+0000","last_unstale":"2024-04-20T21:16:59.609082+0000","last_undegraded":"2024-04-20T21:16:59.609082+0000","last_fullsized":"2024-04-20T21:16:59.609082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:25.219611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604713+0000","last_change":"2024-04-20T21:16:58.595827+0000","last_active":"2024-04-20T21:16:59.604713+0000","last_peered":"2024-04-20T21:16:59.604713+0000","last_clean":"2024-04-20T21:16:59.604713+0000","last_became_active":"2024-04-20T21:16:58.595403+0000","last_became_peered":"2024-04-20T21:16:58.595403+0000","last_unstale":"2024-04-20T21:16:59.604713+0000","last_undegraded":"2024-04-20T21:16:59.604713+0000","last_fullsized":"2024-04-20T21:16:59.604713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:53.227573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918376+0000","last_change":"2024-04-20T21:16:54.562128+0000","last_active":"2024-04-20T21:16:59.918376+0000","last_peered":"2024-04-20T21:16:59.918376+0000","last_clean":"2024-04-20T21:16:59.918376+0000","last_became_active":"2024-04-20T21:16:54.561495+0000","last_became_peered":"2024-04-20T21:16:54.561495+0000","last_unstale":"2024-04-20T21:16:59.918376+0000","last_undegraded":"2024-04-20T21:16:59.918376+0000","last_fullsized":"2024-04-20T21:16:59.918376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:19.582218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911795+0000","last_change":"2024-04-20T21:16:53.554881+0000","last_active":"2024-04-20T21:16:59.911795+0000","last_peered":"2024-04-20T21:16:59.911795+0000","last_clean":"2024-04-20T21:16:59.911795+0000","last_became_active":"2024-04-20T21:16:53.554590+0000","last_became_peered":"2024-04-20T21:16:53.554590+0000","last_unstale":"2024-04-20T21:16:59.911795+0000","last_undegraded":"2024-04-20T21:16:59.911795+0000","last_fullsized":"2024-04-20T21:16:59.911795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:36.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913934+0000","last_change":"2024-04-20T21:16:58.610704+0000","last_active":"2024-04-20T21:16:59.913934+0000","last_peered":"2024-04-20T21:16:59.913934+0000","last_clean":"2024-04-20T21:16:59.913934+0000","last_became_active":"2024-04-20T21:16:58.610309+0000","last_became_peered":"2024-04-20T21:16:58.610309+0000","last_unstale":"2024-04-20T21:16:59.913934+0000","last_undegraded":"2024-04-20T21:16:59.913934+0000","last_fullsized":"2024-04-20T21:16:59.913934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:04.215050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911685+0000","last_change":"2024-04-20T21:16:54.562351+0000","last_active":"2024-04-20T21:16:59.911685+0000","last_peered":"2024-04-20T21:16:59.911685+0000","last_clean":"2024-04-20T21:16:59.911685+0000","last_became_active":"2024-04-20T21:16:54.561947+0000","last_became_peered":"2024-04-20T21:16:54.561947+0000","last_unstale":"2024-04-20T21:16:59.911685+0000","last_undegraded":"2024-04-20T21:16:59.911685+0000","last_fullsized":"2024-04-20T21:16:59.911685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:52.802328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912701+0000","last_change":"2024-04-20T21:16:53.554081+0000","last_active":"2024-04-20T21:16:59.912701+0000","last_peered":"2024-04-20T21:16:59.912701+0000","last_clean":"2024-04-20T21:16:59.912701+0000","last_became_active":"2024-04-20T21:16:53.553800+0000","last_became_peered":"2024-04-20T21:16:53.553800+0000","last_unstale":"2024-04-20T21:16:59.912701+0000","last_undegraded":"2024-04-20T21:16:59.912701+0000","last_fullsized":"2024-04-20T21:16:59.912701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:52.865485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914034+0000","last_change":"2024-04-20T21:16:58.779184+0000","last_active":"2024-04-20T21:16:59.914034+0000","last_peered":"2024-04-20T21:16:59.914034+0000","last_clean":"2024-04-20T21:16:59.914034+0000","last_became_active":"2024-04-20T21:16:58.778627+0000","last_became_peered":"2024-04-20T21:16:58.778627+0000","last_unstale":"2024-04-20T21:16:59.914034+0000","last_undegraded":"2024-04-20T21:16:59.914034+0000","last_fullsized":"2024-04-20T21:16:59.914034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:50:07.247736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917097+0000","last_change":"2024-04-20T21:16:58.774438+0000","last_active":"2024-04-20T21:16:59.917097+0000","last_peered":"2024-04-20T21:16:59.917097+0000","last_clean":"2024-04-20T21:16:59.917097+0000","last_became_active":"2024-04-20T21:16:58.774030+0000","last_became_peered":"2024-04-20T21:16:58.774030+0000","last_unstale":"2024-04-20T21:16:59.917097+0000","last_undegraded":"2024-04-20T21:16:59.917097+0000","last_fullsized":"2024-04-20T21:16:59.917097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:09:22.583239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606553+0000","last_change":"2024-04-20T21:16:54.573779+0000","last_active":"2024-04-20T21:16:59.606553+0000","last_peered":"2024-04-20T21:16:59.606553+0000","last_clean":"2024-04-20T21:16:59.606553+0000","last_became_active":"2024-04-20T21:16:54.573444+0000","last_became_peered":"2024-04-20T21:16:54.573444+0000","last_unstale":"2024-04-20T21:16:59.606553+0000","last_undegraded":"2024-04-20T21:16:59.606553+0000","last_fullsized":"2024-04-20T21:16:59.606553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:27.950525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911680+0000","last_change":"2024-04-20T21:16:53.543678+0000","last_active":"2024-04-20T21:16:59.911680+0000","last_peered":"2024-04-20T21:16:59.911680+0000","last_clean":"2024-04-20T21:16:59.911680+0000","last_became_active":"2024-04-20T21:16:53.543381+0000","last_became_peered":"2024-04-20T21:16:53.543381+0000","last_unstale":"2024-04-20T21:16:59.911680+0000","last_undegraded":"2024-04-20T21:16:59.911680+0000","last_fullsized":"2024-04-20T21:16:59.911680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:02.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912679+0000","last_change":"2024-04-20T21:16:58.626882+0000","last_active":"2024-04-20T21:16:59.912679+0000","last_peered":"2024-04-20T21:16:59.912679+0000","last_clean":"2024-04-20T21:16:59.912679+0000","last_became_active":"2024-04-20T21:16:58.626489+0000","last_became_peered":"2024-04-20T21:16:58.626489+0000","last_unstale":"2024-04-20T21:16:59.912679+0000","last_undegraded":"2024-04-20T21:16:59.912679+0000","last_fullsized":"2024-04-20T21:16:59.912679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:33:22.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913977+0000","last_change":"2024-04-20T21:16:54.580893+0000","last_active":"2024-04-20T21:16:59.913977+0000","last_peered":"2024-04-20T21:16:59.913977+0000","last_clean":"2024-04-20T21:16:59.913977+0000","last_became_active":"2024-04-20T21:16:54.580528+0000","last_became_peered":"2024-04-20T21:16:54.580528+0000","last_unstale":"2024-04-20T21:16:59.913977+0000","last_undegraded":"2024-04-20T21:16:59.913977+0000","last_fullsized":"2024-04-20T21:16:59.913977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:14.699123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914639+0000","last_change":"2024-04-20T21:16:53.563930+0000","last_active":"2024-04-20T21:16:59.914639+0000","last_peered":"2024-04-20T21:16:59.914639+0000","last_clean":"2024-04-20T21:16:59.914639+0000","last_became_active":"2024-04-20T21:16:53.559991+0000","last_became_peered":"2024-04-20T21:16:53.559991+0000","last_unstale":"2024-04-20T21:16:59.914639+0000","last_undegraded":"2024-04-20T21:16:59.914639+0000","last_fullsized":"2024-04-20T21:16:59.914639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:22.529712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912173+0000","last_change":"2024-04-20T21:16:53.552509+0000","last_active":"2024-04-20T21:16:59.912173+0000","last_peered":"2024-04-20T21:16:59.912173+0000","last_clean":"2024-04-20T21:16:59.912173+0000","last_became_active":"2024-04-20T21:16:53.552173+0000","last_became_peered":"2024-04-20T21:16:53.552173+0000","last_unstale":"2024-04-20T21:16:59.912173+0000","last_undegraded":"2024-04-20T21:16:59.912173+0000","last_fullsized":"2024-04-20T21:16:59.912173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:14.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916470+0000","last_change":"2024-04-20T21:16:54.562849+0000","last_active":"2024-04-20T21:16:59.916470+0000","last_peered":"2024-04-20T21:16:59.916470+0000","last_clean":"2024-04-20T21:16:59.916470+0000","last_became_active":"2024-04-20T21:16:54.561283+0000","last_became_peered":"2024-04-20T21:16:54.561283+0000","last_unstale":"2024-04-20T21:16:59.916470+0000","last_undegraded":"2024-04-20T21:16:59.916470+0000","last_fullsized":"2024-04-20T21:16:59.916470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:52.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605952+0000","last_change":"2024-04-20T21:16:58.614752+0000","last_active":"2024-04-20T21:16:59.605952+0000","last_peered":"2024-04-20T21:16:59.605952+0000","last_clean":"2024-04-20T21:16:59.605952+0000","last_became_active":"2024-04-20T21:16:58.614201+0000","last_became_peered":"2024-04-20T21:16:58.614201+0000","last_unstale":"2024-04-20T21:16:59.605952+0000","last_undegraded":"2024-04-20T21:16:59.605952+0000","last_fullsized":"2024-04-20T21:16:59.605952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:46.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913587+0000","last_change":"2024-04-20T21:16:53.564303+0000","last_active":"2024-04-20T21:16:59.913587+0000","last_peered":"2024-04-20T21:16:59.913587+0000","last_clean":"2024-04-20T21:16:59.913587+0000","last_became_active":"2024-04-20T21:16:53.560203+0000","last_became_peered":"2024-04-20T21:16:53.560203+0000","last_unstale":"2024-04-20T21:16:59.913587+0000","last_undegraded":"2024-04-20T21:16:59.913587+0000","last_fullsized":"2024-04-20T21:16:59.913587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:42.691233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604601+0000","last_change":"2024-04-20T21:16:54.554499+0000","last_active":"2024-04-20T21:16:59.604601+0000","last_peered":"2024-04-20T21:16:59.604601+0000","last_clean":"2024-04-20T21:16:59.604601+0000","last_became_active":"2024-04-20T21:16:54.554230+0000","last_became_peered":"2024-04-20T21:16:54.554230+0000","last_unstale":"2024-04-20T21:16:59.604601+0000","last_undegraded":"2024-04-20T21:16:59.604601+0000","last_fullsized":"2024-04-20T21:16:59.604601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:24:31.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915386+0000","last_change":"2024-04-20T21:16:58.610424+0000","last_active":"2024-04-20T21:16:59.915386+0000","last_peered":"2024-04-20T21:16:59.915386+0000","last_clean":"2024-04-20T21:16:59.915386+0000","last_became_active":"2024-04-20T21:16:58.610025+0000","last_became_peered":"2024-04-20T21:16:58.610025+0000","last_unstale":"2024-04-20T21:16:59.915386+0000","last_undegraded":"2024-04-20T21:16:59.915386+0000","last_fullsized":"2024-04-20T21:16:59.915386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:19.290865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911710+0000","last_change":"2024-04-20T21:16:54.578801+0000","last_active":"2024-04-20T21:16:59.911710+0000","last_peered":"2024-04-20T21:16:59.911710+0000","last_clean":"2024-04-20T21:16:59.911710+0000","last_became_active":"2024-04-20T21:16:54.578543+0000","last_became_peered":"2024-04-20T21:16:54.578543+0000","last_unstale":"2024-04-20T21:16:59.911710+0000","last_undegraded":"2024-04-20T21:16:59.911710+0000","last_fullsized":"2024-04-20T21:16:59.911710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:15.042318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605448+0000","last_change":"2024-04-20T21:16:54.565658+0000","last_active":"2024-04-20T21:16:59.605448+0000","last_peered":"2024-04-20T21:16:59.605448+0000","last_clean":"2024-04-20T21:16:59.605448+0000","last_became_active":"2024-04-20T21:16:54.565371+0000","last_became_peered":"2024-04-20T21:16:54.565371+0000","last_unstale":"2024-04-20T21:16:59.605448+0000","last_undegraded":"2024-04-20T21:16:59.605448+0000","last_fullsized":"2024-04-20T21:16:59.605448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:21.781013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912115+0000","last_change":"2024-04-20T21:16:54.568472+0000","last_active":"2024-04-20T21:16:59.912115+0000","last_peered":"2024-04-20T21:16:59.912115+0000","last_clean":"2024-04-20T21:16:59.912115+0000","last_became_active":"2024-04-20T21:16:54.568223+0000","last_became_peered":"2024-04-20T21:16:54.568223+0000","last_unstale":"2024-04-20T21:16:59.912115+0000","last_undegraded":"2024-04-20T21:16:59.912115+0000","last_fullsized":"2024-04-20T21:16:59.912115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:42:53.921935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916243+0000","last_change":"2024-04-20T21:16:54.573747+0000","last_active":"2024-04-20T21:16:59.916243+0000","last_peered":"2024-04-20T21:16:59.916243+0000","last_clean":"2024-04-20T21:16:59.916243+0000","last_became_active":"2024-04-20T21:16:54.573476+0000","last_became_peered":"2024-04-20T21:16:54.573476+0000","last_unstale":"2024-04-20T21:16:59.916243+0000","last_undegraded":"2024-04-20T21:16:59.916243+0000","last_fullsized":"2024-04-20T21:16:59.916243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:02.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913473+0000","last_change":"2024-04-20T21:16:54.577630+0000","last_active":"2024-04-20T21:16:59.913473+0000","last_peered":"2024-04-20T21:16:59.913473+0000","last_clean":"2024-04-20T21:16:59.913473+0000","last_became_active":"2024-04-20T21:16:54.577318+0000","last_became_peered":"2024-04-20T21:16:54.577318+0000","last_unstale":"2024-04-20T21:16:59.913473+0000","last_undegraded":"2024-04-20T21:16:59.913473+0000","last_fullsized":"2024-04-20T21:16:59.913473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:45.584050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608267+0000","last_change":"2024-04-20T21:16:54.579850+0000","last_active":"2024-04-20T21:16:59.608267+0000","last_peered":"2024-04-20T21:16:59.608267+0000","last_clean":"2024-04-20T21:16:59.608267+0000","last_became_active":"2024-04-20T21:16:54.579546+0000","last_became_peered":"2024-04-20T21:16:54.579546+0000","last_unstale":"2024-04-20T21:16:59.608267+0000","last_undegraded":"2024-04-20T21:16:59.608267+0000","last_fullsized":"2024-04-20T21:16:59.608267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:49:53.577077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912501+0000","last_change":"2024-04-20T21:16:54.559174+0000","last_active":"2024-04-20T21:16:59.912501+0000","last_peered":"2024-04-20T21:16:59.912501+0000","last_clean":"2024-04-20T21:16:59.912501+0000","last_became_active":"2024-04-20T21:16:54.558904+0000","last_became_peered":"2024-04-20T21:16:54.558904+0000","last_unstale":"2024-04-20T21:16:59.912501+0000","last_undegraded":"2024-04-20T21:16:59.912501+0000","last_fullsized":"2024-04-20T21:16:59.912501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:09.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916492+0000","last_change":"2024-04-20T21:16:54.574943+0000","last_active":"2024-04-20T21:16:59.916492+0000","last_peered":"2024-04-20T21:16:59.916492+0000","last_clean":"2024-04-20T21:16:59.916492+0000","last_became_active":"2024-04-20T21:16:54.574471+0000","last_became_peered":"2024-04-20T21:16:54.574471+0000","last_unstale":"2024-04-20T21:16:59.916492+0000","last_undegraded":"2024-04-20T21:16:59.916492+0000","last_fullsized":"2024-04-20T21:16:59.916492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:51:44.741339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911545+0000","last_change":"2024-04-20T21:16:54.566222+0000","last_active":"2024-04-20T21:16:59.911545+0000","last_peered":"2024-04-20T21:16:59.911545+0000","last_clean":"2024-04-20T21:16:59.911545+0000","last_became_active":"2024-04-20T21:16:54.566011+0000","last_became_peered":"2024-04-20T21:16:54.566011+0000","last_unstale":"2024-04-20T21:16:59.911545+0000","last_undegraded":"2024-04-20T21:16:59.911545+0000","last_fullsized":"2024-04-20T21:16:59.911545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:53.369950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914398+0000","last_change":"2024-04-20T21:16:54.571179+0000","last_active":"2024-04-20T21:16:59.914398+0000","last_peered":"2024-04-20T21:16:59.914398+0000","last_clean":"2024-04-20T21:16:59.914398+0000","last_became_active":"2024-04-20T21:16:54.570855+0000","last_became_peered":"2024-04-20T21:16:54.570855+0000","last_unstale":"2024-04-20T21:16:59.914398+0000","last_undegraded":"2024-04-20T21:16:59.914398+0000","last_fullsized":"2024-04-20T21:16:59.914398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:59.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911999+0000","last_change":"2024-04-20T21:16:54.559092+0000","last_active":"2024-04-20T21:16:59.911999+0000","last_peered":"2024-04-20T21:16:59.911999+0000","last_clean":"2024-04-20T21:16:59.911999+0000","last_became_active":"2024-04-20T21:16:54.558847+0000","last_became_peered":"2024-04-20T21:16:54.558847+0000","last_unstale":"2024-04-20T21:16:59.911999+0000","last_undegraded":"2024-04-20T21:16:59.911999+0000","last_fullsized":"2024-04-20T21:16:59.911999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:02.752070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915426+0000","last_change":"2024-04-20T21:16:54.571815+0000","last_active":"2024-04-20T21:16:59.915426+0000","last_peered":"2024-04-20T21:16:59.915426+0000","last_clean":"2024-04-20T21:16:59.915426+0000","last_became_active":"2024-04-20T21:16:54.570704+0000","last_became_peered":"2024-04-20T21:16:54.570704+0000","last_unstale":"2024-04-20T21:16:59.915426+0000","last_undegraded":"2024-04-20T21:16:59.915426+0000","last_fullsized":"2024-04-20T21:16:59.915426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:56.511539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912064+0000","last_change":"2024-04-20T21:16:54.555591+0000","last_active":"2024-04-20T21:16:59.912064+0000","last_peered":"2024-04-20T21:16:59.912064+0000","last_clean":"2024-04-20T21:16:59.912064+0000","last_became_active":"2024-04-20T21:16:54.555389+0000","last_became_peered":"2024-04-20T21:16:54.555389+0000","last_unstale":"2024-04-20T21:16:59.912064+0000","last_undegraded":"2024-04-20T21:16:59.912064+0000","last_fullsized":"2024-04-20T21:16:59.912064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:16:25.976707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617803+0000","last_change":"2024-04-20T21:16:54.564088+0000","last_active":"2024-04-20T21:16:59.617803+0000","last_peered":"2024-04-20T21:16:59.617803+0000","last_clean":"2024-04-20T21:16:59.617803+0000","last_became_active":"2024-04-20T21:16:54.563796+0000","last_became_peered":"2024-04-20T21:16:54.563796+0000","last_unstale":"2024-04-20T21:16:59.617803+0000","last_undegraded":"2024-04-20T21:16:59.617803+0000","last_fullsized":"2024-04-20T21:16:59.617803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:07.986736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606878+0000","last_change":"2024-04-20T21:16:54.563578+0000","last_active":"2024-04-20T21:16:59.606878+0000","last_peered":"2024-04-20T21:16:59.606878+0000","last_clean":"2024-04-20T21:16:59.606878+0000","last_became_active":"2024-04-20T21:16:54.563317+0000","last_became_peered":"2024-04-20T21:16:54.563317+0000","last_unstale":"2024-04-20T21:16:59.606878+0000","last_undegraded":"2024-04-20T21:16:59.606878+0000","last_fullsized":"2024-04-20T21:16:59.606878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:06.093675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912217+0000","last_change":"2024-04-20T21:16:54.565168+0000","last_active":"2024-04-20T21:16:59.912217+0000","last_peered":"2024-04-20T21:16:59.912217+0000","last_clean":"2024-04-20T21:16:59.912217+0000","last_became_active":"2024-04-20T21:16:54.564712+0000","last_became_peered":"2024-04-20T21:16:54.564712+0000","last_unstale":"2024-04-20T21:16:59.912217+0000","last_undegraded":"2024-04-20T21:16:59.912217+0000","last_fullsized":"2024-04-20T21:16:59.912217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:07:30.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911433+0000","last_change":"2024-04-20T21:16:54.572211+0000","last_active":"2024-04-20T21:16:59.911433+0000","last_peered":"2024-04-20T21:16:59.911433+0000","last_clean":"2024-04-20T21:16:59.911433+0000","last_became_active":"2024-04-20T21:16:54.571894+0000","last_became_peered":"2024-04-20T21:16:54.571894+0000","last_unstale":"2024-04-20T21:16:59.911433+0000","last_undegraded":"2024-04-20T21:16:59.911433+0000","last_fullsized":"2024-04-20T21:16:59.911433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:59.376514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606682+0000","last_change":"2024-04-20T21:16:54.555195+0000","last_active":"2024-04-20T21:16:59.606682+0000","last_peered":"2024-04-20T21:16:59.606682+0000","last_clean":"2024-04-20T21:16:59.606682+0000","last_became_active":"2024-04-20T21:16:54.554284+0000","last_became_peered":"2024-04-20T21:16:54.554284+0000","last_unstale":"2024-04-20T21:16:59.606682+0000","last_undegraded":"2024-04-20T21:16:59.606682+0000","last_fullsized":"2024-04-20T21:16:59.606682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:42:21.500952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915085+0000","last_change":"2024-04-20T21:16:54.565527+0000","last_active":"2024-04-20T21:16:59.915085+0000","last_peered":"2024-04-20T21:16:59.915085+0000","last_clean":"2024-04-20T21:16:59.915085+0000","last_became_active":"2024-04-20T21:16:54.565211+0000","last_became_peered":"2024-04-20T21:16:54.565211+0000","last_unstale":"2024-04-20T21:16:59.915085+0000","last_undegraded":"2024-04-20T21:16:59.915085+0000","last_fullsized":"2024-04-20T21:16:59.915085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:17:40.462670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913206+0000","last_change":"2024-04-20T21:16:54.576464+0000","last_active":"2024-04-20T21:16:59.913206+0000","last_peered":"2024-04-20T21:16:59.913206+0000","last_clean":"2024-04-20T21:16:59.913206+0000","last_became_active":"2024-04-20T21:16:54.575947+0000","last_became_peered":"2024-04-20T21:16:54.575947+0000","last_unstale":"2024-04-20T21:16:59.913206+0000","last_undegraded":"2024-04-20T21:16:59.913206+0000","last_fullsized":"2024-04-20T21:16:59.913206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:50:29.581938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604732+0000","last_change":"2024-04-20T21:16:54.558162+0000","last_active":"2024-04-20T21:16:59.604732+0000","last_peered":"2024-04-20T21:16:59.604732+0000","last_clean":"2024-04-20T21:16:59.604732+0000","last_became_active":"2024-04-20T21:16:54.557875+0000","last_became_peered":"2024-04-20T21:16:54.557875+0000","last_unstale":"2024-04-20T21:16:59.604732+0000","last_undegraded":"2024-04-20T21:16:59.604732+0000","last_fullsized":"2024-04-20T21:16:59.604732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:42:46.886405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911994+0000","last_change":"2024-04-20T21:16:54.560566+0000","last_active":"2024-04-20T21:16:59.911994+0000","last_peered":"2024-04-20T21:16:59.911994+0000","last_clean":"2024-04-20T21:16:59.911994+0000","last_became_active":"2024-04-20T21:16:54.559409+0000","last_became_peered":"2024-04-20T21:16:54.559409+0000","last_unstale":"2024-04-20T21:16:59.911994+0000","last_undegraded":"2024-04-20T21:16:59.911994+0000","last_fullsized":"2024-04-20T21:16:59.911994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:00.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912405+0000","last_change":"2024-04-20T21:16:54.559491+0000","last_active":"2024-04-20T21:16:59.912405+0000","last_peered":"2024-04-20T21:16:59.912405+0000","last_clean":"2024-04-20T21:16:59.912405+0000","last_became_active":"2024-04-20T21:16:54.558187+0000","last_became_peered":"2024-04-20T21:16:54.558187+0000","last_unstale":"2024-04-20T21:16:59.912405+0000","last_undegraded":"2024-04-20T21:16:59.912405+0000","last_fullsized":"2024-04-20T21:16:59.912405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:33.686248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617814+0000","last_change":"2024-04-20T21:16:54.564258+0000","last_active":"2024-04-20T21:16:59.617814+0000","last_peered":"2024-04-20T21:16:59.617814+0000","last_clean":"2024-04-20T21:16:59.617814+0000","last_became_active":"2024-04-20T21:16:54.563966+0000","last_became_peered":"2024-04-20T21:16:54.563966+0000","last_unstale":"2024-04-20T21:16:59.617814+0000","last_undegraded":"2024-04-20T21:16:59.617814+0000","last_fullsized":"2024-04-20T21:16:59.617814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:13:34.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912591+0000","last_change":"2024-04-20T21:16:54.573554+0000","last_active":"2024-04-20T21:16:59.912591+0000","last_peered":"2024-04-20T21:16:59.912591+0000","last_clean":"2024-04-20T21:16:59.912591+0000","last_became_active":"2024-04-20T21:16:54.571849+0000","last_became_peered":"2024-04-20T21:16:54.571849+0000","last_unstale":"2024-04-20T21:16:59.912591+0000","last_undegraded":"2024-04-20T21:16:59.912591+0000","last_fullsized":"2024-04-20T21:16:59.912591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:09.481779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617599+0000","last_change":"2024-04-20T21:16:54.571471+0000","last_active":"2024-04-20T21:16:59.617599+0000","last_peered":"2024-04-20T21:16:59.617599+0000","last_clean":"2024-04-20T21:16:59.617599+0000","last_became_active":"2024-04-20T21:16:54.571209+0000","last_became_peered":"2024-04-20T21:16:54.571209+0000","last_unstale":"2024-04-20T21:16:59.617599+0000","last_undegraded":"2024-04-20T21:16:59.617599+0000","last_fullsized":"2024-04-20T21:16:59.617599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:35.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911811+0000","last_change":"2024-04-20T21:16:54.572168+0000","last_active":"2024-04-20T21:16:59.911811+0000","last_peered":"2024-04-20T21:16:59.911811+0000","last_clean":"2024-04-20T21:16:59.911811+0000","last_became_active":"2024-04-20T21:16:54.571740+0000","last_became_peered":"2024-04-20T21:16:54.571740+0000","last_unstale":"2024-04-20T21:16:59.911811+0000","last_undegraded":"2024-04-20T21:16:59.911811+0000","last_fullsized":"2024-04-20T21:16:59.911811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:11:35.728824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617393+0000","last_change":"2024-04-20T21:16:54.569081+0000","last_active":"2024-04-20T21:16:59.617393+0000","last_peered":"2024-04-20T21:16:59.617393+0000","last_clean":"2024-04-20T21:16:59.617393+0000","last_became_active":"2024-04-20T21:16:54.568847+0000","last_became_peered":"2024-04-20T21:16:54.568847+0000","last_unstale":"2024-04-20T21:16:59.617393+0000","last_undegraded":"2024-04-20T21:16:59.617393+0000","last_fullsized":"2024-04-20T21:16:59.617393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:35.737237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912610+0000","last_change":"2024-04-20T21:16:54.571525+0000","last_active":"2024-04-20T21:16:59.912610+0000","last_peered":"2024-04-20T21:16:59.912610+0000","last_clean":"2024-04-20T21:16:59.912610+0000","last_became_active":"2024-04-20T21:16:54.570776+0000","last_became_peered":"2024-04-20T21:16:54.570776+0000","last_unstale":"2024-04-20T21:16:59.912610+0000","last_undegraded":"2024-04-20T21:16:59.912610+0000","last_fullsized":"2024-04-20T21:16:59.912610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:03.693476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916192+0000","last_change":"2024-04-20T21:16:54.570895+0000","last_active":"2024-04-20T21:16:59.916192+0000","last_peered":"2024-04-20T21:16:59.916192+0000","last_clean":"2024-04-20T21:16:59.916192+0000","last_became_active":"2024-04-20T21:16:54.570577+0000","last_became_peered":"2024-04-20T21:16:54.570577+0000","last_unstale":"2024-04-20T21:16:59.916192+0000","last_undegraded":"2024-04-20T21:16:59.916192+0000","last_fullsized":"2024-04-20T21:16:59.916192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:26.787487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608644+0000","last_change":"2024-04-20T21:16:54.581648+0000","last_active":"2024-04-20T21:16:59.608644+0000","last_peered":"2024-04-20T21:16:59.608644+0000","last_clean":"2024-04-20T21:16:59.608644+0000","last_became_active":"2024-04-20T21:16:54.581382+0000","last_became_peered":"2024-04-20T21:16:54.581382+0000","last_unstale":"2024-04-20T21:16:59.608644+0000","last_undegraded":"2024-04-20T21:16:59.608644+0000","last_fullsized":"2024-04-20T21:16:59.608644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:21.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617750+0000","last_change":"2024-04-20T21:16:54.563949+0000","last_active":"2024-04-20T21:16:59.617750+0000","last_peered":"2024-04-20T21:16:59.617750+0000","last_clean":"2024-04-20T21:16:59.617750+0000","last_became_active":"2024-04-20T21:16:54.563606+0000","last_became_peered":"2024-04-20T21:16:54.563606+0000","last_unstale":"2024-04-20T21:16:59.617750+0000","last_undegraded":"2024-04-20T21:16:59.617750+0000","last_fullsized":"2024-04-20T21:16:59.617750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:26:05.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:22.696 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:17:22.696 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T21:17:22.697 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-20T21:17:22.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-20T21:17:22.697 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-20T21:17:22.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.939+0000 7f6065c00640 1 Processor -- start 2024-04-20T21:17:22.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.939+0000 7f6065c00640 1 -- start start 2024-04-20T21:17:22.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f6065c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f60600cc470 0x7f60600cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f6065c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60600cdfc0 0x7f60600cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f6065c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606015ef60 con 0x7f6060155530 2024-04-20T21:17:22.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f6065c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f60600c5ca0 con 0x7f60600cc470 2024-04-20T21:17:22.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f6065c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f60600cd5f0 con 0x7f60600cdfc0 2024-04-20T21:17:22.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f605f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f60600cc470 0x7f60600cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f605f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f60600cc470 0x7f60600cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51328/0 (socket says 172.21.15.102:51328) 2024-04-20T21:17:22.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f605f400640 1 -- 172.21.15.102:0/339750435 learned_addr learned my addr 172.21.15.102:0/339750435 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:22.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f6065200640 1 --2- 172.21.15.102:0/339750435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60600cdfc0 0x7f60600cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f605f400640 1 -- 172.21.15.102:0/339750435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60600cdfc0 msgr2=0x7f60600cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:22.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f605f400640 1 --2- 172.21.15.102:0/339750435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60600cdfc0 0x7f60600cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f605f400640 1 -- 172.21.15.102:0/339750435 >> v1:172.21.15.193:6789/0 conn(0x7f6060155530 legacy=0x7f606015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f605f400640 1 -- 172.21.15.102:0/339750435 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606015f240 con 0x7f60600cc470 2024-04-20T21:17:22.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.941+0000 7f6065200640 1 --2- 172.21.15.102:0/339750435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60600cdfc0 0x7f60600cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:22.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.943+0000 7f605f400640 1 --2- 172.21.15.102:0/339750435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 0x7f60600cda80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f604c009870 tx=0x7f604c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba1ba045ef2d00b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:22.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.943+0000 7f605ea00640 1 -- 172.21.15.102:0/339750435 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c034070 con 0x7f60600cc470 2024-04-20T21:17:22.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.943+0000 7f605ea00640 1 -- 172.21.15.102:0/339750435 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604c030840 con 0x7f60600cc470 2024-04-20T21:17:22.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.943+0000 7f605ea00640 1 -- 172.21.15.102:0/339750435 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c030c10 con 0x7f60600cc470 2024-04-20T21:17:22.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.943+0000 7f6065c00640 1 -- 172.21.15.102:0/339750435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 msgr2=0x7f60600cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.943+0000 7f6065c00640 1 --2- 172.21.15.102:0/339750435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 0x7f60600cda80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f604c009870 tx=0x7f604c02b980 comp rx=0 tx=0).stop 2024-04-20T21:17:22.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.944+0000 7f6065c00640 1 -- 172.21.15.102:0/339750435 shutdown_connections 2024-04-20T21:17:22.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.944+0000 7f6065c00640 1 --2- 172.21.15.102:0/339750435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60600cdfc0 0x7f60600cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.944+0000 7f6065c00640 1 --2- 172.21.15.102:0/339750435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 0x7f60600cda80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.944+0000 7f6065c00640 1 -- 172.21.15.102:0/339750435 >> 172.21.15.102:0/339750435 conn(0x7f6060075080 msgr2=0x7f6060075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:22.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.944+0000 7f6065c00640 1 -- 172.21.15.102:0/339750435 shutdown_connections 2024-04-20T21:17:22.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.944+0000 7f6065c00640 1 -- 172.21.15.102:0/339750435 wait complete. 2024-04-20T21:17:22.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.945+0000 7f6065c00640 1 Processor -- start 2024-04-20T21:17:22.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.945+0000 7f6065c00640 1 -- start start 2024-04-20T21:17:22.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.946+0000 7f6065c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 0x7f60600ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.946+0000 7f6065c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60600cdfc0 0x7f606015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.946+0000 7f6065c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6060155530 0x7f606015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.946+0000 7f6065c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60600cd280 con 0x7f60600cc470 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.946+0000 7f6065c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60600ccfa0 con 0x7f60600cdfc0 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.946+0000 7f6065c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f60600ccbe0 con 0x7f6060155530 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f605fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 0x7f60600ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f605f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60600cdfc0 0x7f606015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6060155530 0x7f606015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6060155530 0x7f606015c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55162/0 (socket says 172.21.15.102:55162) 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f605f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60600cdfc0 0x7f606015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56668/0 (socket says 172.21.15.102:56668) 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 -- 172.21.15.102:0/699227517 learned_addr learned my addr 172.21.15.102:0/699227517 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 -- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60600cdfc0 msgr2=0x7f606015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60600cdfc0 0x7f606015c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 -- 172.21.15.102:0/699227517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 msgr2=0x7f60600ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:22.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 0x7f60600ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 -- 172.21.15.102:0/699227517 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60601f4f30 con 0x7f6060155530 2024-04-20T21:17:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f605fe00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 0x7f60600ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.947+0000 7f6065200640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6060155530 0x7f606015c9e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6058007f20 tx=0x7f605800d560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.948+0000 7f6057e00640 1 -- 172.21.15.102:0/699227517 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605801a070 con 0x7f6060155530 2024-04-20T21:17:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.948+0000 7f6057e00640 1 -- 172.21.15.102:0/699227517 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6058003b90 con 0x7f6060155530 2024-04-20T21:17:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.948+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606015d430 con 0x7f6060155530 2024-04-20T21:17:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.948+0000 7f6057e00640 1 -- 172.21.15.102:0/699227517 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058004030 con 0x7f6060155530 2024-04-20T21:17:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.948+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f606015d6d0 con 0x7f6060155530 2024-04-20T21:17:22.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.949+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606004fd60 con 0x7f6060155530 2024-04-20T21:17:22.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.950+0000 7f6057e00640 1 -- 172.21.15.102:0/699227517 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f605801d070 con 0x7f6060155530 2024-04-20T21:17:22.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.951+0000 7f6057e00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f604006c920 0x7f604006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:22.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.951+0000 7f6057e00640 1 -- 172.21.15.102:0/699227517 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f605808c180 con 0x7f6060155530 2024-04-20T21:17:22.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.951+0000 7f605fe00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f604006c920 0x7f604006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:22.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.952+0000 7f605fe00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f604006c920 0x7f604006ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6050000f60 tx=0x7f6050009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:22.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:22.955+0000 7f6057e00640 1 -- 172.21.15.102:0/699227517 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6058010070 con 0x7f6060155530 2024-04-20T21:17:23.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.208+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f60600cc9c0 con 0x7f6060155530 2024-04-20T21:17:23.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.211+0000 7f6057e00640 1 -- 172.21.15.102:0/699227517 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174142 (secure 0 0 0) 0x7f6058059b80 con 0x7f6060155530 2024-04-20T21:17:23.213 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:23.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f604006c920 msgr2=0x7f604006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f604006c920 0x7f604006ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6050000f60 tx=0x7f6050009040 comp rx=0 tx=0).stop 2024-04-20T21:17:23.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6060155530 msgr2=0x7f606015c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6060155530 0x7f606015c9e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6058007f20 tx=0x7f605800d560 comp rx=0 tx=0).stop 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 shutdown_connections 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6060155530 0x7f606015c9e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f604006c920 0x7f604006ede0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60600cdfc0 0x7f606015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 --2- 172.21.15.102:0/699227517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60600cc470 0x7f60600ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.215+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 >> 172.21.15.102:0/699227517 conn(0x7f6060075080 msgr2=0x7f60601545c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.216+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 shutdown_connections 2024-04-20T21:17:23.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.216+0000 7f6065c00640 1 -- 172.21.15.102:0/699227517 wait complete. 2024-04-20T21:17:23.233 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"flags":0,"active_gid":4112,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2803621564},{"type":"v1","addr":"172.21.15.193:6833","nonce":2803621564}]},"active_addr":"172.21.15.193:6833/2803621564","active_change":"2024-04-20T21:16:34.648042+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.193:0","nonce":2571850227}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.193:0","nonce":239601120}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.193:0","nonce":816794684}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.193:0","nonce":3908462650}]}]} 2024-04-20T21:17:23.237 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-20T21:17:23.237 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T21:17:23.237 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.433+0000 7f64ca600640 1 Processor -- start 2024-04-20T21:17:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.433+0000 7f64ca600640 1 -- start start 2024-04-20T21:17:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64ca600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f64c41557f0 0x7f64c415c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64ca600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f64c40c8040 0x7f64c406d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64ca600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c4155bd0 con 0x7f64c40c8620 2024-04-20T21:17:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64ca600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f64c40c94b0 con 0x7f64c41557f0 2024-04-20T21:17:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64ca600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f64c415c050 con 0x7f64c40c8040 2024-04-20T21:17:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64c3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f64c41557f0 0x7f64c415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64c3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f64c41557f0 0x7f64c415c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51358/0 (socket says 172.21.15.102:51358) 2024-04-20T21:17:23.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64c3400640 1 -- 172.21.15.102:0/1496669882 learned_addr learned my addr 172.21.15.102:0/1496669882 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:23.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64c3e00640 1 --2- 172.21.15.102:0/1496669882 >> v2:172.21.15.193:3300/0 conn(0x7f64c40c8040 0x7f64c406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64c3400640 1 -- 172.21.15.102:0/1496669882 >> v2:172.21.15.193:3300/0 conn(0x7f64c40c8040 msgr2=0x7f64c406d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:23.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64c3400640 1 --2- 172.21.15.102:0/1496669882 >> v2:172.21.15.193:3300/0 conn(0x7f64c40c8040 0x7f64c406d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64c3400640 1 -- 172.21.15.102:0/1496669882 >> v1:172.21.15.193:6789/0 conn(0x7f64c40c8620 legacy=0x7f64c406c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.435+0000 7f64c3400640 1 -- 172.21.15.102:0/1496669882 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c415c330 con 0x7f64c41557f0 2024-04-20T21:17:23.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.437+0000 7f64c3400640 1 --2- 172.21.15.102:0/1496669882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c41557f0 0x7f64c415c910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64c415fab0 tx=0x7f64b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703bd0650cefa790 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:23.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.437+0000 7f64c2000640 1 -- 172.21.15.102:0/1496669882 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4034070 con 0x7f64c41557f0 2024-04-20T21:17:23.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.437+0000 7f64c2000640 1 -- 172.21.15.102:0/1496669882 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4030860 con 0x7f64c41557f0 2024-04-20T21:17:23.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.437+0000 7f64c2000640 1 -- 172.21.15.102:0/1496669882 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4030c30 con 0x7f64c41557f0 2024-04-20T21:17:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.437+0000 7f64ca600640 1 -- 172.21.15.102:0/1496669882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c41557f0 msgr2=0x7f64c415c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.438+0000 7f64ca600640 1 --2- 172.21.15.102:0/1496669882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c41557f0 0x7f64c415c910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64c415fab0 tx=0x7f64b402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.438+0000 7f64ca600640 1 -- 172.21.15.102:0/1496669882 shutdown_connections 2024-04-20T21:17:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.438+0000 7f64ca600640 1 --2- 172.21.15.102:0/1496669882 >> v2:172.21.15.193:3300/0 conn(0x7f64c40c8040 0x7f64c406d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.438+0000 7f64ca600640 1 --2- 172.21.15.102:0/1496669882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c41557f0 0x7f64c415c910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.438+0000 7f64ca600640 1 -- 172.21.15.102:0/1496669882 >> 172.21.15.102:0/1496669882 conn(0x7f64c40d0a90 msgr2=0x7f64c40d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.438+0000 7f64ca600640 1 -- 172.21.15.102:0/1496669882 shutdown_connections 2024-04-20T21:17:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.438+0000 7f64ca600640 1 -- 172.21.15.102:0/1496669882 wait complete. 2024-04-20T21:17:23.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.439+0000 7f64ca600640 1 Processor -- start 2024-04-20T21:17:23.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.440+0000 7f64ca600640 1 -- start start 2024-04-20T21:17:23.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.440+0000 7f64ca600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64c40c8040 0x7f64c40c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.440+0000 7f64ca600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c40c8620 0x7f64c414cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.440+0000 7f64ca600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64c41557f0 0x7f64c414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64ca600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64c40c9140 con 0x7f64c40c8620 2024-04-20T21:17:23.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64ca600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f64c40c8e60 con 0x7f64c40c8040 2024-04-20T21:17:23.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64ca600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f64c406ca90 con 0x7f64c41557f0 2024-04-20T21:17:23.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c40c8620 0x7f64c414cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64c41557f0 0x7f64c414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64c40c8040 0x7f64c40c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c40c8620 0x7f64c414cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51362/0 (socket says 172.21.15.102:51362) 2024-04-20T21:17:23.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c2a00640 1 -- 172.21.15.102:0/3977481467 learned_addr learned my addr 172.21.15.102:0/3977481467 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:23.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c3400640 1 -- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64c41557f0 msgr2=0x7f64c414d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c3400640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64c41557f0 0x7f64c414d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c3400640 1 -- 172.21.15.102:0/3977481467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c40c8620 msgr2=0x7f64c414cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c3400640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c40c8620 0x7f64c414cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c3400640 1 -- 172.21.15.102:0/3977481467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c41e3520 con 0x7f64c40c8040 2024-04-20T21:17:23.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.441+0000 7f64c2a00640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c40c8620 0x7f64c414cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:23.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.442+0000 7f64c3400640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64c40c8040 0x7f64c40c47c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f64c40729d0 tx=0x7f64b40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:23.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.442+0000 7f64bb400640 1 -- 172.21.15.102:0/3977481467 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4034040 con 0x7f64c40c8040 2024-04-20T21:17:23.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.442+0000 7f64bb400640 1 -- 172.21.15.102:0/3977481467 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4039d90 con 0x7f64c40c8040 2024-04-20T21:17:23.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.442+0000 7f64bb400640 1 -- 172.21.15.102:0/3977481467 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4043630 con 0x7f64c40c8040 2024-04-20T21:17:23.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.442+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c41e37e0 con 0x7f64c40c8040 2024-04-20T21:17:23.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.442+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c4156b00 con 0x7f64c40c8040 2024-04-20T21:17:23.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.443+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c40b6840 con 0x7f64c40c8040 2024-04-20T21:17:23.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.444+0000 7f64bb400640 1 -- 172.21.15.102:0/3977481467 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64b4037070 con 0x7f64c40c8040 2024-04-20T21:17:23.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.445+0000 7f64bb400640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f64a006c8f0 0x7f64a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.445+0000 7f64bb400640 1 -- 172.21.15.102:0/3977481467 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f64b40adec0 con 0x7f64c40c8040 2024-04-20T21:17:23.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.445+0000 7f64c2a00640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f64a006c8f0 0x7f64a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.446+0000 7f64c2a00640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f64a006c8f0 0x7f64a006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f64c41581b0 tx=0x7f64ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:23.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.450+0000 7f64bb400640 1 -- 172.21.15.102:0/3977481467 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f64b403e070 con 0x7f64c40c8040 2024-04-20T21:17:23.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.650+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64c406d770 con 0x7f64c40c8040 2024-04-20T21:17:23.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.652+0000 7f64bb400640 1 -- 172.21.15.102:0/3977481467 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f64b407b9e0 con 0x7f64c40c8040 2024-04-20T21:17:23.652 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:23.652 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:23.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.654+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f64a006c8f0 msgr2=0x7f64a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.654+0000 7f64ca600640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f64a006c8f0 0x7f64a006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f64c41581b0 tx=0x7f64ac009040 comp rx=0 tx=0).stop 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.654+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64c40c8040 msgr2=0x7f64c40c47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.654+0000 7f64ca600640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64c40c8040 0x7f64c40c47c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f64c40729d0 tx=0x7f64b40046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.655+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 shutdown_connections 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.655+0000 7f64ca600640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64c41557f0 0x7f64c414d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.655+0000 7f64ca600640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64c40c8620 0x7f64c414cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.655+0000 7f64ca600640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f64a006c8f0 0x7f64a006edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.655+0000 7f64ca600640 1 --2- 172.21.15.102:0/3977481467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64c40c8040 0x7f64c40c47c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.655+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 >> 172.21.15.102:0/3977481467 conn(0x7f64c40d0a90 msgr2=0x7f64c41548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.655+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 shutdown_connections 2024-04-20T21:17:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.655+0000 7f64ca600640 1 -- 172.21.15.102:0/3977481467 wait complete. 2024-04-20T21:17:23.673 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T21:17:23.673 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:23.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.870+0000 7fb922800640 1 Processor -- start 2024-04-20T21:17:23.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.870+0000 7fb922800640 1 -- start start 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb922800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb91c1557f0 0x7fb91c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb922800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb91c0c8040 0x7fb91c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb922800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb91c155bd0 con 0x7fb91c0c8620 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb922800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb91c0c94b0 con 0x7fb91c0c8040 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb922800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb91c15c050 con 0x7fb91c1557f0 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb91be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb91c0c8040 0x7fb91c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb91be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb91c0c8040 0x7fb91c06d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51372/0 (socket says 172.21.15.102:51372) 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb91be00640 1 -- 172.21.15.102:0/2518949306 learned_addr learned my addr 172.21.15.102:0/2518949306 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb91b400640 1 --2- 172.21.15.102:0/2518949306 >> v2:172.21.15.193:3300/0 conn(0x7fb91c1557f0 0x7fb91c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb91be00640 1 -- 172.21.15.102:0/2518949306 >> v2:172.21.15.193:3300/0 conn(0x7fb91c1557f0 msgr2=0x7fb91c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:23.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb91be00640 1 --2- 172.21.15.102:0/2518949306 >> v2:172.21.15.193:3300/0 conn(0x7fb91c1557f0 0x7fb91c15c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb91be00640 1 -- 172.21.15.102:0/2518949306 >> v1:172.21.15.193:6789/0 conn(0x7fb91c0c8620 legacy=0x7fb91c06c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.872+0000 7fb91be00640 1 -- 172.21.15.102:0/2518949306 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c15c330 con 0x7fb91c0c8040 2024-04-20T21:17:23.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb91be00640 1 --2- 172.21.15.102:0/2518949306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c06d110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb91c08e350 tx=0x7fb91402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f51b2e73c30a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:23.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb91aa00640 1 -- 172.21.15.102:0/2518949306 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914034070 con 0x7fb91c0c8040 2024-04-20T21:17:23.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb91aa00640 1 -- 172.21.15.102:0/2518949306 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb914030860 con 0x7fb91c0c8040 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb91aa00640 1 -- 172.21.15.102:0/2518949306 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914030c30 con 0x7fb91c0c8040 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb922800640 1 -- 172.21.15.102:0/2518949306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 msgr2=0x7fb91c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb922800640 1 --2- 172.21.15.102:0/2518949306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c06d110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb91c08e350 tx=0x7fb91402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb922800640 1 -- 172.21.15.102:0/2518949306 shutdown_connections 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb922800640 1 --2- 172.21.15.102:0/2518949306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c06d110 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.874+0000 7fb922800640 1 --2- 172.21.15.102:0/2518949306 >> v2:172.21.15.193:3300/0 conn(0x7fb91c1557f0 0x7fb91c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.875+0000 7fb922800640 1 -- 172.21.15.102:0/2518949306 >> 172.21.15.102:0/2518949306 conn(0x7fb91c0d0a90 msgr2=0x7fb91c0d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.875+0000 7fb922800640 1 -- 172.21.15.102:0/2518949306 shutdown_connections 2024-04-20T21:17:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.875+0000 7fb922800640 1 -- 172.21.15.102:0/2518949306 wait complete. 2024-04-20T21:17:23.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.876+0000 7fb922800640 1 Processor -- start 2024-04-20T21:17:23.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.876+0000 7fb922800640 1 -- start start 2024-04-20T21:17:23.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.877+0000 7fb922800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c1e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.877+0000 7fb922800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb91c0c8620 0x7fb91c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.877+0000 7fb922800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c1557f0 0x7fb91c0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.877+0000 7fb922800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c0c9140 con 0x7fb91c0c8040 2024-04-20T21:17:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.877+0000 7fb922800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c0c8e60 con 0x7fb91c1557f0 2024-04-20T21:17:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.877+0000 7fb922800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c06ca90 con 0x7fb91c0c8620 2024-04-20T21:17:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c1e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c1557f0 0x7fb91c0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c1e6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51376/0 (socket says 172.21.15.102:51376) 2024-04-20T21:17:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 -- 172.21.15.102:0/450005706 learned_addr learned my addr 172.21.15.102:0/450005706 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb912a00640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb91c0c8620 0x7fb91c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 -- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb91c0c8620 msgr2=0x7fb91c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb91c0c8620 0x7fb91c0d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 -- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c1557f0 msgr2=0x7fb91c0c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c1557f0 0x7fb91c0c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 -- 172.21.15.102:0/450005706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c15b060 con 0x7fb91c0c8040 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91be00640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c1557f0 0x7fb91c0c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.878+0000 7fb91b400640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c1e6c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00efc0 tx=0x7fb90c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.879+0000 7fb913e00640 1 -- 172.21.15.102:0/450005706 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c018020 con 0x7fb91c0c8040 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.879+0000 7fb913e00640 1 -- 172.21.15.102:0/450005706 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c014480 con 0x7fb91c0c8040 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.879+0000 7fb913e00640 1 -- 172.21.15.102:0/450005706 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c014850 con 0x7fb91c0c8040 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.879+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91c15b300 con 0x7fb91c0c8040 2024-04-20T21:17:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.879+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91c0c7680 con 0x7fb91c0c8040 2024-04-20T21:17:23.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.880+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91c06cd90 con 0x7fb91c0c8040 2024-04-20T21:17:23.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.881+0000 7fb913e00640 1 -- 172.21.15.102:0/450005706 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb90c005d10 con 0x7fb91c0c8040 2024-04-20T21:17:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.882+0000 7fb913e00640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb8f806c8f0 0x7fb8f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.882+0000 7fb913e00640 1 -- 172.21.15.102:0/450005706 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb90c08cbb0 con 0x7fb91c0c8040 2024-04-20T21:17:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.882+0000 7fb912a00640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb8f806c8f0 0x7fb8f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.883+0000 7fb912a00640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb8f806c8f0 0x7fb8f806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb91c1ef120 tx=0x7fb904009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:23.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:23.886+0000 7fb913e00640 1 -- 172.21.15.102:0/450005706 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb91c06cd90 con 0x7fb91c0c8040 2024-04-20T21:17:24.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.072+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb91c0caea0 con 0x7fb91c0c8040 2024-04-20T21:17:24.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.075+0000 7fb913e00640 1 -- 172.21.15.102:0/450005706 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7fb91c0caea0 con 0x7fb91c0c8040 2024-04-20T21:17:24.075 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:24.075 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb8f806c8f0 msgr2=0x7fb8f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb8f806c8f0 0x7fb8f806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb91c1ef120 tx=0x7fb904009040 comp rx=0 tx=0).stop 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 msgr2=0x7fb91c1e6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c1e6c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00efc0 tx=0x7fb90c00c490 comp rx=0 tx=0).stop 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 shutdown_connections 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb8f806c8f0 0x7fb8f806edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c1557f0 0x7fb91c0c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb91c0c8620 0x7fb91c0d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 --2- 172.21.15.102:0/450005706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0c8040 0x7fb91c1e6c50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 >> 172.21.15.102:0/450005706 conn(0x7fb91c0d0a90 msgr2=0x7fb91c15d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 shutdown_connections 2024-04-20T21:17:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.078+0000 7fb922800640 1 -- 172.21.15.102:0/450005706 wait complete. 2024-04-20T21:17:24.097 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T21:17:24.097 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T21:17:24.097 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T21:17:24.098 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T21:17:24.098 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T21:17:24.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T21:17:24.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-20T21:17:24.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-20T21:17:24.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.377+0000 7f61cfc00640 1 Processor -- start 2024-04-20T21:17:24.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.377+0000 7f61cfc00640 1 -- start start 2024-04-20T21:17:24.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.380+0000 7f61cfc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81515d0 0x7f61c81570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.380+0000 7f61cfc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c80c7330 con 0x7f61c81511f0 2024-04-20T21:17:24.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.380+0000 7f61cfc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c806f170 con 0x7f61c80c7dd0 2024-04-20T21:17:24.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.380+0000 7f61cfc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f61c806f450 con 0x7f61c81515d0 2024-04-20T21:17:24.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.381+0000 7f1431e00640 1 Processor -- start 2024-04-20T21:17:24.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.381+0000 7f61cd200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f61c81511f0 0x7f61c806e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38650/0 (socket says 172.21.15.102:38650) 2024-04-20T21:17:24.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.381+0000 7f61cd200640 1 -- 172.21.15.102:0/3846928826 learned_addr learned my addr 172.21.15.102:0/3846928826 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:24.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.381+0000 7f61c7e00640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81515d0 0x7f61c81570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.382+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343216876 0 0) 0x7f61c80c7330 con 0x7f61c81511f0 2024-04-20T21:17:24.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.382+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61ac003520 con 0x7f61c81511f0 2024-04-20T21:17:24.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639739506 0 0) 0x7f61c806f170 con 0x7f61c80c7dd0 2024-04-20T21:17:24.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61ac003880 con 0x7f61c80c7dd0 2024-04-20T21:17:24.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3383016717 0 0) 0x7f61ac003520 con 0x7f61c81511f0 2024-04-20T21:17:24.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81515d0 msgr2=0x7f61c81570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f61c7400640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81515d0 0x7f61c81570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 >> v1:172.21.15.193:6789/0 conn(0x7f61c80c7dd0 legacy=0x7f61c80c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c80c75d0 con 0x7f61c81511f0 2024-04-20T21:17:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.381+0000 7f1431e00640 1 -- start start 2024-04-20T21:17:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f1431e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f14240d8eb0 0x7f142404ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f1431e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f14240061f0 0x7f14240dec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f1431e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f142404b3c0 con 0x7f1424050900 2024-04-20T21:17:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f1431e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f142404f4e0 con 0x7f14240061f0 2024-04-20T21:17:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.383+0000 7f1431e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f142404f7c0 con 0x7f14240d8eb0 2024-04-20T21:17:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.384+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7f61c0001250 con 0x7f61c81511f0 2024-04-20T21:17:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.384+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 >> v1:172.21.15.102:6789/0 conn(0x7f61c81511f0 legacy=0x7f61c806e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.384+0000 7f61c7400640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61ac003d60 0x7f61ac004180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.384+0000 7f61c7400640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61ac004800 0x7f61ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.384+0000 7f61c7400640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81515d0 0x7f61ac005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.384+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f61c806f450 con 0x7f61ac003d60 2024-04-20T21:17:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.384+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac010190 con 0x7f61ac004800 2024-04-20T21:17:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.384+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f61ac010390 con 0x7f61c81515d0 2024-04-20T21:17:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.385+0000 7f61c7e00640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61ac004800 0x7f61ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.386+0000 7f61cdc00640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81515d0 0x7f61ac005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.385+0000 7f142aa00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f14240d8eb0 0x7f142404ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.385+0000 7f142aa00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f14240d8eb0 0x7f142404ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56718/0 (socket says 172.21.15.102:56718) 2024-04-20T21:17:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.385+0000 7f142aa00640 1 -- 172.21.15.102:0/1854763867 learned_addr learned my addr 172.21.15.102:0/1854763867 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.386+0000 7f142aa00640 1 -- 172.21.15.102:0/1854763867 >> v2:172.21.15.102:3300/0 conn(0x7f14240061f0 msgr2=0x7f14240dec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.386+0000 7f142aa00640 1 --2- 172.21.15.102:0/1854763867 >> v2:172.21.15.102:3300/0 conn(0x7f14240061f0 0x7f14240dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.386+0000 7f142aa00640 1 -- 172.21.15.102:0/1854763867 >> v1:172.21.15.102:6789/0 conn(0x7f1424050900 legacy=0x7f14240df1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.386+0000 7f142aa00640 1 -- 172.21.15.102:0/1854763867 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142404a3a0 con 0x7f14240d8eb0 2024-04-20T21:17:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f61cd200640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61ac003d60 0x7f61ac004180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f8771800640 1 Processor -- start 2024-04-20T21:17:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f8771800640 1 -- start start 2024-04-20T21:17:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f142aa00640 1 --2- 172.21.15.102:0/1854763867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14240d8eb0 0x7f142404ae80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f142c06a340 tx=0x7f142c0a8170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5edc33811a038fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f61c7e00640 1 -- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81515d0 msgr2=0x7f61ac005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f61c7e00640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81515d0 0x7f61ac005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.388+0000 7f1429600640 1 -- 172.21.15.102:0/1854763867 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c0b5070 con 0x7f14240d8eb0 2024-04-20T21:17:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.388+0000 7f1429600640 1 -- 172.21.15.102:0/1854763867 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f142c0b0440 con 0x7f14240d8eb0 2024-04-20T21:17:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.388+0000 7f1429600640 1 -- 172.21.15.102:0/1854763867 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c0b08b0 con 0x7f14240d8eb0 2024-04-20T21:17:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f8771800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f8771800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f876c170600 0x7f876c1729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f8771800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f876c0c7440 con 0x7f876c0c9220 2024-04-20T21:17:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f8771800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f876c16c100 con 0x7f876c170600 2024-04-20T21:17:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f8771800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f876c0c76e0 con 0x7f876c08d350 2024-04-20T21:17:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f876be00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f876c0c9220 0x7f876c167d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49088/0 (socket says 172.21.15.102:49088) 2024-04-20T21:17:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f876be00640 1 -- 172.21.15.102:0/1499681349 learned_addr learned my addr 172.21.15.102:0/1499681349 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f876b400640 1 --2- 172.21.15.102:0/1499681349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f1431e00640 1 -- 172.21.15.102:0/1854763867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14240d8eb0 msgr2=0x7f142404ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f1431e00640 1 --2- 172.21.15.102:0/1854763867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14240d8eb0 0x7f142404ae80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f142c06a340 tx=0x7f142c0a8170 comp rx=0 tx=0).stop 2024-04-20T21:17:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f1431e00640 1 -- 172.21.15.102:0/1854763867 shutdown_connections 2024-04-20T21:17:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f1431e00640 1 --2- 172.21.15.102:0/1854763867 >> v2:172.21.15.102:3300/0 conn(0x7f14240061f0 0x7f14240dec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f1431e00640 1 --2- 172.21.15.102:0/1854763867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14240d8eb0 0x7f142404ae80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.389+0000 7f1431e00640 1 -- 172.21.15.102:0/1854763867 >> 172.21.15.102:0/1854763867 conn(0x7f1424054510 msgr2=0x7f1424054920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f61c7e00640 1 -- 172.21.15.102:0/3846928826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61ac003d60 msgr2=0x7f61ac004180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f61c7e00640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61ac003d60 0x7f61ac004180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.387+0000 7f61c7e00640 1 -- 172.21.15.102:0/3846928826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac003520 con 0x7f61ac004800 2024-04-20T21:17:24.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.388+0000 7f61cd200640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61ac003d60 0x7f61ac004180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:24.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f1431e00640 1 -- 172.21.15.102:0/1854763867 shutdown_connections 2024-04-20T21:17:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f8770e00640 1 --2- 172.21.15.102:0/1499681349 >> v2:172.21.15.102:3300/0 conn(0x7f876c170600 0x7f876c1729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f876b400640 1 -- 172.21.15.102:0/1499681349 >> v2:172.21.15.102:3300/0 conn(0x7f876c170600 msgr2=0x7f876c1729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f876b400640 1 --2- 172.21.15.102:0/1499681349 >> v2:172.21.15.102:3300/0 conn(0x7f876c170600 0x7f876c1729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f876b400640 1 -- 172.21.15.102:0/1499681349 >> v1:172.21.15.193:6789/0 conn(0x7f876c0c9220 legacy=0x7f876c167d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f876b400640 1 -- 172.21.15.102:0/1499681349 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c0c79c0 con 0x7f876c08d350 2024-04-20T21:17:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f8770e00640 1 --2- 172.21.15.102:0/1499681349 >> v2:172.21.15.102:3300/0 conn(0x7f876c170600 0x7f876c1729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.391+0000 7f61c7e00640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61ac004800 0x7f61ac004c80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b9f0 tx=0x7f61ac005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c748bc81d38c2c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.391+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8039070 con 0x7f61ac004800 2024-04-20T21:17:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.392+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b802c810 con 0x7f61ac004800 2024-04-20T21:17:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.392+0000 7f61c7400640 1 -- 172.21.15.102:0/3846928826 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b800b040 con 0x7f61ac004800 2024-04-20T21:17:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.392+0000 7f876b400640 1 --2- 172.21.15.102:0/1499681349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c9600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f875c00a8c0 tx=0x7f875c02ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1332715c5e0c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.392+0000 7f876aa00640 1 -- 172.21.15.102:0/1499681349 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c02e850 con 0x7f876c08d350 2024-04-20T21:17:24.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.392+0000 7f876aa00640 1 -- 172.21.15.102:0/1499681349 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f875c02e9f0 con 0x7f876c08d350 2024-04-20T21:17:24.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.392+0000 7f876aa00640 1 -- 172.21.15.102:0/1499681349 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c02ee00 con 0x7f876c08d350 2024-04-20T21:17:24.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.390+0000 7f1431e00640 1 -- 172.21.15.102:0/1854763867 wait complete. 2024-04-20T21:17:24.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.392+0000 7f1431e00640 1 Processor -- start 2024-04-20T21:17:24.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.393+0000 7f1431e00640 1 -- start start 2024-04-20T21:17:24.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.393+0000 7f8771800640 1 -- 172.21.15.102:0/1499681349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 msgr2=0x7f876c0c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.393+0000 7f8771800640 1 --2- 172.21.15.102:0/1499681349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c9600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f875c00a8c0 tx=0x7f875c02ca90 comp rx=0 tx=0).stop 2024-04-20T21:17:24.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.393+0000 7f61cfc00640 1 -- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61ac004800 msgr2=0x7f61ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.394+0000 7f61cfc00640 1 --2- 172.21.15.102:0/3846928826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61ac004800 0x7f61ac004c80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b9f0 tx=0x7f61ac005830 comp rx=0 tx=0).stop 2024-04-20T21:17:24.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.394+0000 7f61cdc00640 1 -- 172.21.15.102:0/3846928826 reap_dead start 2024-04-20T21:17:24.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.394+0000 7f8771800640 1 -- 172.21.15.102:0/1499681349 shutdown_connections 2024-04-20T21:17:24.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.394+0000 7f8771800640 1 --2- 172.21.15.102:0/1499681349 >> v2:172.21.15.102:3300/0 conn(0x7f876c170600 0x7f876c1729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.394+0000 7f8771800640 1 --2- 172.21.15.102:0/1499681349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c9600 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.394+0000 7f61cfc00640 1 -- 172.21.15.102:0/3846928826 shutdown_connections 2024-04-20T21:17:24.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f61cfc00640 1 -- 172.21.15.102:0/3846928826 >> 172.21.15.102:0/3846928826 conn(0x7f61c80cc890 msgr2=0x7f61c80ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f1431e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14240061f0 0x7f14240d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.394+0000 7f8771800640 1 -- 172.21.15.102:0/1499681349 >> 172.21.15.102:0/1499681349 conn(0x7f876c074ea0 msgr2=0x7f876c075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f1431e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1424050900 0x7f14240c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f1431e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14240d8eb0 0x7f14240c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f1431e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f14240d9290 con 0x7f1424050900 2024-04-20T21:17:24.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f5a25600640 1 Processor -- start 2024-04-20T21:17:24.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f1431e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f142404a960 con 0x7f14240d8eb0 2024-04-20T21:17:24.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f1431e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f142404fc70 con 0x7f14240061f0 2024-04-20T21:17:24.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f142aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14240061f0 0x7f14240d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f8771800640 1 -- 172.21.15.102:0/1499681349 shutdown_connections 2024-04-20T21:17:24.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f8771800640 1 -- 172.21.15.102:0/1499681349 wait complete. 2024-04-20T21:17:24.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f142aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14240061f0 0x7f14240d52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55222/0 (socket says 172.21.15.102:55222) 2024-04-20T21:17:24.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f142aa00640 1 -- 172.21.15.102:0/1598010015 learned_addr learned my addr 172.21.15.102:0/1598010015 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f142a000640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1424050900 0x7f14240c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f142b400640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14240d8eb0 0x7f14240c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f5a25600640 1 -- start start 2024-04-20T21:17:24.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.395+0000 7f61cfc00640 1 -- 172.21.15.102:0/3846928826 shutdown_connections 2024-04-20T21:17:24.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f142a000640 1 -- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14240061f0 msgr2=0x7f14240d52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.396+0000 7f142a000640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14240061f0 0x7f14240d52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.397+0000 7f142a000640 1 -- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14240d8eb0 msgr2=0x7f14240c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.397+0000 7f142a000640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14240d8eb0 0x7f14240c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.397+0000 7f142a000640 1 -- 172.21.15.102:0/1598010015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14240540d0 con 0x7f1424050900 2024-04-20T21:17:24.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.397+0000 7f61cfc00640 1 -- 172.21.15.102:0/3846928826 wait complete. 2024-04-20T21:17:24.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.398+0000 7f5a25600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a200c8f80 0x7f5a200c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.398+0000 7f142a000640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1424050900 0x7f14240c71e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b410 tx=0x7f141c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.398+0000 7f5a1e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a200c8f80 0x7f5a200c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f5a25600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a200c7ca0 con 0x7f5a20071370 2024-04-20T21:17:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f5a25600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a20163870 con 0x7f5a20162d80 2024-04-20T21:17:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f5a1e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a200c8f80 0x7f5a200c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55230/0 (socket says 172.21.15.102:55230) 2024-04-20T21:17:24.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f5a25600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5a200c8220 con 0x7f5a200c8f80 2024-04-20T21:17:24.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f5a1e000640 1 -- 172.21.15.102:0/1036448861 learned_addr learned my addr 172.21.15.102:0/1036448861 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.398+0000 7fbe6b200640 1 Processor -- start 2024-04-20T21:17:24.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.398+0000 7fbe6b200640 1 -- start start 2024-04-20T21:17:24.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.398+0000 7f8771800640 1 Processor -- start 2024-04-20T21:17:24.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f8771800640 1 -- start start 2024-04-20T21:17:24.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235216791 0 0) 0x7f5a20163870 con 0x7f5a20162d80 2024-04-20T21:17:24.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c015020 con 0x7f1424050900 2024-04-20T21:17:24.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a04003520 con 0x7f5a20162d80 2024-04-20T21:17:24.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f141c011850 con 0x7f1424050900 2024-04-20T21:17:24.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c011c40 con 0x7f1424050900 2024-04-20T21:17:24.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.399+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1424178be0 con 0x7f1424050900 2024-04-20T21:17:24.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7fadd7e00640 1 Processor -- start 2024-04-20T21:17:24.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7fadd7e00640 1 -- start start 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1880071453 0 0) 0x7f5a04003520 con 0x7f5a20162d80 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a200c8f80 msgr2=0x7f5a200c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a200c8f80 0x7f5a200c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 >> v1:172.21.15.102:6789/0 conn(0x7f5a20071370 legacy=0x7f5a20163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 --> v1:172.21.15.193:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a200c7f40 con 0x7f5a20162d80 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 <== mon.1 v1:172.21.15.193:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7f5a10001670 con 0x7f5a20162d80 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 >> v1:172.21.15.193:6789/0 conn(0x7f5a20162d80 legacy=0x7f5a2006d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a040039b0 0x7f5a04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a04004800 0x7f5a04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a04005210 0x7f5a0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5a200c8220 con 0x7f5a040039b0 2024-04-20T21:17:24.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0400c120 con 0x7f5a04005210 2024-04-20T21:17:24.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0400c320 con 0x7f5a04004800 2024-04-20T21:17:24.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142404e7e0 con 0x7f1424050900 2024-04-20T21:17:24.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7f61cfc00640 1 Processor -- start 2024-04-20T21:17:24.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7f5a1e000640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a200c8f80 0x7f5a200c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:24.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7f5a1e000640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a04004800 0x7f5a04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7fbe6b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe640c8f80 0x7fbe640c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7fbe6b200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe640c7ca0 con 0x7fbe64071370 2024-04-20T21:17:24.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7fbe6b200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe64163870 con 0x7fbe64162d80 2024-04-20T21:17:24.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.400+0000 7fbe6b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbe640c8220 con 0x7fbe640c8f80 2024-04-20T21:17:24.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe63e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fbe64162d80 0x7fbe6406d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49116/0 (socket says 172.21.15.102:49116) 2024-04-20T21:17:24.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe63e00640 1 -- 172.21.15.102:0/1815580750 learned_addr learned my addr 172.21.15.102:0/1815580750 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:24.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876478926 0 0) 0x7fbe64163870 con 0x7fbe64162d80 2024-04-20T21:17:24.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe48003520 con 0x7fbe64162d80 2024-04-20T21:17:24.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7f5a1e000640 1 -- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a04005210 msgr2=0x7f5a0400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:17:24.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2122956476 0 0) 0x7fbe48003520 con 0x7fbe64162d80 2024-04-20T21:17:24.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe640c8f80 msgr2=0x7fbe640c8470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:17:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe640c8f80 0x7fbe640c8470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 >> v1:172.21.15.102:6789/0 conn(0x7fbe64071370 legacy=0x7fbe64163160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:17:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7f5a1e000640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a04005210 0x7f5a0400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7f5a1e000640 1 -- 172.21.15.102:0/1036448861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a040039b0 msgr2=0x7f5a04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7f5a1e000640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a040039b0 0x7f5a04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7f5a1e000640 1 -- 172.21.15.102:0/1036448861 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a200c7f40 con 0x7f5a04004800 2024-04-20T21:17:24.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 --> v1:172.21.15.193:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe640c7f40 con 0x7fbe64162d80 2024-04-20T21:17:24.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 <== mon.1 v1:172.21.15.193:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7fbe5c0019f0 con 0x7fbe64162d80 2024-04-20T21:17:24.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 >> v1:172.21.15.193:6789/0 conn(0x7fbe64162d80 legacy=0x7fbe6406d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe480039b0 0x7fbe48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7fbe62a00640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f8771800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1404001030 con 0x7f1424050900 2024-04-20T21:17:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.401+0000 7f61cfc00640 1 -- start start 2024-04-20T21:17:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f61cfc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61ac004800 0x7f61c8155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f61cfc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81f9160 0x7f61c80c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f61cfc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c8153280 0x7f61c814f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f61cfc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f61c806ee40 con 0x7f61ac004800 2024-04-20T21:17:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f61cfc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f61c806f730 con 0x7f61c8153280 2024-04-20T21:17:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f8771800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f876c0c9220 0x7f876c2076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f8771800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f876c170600 0x7f876c16a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f8771800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f876c16b900 con 0x7f876c0c9220 2024-04-20T21:17:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f8771800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f876c16b620 con 0x7f876c170600 2024-04-20T21:17:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f8771800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f876c16bbe0 con 0x7f876c08d350 2024-04-20T21:17:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7f61cfc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f61c81585e0 con 0x7f61c81f9160 2024-04-20T21:17:24.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f876be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f876be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55262/0 (socket says 172.21.15.102:55262) 2024-04-20T21:17:24.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f876be00640 1 -- 172.21.15.102:0/3965422801 learned_addr learned my addr 172.21.15.102:0/3965422801 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f8770e00640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f876c170600 0x7f876c16a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7fadd7e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fadd00d5ac0 0x7fadd004c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7fadd7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fadd0035100 0x7fadd00dc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f8770e00640 1 -- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 msgr2=0x7f876c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f61c7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81f9160 0x7f61c80c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f8770e00640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f876c08d350 0x7f876c0c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f5a1e000640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a04004800 0x7f5a04004c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5a2015d1d0 tx=0x7f5a04005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9878c25ede613467 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f8770e00640 1 -- 172.21.15.102:0/3965422801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f876c0c9220 msgr2=0x7f876c2076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c035020 con 0x7f5a04004800 2024-04-20T21:17:24.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7fbe62a00640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe640c8f80 0x7fbe4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbe640c8220 con 0x7fbe480039b0 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe480055d0 con 0x7fbe48004800 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbe4800bf00 con 0x7fbe640c8f80 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7fbe69200640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe640c8f80 0x7fbe4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.402+0000 7fbe63400640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7fbe69200640 1 -- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe48004800 msgr2=0x7fbe48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7fbe69200640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7fbe69200640 1 -- 172.21.15.102:0/1815580750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe480039b0 msgr2=0x7fbe48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7fbe69200640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe480039b0 0x7fbe48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7fbe69200640 1 -- 172.21.15.102:0/1815580750 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe640c7f40 con 0x7fbe640c8f80 2024-04-20T21:17:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7fadd7e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadd0036240 con 0x7fadd0006240 2024-04-20T21:17:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7fadd7e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fadd0046cd0 con 0x7fadd0035100 2024-04-20T21:17:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7fadd7e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fadd00d5ea0 con 0x7fadd00d5ac0 2024-04-20T21:17:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7fbe63400640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7fbe69200640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe640c8f80 0x7fbe4800b9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009860 tx=0x7fbe48005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=750d77dcba3c7d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f61cdc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c8153280 0x7f61c814f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f61cdc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c8153280 0x7f61c814f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56754/0 (socket says 172.21.15.102:56754) 2024-04-20T21:17:24.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f61cdc00640 1 -- 172.21.15.102:0/4265435836 learned_addr learned my addr 172.21.15.102:0/4265435836 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f61cdc00640 1 -- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81f9160 msgr2=0x7f61c80c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f61cdc00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61c81f9160 0x7f61c80c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f61cdc00640 1 -- 172.21.15.102:0/4265435836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61ac004800 msgr2=0x7f61c8155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f61cdc00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61ac004800 0x7f61c8155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f61cdc00640 1 -- 172.21.15.102:0/4265435836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c81521e0 con 0x7f61c8153280 2024-04-20T21:17:24.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fadd6a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fadd00d5ac0 0x7fadd004c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54037070 con 0x7fbe640c8f80 2024-04-20T21:17:24.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe54032620 con 0x7fbe640c8f80 2024-04-20T21:17:24.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fbe62a00640 1 -- 172.21.15.102:0/1815580750 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540093b0 con 0x7fbe640c8f80 2024-04-20T21:17:24.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f8770e00640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f876c0c9220 0x7f876c2076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7f61cdc00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c8153280 0x7f61c814f8e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f61c8069d00 tx=0x7f61b400fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fdff9400640 1 Processor -- start 2024-04-20T21:17:24.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.403+0000 7f8770e00640 1 -- 172.21.15.102:0/3965422801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c2063f0 con 0x7f876c170600 2024-04-20T21:17:24.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f141c018020 con 0x7f1424050900 2024-04-20T21:17:24.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f8770e00640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f876c170600 0x7f876c16a360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f876400cc60 tx=0x7f876400a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876400f680 con 0x7f876c170600 2024-04-20T21:17:24.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c031610 con 0x7f5a04004800 2024-04-20T21:17:24.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f5a1d600640 1 -- 172.21.15.102:0/1036448861 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00a590 con 0x7f5a04004800 2024-04-20T21:17:24.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7f5a25600640 1 -- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a04004800 msgr2=0x7f5a04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7f5a25600640 1 --2- 172.21.15.102:0/1036448861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a04004800 0x7f5a04004c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5a2015d1d0 tx=0x7f5a04005830 comp rx=0 tx=0).stop 2024-04-20T21:17:24.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7f5a1f400640 1 -- 172.21.15.102:0/1036448861 reap_dead start 2024-04-20T21:17:24.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fadd6a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fadd00d5ac0 0x7fadd004c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56768/0 (socket says 172.21.15.102:56768) 2024-04-20T21:17:24.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fadd6a00640 1 -- 172.21.15.102:0/519393267 learned_addr learned my addr 172.21.15.102:0/519393267 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fadd7400640 1 --2- 172.21.15.102:0/519393267 >> v2:172.21.15.102:3300/0 conn(0x7fadd0035100 0x7fadd00dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7fdff9400640 1 -- start start 2024-04-20T21:17:24.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876400f820 con 0x7f876c170600 2024-04-20T21:17:24.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7f5a25600640 1 -- 172.21.15.102:0/1036448861 shutdown_connections 2024-04-20T21:17:24.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7f5a25600640 1 -- 172.21.15.102:0/1036448861 >> 172.21.15.102:0/1036448861 conn(0x7f5a200c9390 msgr2=0x7f5a200c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.404+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876400fc10 con 0x7f876c170600 2024-04-20T21:17:24.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7fbe6b200640 1 -- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe640c8f80 msgr2=0x7fbe4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7fbe6b200640 1 --2- 172.21.15.102:0/1815580750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe640c8f80 0x7fbe4800b9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009860 tx=0x7fbe48005830 comp rx=0 tx=0).stop 2024-04-20T21:17:24.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7f5a25600640 1 -- 172.21.15.102:0/1036448861 shutdown_connections 2024-04-20T21:17:24.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c0c3d30 con 0x7f876c170600 2024-04-20T21:17:24.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.405+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c0c42f0 con 0x7f876c170600 2024-04-20T21:17:24.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b40050d0 con 0x7f61c8153280 2024-04-20T21:17:24.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7f1423400640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f140c06c9c0 0x7f140c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7fbe69200640 1 -- 172.21.15.102:0/1815580750 reap_dead start 2024-04-20T21:17:24.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7fbe6b200640 1 -- 172.21.15.102:0/1815580750 shutdown_connections 2024-04-20T21:17:24.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7fbe6b200640 1 -- 172.21.15.102:0/1815580750 >> 172.21.15.102:0/1815580750 conn(0x7fbe640c9390 msgr2=0x7fbe640c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7fbe6b200640 1 -- 172.21.15.102:0/1815580750 shutdown_connections 2024-04-20T21:17:24.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7fbe6b200640 1 -- 172.21.15.102:0/1815580750 wait complete. 2024-04-20T21:17:24.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7f75cd200640 1 Processor -- start 2024-04-20T21:17:24.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7fadd6a00640 1 -- 172.21.15.102:0/519393267 >> v2:172.21.15.102:3300/0 conn(0x7fadd0035100 msgr2=0x7fadd00dc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7f75cd200640 1 -- start start 2024-04-20T21:17:24.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7fadd6a00640 1 --2- 172.21.15.102:0/519393267 >> v2:172.21.15.102:3300/0 conn(0x7fadd0035100 0x7fadd00dc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.406+0000 7f5a25600640 1 -- 172.21.15.102:0/1036448861 wait complete. 2024-04-20T21:17:24.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7fadd6a00640 1 -- 172.21.15.102:0/519393267 >> v1:172.21.15.193:6789/0 conn(0x7fadd0006240 legacy=0x7fadd0045ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7fadd6a00640 1 -- 172.21.15.102:0/519393267 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd0046f30 con 0x7fadd00d5ac0 2024-04-20T21:17:24.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7fadd5600640 1 -- 172.21.15.102:0/519393267 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201489337 0 0) 0x7fadc8001110 con 0x7fadd0006240 2024-04-20T21:17:24.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.408+0000 7fadd7400640 1 --2- 172.21.15.102:0/519393267 >> v2:172.21.15.102:3300/0 conn(0x7fadd0035100 0x7fadd00dc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:24.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.408+0000 7fadd6a00640 1 --2- 172.21.15.102:0/519393267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd00d5ac0 0x7fadd004c6c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadd806b2e0 tx=0x7fadd80aa440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891dd442e63ffc8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f5a25600640 1 Processor -- start 2024-04-20T21:17:24.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7fdff9400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdff415bd10 0x7fdff40751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7fadd5600640 1 -- 172.21.15.102:0/519393267 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80b6070 con 0x7fadd00d5ac0 2024-04-20T21:17:24.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f5a25600640 1 -- start start 2024-04-20T21:17:24.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f876c0c46d0 con 0x7f876c170600 2024-04-20T21:17:24.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f75cd200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f75c80cb490 0x7f75c815b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f75cd200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c806c420 con 0x7f75c81554d0 2024-04-20T21:17:24.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f75cd200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c81558b0 con 0x7f75c80ce1b0 2024-04-20T21:17:24.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f75cd200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f75c806c750 con 0x7f75c80cb490 2024-04-20T21:17:24.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7fadd5600640 1 -- 172.21.15.102:0/519393267 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd80af7d0 con 0x7fadd00d5ac0 2024-04-20T21:17:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7fadd5600640 1 -- 172.21.15.102:0/519393267 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80afc40 con 0x7fadd00d5ac0 2024-04-20T21:17:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f75c7400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f75c81554d0 0x7f75c806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38676/0 (socket says 172.21.15.102:38676) 2024-04-20T21:17:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f75c7400640 1 -- 172.21.15.102:0/851003685 learned_addr learned my addr 172.21.15.102:0/851003685 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f75c7e00640 1 --2- 172.21.15.102:0/851003685 >> v2:172.21.15.193:3300/0 conn(0x7f75c80cb490 0x7f75c815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7f75c7e00640 1 -- 172.21.15.102:0/851003685 >> v1:172.21.15.193:6789/0 conn(0x7f75c80ce1b0 legacy=0x7f75c80cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7f75c7e00640 1 -- 172.21.15.102:0/851003685 >> v1:172.21.15.102:6789/0 conn(0x7f75c81554d0 legacy=0x7f75c806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7f75c6000640 1 -- 172.21.15.102:0/851003685 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547421774 0 0) 0x7f75c806c420 con 0x7f75c81554d0 2024-04-20T21:17:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7f75c7e00640 1 -- 172.21.15.102:0/851003685 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c80ccaa0 con 0x7f75c80cb490 2024-04-20T21:17:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7f5a25600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a201ec3b0 0x7f5a200c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7fadd7e00640 1 -- 172.21.15.102:0/519393267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd00d5ac0 msgr2=0x7fadd004c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7fadd7e00640 1 --2- 172.21.15.102:0/519393267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd00d5ac0 0x7fadd004c6c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadd806b2e0 tx=0x7fadd80aa440 comp rx=0 tx=0).stop 2024-04-20T21:17:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7fdff9400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff415c150 con 0x7fdff4075710 2024-04-20T21:17:24.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7fdff9400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff40d0b30 con 0x7fdff4155690 2024-04-20T21:17:24.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7fdff9400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fdff40cc2f0 con 0x7fdff415bd10 2024-04-20T21:17:24.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a25600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a201ec790 0x7f5a200cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a25600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a20152920 0x7f5a200cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a25600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2006cff0 con 0x7f5a20152920 2024-04-20T21:17:24.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a25600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2006cd10 con 0x7f5a201ec790 2024-04-20T21:17:24.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a25600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5a200c8470 con 0x7f5a201ec3b0 2024-04-20T21:17:24.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fdff3200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdff4075710 0x7fdff40cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38678/0 (socket says 172.21.15.102:38678) 2024-04-20T21:17:24.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fdff3200640 1 -- 172.21.15.102:0/518246751 learned_addr learned my addr 172.21.15.102:0/518246751 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:24.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a1ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a201ec3b0 0x7f5a200c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a1e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a201ec790 0x7f5a200cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a1ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a201ec3b0 0x7f5a200c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55278/0 (socket says 172.21.15.102:55278) 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a1ea00640 1 -- 172.21.15.102:0/2192469739 learned_addr learned my addr 172.21.15.102:0/2192469739 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a1e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a201ec790 0x7f5a200cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56800/0 (socket says 172.21.15.102:56800) 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f5a1f400640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a20152920 0x7f5a200cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f141c08bb60 con 0x7f1424050900 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7f1423400640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:6817/1144653737,v1:172.21.15.102:6819/1144653737] conn(0x7f140c076130 0x7f140c078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 --> [v2:172.21.15.102:6817/1144653737,v1:172.21.15.102:6819/1144653737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f141c012e50 con 0x7f140c076130 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f141c0902a0 con 0x7f1424050900 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.407+0000 7f142aa00640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f140c06c9c0 0x7f140c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.408+0000 7f142b400640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:6817/1144653737,v1:172.21.15.102:6819/1144653737] conn(0x7f140c076130 0x7f140c078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7f142b400640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:6817/1144653737,v1:172.21.15.102:6819/1144653737] conn(0x7f140c076130 0x7f140c078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f142aa00640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f140c06c9c0 0x7f140c06ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f142c06a340 tx=0x7f142c0ac000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.410+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c81524c0 con 0x7f61c8153280 2024-04-20T21:17:24.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c8152a80 con 0x7f61c8153280 2024-04-20T21:17:24.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b4018070 con 0x7f61c8153280 2024-04-20T21:17:24.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fadd7e00640 1 -- 172.21.15.102:0/519393267 shutdown_connections 2024-04-20T21:17:24.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fadd7e00640 1 --2- 172.21.15.102:0/519393267 >> v2:172.21.15.102:3300/0 conn(0x7fadd0035100 0x7fadd00dc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fadd7e00640 1 --2- 172.21.15.102:0/519393267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd00d5ac0 0x7fadd004c6c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fadd7e00640 1 -- 172.21.15.102:0/519393267 >> 172.21.15.102:0/519393267 conn(0x7fadd0050180 msgr2=0x7fadd0050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7fbe6b200640 1 Processor -- start 2024-04-20T21:17:24.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fdfe9e00640 1 --2- 172.21.15.102:0/518246751 >> v2:172.21.15.193:3300/0 conn(0x7fdff415bd10 0x7fdff40751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 <== osd.2 v2:172.21.15.102:6817/1144653737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f141c012e50 con 0x7f140c076130 2024-04-20T21:17:24.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f61a0001030 con 0x7f61c8153280 2024-04-20T21:17:24.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7f5a1e000640 1 -- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a201ec3b0 msgr2=0x7f5a200c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7f5a1e000640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a201ec3b0 0x7f5a200c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7f5a1e000640 1 -- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a20152920 msgr2=0x7f5a200cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.409+0000 7fbe6b200640 1 -- start start 2024-04-20T21:17:24.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fbe6b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe640c8f80 0x7fbe641540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fbe6b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe6415dd40 0x7fbe6415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fbe6b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe64160700 0x7fbe64160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fbe63e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe640c8f80 0x7fbe641540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fbe63e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe640c8f80 0x7fbe641540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56802/0 (socket says 172.21.15.102:56802) 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fbe69200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe64160700 0x7fbe64160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.411+0000 7fbe69200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe64160700 0x7fbe64160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55286/0 (socket says 172.21.15.102:55286) 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fbe6b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6406cff0 con 0x7fbe6415dd40 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fbe6b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6406cd10 con 0x7fbe640c8f80 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fbe6b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbe640c8470 con 0x7fbe64160700 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fbe63e00640 1 -- 172.21.15.102:0/661498208 learned_addr learned my addr 172.21.15.102:0/661498208 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fbe63400640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe6415dd40 0x7fbe6415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 672924999 0 0) 0x7fdfec001110 con 0x7fdff4075710 2024-04-20T21:17:24.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfd8003520 con 0x7fdff4075710 2024-04-20T21:17:24.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7f5a1e000640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a20152920 0x7f5a200cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b40050d0 con 0x7f61c8153280 2024-04-20T21:17:24.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7f5a1e000640 1 -- 172.21.15.102:0/2192469739 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a201f60d0 con 0x7f5a201ec790 2024-04-20T21:17:24.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7f5a1f400640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a20152920 0x7f5a200cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:24.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1564980878 0 0) 0x7fdff40d0b30 con 0x7fdff4155690 2024-04-20T21:17:24.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfec001110 con 0x7fdff4155690 2024-04-20T21:17:24.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fadd7e00640 1 -- 172.21.15.102:0/519393267 shutdown_connections 2024-04-20T21:17:24.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fbe69200640 1 -- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe640c8f80 msgr2=0x7fbe641540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7fbe69200640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe640c8f80 0x7fbe641540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fbe69200640 1 -- 172.21.15.102:0/661498208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe6415dd40 msgr2=0x7fbe6415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fbe69200640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe6415dd40 0x7fbe6415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fbe69200640 1 -- 172.21.15.102:0/661498208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6415e6e0 con 0x7fbe64160700 2024-04-20T21:17:24.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fbe63400640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe6415dd40 0x7fbe6415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:24.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fadd7e00640 1 -- 172.21.15.102:0/519393267 wait complete. 2024-04-20T21:17:24.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1172052723 0 0) 0x7fdfd8003520 con 0x7fdff4075710 2024-04-20T21:17:24.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 >> v2:172.21.15.193:3300/0 conn(0x7fdff415bd10 msgr2=0x7fdff40751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 --2- 172.21.15.102:0/518246751 >> v2:172.21.15.193:3300/0 conn(0x7fdff415bd10 0x7fdff40751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 >> v1:172.21.15.193:6789/0 conn(0x7fdff4155690 legacy=0x7fdff40cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff40d0e10 con 0x7fdff4075710 2024-04-20T21:17:24.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fbe69200640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe64160700 0x7fbe64160b80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009830 tx=0x7fbe54004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7f75c7e00640 1 --2- 172.21.15.102:0/851003685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 0x7f75c815b150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009d90 tx=0x7f75c002bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a1ba98693c026fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.412+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8764018400 con 0x7f876c170600 2024-04-20T21:17:24.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7f5a1e000640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a201ec790 0x7f5a200cae00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c4f0 tx=0x7f5a0c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54041070 con 0x7fbe64160700 2024-04-20T21:17:24.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7f75c6000640 1 -- 172.21.15.102:0/851003685 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0039070 con 0x7f75c80cb490 2024-04-20T21:17:24.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7f75c6000640 1 -- 172.21.15.102:0/851003685 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c002cda0 con 0x7f75c80cb490 2024-04-20T21:17:24.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7f75c6000640 1 -- 172.21.15.102:0/851003685 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0034660 con 0x7f75c80cb490 2024-04-20T21:17:24.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5403d5f0 con 0x7fbe64160700 2024-04-20T21:17:24.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54037040 con 0x7fbe64160700 2024-04-20T21:17:24.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe64155020 con 0x7fbe64160700 2024-04-20T21:17:24.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641555e0 con 0x7fbe64160700 2024-04-20T21:17:24.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c035040 con 0x7f5a201ec790 2024-04-20T21:17:24.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c03b9f0 con 0x7f5a201ec790 2024-04-20T21:17:24.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00a590 con 0x7f5a201ec790 2024-04-20T21:17:24.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a200c6860 con 0x7f5a201ec790 2024-04-20T21:17:24.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.414+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a200ca8e0 con 0x7f5a201ec790 2024-04-20T21:17:24.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.415+0000 7fadd7e00640 1 Processor -- start 2024-04-20T21:17:24.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.415+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61b4005270 con 0x7f61c8153280 2024-04-20T21:17:24.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.415+0000 7fadd7e00640 1 -- start start 2024-04-20T21:17:24.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.415+0000 7f75cd200640 1 -- 172.21.15.102:0/851003685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 msgr2=0x7f75c815b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.415+0000 7f75cd200640 1 --2- 172.21.15.102:0/851003685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 0x7f75c815b150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009d90 tx=0x7f75c002bff0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.415+0000 7f8763e00640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874806c8f0 0x7f874806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f876408cb30 con 0x7f876c170600 2024-04-20T21:17:24.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7fadd7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadd0006240 0x7fadd0165a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7f75cd200640 1 -- 172.21.15.102:0/851003685 shutdown_connections 2024-04-20T21:17:24.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7f75cd200640 1 --2- 172.21.15.102:0/851003685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 0x7f75c815b150 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7f75cd200640 1 -- 172.21.15.102:0/851003685 >> 172.21.15.102:0/851003685 conn(0x7f75c8073060 msgr2=0x7f75c8073470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7fadd7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0035100 0x7fadd00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7fadd7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fadd00d5ac0 0x7fadd00cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7fadd7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fadd00dc000 con 0x7fadd0006240 2024-04-20T21:17:24.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7fadd7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fadd00dbd20 con 0x7fadd0035100 2024-04-20T21:17:24.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7fadd7e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fadd00db9d0 con 0x7fadd00d5ac0 2024-04-20T21:17:24.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7f8763e00640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.102:6800/100719841,v1:172.21.15.102:6801/100719841] conn(0x7f8748075f60 0x7f87480783a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7f75cd200640 1 -- 172.21.15.102:0/851003685 shutdown_connections 2024-04-20T21:17:24.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7fdfec001840 con 0x7fdff4075710 2024-04-20T21:17:24.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.413+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 >> v1:172.21.15.102:6789/0 conn(0x7fdff4075710 legacy=0x7fdff40cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 --> [v2:172.21.15.102:6800/100719841,v1:172.21.15.102:6801/100719841] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f876401ce50 con 0x7f8748075f60 2024-04-20T21:17:24.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.416+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe64074340 con 0x7fbe64160700 2024-04-20T21:17:24.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7f75cd200640 1 -- 172.21.15.102:0/851003685 wait complete. 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fadd7400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fadd00d5ac0 0x7fadd00cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fadd6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0035100 0x7fadd00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fadd7400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fadd00d5ac0 0x7fadd00cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55296/0 (socket says 172.21.15.102:55296) 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fadd7400640 1 -- 172.21.15.102:0/170594620 learned_addr learned my addr 172.21.15.102:0/170594620 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fdff1e00640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfd8004ba0 0x7fdfd8004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fdff1e00640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd80037a0 0x7fdfd8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fdff1e00640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd8004160 0x7fdfd80141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40cc2f0 con 0x7fdfd8004ba0 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd80147c0 con 0x7fdfd8004160 2024-04-20T21:17:24.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd80149c0 con 0x7fdfd80037a0 2024-04-20T21:17:24.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fadd6a00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadd0006240 0x7fadd0165a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a0c038020 con 0x7f5a201ec790 2024-04-20T21:17:24.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fdff2800640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfd8004ba0 0x7fdfd8004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fdfe9e00640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd80037a0 0x7fdfd8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fadd6a00640 1 -- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fadd00d5ac0 msgr2=0x7fadd00cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fadd6a00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fadd00d5ac0 0x7fadd00cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fadd6a00640 1 -- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0035100 msgr2=0x7fadd00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fdff3200640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd8004160 0x7fdfd80141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7f5a0be00640 1 -- 172.21.15.102:0/2192469739 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a20074340 con 0x7f5a201ec790 2024-04-20T21:17:24.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.417+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe5403d9a0 con 0x7fbe64160700 2024-04-20T21:17:24.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7f61c4c00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f61a806caf0 0x7f61a806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f876b400640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.102:6800/100719841,v1:172.21.15.102:6801/100719841] conn(0x7f8748075f60 0x7f87480783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f876be00640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874806c8f0 0x7f874806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f61b408c1f0 con 0x7f61c8153280 2024-04-20T21:17:24.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f61cd200640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f61a806caf0 0x7f61a806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8764056870 con 0x7f876c170600 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7fadd6a00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0035100 0x7fadd00cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7fadd6a00640 1 -- 172.21.15.102:0/170594620 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd01673e0 con 0x7fadd0006240 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7fadd6000640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd0035100 0x7fadd00cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fdff2800640 1 -- 172.21.15.102:0/518246751 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd80037a0 msgr2=0x7fdfd8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fdff2800640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd80037a0 0x7fdfd8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fdff2800640 1 -- 172.21.15.102:0/518246751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd8004160 msgr2=0x7fdfd80141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fdff2800640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd8004160 0x7fdfd80141d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.418+0000 7fdff2800640 1 -- 172.21.15.102:0/518246751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff40d0e10 con 0x7fdfd8004ba0 2024-04-20T21:17:24.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7fdff3200640 1 -- 172.21.15.102:0/518246751 reap_dead start 2024-04-20T21:17:24.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7f61c4c00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.102:6808/1191557030,v1:172.21.15.102:6809/1191557030] conn(0x7f61a80761f0 0x7f61a8078630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7fbe5be00640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbe4406c920 0x7fbe4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbe540ad210 con 0x7fbe64160700 2024-04-20T21:17:24.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7fbe63e00640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbe4406c920 0x7fbe4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fbe5be00640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6800/2469990595,v1:172.21.15.193:6801/2469990595] conn(0x7fbe44076010 0x7fbe44078450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fadd6a00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadd0006240 0x7fadd0165a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fadd8050210 tx=0x7fadd806ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 --> [v2:172.21.15.102:6808/1191557030,v1:172.21.15.102:6809/1191557030] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61b401de50 con 0x7f61a80761f0 2024-04-20T21:17:24.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f61b408c5e0 con 0x7f61c8153280 2024-04-20T21:17:24.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 --> [v2:172.21.15.193:6800/2469990595,v1:172.21.15.193:6801/2469990595] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe54004510 con 0x7fbe44076010 2024-04-20T21:17:24.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbe540b4050 con 0x7fbe64160700 2024-04-20T21:17:24.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f5a16a00640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a0006c940 0x7f5a0006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fbe63400640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6800/2469990595,v1:172.21.15.193:6801/2469990595] conn(0x7fbe44076010 0x7fbe44078450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80bdd90 con 0x7fadd0006240 2024-04-20T21:17:24.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd80b17b0 con 0x7fadd0006240 2024-04-20T21:17:24.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80b1b60 con 0x7fadd0006240 2024-04-20T21:17:24.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7fadd7e00640 1 -- 172.21.15.102:0/170594620 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd004d9c0 con 0x7fadd0006240 2024-04-20T21:17:24.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7f61c7e00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.102:6808/1191557030,v1:172.21.15.102:6809/1191557030] conn(0x7f61a80761f0 0x7f61a8078630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.421+0000 7fbe63e00640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbe4406c920 0x7fbe4406ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c001e40 tx=0x7fbe5c007d70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.421+0000 7f61cd200640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f61a806caf0 0x7f61a806efb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61c00013c0 tx=0x7f61c000a040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7f876be00640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874806c8f0 0x7f874806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8754000f60 tx=0x7f875400b210 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7f876b400640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.102:6800/100719841,v1:172.21.15.102:6801/100719841] conn(0x7f8748075f60 0x7f87480783a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.421+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 <== osd.1 v2:172.21.15.102:6800/100719841 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f876401ce50 con 0x7f8748075f60 2024-04-20T21:17:24.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5a0c0ad920 con 0x7f5a201ec790 2024-04-20T21:17:24.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f5a16a00640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:6816/2815998409,v1:172.21.15.102:6818/2815998409] conn(0x7f5a000760b0 0x7f5a000784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 --> [v2:172.21.15.102:6816/2815998409,v1:172.21.15.102:6818/2815998409] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a0c03b280 con 0x7f5a000760b0 2024-04-20T21:17:24.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.419+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c0b22a0 con 0x7f5a201ec790 2024-04-20T21:17:24.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7f5a1f400640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:6816/2815998409,v1:172.21.15.102:6818/2815998409] conn(0x7f5a000760b0 0x7f5a000784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.420+0000 7f5a1ea00640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a0006c940 0x7f5a0006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.421+0000 7f5a1f400640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:6816/2815998409,v1:172.21.15.102:6818/2815998409] conn(0x7f5a000760b0 0x7f5a000784f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.421+0000 7fadd7e00640 1 -- 172.21.15.102:0/170594620 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd004ef80 con 0x7fadd0006240 2024-04-20T21:17:24.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.422+0000 7fbe63400640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6800/2469990595,v1:172.21.15.193:6801/2469990595] conn(0x7fbe44076010 0x7fbe44078450 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.422+0000 7f61c7e00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.102:6808/1191557030,v1:172.21.15.102:6809/1191557030] conn(0x7f61a80761f0 0x7f61a8078630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.422+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 <== osd.3 v2:172.21.15.102:6816/2815998409 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a0c03b280 con 0x7f5a000760b0 2024-04-20T21:17:24.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.422+0000 7fdff2800640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfd8004ba0 0x7fdfd8004fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4001fa0 tx=0x7fdfd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c437c221c0789bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.422+0000 7f5a1ea00640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a0006c940 0x7f5a0006ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a100017e0 tx=0x7f5a10009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.423+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 <== osd.4 v2:172.21.15.193:6800/2469990595 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe54004510 con 0x7fbe44076010 2024-04-20T21:17:24.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.423+0000 7f75cd200640 1 Processor -- start 2024-04-20T21:17:24.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.422+0000 7fadd7e00640 1 -- 172.21.15.102:0/170594620 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fadd0005390 con 0x7fadd0006240 2024-04-20T21:17:24.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.423+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4001bf0 con 0x7fdfd8004ba0 2024-04-20T21:17:24.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.423+0000 7fdff1e00640 1 -- 172.21.15.102:0/518246751 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe4003b70 con 0x7fdfd8004ba0 2024-04-20T21:17:24.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.424+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 <== osd.0 v2:172.21.15.102:6808/1191557030 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61b401de50 con 0x7f61a80761f0 2024-04-20T21:17:24.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.424+0000 7fdff9400640 1 -- 172.21.15.102:0/518246751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfd8004ba0 msgr2=0x7fdfd8004fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.424+0000 7fdff9400640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfd8004ba0 0x7fdfd8004fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4001fa0 tx=0x7fdfd8005830 comp rx=0 tx=0).stop 2024-04-20T21:17:24.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.424+0000 7fdff9400640 1 -- 172.21.15.102:0/518246751 shutdown_connections 2024-04-20T21:17:24.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.424+0000 7fdff9400640 1 --2- 172.21.15.102:0/518246751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfd8004ba0 0x7fdfd8004fa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.424+0000 7fdff9400640 1 -- 172.21.15.102:0/518246751 >> 172.21.15.102:0/518246751 conn(0x7fdff40685f0 msgr2=0x7fdff406ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.424+0000 7fdff9400640 1 -- 172.21.15.102:0/518246751 shutdown_connections 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.425+0000 7fdff9400640 1 -- 172.21.15.102:0/518246751 wait complete. 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.425+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadd80af690 con 0x7fadd0006240 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.427+0000 7fadcea00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fadb806c940 0x7fadb806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.427+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fadd8132ec0 con 0x7fadd0006240 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.427+0000 7fadcea00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6804/1109454637,v1:172.21.15.193:6806/1109454637] conn(0x7fadb80760b0 0x7fadb80784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.427+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 --> [v2:172.21.15.193:6804/1109454637,v1:172.21.15.193:6806/1109454637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadd8070f30 con 0x7fadb80760b0 2024-04-20T21:17:24.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.427+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fadd8133270 con 0x7fadd0006240 2024-04-20T21:17:24.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.427+0000 7fdff9400640 1 Processor -- start 2024-04-20T21:17:24.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.427+0000 7fdff9400640 1 -- start start 2024-04-20T21:17:24.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.427+0000 7f75cd200640 1 -- start start 2024-04-20T21:17:24.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.428+0000 7f75cd200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 0x7f75c81f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.428+0000 7f75cd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75c80ce1b0 0x7f75c81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.428+0000 7f75cd200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75c81554d0 0x7f75c8144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.428+0000 7f75cd200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f75c815bd50 con 0x7f75c80ce1b0 2024-04-20T21:17:24.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.428+0000 7f75cd200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75c815ba70 con 0x7f75c80cb490 2024-04-20T21:17:24.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7f75c7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 0x7f75c81f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff9400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd8004ba0 0x7fdff4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7f75c7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 0x7f75c81f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56830/0 (socket says 172.21.15.102:56830) 2024-04-20T21:17:24.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7f75c7400640 1 -- 172.21.15.102:0/3850733823 learned_addr learned my addr 172.21.15.102:0/3850733823 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdff41560c0 0x7fdff4157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f75c806d2f0 con 0x7f75c81554d0 2024-04-20T21:17:24.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fadd7400640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6804/1109454637,v1:172.21.15.193:6806/1109454637] conn(0x7fadb80760b0 0x7fadb80784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7f75c7e00640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75c81554d0 0x7f75c8144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff2800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd8004ba0 0x7fdff4156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff40c7d60 0x7fdff41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff9400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40d16b0 con 0x7fdff41560c0 2024-04-20T21:17:24.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff2800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd8004ba0 0x7fdff4156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55322/0 (socket says 172.21.15.102:55322) 2024-04-20T21:17:24.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff9400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40d13d0 con 0x7fdff40c7d60 2024-04-20T21:17:24.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff2800640 1 -- 172.21.15.102:0/1620649036 learned_addr learned my addr 172.21.15.102:0/1620649036 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.429+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdff40751d0 con 0x7fdfd8004ba0 2024-04-20T21:17:24.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7fdff3200640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff40c7d60 0x7fdff41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7f75c6a00640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75c80ce1b0 0x7f75c81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7fadd7400640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6804/1109454637,v1:172.21.15.193:6806/1109454637] conn(0x7fadb80760b0 0x7fadb80784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7fdfe9e00640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdff41560c0 0x7fdff4157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7fadd6000640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fadb806c940 0x7fadb806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7fdff2800640 1 -- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff40c7d60 msgr2=0x7fdff41584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7fdff2800640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff40c7d60 0x7fdff41584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7fdff2800640 1 -- 172.21.15.102:0/1620649036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdff41560c0 msgr2=0x7fdff4157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7fdff2800640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdff41560c0 0x7fdff4157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.430+0000 7fdff2800640 1 -- 172.21.15.102:0/1620649036 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff4156c30 con 0x7fdfd8004ba0 2024-04-20T21:17:24.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7f75c7e00640 1 -- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 msgr2=0x7f75c81f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7f75c7e00640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75c80cb490 0x7f75c81f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7f75c7e00640 1 -- 172.21.15.102:0/3850733823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75c80ce1b0 msgr2=0x7f75c81563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7f75c7e00640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75c80ce1b0 0x7f75c81563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7f75c7e00640 1 -- 172.21.15.102:0/3850733823 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c8144cd0 con 0x7f75c81554d0 2024-04-20T21:17:24.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7f75c6a00640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75c80ce1b0 0x7f75c81563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:24.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 <== osd.5 v2:172.21.15.193:6804/1109454637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fadd8070f30 con 0x7fadb80760b0 2024-04-20T21:17:24.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7fdfe9e00640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdff41560c0 0x7fdff4157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:24.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7f75c7e00640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75c81554d0 0x7f75c8144790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f75c0033000 tx=0x7f75c0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0044020 con 0x7f75c81554d0 2024-04-20T21:17:24.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c003f670 con 0x7f75c81554d0 2024-04-20T21:17:24.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0039040 con 0x7f75c81554d0 2024-04-20T21:17:24.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c8159bf0 con 0x7f75c81554d0 2024-04-20T21:17:24.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c8159350 con 0x7f75c81554d0 2024-04-20T21:17:24.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.431+0000 7fdff2800640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd8004ba0 0x7fdff4156f10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005040 tx=0x7fdfe4001650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4039040 con 0x7fdfd8004ba0 2024-04-20T21:17:24.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe4042070 con 0x7fdfd8004ba0 2024-04-20T21:17:24.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff41ed440 con 0x7fdfd8004ba0 2024-04-20T21:17:24.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4001bf0 con 0x7fdfd8004ba0 2024-04-20T21:17:24.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.432+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41ed700 con 0x7fdfd8004ba0 2024-04-20T21:17:24.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.436+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdff4074340 con 0x7fdfd8004ba0 2024-04-20T21:17:24.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.436+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfe4003b70 con 0x7fdfd8004ba0 2024-04-20T21:17:24.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.437+0000 7fdfeb400640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdfd006c920 0x7fdfd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.437+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdfe40ad800 con 0x7fdfd8004ba0 2024-04-20T21:17:24.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.437+0000 7fdfe9e00640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdfd006c920 0x7fdfd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.434+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75c8155030 con 0x7f75c81554d0 2024-04-20T21:17:24.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.438+0000 7fdfeb400640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6820/2963386825,v1:172.21.15.193:6823/2963386825] conn(0x7fdfd0075fa0 0x7fdfd00783e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.438+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 --> [v2:172.21.15.193:6820/2963386825,v1:172.21.15.193:6823/2963386825] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdfe403ae50 con 0x7fdfd0075fa0 2024-04-20T21:17:24.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.438+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdfe40b4050 con 0x7fdfd8004ba0 2024-04-20T21:17:24.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.438+0000 7fdfe9e00640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdfd006c920 0x7fdfd006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdff414f6b0 tx=0x7fdfdc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.440+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75c0047020 con 0x7f75c81554d0 2024-04-20T21:17:24.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.440+0000 7fadd6000640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fadb806c940 0x7fadb806ee00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fadd0174b30 tx=0x7fadc8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.440+0000 7fdff3200640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6820/2963386825,v1:172.21.15.193:6823/2963386825] conn(0x7fdfd0075fa0 0x7fdfd00783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.442+0000 7fdff3200640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6820/2963386825,v1:172.21.15.193:6823/2963386825] conn(0x7fdfd0075fa0 0x7fdfd00783e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.442+0000 7f75bf400640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f75a806c920 0x7f75a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.442+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f75c00aeb30 con 0x7f75c81554d0 2024-04-20T21:17:24.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.442+0000 7f75bf400640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6816/1341503027,v1:172.21.15.193:6817/1341503027] conn(0x7f75a8076010 0x7f75a8078450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.443+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 <== osd.7 v2:172.21.15.193:6820/2963386825 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfe403ae50 con 0x7fdfd0075fa0 2024-04-20T21:17:24.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.443+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 --> [v2:172.21.15.193:6816/1341503027,v1:172.21.15.193:6817/1341503027] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75c00352a0 con 0x7f75a8076010 2024-04-20T21:17:24.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.443+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f75c00aeee0 con 0x7f75c81554d0 2024-04-20T21:17:24.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.443+0000 7f75c7400640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f75a806c920 0x7f75a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.443+0000 7f75c6a00640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6816/1341503027,v1:172.21.15.193:6817/1341503027] conn(0x7f75a8076010 0x7f75a8078450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.444+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 --> [v2:172.21.15.102:6800/100719841,v1:172.21.15.102:6801/100719841] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f876c0c9830 con 0x7f8748075f60 2024-04-20T21:17:24.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.447+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 --> [v2:172.21.15.102:6817/1144653737,v1:172.21.15.102:6819/1144653737] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1404002dd0 con 0x7f140c076130 2024-04-20T21:17:24.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.447+0000 7f75c6a00640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6816/1341503027,v1:172.21.15.193:6817/1341503027] conn(0x7f75a8076010 0x7f75a8078450 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.447+0000 7f75c7400640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f75a806c920 0x7f75a806ede0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75b8000f60 tx=0x7f75b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.447+0000 7f8763e00640 1 -- 172.21.15.102:0/3965422801 <== osd.1 v2:172.21.15.102:6800/100719841 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f876c0c9830 con 0x7f8748075f60 2024-04-20T21:17:24.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.447+0000 7f1423400640 1 -- 172.21.15.102:0/1598010015 <== osd.2 v2:172.21.15.102:6817/1144653737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1404002dd0 con 0x7f140c076130 2024-04-20T21:17:24.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:6817/1144653737,v1:172.21.15.102:6819/1144653737] conn(0x7f140c076130 msgr2=0x7f140c078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f1431e00640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:6817/1144653737,v1:172.21.15.102:6819/1144653737] conn(0x7f140c076130 0x7f140c078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f140c06c9c0 msgr2=0x7f140c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f1431e00640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f140c06c9c0 0x7f140c06ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f142c06a340 tx=0x7f142c0ac000 comp rx=0 tx=0).stop 2024-04-20T21:17:24.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1424050900 msgr2=0x7f14240c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f1431e00640 1 --2- 172.21.15.102:0/1598010015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1424050900 0x7f14240c71e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b410 tx=0x7f141c00b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 <== osd.6 v2:172.21.15.193:6816/1341503027 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75c00352a0 con 0x7f75a8076010 2024-04-20T21:17:24.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f142b400640 1 -- 172.21.15.102:0/1598010015 reap_dead start 2024-04-20T21:17:24.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 >> [v2:172.21.15.102:6800/100719841,v1:172.21.15.102:6801/100719841] conn(0x7f8748075f60 msgr2=0x7f87480783a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 shutdown_connections 2024-04-20T21:17:24.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 >> 172.21.15.102:0/1598010015 conn(0x7f1424054510 msgr2=0x7f142404ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.448+0000 7f8771800640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.102:6800/100719841,v1:172.21.15.102:6801/100719841] conn(0x7f8748075f60 0x7f87480783a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.449+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 shutdown_connections 2024-04-20T21:17:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.449+0000 7f1431e00640 1 -- 172.21.15.102:0/1598010015 wait complete. 2024-04-20T21:17:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.449+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874806c8f0 msgr2=0x7f874806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.449+0000 7f8771800640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874806c8f0 0x7f874806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8754000f60 tx=0x7f875400b210 comp rx=0 tx=0).stop 2024-04-20T21:17:24.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.449+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f876c170600 msgr2=0x7f876c16a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.449+0000 7f8771800640 1 --2- 172.21.15.102:0/3965422801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f876c170600 0x7f876c16a360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f876400cc60 tx=0x7f876400a650 comp rx=0 tx=0).stop 2024-04-20T21:17:24.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.449+0000 7f8770e00640 1 -- 172.21.15.102:0/3965422801 reap_dead start 2024-04-20T21:17:24.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.450+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 --> [v2:172.21.15.102:6808/1191557030,v1:172.21.15.102:6809/1191557030] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f61a0002dd0 con 0x7f61a80761f0 2024-04-20T21:17:24.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.450+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 shutdown_connections 2024-04-20T21:17:24.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.450+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 >> 172.21.15.102:0/3965422801 conn(0x7f876c074ea0 msgr2=0x7f876c071870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.450+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 shutdown_connections 2024-04-20T21:17:24.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.450+0000 7f8771800640 1 -- 172.21.15.102:0/3965422801 wait complete. 2024-04-20T21:17:24.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.451+0000 7f61c4c00640 1 -- 172.21.15.102:0/4265435836 <== osd.0 v2:172.21.15.102:6808/1191557030 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f61a0002dd0 con 0x7f61a80761f0 2024-04-20T21:17:24.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.451+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 >> [v2:172.21.15.102:6808/1191557030,v1:172.21.15.102:6809/1191557030] conn(0x7f61a80761f0 msgr2=0x7f61a8078630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.451+0000 7f61cfc00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.102:6808/1191557030,v1:172.21.15.102:6809/1191557030] conn(0x7f61a80761f0 0x7f61a8078630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.452+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f61a806caf0 msgr2=0x7f61a806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.452+0000 7f61cfc00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f61a806caf0 0x7f61a806efb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61c00013c0 tx=0x7f61c000a040 comp rx=0 tx=0).stop 2024-04-20T21:17:24.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.452+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c8153280 msgr2=0x7f61c814f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.452+0000 7f61cfc00640 1 --2- 172.21.15.102:0/4265435836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c8153280 0x7f61c814f8e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f61c8069d00 tx=0x7f61b400fc00 comp rx=0 tx=0).stop 2024-04-20T21:17:24.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.452+0000 7f61cdc00640 1 -- 172.21.15.102:0/4265435836 reap_dead start 2024-04-20T21:17:24.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.452+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 shutdown_connections 2024-04-20T21:17:24.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.452+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 >> 172.21.15.102:0/4265435836 conn(0x7f61c80cc890 msgr2=0x7f61c80cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.452+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 shutdown_connections 2024-04-20T21:17:24.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.453+0000 7f61cfc00640 1 -- 172.21.15.102:0/4265435836 wait complete. 2024-04-20T21:17:24.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.462+0000 7f5a0be00640 1 -- 172.21.15.102:0/2192469739 --> [v2:172.21.15.102:6816/2815998409,v1:172.21.15.102:6818/2815998409] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5a20075c90 con 0x7f5a000760b0 2024-04-20T21:17:24.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.463+0000 7f5a16a00640 1 -- 172.21.15.102:0/2192469739 <== osd.3 v2:172.21.15.102:6816/2815998409 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5a20075c90 con 0x7f5a000760b0 2024-04-20T21:17:24.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 --> [v2:172.21.15.193:6800/2469990595,v1:172.21.15.193:6801/2469990595] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe640c6cc0 con 0x7fbe44076010 2024-04-20T21:17:24.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7fadd7e00640 1 -- 172.21.15.102:0/170594620 --> [v2:172.21.15.193:6804/1109454637,v1:172.21.15.193:6806/1109454637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fadd00d9380 con 0x7fadb80760b0 2024-04-20T21:17:24.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:6816/2815998409,v1:172.21.15.102:6818/2815998409] conn(0x7f5a000760b0 msgr2=0x7f5a000784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a25600640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.102:6816/2815998409,v1:172.21.15.102:6818/2815998409] conn(0x7f5a000760b0 0x7f5a000784f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a0006c940 msgr2=0x7f5a0006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a25600640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a0006c940 0x7f5a0006ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a100017e0 tx=0x7f5a10009040 comp rx=0 tx=0).stop 2024-04-20T21:17:24.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a201ec790 msgr2=0x7f5a200cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a25600640 1 --2- 172.21.15.102:0/2192469739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a201ec790 0x7f5a200cae00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c4f0 tx=0x7f5a0c004730 comp rx=0 tx=0).stop 2024-04-20T21:17:24.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a1f400640 1 -- 172.21.15.102:0/2192469739 reap_dead start 2024-04-20T21:17:24.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.465+0000 7fbe5be00640 1 -- 172.21.15.102:0/661498208 <== osd.4 v2:172.21.15.193:6800/2469990595 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbe640c6cc0 con 0x7fbe44076010 2024-04-20T21:17:24.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.465+0000 7fadcea00640 1 -- 172.21.15.102:0/170594620 <== osd.5 v2:172.21.15.193:6804/1109454637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fadd00d9380 con 0x7fadb80760b0 2024-04-20T21:17:24.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 shutdown_connections 2024-04-20T21:17:24.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.464+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 >> 172.21.15.102:0/2192469739 conn(0x7f5a200c9390 msgr2=0x7f5a20157880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.465+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 shutdown_connections 2024-04-20T21:17:24.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.465+0000 7f5a25600640 1 -- 172.21.15.102:0/2192469739 wait complete. 2024-04-20T21:17:24.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.465+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6800/2469990595,v1:172.21.15.193:6801/2469990595] conn(0x7fbe44076010 msgr2=0x7fbe44078450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.465+0000 7fbe6b200640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6800/2469990595,v1:172.21.15.193:6801/2469990595] conn(0x7fbe44076010 0x7fbe44078450 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbe4406c920 msgr2=0x7fbe4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fbe6b200640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbe4406c920 0x7fbe4406ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c001e40 tx=0x7fbe5c007d70 comp rx=0 tx=0).stop 2024-04-20T21:17:24.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fadc3e00640 1 -- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6804/1109454637,v1:172.21.15.193:6806/1109454637] conn(0x7fadb80760b0 msgr2=0x7fadb80784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fadc3e00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6804/1109454637,v1:172.21.15.193:6806/1109454637] conn(0x7fadb80760b0 0x7fadb80784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe64160700 msgr2=0x7fbe64160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fbe6b200640 1 --2- 172.21.15.102:0/661498208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe64160700 0x7fbe64160b80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009830 tx=0x7fbe54004700 comp rx=0 tx=0).stop 2024-04-20T21:17:24.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fadc3e00640 1 -- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fadb806c940 msgr2=0x7fadb806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fadc3e00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fadb806c940 0x7fadb806ee00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fadd0174b30 tx=0x7fadc8009040 comp rx=0 tx=0).stop 2024-04-20T21:17:24.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fadc3e00640 1 -- 172.21.15.102:0/170594620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadd0006240 msgr2=0x7fadd0165a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fadc3e00640 1 --2- 172.21.15.102:0/170594620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadd0006240 0x7fadd0165a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fadd8050210 tx=0x7fadd806ae40 comp rx=0 tx=0).stop 2024-04-20T21:17:24.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.466+0000 7fadd7400640 1 -- 172.21.15.102:0/170594620 reap_dead start 2024-04-20T21:17:24.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.467+0000 7fadc3e00640 1 -- 172.21.15.102:0/170594620 shutdown_connections 2024-04-20T21:17:24.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.467+0000 7fadc3e00640 1 -- 172.21.15.102:0/170594620 >> 172.21.15.102:0/170594620 conn(0x7fadd0050180 msgr2=0x7fadd004a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.467+0000 7fadc3e00640 1 -- 172.21.15.102:0/170594620 shutdown_connections 2024-04-20T21:17:24.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.467+0000 7fadc3e00640 1 -- 172.21.15.102:0/170594620 wait complete. 2024-04-20T21:17:24.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.468+0000 7fbe69200640 1 -- 172.21.15.102:0/661498208 reap_dead start 2024-04-20T21:17:24.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.468+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 shutdown_connections 2024-04-20T21:17:24.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.468+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 >> 172.21.15.102:0/661498208 conn(0x7fbe640c9390 msgr2=0x7fbe64157020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.468+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 shutdown_connections 2024-04-20T21:17:24.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.469+0000 7fbe6b200640 1 -- 172.21.15.102:0/661498208 wait complete. 2024-04-20T21:17:24.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.472+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 --> [v2:172.21.15.193:6820/2963386825,v1:172.21.15.193:6823/2963386825] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdff41ed900 con 0x7fdfd0075fa0 2024-04-20T21:17:24.500 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:24.500 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T21:17:24.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.473+0000 7fdfeb400640 1 -- 172.21.15.102:0/1620649036 <== osd.7 v2:172.21.15.193:6820/2963386825 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdff41ed900 con 0x7fdfd0075fa0 2024-04-20T21:17:24.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.473+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6820/2963386825,v1:172.21.15.193:6823/2963386825] conn(0x7fdfd0075fa0 msgr2=0x7fdfd00783e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6820/2963386825,v1:172.21.15.193:6823/2963386825] conn(0x7fdfd0075fa0 0x7fdfd00783e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdfd006c920 msgr2=0x7fdfd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdfd006c920 0x7fdfd006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdff414f6b0 tx=0x7fdfdc008040 comp rx=0 tx=0).stop 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd8004ba0 msgr2=0x7fdff4156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 --2- 172.21.15.102:0/1620649036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfd8004ba0 0x7fdff4156f10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005040 tx=0x7fdfe4001650 comp rx=0 tx=0).stop 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff3200640 1 -- 172.21.15.102:0/1620649036 reap_dead start 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 shutdown_connections 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 >> 172.21.15.102:0/1620649036 conn(0x7fdff40685f0 msgr2=0x7fdff41503b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 shutdown_connections 2024-04-20T21:17:24.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.474+0000 7fdff9400640 1 -- 172.21.15.102:0/1620649036 wait complete. 2024-04-20T21:17:24.503 INFO:teuthology.orchestra.run.smithi102.stdout:55834574861 2024-04-20T21:17:24.503 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T21:17:24.504 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:24.504 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T21:17:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.482+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 --> [v2:172.21.15.193:6816/1341503027,v1:172.21.15.193:6817/1341503027] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75c804fd60 con 0x7f75a8076010 2024-04-20T21:17:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.483+0000 7f75bf400640 1 -- 172.21.15.102:0/3850733823 <== osd.6 v2:172.21.15.193:6816/1341503027 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f75c804fd60 con 0x7f75a8076010 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6816/1341503027,v1:172.21.15.193:6817/1341503027] conn(0x7f75a8076010 msgr2=0x7f75a8078450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6816/1341503027,v1:172.21.15.193:6817/1341503027] conn(0x7f75a8076010 0x7f75a8078450 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f75a806c920 msgr2=0x7f75a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f75a806c920 0x7f75a806ede0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75b8000f60 tx=0x7f75b8009040 comp rx=0 tx=0).stop 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75c81554d0 msgr2=0x7f75c8144790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 --2- 172.21.15.102:0/3850733823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75c81554d0 0x7f75c8144790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f75c0033000 tx=0x7f75c0004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75c7e00640 1 -- 172.21.15.102:0/3850733823 reap_dead start 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 shutdown_connections 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 >> 172.21.15.102:0/3850733823 conn(0x7f75c8073060 msgr2=0x7f75c8147740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.484+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 shutdown_connections 2024-04-20T21:17:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.485+0000 7f75cd200640 1 -- 172.21.15.102:0/3850733823 wait complete. 2024-04-20T21:17:24.507 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:24.507 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T21:17:24.508 INFO:teuthology.orchestra.run.smithi102.stdout:55834574859 2024-04-20T21:17:24.508 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T21:17:24.509 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:24.509 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T21:17:24.510 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:24.511 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T21:17:24.511 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:24.512 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T21:17:24.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.814+0000 7f421ce00640 1 Processor -- start 2024-04-20T21:17:24.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.814+0000 7f421ce00640 1 -- start start 2024-04-20T21:17:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.819+0000 7f421ce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4218155110 0x7f421806ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.819+0000 7f421ce00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f42180c4040 0x7f421815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.819+0000 7f421ce00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421815f3c0 con 0x7f421806f3d0 2024-04-20T21:17:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.819+0000 7f421ce00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f42180c7c30 con 0x7f42180c4040 2024-04-20T21:17:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.819+0000 7f421ce00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f421806ea00 con 0x7f4218155110 2024-04-20T21:17:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.820+0000 7f4217400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4218155110 0x7f421806ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.820+0000 7f4217400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4218155110 0x7f421806ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55330/0 (socket says 172.21.15.102:55330) 2024-04-20T21:17:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.820+0000 7f4217400640 1 -- 172.21.15.102:0/2595822511 learned_addr learned my addr 172.21.15.102:0/2595822511 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.820+0000 7f4217e00640 1 --2- 172.21.15.102:0/2595822511 >> v2:172.21.15.193:3300/0 conn(0x7f42180c4040 0x7f421815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.821+0000 7f4217e00640 1 -- 172.21.15.102:0/2595822511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4218155110 msgr2=0x7f421806ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.821+0000 7f4217e00640 1 --2- 172.21.15.102:0/2595822511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4218155110 0x7f421806ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.821+0000 7f4217e00640 1 -- 172.21.15.102:0/2595822511 >> v1:172.21.15.102:6789/0 conn(0x7f421806f3d0 legacy=0x7f421815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.821+0000 7f4217e00640 1 -- 172.21.15.102:0/2595822511 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421806e6d0 con 0x7f42180c4040 2024-04-20T21:17:24.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.822+0000 7f9222200640 1 Processor -- start 2024-04-20T21:17:24.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.822+0000 7f9222200640 1 -- start start 2024-04-20T21:17:24.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.824+0000 7f4217e00640 1 --2- 172.21.15.102:0/2595822511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421815ec90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f42180c7db0 tx=0x7f421002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf7b5c317a06303 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.824+0000 7f9222200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f921c0cb680 0x7f921c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.825+0000 7f9222200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f921c073c00 0x7f921c152db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.825+0000 7f9222200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f921c15c5d0 con 0x7f921c0cba60 2024-04-20T21:17:24.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.825+0000 7f9222200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f921c0c74f0 con 0x7f921c0cb680 2024-04-20T21:17:24.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.825+0000 7f9222200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f921c0c5bb0 con 0x7f921c073c00 2024-04-20T21:17:24.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.825+0000 7f4216000640 1 -- 172.21.15.102:0/2595822511 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210039070 con 0x7f42180c4040 2024-04-20T21:17:24.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.825+0000 7f4216000640 1 -- 172.21.15.102:0/2595822511 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421002cc60 con 0x7f42180c4040 2024-04-20T21:17:24.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.825+0000 7f4216000640 1 -- 172.21.15.102:0/2595822511 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210034660 con 0x7f42180c4040 2024-04-20T21:17:24.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.826+0000 7f9221800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f921c0cba60 0x7f921c15ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38698/0 (socket says 172.21.15.102:38698) 2024-04-20T21:17:24.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.826+0000 7f9221800640 1 -- 172.21.15.102:0/1146529854 learned_addr learned my addr 172.21.15.102:0/1146529854 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:24.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.826+0000 7fb9ba200640 1 Processor -- start 2024-04-20T21:17:24.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.826+0000 7fb9ba200640 1 -- start start 2024-04-20T21:17:24.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.826+0000 7f421ce00640 1 -- 172.21.15.102:0/2595822511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 msgr2=0x7f421815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.826+0000 7f421ce00640 1 --2- 172.21.15.102:0/2595822511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421815ec90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f42180c7db0 tx=0x7f421002beb0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f1d9c600640 1 Processor -- start 2024-04-20T21:17:24.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f421ce00640 1 -- 172.21.15.102:0/2595822511 shutdown_connections 2024-04-20T21:17:24.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f421ce00640 1 --2- 172.21.15.102:0/2595822511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421815ec90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f421ce00640 1 --2- 172.21.15.102:0/2595822511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4218155110 0x7f421806ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f421ce00640 1 -- 172.21.15.102:0/2595822511 >> 172.21.15.102:0/2595822511 conn(0x7f42180cc820 msgr2=0x7f42180ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f1d9c600640 1 -- start start 2024-04-20T21:17:24.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f421ce00640 1 -- 172.21.15.102:0/2595822511 shutdown_connections 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.826+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089403990 0 0) 0x7f921c15c5d0 con 0x7f921c0cba60 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f9220e00640 1 --2- 172.21.15.102:0/1146529854 >> v2:172.21.15.102:3300/0 conn(0x7f921c0cb680 0x7f921c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9200003520 con 0x7f921c0cba60 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921be00640 1 --2- 172.21.15.102:0/1146529854 >> v2:172.21.15.193:3300/0 conn(0x7f921c073c00 0x7f921c152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3440943333 0 0) 0x7f9200003520 con 0x7f921c0cba60 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 >> v2:172.21.15.193:3300/0 conn(0x7f921c073c00 msgr2=0x7f921c152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 --2- 172.21.15.102:0/1146529854 >> v2:172.21.15.193:3300/0 conn(0x7f921c073c00 0x7f921c152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 >> v2:172.21.15.102:3300/0 conn(0x7f921c0cb680 msgr2=0x7f921c0736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 --2- 172.21.15.102:0/1146529854 >> v2:172.21.15.102:3300/0 conn(0x7f921c0cb680 0x7f921c0736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c0c8280 con 0x7f921c0cba60 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7f9204001670 con 0x7f921c0cba60 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 >> v1:172.21.15.102:6789/0 conn(0x7f921c0cba60 legacy=0x7f921c15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200003a70 0x7f9200003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9200004800 0x7f9200004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92000051c0 0x7f920001c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f921c0c74f0 con 0x7f92000051c0 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f921c0c5bb0 con 0x7f9200004800 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.827+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f920001d0d0 con 0x7f9200003a70 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f921be00640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9200004800 0x7f9200004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f9221800640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92000051c0 0x7f920001c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f421ce00640 1 -- 172.21.15.102:0/2595822511 wait complete. 2024-04-20T21:17:24.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f921be00640 1 -- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200003a70 msgr2=0x7f9200003e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:17:24.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f921be00640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200003a70 0x7f9200003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f921be00640 1 -- 172.21.15.102:0/1146529854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92000051c0 msgr2=0x7f920001c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f921be00640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92000051c0 0x7f920001c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f921be00640 1 -- 172.21.15.102:0/1146529854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9200003520 con 0x7f9200004800 2024-04-20T21:17:24.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f9221800640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92000051c0 0x7f920001c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:24.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.828+0000 7f9221800640 1 -- 172.21.15.102:0/1146529854 reap_dead start 2024-04-20T21:17:24.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.829+0000 7f1d9c600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9415a930 0x7f1d940c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.829+0000 7f1d9c600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1d940c9eb0 0x7f1d94150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.829+0000 7fb9ba200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb9b4163220 0x7fb9b4159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.829+0000 7f1d9c600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d940c7430 con 0x7f1d9415b2c0 2024-04-20T21:17:24.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.829+0000 7f1d9c600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1d94158650 con 0x7f1d940c9eb0 2024-04-20T21:17:24.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.829+0000 7f1d9c600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1d940c76d0 con 0x7f1d9415a930 2024-04-20T21:17:24.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.829+0000 7f1d99200640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f1d9415b2c0 0x7f1d9406d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49154/0 (socket says 172.21.15.102:49154) 2024-04-20T21:17:24.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f1d99200640 1 -- 172.21.15.102:0/2543229085 learned_addr learned my addr 172.21.15.102:0/2543229085 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:24.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f1d99c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9415a930 0x7f1d940c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f1d9a600640 1 --2- 172.21.15.102:0/2543229085 >> v2:172.21.15.102:3300/0 conn(0x7f1d940c9eb0 0x7f1d94150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f13b9200640 1 Processor -- start 2024-04-20T21:17:24.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7fb9ba200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4163660 0x7fb9b4159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f1d99c00640 1 -- 172.21.15.102:0/2543229085 >> v2:172.21.15.102:3300/0 conn(0x7f1d940c9eb0 msgr2=0x7f1d94150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7fb9ba200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b406d2e0 con 0x7fb9b4162e40 2024-04-20T21:17:24.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f13b9200640 1 -- start start 2024-04-20T21:17:24.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f1d99c00640 1 --2- 172.21.15.102:0/2543229085 >> v2:172.21.15.102:3300/0 conn(0x7f1d940c9eb0 0x7f1d94150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f1d99c00640 1 -- 172.21.15.102:0/2543229085 >> v1:172.21.15.193:6789/0 conn(0x7f1d9415b2c0 legacy=0x7f1d9406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7fb9ba200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b40c7290 con 0x7fb9b4163220 2024-04-20T21:17:24.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.830+0000 7f1d99c00640 1 -- 172.21.15.102:0/2543229085 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d940c79b0 con 0x7f1d9415a930 2024-04-20T21:17:24.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.831+0000 7fb9b2a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb9b4163220 0x7fb9b4159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.831+0000 7fb9b2a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb9b4163220 0x7fb9b4159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51516/0 (socket says 172.21.15.102:51516) 2024-04-20T21:17:24.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.831+0000 7fb9ba200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b40c7570 con 0x7fb9b4163660 2024-04-20T21:17:24.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.831+0000 7fb9b2a00640 1 -- 172.21.15.102:0/1935640215 learned_addr learned my addr 172.21.15.102:0/1935640215 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.831+0000 7f1d9a600640 1 --2- 172.21.15.102:0/2543229085 >> v2:172.21.15.102:3300/0 conn(0x7f1d940c9eb0 0x7f1d94150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:24.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.831+0000 7f921be00640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9200004800 0x7f9200004c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9214007460 tx=0x7f9200005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76aa2c7361f1b5da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.831+0000 7f421ce00640 1 Processor -- start 2024-04-20T21:17:24.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.832+0000 7f421ce00640 1 -- start start 2024-04-20T21:17:24.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.831+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214032070 con 0x7f9200004800 2024-04-20T21:17:24.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.832+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9214036400 con 0x7f9200004800 2024-04-20T21:17:24.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.832+0000 7fb9b3e00640 1 --2- 172.21.15.102:0/1935640215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4163660 0x7fb9b4159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.832+0000 7f1d99c00640 1 --2- 172.21.15.102:0/2543229085 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9415a930 0x7f1d940c8770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d880097a0 tx=0x7f1d8802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f0e54c1af8d943 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.832+0000 7f1d93e00640 1 -- 172.21.15.102:0/2543229085 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88034070 con 0x7f1d9415a930 2024-04-20T21:17:24.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.832+0000 7f1d93e00640 1 -- 172.21.15.102:0/2543229085 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d88030800 con 0x7f1d9415a930 2024-04-20T21:17:24.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.832+0000 7f1d93e00640 1 -- 172.21.15.102:0/2543229085 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88030bd0 con 0x7f1d9415a930 2024-04-20T21:17:24.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f921b400640 1 -- 172.21.15.102:0/1146529854 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214009d60 con 0x7f9200004800 2024-04-20T21:17:24.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f13b9200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f13b4159850 0x7f13b406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7fb9b2a00640 1 -- 172.21.15.102:0/1935640215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4163660 msgr2=0x7fb9b4159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f13b9200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f13b406f3d0 0x7f13b415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f9222200640 1 -- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9200004800 msgr2=0x7f9200004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f9222200640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9200004800 0x7f9200004c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9214007460 tx=0x7f9200005830 comp rx=0 tx=0).stop 2024-04-20T21:17:24.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f13b3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f13b406f3d0 0x7f13b415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f9222200640 1 -- 172.21.15.102:0/1146529854 shutdown_connections 2024-04-20T21:17:24.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f9222200640 1 --2- 172.21.15.102:0/1146529854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9200004800 0x7f9200004c80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f9222200640 1 -- 172.21.15.102:0/1146529854 >> 172.21.15.102:0/1146529854 conn(0x7f921c074ea0 msgr2=0x7f921c075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f13b9200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b406c2d0 con 0x7f13b40c8600 2024-04-20T21:17:24.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f13b9200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f13b40c7c00 con 0x7f13b4159850 2024-04-20T21:17:24.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7f13b9200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f13b406e6d0 con 0x7f13b406f3d0 2024-04-20T21:17:24.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f13b3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f13b406f3d0 0x7f13b415fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56880/0 (socket says 172.21.15.102:56880) 2024-04-20T21:17:24.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f421ce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f421806f3d0 0x7f42181f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f13b3e00640 1 -- 172.21.15.102:0/392507035 learned_addr learned my addr 172.21.15.102:0/392507035 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f421ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421814a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7fb9b2a00640 1 --2- 172.21.15.102:0/1935640215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4163660 0x7fb9b4159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7fb9b2a00640 1 -- 172.21.15.102:0/1935640215 >> v1:172.21.15.193:6789/0 conn(0x7fb9b4162e40 legacy=0x7fb9b406c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.833+0000 7fb9b2a00640 1 -- 172.21.15.102:0/1935640215 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b406d5c0 con 0x7fb9b4163220 2024-04-20T21:17:24.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f421ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4218155110 0x7f42181561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7fb9b3e00640 1 --2- 172.21.15.102:0/1935640215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4163660 0x7fb9b4159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:24.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f421ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f42181554f0 con 0x7f4218155110 2024-04-20T21:17:24.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f13b3e00640 1 -- 172.21.15.102:0/392507035 >> v2:172.21.15.102:3300/0 conn(0x7f13b4159850 msgr2=0x7f13b406ee90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:17:24.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f13b3e00640 1 --2- 172.21.15.102:0/392507035 >> v2:172.21.15.102:3300/0 conn(0x7f13b4159850 0x7f13b406ee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f13b3e00640 1 -- 172.21.15.102:0/392507035 >> v1:172.21.15.102:6789/0 conn(0x7f13b40c8600 legacy=0x7f13b415f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f13b3e00640 1 -- 172.21.15.102:0/392507035 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b40c3810 con 0x7f13b406f3d0 2024-04-20T21:17:24.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f4216a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421814a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f4216a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421814a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56882/0 (socket says 172.21.15.102:56882) 2024-04-20T21:17:24.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.834+0000 7f4216a00640 1 -- 172.21.15.102:0/968934647 learned_addr learned my addr 172.21.15.102:0/968934647 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7f421ce00640 1 -- 172.21.15.102:0/968934647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f42180c8160 con 0x7f42180c4040 2024-04-20T21:17:24.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7f421ce00640 1 -- 172.21.15.102:0/968934647 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4218155670 con 0x7f421806f3d0 2024-04-20T21:17:24.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7f4217e00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4218155110 0x7f42181561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7f4216a00640 1 -- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f421806f3d0 msgr2=0x7f42181f5200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T21:17:24.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7fb9b2a00640 1 --2- 172.21.15.102:0/1935640215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163220 0x7fb9b4159440 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40c7410 tx=0x7fb9a402b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c3e491e899e1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7f4216a00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f421806f3d0 0x7f42181f5200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7f4216a00640 1 -- 172.21.15.102:0/968934647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4218155110 msgr2=0x7f42181561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7f4216a00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4218155110 0x7f42181561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.835+0000 7f4216a00640 1 -- 172.21.15.102:0/968934647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42181f2760 con 0x7f42180c4040 2024-04-20T21:17:24.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f4216a00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421814a060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f420400c430 tx=0x7f420400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7fb9b2000640 1 -- 172.21.15.102:0/1935640215 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4037070 con 0x7fb9b4163220 2024-04-20T21:17:24.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f9222200640 1 -- 172.21.15.102:0/1146529854 shutdown_connections 2024-04-20T21:17:24.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f9222200640 1 -- 172.21.15.102:0/1146529854 wait complete. 2024-04-20T21:17:24.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7fb9b2000640 1 -- 172.21.15.102:0/1935640215 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a4030710 con 0x7fb9b4163220 2024-04-20T21:17:24.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7fb9b2000640 1 -- 172.21.15.102:0/1935640215 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4030ae0 con 0x7fb9b4163220 2024-04-20T21:17:24.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f13b3e00640 1 --2- 172.21.15.102:0/392507035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b415fab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13b40ca6d0 tx=0x7f13ac02b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be972650704f1575 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f13b2a00640 1 -- 172.21.15.102:0/392507035 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac035020 con 0x7f13b406f3d0 2024-04-20T21:17:24.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f1d9c600640 1 -- 172.21.15.102:0/2543229085 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9415a930 msgr2=0x7f1d940c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f1d9c600640 1 --2- 172.21.15.102:0/2543229085 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9415a930 0x7f1d940c8770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d880097a0 tx=0x7f1d8802b940 comp rx=0 tx=0).stop 2024-04-20T21:17:24.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7fb9ba200640 1 -- 172.21.15.102:0/1935640215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163220 msgr2=0x7fb9b4159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7fb9ba200640 1 --2- 172.21.15.102:0/1935640215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163220 0x7fb9b4159440 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40c7410 tx=0x7fb9a402b7c0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f420f400640 1 -- 172.21.15.102:0/968934647 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204011790 con 0x7f42180c4040 2024-04-20T21:17:24.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f420f400640 1 -- 172.21.15.102:0/968934647 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4204017070 con 0x7f42180c4040 2024-04-20T21:17:24.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f420f400640 1 -- 172.21.15.102:0/968934647 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204011bb0 con 0x7f42180c4040 2024-04-20T21:17:24.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f13b2a00640 1 -- 172.21.15.102:0/392507035 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ac031890 con 0x7f13b406f3d0 2024-04-20T21:17:24.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.836+0000 7f13b2a00640 1 -- 172.21.15.102:0/392507035 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac031c60 con 0x7f13b406f3d0 2024-04-20T21:17:24.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f1d9c600640 1 -- 172.21.15.102:0/2543229085 shutdown_connections 2024-04-20T21:17:24.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f1d9c600640 1 --2- 172.21.15.102:0/2543229085 >> v2:172.21.15.102:3300/0 conn(0x7f1d940c9eb0 0x7f1d94150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f1d9c600640 1 --2- 172.21.15.102:0/2543229085 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d9415a930 0x7f1d940c8770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f1d9c600640 1 -- 172.21.15.102:0/2543229085 >> 172.21.15.102:0/2543229085 conn(0x7f1d940c9390 msgr2=0x7f1d940c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f1d9c600640 1 -- 172.21.15.102:0/2543229085 shutdown_connections 2024-04-20T21:17:24.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.837+0000 7f1d9c600640 1 -- 172.21.15.102:0/2543229085 wait complete. 2024-04-20T21:17:24.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f13b9200640 1 -- 172.21.15.102:0/392507035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 msgr2=0x7f13b415fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f13b9200640 1 --2- 172.21.15.102:0/392507035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b415fab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f13b40ca6d0 tx=0x7f13ac02b930 comp rx=0 tx=0).stop 2024-04-20T21:17:24.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f13b9200640 1 -- 172.21.15.102:0/392507035 shutdown_connections 2024-04-20T21:17:24.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f13b9200640 1 --2- 172.21.15.102:0/392507035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b415fab0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f13b9200640 1 --2- 172.21.15.102:0/392507035 >> v2:172.21.15.102:3300/0 conn(0x7f13b4159850 0x7f13b406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f13b9200640 1 -- 172.21.15.102:0/392507035 >> 172.21.15.102:0/392507035 conn(0x7f13b40cc820 msgr2=0x7f13b40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f13b9200640 1 -- 172.21.15.102:0/392507035 shutdown_connections 2024-04-20T21:17:24.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f421ce00640 1 -- 172.21.15.102:0/968934647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4218157f30 con 0x7f42180c4040 2024-04-20T21:17:24.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f421ce00640 1 -- 172.21.15.102:0/968934647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42181584f0 con 0x7f42180c4040 2024-04-20T21:17:24.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.839+0000 7fb9ba200640 1 -- 172.21.15.102:0/1935640215 shutdown_connections 2024-04-20T21:17:24.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.839+0000 7fb9ba200640 1 --2- 172.21.15.102:0/1935640215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4163660 0x7fb9b4159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.839+0000 7fb9ba200640 1 --2- 172.21.15.102:0/1935640215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163220 0x7fb9b4159440 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.839+0000 7fb9ba200640 1 -- 172.21.15.102:0/1935640215 >> 172.21.15.102:0/1935640215 conn(0x7fb9b40c9390 msgr2=0x7fb9b40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.839+0000 7f9222200640 1 Processor -- start 2024-04-20T21:17:24.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.840+0000 7f1d9c600640 1 Processor -- start 2024-04-20T21:17:24.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.840+0000 7f9222200640 1 -- start start 2024-04-20T21:17:24.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.840+0000 7f7940600640 1 Processor -- start 2024-04-20T21:17:24.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.840+0000 7f1d9c600640 1 -- start start 2024-04-20T21:17:24.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.838+0000 7f13b9200640 1 -- 172.21.15.102:0/392507035 wait complete. 2024-04-20T21:17:24.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.840+0000 7fb9ba200640 1 -- 172.21.15.102:0/1935640215 shutdown_connections 2024-04-20T21:17:24.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.841+0000 7fb9ba200640 1 -- 172.21.15.102:0/1935640215 wait complete. 2024-04-20T21:17:24.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.840+0000 7f13b9200640 1 Processor -- start 2024-04-20T21:17:24.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.841+0000 7f13b9200640 1 -- start start 2024-04-20T21:17:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.841+0000 7f1d9c600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d940c9eb0 0x7f1d9415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.841+0000 7f1d9c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9415a930 0x7f1d9415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.841+0000 7f1d9c600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9415b2c0 0x7f1d9415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.841+0000 7f1d9c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d940c8250 con 0x7f1d9415a930 2024-04-20T21:17:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.841+0000 7f1d9c600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d940c7f70 con 0x7f1d9415b2c0 2024-04-20T21:17:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.841+0000 7f1d9c600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9406db00 con 0x7f1d940c9eb0 2024-04-20T21:17:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f9222200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200004800 0x7f921c0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f9222200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f921c06c800 0x7f921c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f9222200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f921c1441a0 0x7f921c144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f9222200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f921c0c8b20 con 0x7f921c06c800 2024-04-20T21:17:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f9222200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f921c0c8840 con 0x7f921c1441a0 2024-04-20T21:17:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f9222200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f921c0c8e00 con 0x7f9200004800 2024-04-20T21:17:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f1d99c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d940c9eb0 0x7f1d9415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f835e000640 1 Processor -- start 2024-04-20T21:17:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f1d99c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d940c9eb0 0x7f1d9415bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55396/0 (socket says 172.21.15.102:55396) 2024-04-20T21:17:24.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f13b9200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b40725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f835e000640 1 -- start start 2024-04-20T21:17:24.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f1d9a600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9415b2c0 0x7f1d9415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f1d9a600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9415b2c0 0x7f1d9415d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56888/0 (socket says 172.21.15.102:56888) 2024-04-20T21:17:24.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f13b9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13b40c8600 0x7f13b40cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f1d9a600640 1 -- 172.21.15.102:0/3599169512 learned_addr learned my addr 172.21.15.102:0/3599169512 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f921be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f921c06c800 0x7f921c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f1d99200640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9415a930 0x7f1d9415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f921be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f921c06c800 0x7f921c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51548/0 (socket says 172.21.15.102:51548) 2024-04-20T21:17:24.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f921be00640 1 -- 172.21.15.102:0/2634283324 learned_addr learned my addr 172.21.15.102:0/2634283324 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7f7574e00640 1 Processor -- start 2024-04-20T21:17:24.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f7574e00640 1 -- start start 2024-04-20T21:17:24.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f7940600640 1 -- start start 2024-04-20T21:17:24.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f7940600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cb980 0x7f793815e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f7940600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f79380cdb30 0x7f79380cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f7940600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79380ccdd0 con 0x7f79380cbd60 2024-04-20T21:17:24.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f7940600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f793815f7e0 con 0x7f79380cdb30 2024-04-20T21:17:24.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f7940600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f793815ef40 con 0x7f79380cb980 2024-04-20T21:17:24.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.843+0000 7fb9ba200640 1 Processor -- start 2024-04-20T21:17:24.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7fb9ba200640 1 -- start start 2024-04-20T21:17:24.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f13aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13b40c8600 0x7f13b40cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f13b3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b40725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f13b3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b40725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56902/0 (socket says 172.21.15.102:56902) 2024-04-20T21:17:24.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f13b3400640 1 -- 172.21.15.102:0/2386627974 learned_addr learned my addr 172.21.15.102:0/2386627974 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f13b9200640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13b4159850 0x7f13b4073010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f793dc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cb980 0x7f793815e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f793dc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cb980 0x7f793815e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55410/0 (socket says 172.21.15.102:55410) 2024-04-20T21:17:24.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f793dc00640 1 -- 172.21.15.102:0/4175568906 learned_addr learned my addr 172.21.15.102:0/4175568906 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f793e600640 1 --2- 172.21.15.102:0/4175568906 >> v2:172.21.15.102:3300/0 conn(0x7f79380cdb30 0x7f79380cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f9221800640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f921c1441a0 0x7f921c144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f1d9a600640 1 -- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d940c9eb0 msgr2=0x7f1d9415bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f1d9a600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d940c9eb0 0x7f1d9415bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f1d9a600640 1 -- 172.21.15.102:0/3599169512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9415a930 msgr2=0x7f1d9415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f1d9a600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9415a930 0x7f1d9415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.844+0000 7f1d9a600640 1 -- 172.21.15.102:0/3599169512 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d94160be0 con 0x7f1d9415b2c0 2024-04-20T21:17:24.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f9220e00640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200004800 0x7f921c0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7fb9ba200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4162e40 0x7fb9b415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f420f400640 1 -- 172.21.15.102:0/968934647 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f420401c460 con 0x7f42180c4040 2024-04-20T21:17:24.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.842+0000 7f421ce00640 1 -- 172.21.15.102:0/968934647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421806c120 con 0x7f42180c4040 2024-04-20T21:17:24.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f420f400640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41f806c9c0 0x7f41f806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f420f400640 1 -- 172.21.15.102:0/968934647 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f420408bca0 con 0x7f42180c4040 2024-04-20T21:17:24.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f13b3e00640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13b4159850 0x7f13b4073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13b40c8160 con 0x7f13b40c8600 2024-04-20T21:17:24.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13b406e970 con 0x7f13b406f3d0 2024-04-20T21:17:24.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f13b40c3460 con 0x7f13b4159850 2024-04-20T21:17:24.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7fb9ba200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b4163220 0x7fb9b41fdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7fb9ba200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163660 0x7fb9b40c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7fb9ba200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b40c9080 con 0x7fb9b4163660 2024-04-20T21:17:24.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7fb9ba200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b406db80 con 0x7fb9b4163220 2024-04-20T21:17:24.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7fb9ba200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b40c7a00 con 0x7fb9b4162e40 2024-04-20T21:17:24.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f921be00640 1 -- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200004800 msgr2=0x7f921c0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f1d9a600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9415b2c0 0x7f1d9415d1c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00ceb0 tx=0x7f1d8c00a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f921be00640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200004800 0x7f921c0cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f7574e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f75701555e0 0x7f75700cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f835e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f835815ea20 0x7f835806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4162e40 0x7fb9b415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4162e40 0x7fb9b415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55436/0 (socket says 172.21.15.102:55436) 2024-04-20T21:17:24.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7fb9b3400640 1 -- 172.21.15.102:0/2237511363 learned_addr learned my addr 172.21.15.102:0/2237511363 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f7574e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f757015b750 0x7f757015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f7574e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757015b250 con 0x7f75700cc300 2024-04-20T21:17:24.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f7574e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f7570073a90 con 0x7f757015b750 2024-04-20T21:17:24.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f7574e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f757015e840 con 0x7f75701555e0 2024-04-20T21:17:24.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f921be00640 1 -- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f921c1441a0 msgr2=0x7f921c144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f793dc00640 1 -- 172.21.15.102:0/4175568906 >> v2:172.21.15.102:3300/0 conn(0x7f79380cdb30 msgr2=0x7f79380cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f793dc00640 1 --2- 172.21.15.102:0/4175568906 >> v2:172.21.15.102:3300/0 conn(0x7f79380cdb30 0x7f79380cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f793dc00640 1 -- 172.21.15.102:0/4175568906 >> v1:172.21.15.193:6789/0 conn(0x7f79380cbd60 legacy=0x7f79380cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f793dc00640 1 -- 172.21.15.102:0/4175568906 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793815f220 con 0x7f79380cb980 2024-04-20T21:17:24.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f793e600640 1 --2- 172.21.15.102:0/4175568906 >> v2:172.21.15.102:3300/0 conn(0x7f79380cdb30 0x7f79380cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:24.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f835e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83580c8f20 con 0x7f835815f2b0 2024-04-20T21:17:24.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f835e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835806cb80 con 0x7f835815ee00 2024-04-20T21:17:24.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f835e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f835806ce60 con 0x7f835815ea20 2024-04-20T21:17:24.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f921be00640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f921c1441a0 0x7f921c144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f4217400640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41f806c9c0 0x7f41f806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f756de00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f75701555e0 0x7f75700cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f756de00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f75701555e0 0x7f75700cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56920/0 (socket says 172.21.15.102:56920) 2024-04-20T21:17:24.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f756de00640 1 -- 172.21.15.102:0/2286181848 learned_addr learned my addr 172.21.15.102:0/2286181848 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.845+0000 7f921be00640 1 -- 172.21.15.102:0/2634283324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c1ecc80 con 0x7f921c06c800 2024-04-20T21:17:24.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7f9220e00640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200004800 0x7f921c0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:24.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f756e800640 1 --2- 172.21.15.102:0/2286181848 >> v2:172.21.15.102:3300/0 conn(0x7f757015b750 0x7f757015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f793dc00640 1 --2- 172.21.15.102:0/4175568906 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cb980 0x7f793815e890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7930009e50 tx=0x7f793002bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68e6e0e8780fa52 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f8357400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f835815ea20 0x7f835806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f8357400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f835815ea20 0x7f835806d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56930/0 (socket says 172.21.15.102:56930) 2024-04-20T21:17:24.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f8357e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f835815f2b0 0x7f8358154e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38718/0 (socket says 172.21.15.102:38718) 2024-04-20T21:17:24.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f8357400640 1 -- 172.21.15.102:0/2217319732 learned_addr learned my addr 172.21.15.102:0/2217319732 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7fb9b2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b4163220 0x7fb9b41fdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.846+0000 7fb9b3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163660 0x7fb9b40c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f13b3e00640 1 -- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 msgr2=0x7f13b40725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f13b3e00640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b40725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f13b3e00640 1 -- 172.21.15.102:0/2386627974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13b40c8600 msgr2=0x7f13b40cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f13b3e00640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13b40c8600 0x7f13b40cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7fb9b2a00640 1 -- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4162e40 msgr2=0x7fb9b415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f13b3e00640 1 -- 172.21.15.102:0/2386627974 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b414c0f0 con 0x7f13b4159850 2024-04-20T21:17:24.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f7937e00640 1 -- 172.21.15.102:0/4175568906 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793002cb20 con 0x7f79380cb980 2024-04-20T21:17:24.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7fb9b2a00640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4162e40 0x7fb9b415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7fb9b2a00640 1 -- 172.21.15.102:0/2237511363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163660 msgr2=0x7fb9b40c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f13aaa00640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13b40c8600 0x7f13b40cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:24.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f756e800640 1 -- 172.21.15.102:0/2286181848 >> v2:172.21.15.193:3300/0 conn(0x7f75701555e0 msgr2=0x7f75700cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:24.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f756e800640 1 --2- 172.21.15.102:0/2286181848 >> v2:172.21.15.193:3300/0 conn(0x7f75701555e0 0x7f75700cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f756e800640 1 -- 172.21.15.102:0/2286181848 >> v1:172.21.15.193:6789/0 conn(0x7f75700cc300 legacy=0x7f75700cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f7937e00640 1 -- 172.21.15.102:0/4175568906 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793002ccc0 con 0x7f79380cb980 2024-04-20T21:17:24.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f7937e00640 1 -- 172.21.15.102:0/4175568906 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930034610 con 0x7f79380cb980 2024-04-20T21:17:24.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f13b3400640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b40725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:24.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f8356000640 1 -- 172.21.15.102:0/2217319732 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696705730 0 0) 0x7f83580c8f20 con 0x7f835815f2b0 2024-04-20T21:17:24.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f8356000640 1 -- 172.21.15.102:0/2217319732 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f833c003520 con 0x7f835815f2b0 2024-04-20T21:17:24.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7f921be00640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f921c06c800 0x7f921c0cda80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f921402c8a0 tx=0x7f9214004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.847+0000 7fb9b2a00640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163660 0x7fb9b40c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7fb9b2a00640 1 -- 172.21.15.102:0/2237511363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b415ceb0 con 0x7fb9b4163220 2024-04-20T21:17:24.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f756e800640 1 -- 172.21.15.102:0/2286181848 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757015ea90 con 0x7f757015b750 2024-04-20T21:17:24.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7fb9b3400640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4162e40 0x7fb9b415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:24.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f9218c00640 1 -- 172.21.15.102:0/2634283324 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214045030 con 0x7f921c06c800 2024-04-20T21:17:24.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f4217400640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41f806c9c0 0x7f41f806ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4208009940 tx=0x7f4208009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f13b3e00640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13b4159850 0x7f13b4073010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13b406e850 tx=0x7f13ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7fb9b3e00640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163660 0x7fb9b40c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:24.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f8357400640 1 -- 172.21.15.102:0/2217319732 >> v1:172.21.15.193:6789/0 conn(0x7f835815ee00 legacy=0x7f83580c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f8357400640 1 -- 172.21.15.102:0/2217319732 >> v1:172.21.15.102:6789/0 conn(0x7f835815f2b0 legacy=0x7f8358154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f8357400640 1 -- 172.21.15.102:0/2217319732 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835806d140 con 0x7f835815ea20 2024-04-20T21:17:24.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7fb9b2a00640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b4163220 0x7fb9b41fdfe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4032000 tx=0x7fb9a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7fb9ab400640 1 -- 172.21.15.102:0/2237511363 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4040020 con 0x7fb9b4163220 2024-04-20T21:17:24.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7fb9ab400640 1 -- 172.21.15.102:0/2237511363 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a40327b0 con 0x7fb9b4163220 2024-04-20T21:17:24.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7fb9ab400640 1 -- 172.21.15.102:0/2237511363 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4037040 con 0x7fb9b4163220 2024-04-20T21:17:24.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c0cb180 con 0x7f921c06c800 2024-04-20T21:17:24.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c0c95a0 con 0x7f921c06c800 2024-04-20T21:17:24.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f9218c00640 1 -- 172.21.15.102:0/2634283324 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9214003990 con 0x7f921c06c800 2024-04-20T21:17:24.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.848+0000 7f9218c00640 1 -- 172.21.15.102:0/2634283324 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214045030 con 0x7f921c06c800 2024-04-20T21:17:24.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7f1d91600640 1 -- 172.21.15.102:0/3599169512 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c01a070 con 0x7f1d9415b2c0 2024-04-20T21:17:24.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7f1d91600640 1 -- 172.21.15.102:0/3599169512 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00d840 con 0x7f1d9415b2c0 2024-04-20T21:17:24.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7f1d91600640 1 -- 172.21.15.102:0/3599169512 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00dc30 con 0x7f1d9415b2c0 2024-04-20T21:17:24.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7f7940600640 1 -- 172.21.15.102:0/4175568906 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cb980 msgr2=0x7f793815e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7f7940600640 1 --2- 172.21.15.102:0/4175568906 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cb980 0x7f793815e890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7930009e50 tx=0x7f793002bf80 comp rx=0 tx=0).stop 2024-04-20T21:17:24.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7f420f400640 1 -- 172.21.15.102:0/968934647 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f421806c120 con 0x7f42180c4040 2024-04-20T21:17:24.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.849+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d940c5560 con 0x7f1d9415b2c0 2024-04-20T21:17:24.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d940c5b00 con 0x7f1d9415b2c0 2024-04-20T21:17:24.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f7940600640 1 -- 172.21.15.102:0/4175568906 shutdown_connections 2024-04-20T21:17:24.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f7940600640 1 --2- 172.21.15.102:0/4175568906 >> v2:172.21.15.102:3300/0 conn(0x7f79380cdb30 0x7f79380cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f7940600640 1 --2- 172.21.15.102:0/4175568906 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cb980 0x7f793815e890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f7940600640 1 -- 172.21.15.102:0/4175568906 >> 172.21.15.102:0/4175568906 conn(0x7f7938075080 msgr2=0x7f7938075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f7940600640 1 -- 172.21.15.102:0/4175568906 shutdown_connections 2024-04-20T21:17:24.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f7940600640 1 -- 172.21.15.102:0/4175568906 wait complete. 2024-04-20T21:17:24.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b415d250 con 0x7fb9b4163220 2024-04-20T21:17:24.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f13abe00640 1 -- 172.21.15.102:0/2386627974 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0443f0 con 0x7f13b4159850 2024-04-20T21:17:24.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f13abe00640 1 -- 172.21.15.102:0/2386627974 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ac044590 con 0x7f13b4159850 2024-04-20T21:17:24.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f13abe00640 1 -- 172.21.15.102:0/2386627974 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac044940 con 0x7f13b4159850 2024-04-20T21:17:24.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f756e800640 1 --2- 172.21.15.102:0/2286181848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f757015b750 0x7f757015bb60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f757006d580 tx=0x7f755802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe661b99c9ad024 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b414c3d0 con 0x7f13b4159850 2024-04-20T21:17:24.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.850+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41fe0e0 con 0x7f13b4159850 2024-04-20T21:17:24.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.851+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b415d810 con 0x7fb9b4163220 2024-04-20T21:17:24.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.851+0000 7f8357400640 1 --2- 172.21.15.102:0/2217319732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815ea20 0x7f835806d940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8344009870 tx=0x7f834402bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f400199c95264928 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7f7940600640 1 Processor -- start 2024-04-20T21:17:24.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7f8356000640 1 -- 172.21.15.102:0/2217319732 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834402d870 con 0x7f835815ea20 2024-04-20T21:17:24.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f4005290 con 0x7f921c06c800 2024-04-20T21:17:24.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c005290 con 0x7f13b4159850 2024-04-20T21:17:24.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7f8356000640 1 -- 172.21.15.102:0/2217319732 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834402da10 con 0x7f835815ea20 2024-04-20T21:17:24.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7f8356000640 1 -- 172.21.15.102:0/2217319732 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834402dde0 con 0x7f835815ea20 2024-04-20T21:17:24.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7fb9ab400640 1 -- 172.21.15.102:0/2237511363 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9a4043020 con 0x7fb9b4163220 2024-04-20T21:17:24.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b4074340 con 0x7fb9b4163220 2024-04-20T21:17:24.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.853+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d940c5d00 con 0x7f1d9415b2c0 2024-04-20T21:17:24.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.854+0000 7f7940600640 1 -- start start 2024-04-20T21:17:24.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.854+0000 7f13abe00640 1 -- 172.21.15.102:0/2386627974 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13ac038020 con 0x7f13b4159850 2024-04-20T21:17:24.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.854+0000 7f756ca00640 1 -- 172.21.15.102:0/2286181848 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558039070 con 0x7f757015b750 2024-04-20T21:17:24.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.854+0000 7f756ca00640 1 -- 172.21.15.102:0/2286181848 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f755802cd10 con 0x7f757015b750 2024-04-20T21:17:24.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.854+0000 7f756ca00640 1 -- 172.21.15.102:0/2286181848 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558034660 con 0x7f757015b750 2024-04-20T21:17:24.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7940600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79380cb980 0x7f79381f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7940600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79380cbd60 0x7f79381514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.854+0000 7fb9ab400640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb99406c8f0 0x7fb99406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7fb9ab400640 1 -- 172.21.15.102:0/2237511363 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb9a40ad210 con 0x7fb9b4163220 2024-04-20T21:17:24.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7574e00640 1 -- 172.21.15.102:0/2286181848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f757015b750 msgr2=0x7f757015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7574e00640 1 --2- 172.21.15.102:0/2286181848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f757015b750 0x7f757015bb60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f757006d580 tx=0x7f755802bf60 comp rx=0 tx=0).stop 2024-04-20T21:17:24.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f835e000640 1 -- 172.21.15.102:0/2217319732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815ea20 msgr2=0x7f835806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f13abe00640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f139006c940 0x7f139006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f793dc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79380cb980 0x7f79381f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7fb9b3400640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb99406c8f0 0x7fb99406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7574e00640 1 -- 172.21.15.102:0/2286181848 shutdown_connections 2024-04-20T21:17:24.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7574e00640 1 --2- 172.21.15.102:0/2286181848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f757015b750 0x7f757015bb60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7574e00640 1 --2- 172.21.15.102:0/2286181848 >> v2:172.21.15.193:3300/0 conn(0x7f75701555e0 0x7f75700cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7574e00640 1 -- 172.21.15.102:0/2286181848 >> 172.21.15.102:0/2286181848 conn(0x7f75700754d0 msgr2=0x7f7570072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f9218c00640 1 -- 172.21.15.102:0/2634283324 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9214048070 con 0x7f921c06c800 2024-04-20T21:17:24.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f793dc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79380cb980 0x7f79381f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:56936/0 (socket says 172.21.15.102:56936) 2024-04-20T21:17:24.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7940600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cdb30 0x7f793814ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f793dc00640 1 -- 172.21.15.102:0/2309558529 learned_addr learned my addr 172.21.15.102:0/2309558529 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cd610 con 0x7f79380cbd60 2024-04-20T21:17:24.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cd330 con 0x7f79380cb980 2024-04-20T21:17:24.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f79380cc1a0 con 0x7f79380cdb30 2024-04-20T21:17:24.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f7574e00640 1 -- 172.21.15.102:0/2286181848 shutdown_connections 2024-04-20T21:17:24.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f793e600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cdb30 0x7f793814ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f835e000640 1 --2- 172.21.15.102:0/2217319732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815ea20 0x7f835806d940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8344009870 tx=0x7f834402bab0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f793d200640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79380cbd60 0x7f79381514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.855+0000 7f13abe00640 1 -- 172.21.15.102:0/2386627974 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f13ac0ae140 con 0x7f13b4159850 2024-04-20T21:17:24.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f1d91600640 1 -- 172.21.15.102:0/3599169512 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d8c01d070 con 0x7f1d9415b2c0 2024-04-20T21:17:24.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7fb9b3400640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb99406c8f0 0x7fb99406edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac007fd0 tx=0x7fb9ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f835e000640 1 -- 172.21.15.102:0/2217319732 shutdown_connections 2024-04-20T21:17:24.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f835e000640 1 --2- 172.21.15.102:0/2217319732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815ea20 0x7f835806d940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f835e000640 1 -- 172.21.15.102:0/2217319732 >> 172.21.15.102:0/2217319732 conn(0x7f83580c9390 msgr2=0x7f83580c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:24.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f835e000640 1 -- 172.21.15.102:0/2217319732 shutdown_connections 2024-04-20T21:17:24.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f835e000640 1 -- 172.21.15.102:0/2217319732 wait complete. 2024-04-20T21:17:24.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f793e600640 1 -- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79380cb980 msgr2=0x7f79381f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f793e600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79380cb980 0x7f79381f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f793e600640 1 -- 172.21.15.102:0/2309558529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79380cbd60 msgr2=0x7f79381514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f793e600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79380cbd60 0x7f79381514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f793e600640 1 -- 172.21.15.102:0/2309558529 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79381e5b20 con 0x7f79380cdb30 2024-04-20T21:17:24.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.857+0000 7f9218c00640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f91f806cb20 0x7f91f806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.857+0000 7f9218c00640 1 -- 172.21.15.102:0/2634283324 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f92140ae9a0 con 0x7f921c06c800 2024-04-20T21:17:24.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.857+0000 7f793e600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cdb30 0x7f793814ee10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f792c00e7d0 tx=0x7f792c00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f13b3400640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f139006c940 0x7f139006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.857+0000 7f9220e00640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f91f806cb20 0x7f91f806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.856+0000 7f7574e00640 1 -- 172.21.15.102:0/2286181848 wait complete. 2024-04-20T21:17:24.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.858+0000 7f1d91600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7406c8f0 0x7f1d7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.858+0000 7f9220e00640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f91f806cb20 0x7f91f806efe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f920c006fd0 tx=0x7f920c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.858+0000 7f1d91600640 1 -- 172.21.15.102:0/3599169512 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1d8c08c5e0 con 0x7f1d9415b2c0 2024-04-20T21:17:24.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.859+0000 7f13b3400640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f139006c940 0x7f139006ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f13a400c550 tx=0x7f13a400b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.858+0000 7f1d99c00640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7406c8f0 0x7f1d7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.859+0000 7f7574e00640 1 Processor -- start 2024-04-20T21:17:24.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.858+0000 7f835e000640 1 Processor -- start 2024-04-20T21:17:24.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.858+0000 7f835e000640 1 -- start start 2024-04-20T21:17:24.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.859+0000 7f7574e00640 1 -- start start 2024-04-20T21:17:24.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f1d99c00640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7406c8f0 0x7f1d7406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1d88036000 tx=0x7f1d8803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f835e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f835815ea20 0x7f8358153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f835e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835815ee00 0x7f83580c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f835e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815f2b0 0x7f83581598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f835e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f83580c8c80 con 0x7f835815ea20 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f835e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83580c8b00 con 0x7f835815f2b0 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f835e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8358155590 con 0x7f835815ee00 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8356a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835815ee00 0x7f83580c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8356a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835815ee00 0x7f83580c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55462/0 (socket says 172.21.15.102:55462) 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8356a00640 1 -- 172.21.15.102:0/1028278993 learned_addr learned my addr 172.21.15.102:0/1028278993 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8357e00640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815f2b0 0x7f83581598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8357400640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f835815ea20 0x7f8358153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8356a00640 1 -- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815f2b0 msgr2=0x7f83581598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8356a00640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815f2b0 0x7f83581598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8356a00640 1 -- 172.21.15.102:0/1028278993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f835815ea20 msgr2=0x7f8358153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8356a00640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f835815ea20 0x7f8358153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.860+0000 7f8356a00640 1 -- 172.21.15.102:0/1028278993 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835812dfc0 con 0x7f835815ee00 2024-04-20T21:17:24.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f8356a00640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835815ee00 0x7f83580c64c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f834800c440 tx=0x7f8348001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f7574e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75700cc300 0x7f75700b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f7574e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75701555e0 0x7f75700b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f7574e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f757015b750 0x7f757014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f7574e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f757015f050 con 0x7f75701555e0 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f7574e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75701559c0 con 0x7f757015b750 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f7574e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f757015f330 con 0x7f75700cc300 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f756de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75700cc300 0x7f75700b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f756de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75700cc300 0x7f75700b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55468/0 (socket says 172.21.15.102:55468) 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f756de00640 1 -- 172.21.15.102:0/3886159223 learned_addr learned my addr 172.21.15.102:0/3886159223 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f756e800640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f757015b750 0x7f757014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.861+0000 7f756d400640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75701555e0 0x7f75700b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.862+0000 7f756de00640 1 -- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f757015b750 msgr2=0x7f757014f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.862+0000 7f756de00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f757015b750 0x7f757014f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.862+0000 7f756de00640 1 -- 172.21.15.102:0/3886159223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75701555e0 msgr2=0x7f75700b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.862+0000 7f756de00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75701555e0 0x7f75700b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.862+0000 7f756de00640 1 -- 172.21.15.102:0/3886159223 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75701445b0 con 0x7f75700cc300 2024-04-20T21:17:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.862+0000 7f756de00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75700cc300 0x7f75700b54c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f756800e7e0 tx=0x7f756800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.863+0000 7f9218c00640 1 -- 172.21.15.102:0/2634283324 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f921407c960 con 0x7f921c06c800 2024-04-20T21:17:24.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.864+0000 7fb9ab400640 1 -- 172.21.15.102:0/2237511363 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb9a404a030 con 0x7fb9b4163220 2024-04-20T21:17:24.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.864+0000 7f834f400640 1 -- 172.21.15.102:0/1028278993 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83480052d0 con 0x7f835815ee00 2024-04-20T21:17:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.864+0000 7f7566000640 1 -- 172.21.15.102:0/3886159223 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568017070 con 0x7f75700cc300 2024-04-20T21:17:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.865+0000 7f7566000640 1 -- 172.21.15.102:0/3886159223 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756800c8d0 con 0x7f75700cc300 2024-04-20T21:17:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.865+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75701e5a10 con 0x7f75700cc300 2024-04-20T21:17:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.865+0000 7f7566000640 1 -- 172.21.15.102:0/3886159223 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800f040 con 0x7f75700cc300 2024-04-20T21:17:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.865+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701e5cd0 con 0x7f75700cc300 2024-04-20T21:17:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.865+0000 7f13abe00640 1 -- 172.21.15.102:0/2386627974 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f13ac03f070 con 0x7f13b4159850 2024-04-20T21:17:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.865+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835815bbf0 con 0x7f835815ee00 2024-04-20T21:17:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.865+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f835815e2e0 con 0x7f835815ee00 2024-04-20T21:17:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.865+0000 7f7935600640 1 -- 172.21.15.102:0/2309558529 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c017070 con 0x7f79380cdb30 2024-04-20T21:17:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.866+0000 7f7935600640 1 -- 172.21.15.102:0/2309558529 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f792c004020 con 0x7f79380cdb30 2024-04-20T21:17:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.866+0000 7f834f400640 1 -- 172.21.15.102:0/1028278993 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348005470 con 0x7f835815ee00 2024-04-20T21:17:24.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.866+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7570068c30 con 0x7f75700cc300 2024-04-20T21:17:24.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.866+0000 7f834f400640 1 -- 172.21.15.102:0/1028278993 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348007260 con 0x7f835815ee00 2024-04-20T21:17:24.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.866+0000 7f7935600640 1 -- 172.21.15.102:0/2309558529 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c0043f0 con 0x7f79380cdb30 2024-04-20T21:17:24.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.866+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79381f6890 con 0x7f79380cdb30 2024-04-20T21:17:24.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.866+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79381e5d90 con 0x7f79380cdb30 2024-04-20T21:17:24.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.867+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8330005290 con 0x7f835815ee00 2024-04-20T21:17:24.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.868+0000 7f7566000640 1 -- 172.21.15.102:0/3886159223 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f756801a070 con 0x7f75700cc300 2024-04-20T21:17:24.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.869+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7938074340 con 0x7f79380cdb30 2024-04-20T21:17:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.870+0000 7f834f400640 1 -- 172.21.15.102:0/1028278993 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8348007400 con 0x7f835815ee00 2024-04-20T21:17:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.870+0000 7f7566000640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f754c06c8f0 0x7f754c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.870+0000 7f7566000640 1 -- 172.21.15.102:0/3886159223 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f756808bad0 con 0x7f75700cc300 2024-04-20T21:17:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.870+0000 7f1d91600640 1 -- 172.21.15.102:0/3599169512 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d940c5d00 con 0x7f1d9415b2c0 2024-04-20T21:17:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.871+0000 7f756d400640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f754c06c8f0 0x7f754c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.871+0000 7f7935600640 1 -- 172.21.15.102:0/2309558529 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f792c00c8e0 con 0x7f79380cdb30 2024-04-20T21:17:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.871+0000 7f834f400640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833806c920 0x7f833806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.871+0000 7f834f400640 1 -- 172.21.15.102:0/1028278993 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f834808b270 con 0x7f835815ee00 2024-04-20T21:17:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.872+0000 7f756d400640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f754c06c8f0 0x7f754c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f75700b3880 tx=0x7f7560009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.872+0000 7f8357400640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833806c920 0x7f833806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.872+0000 7f7935600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f791806c920 0x7f791806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.873+0000 7f7935600640 1 -- 172.21.15.102:0/2309558529 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f792c08bf10 con 0x7f79380cdb30 2024-04-20T21:17:24.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.873+0000 7f8357400640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833806c920 0x7f833806ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8344036000 tx=0x7f8344008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.873+0000 7f793dc00640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f791806c920 0x7f791806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:24.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.874+0000 7f7566000640 1 -- 172.21.15.102:0/3886159223 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7568059450 con 0x7f75700cc300 2024-04-20T21:17:24.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.874+0000 7f793dc00640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f791806c920 0x7f791806ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7930009e50 tx=0x7f7930030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:24.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.878+0000 7f834f400640 1 -- 172.21.15.102:0/1028278993 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8348058d00 con 0x7f835815ee00 2024-04-20T21:17:24.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:24.883+0000 7f7935600640 1 -- 172.21.15.102:0/2309558529 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f792c059880 con 0x7f79380cdb30 2024-04-20T21:17:25.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.287+0000 7f421ce00640 1 -- 172.21.15.102:0/968934647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f42181586c0 con 0x7f42180c4040 2024-04-20T21:17:25.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.288+0000 7f420f400640 1 -- 172.21.15.102:0/968934647 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4204059e00 con 0x7f42180c4040 2024-04-20T21:17:25.290 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:25.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.291+0000 7f420cc00640 1 -- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41f806c9c0 msgr2=0x7f41f806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.291+0000 7f420cc00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41f806c9c0 0x7f41f806ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4208009940 tx=0x7f4208009240 comp rx=0 tx=0).stop 2024-04-20T21:17:25.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.291+0000 7f420cc00640 1 -- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 msgr2=0x7f421814a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.291+0000 7f420cc00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421814a060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f420400c430 tx=0x7f420400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.292+0000 7f420cc00640 1 -- 172.21.15.102:0/968934647 shutdown_connections 2024-04-20T21:17:25.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.292+0000 7f420cc00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4218155110 0x7f42181561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.292+0000 7f420cc00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41f806c9c0 0x7f41f806ee80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.292+0000 7f420cc00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42180c4040 0x7f421814a060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.292+0000 7f420cc00640 1 --2- 172.21.15.102:0/968934647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f421806f3d0 0x7f42181f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.292+0000 7f420cc00640 1 -- 172.21.15.102:0/968934647 >> 172.21.15.102:0/968934647 conn(0x7f42180cc820 msgr2=0x7f421815dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.293+0000 7f420cc00640 1 -- 172.21.15.102:0/968934647 shutdown_connections 2024-04-20T21:17:25.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.293+0000 7f420cc00640 1 -- 172.21.15.102:0/968934647 wait complete. 2024-04-20T21:17:25.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.301+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f91f4005580 con 0x7f921c06c800 2024-04-20T21:17:25.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.302+0000 7f9218c00640 1 -- 172.21.15.102:0/2634283324 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f921407c260 con 0x7f921c06c800 2024-04-20T21:17:25.305 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:25.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f138c005580 con 0x7f13b4159850 2024-04-20T21:17:25.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f91f806cb20 msgr2=0x7f91f806efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f91f806cb20 0x7f91f806efe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f920c006fd0 tx=0x7f920c008040 comp rx=0 tx=0).stop 2024-04-20T21:17:25.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f921c06c800 msgr2=0x7f921c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f921c06c800 0x7f921c0cda80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f921402c8a0 tx=0x7f9214004ab0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 shutdown_connections 2024-04-20T21:17:25.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f91f806cb20 0x7f91f806efe0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f921c1441a0 0x7f921c144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f921c06c800 0x7f921c0cda80 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f921402c8a0 tx=0x7f9214004ab0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 --2- 172.21.15.102:0/2634283324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9200004800 0x7f921c0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.306+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 >> 172.21.15.102:0/2634283324 conn(0x7f921c074ea0 msgr2=0x7f921c0c5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.307+0000 7f13abe00640 1 -- 172.21.15.102:0/2386627974 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13ac02d040 con 0x7f13b4159850 2024-04-20T21:17:25.310 INFO:teuthology.orchestra.run.smithi102.stdout:55834574861 2024-04-20T21:17:25.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.308+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb9b4072530 con 0x7fb9b4163220 2024-04-20T21:17:25.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.309+0000 7fb9ab400640 1 -- 172.21.15.102:0/2237511363 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb9a407b390 con 0x7fb9b4163220 2024-04-20T21:17:25.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.307+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 shutdown_connections 2024-04-20T21:17:25.311 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:25.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.310+0000 7f9222200640 1 -- 172.21.15.102:0/2634283324 wait complete. 2024-04-20T21:17:25.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.310+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f139006c940 msgr2=0x7f139006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.310+0000 7f13b9200640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f139006c940 0x7f139006ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f13a400c550 tx=0x7f13a400b040 comp rx=0 tx=0).stop 2024-04-20T21:17:25.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.310+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13b4159850 msgr2=0x7f13b4073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.310+0000 7f13b9200640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13b4159850 0x7f13b4073010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13b406e850 tx=0x7f13ac0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.311+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 shutdown_connections 2024-04-20T21:17:25.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.311+0000 7f13b9200640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13b4159850 0x7f13b4073010 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.311+0000 7f13b9200640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13b40c8600 0x7f13b40cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.311+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f75700caf70 con 0x7f75700cc300 2024-04-20T21:17:25.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.311+0000 7f13b9200640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f139006c940 0x7f139006ee00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.311+0000 7f13b9200640 1 --2- 172.21.15.102:0/2386627974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b406f3d0 0x7f13b40725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.311+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 >> 172.21.15.102:0/2386627974 conn(0x7f13b40cc820 msgr2=0x7f13b4145430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.312+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 shutdown_connections 2024-04-20T21:17:25.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.312+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb99406c8f0 msgr2=0x7fb99406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.312+0000 7fb9ba200640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb99406c8f0 0x7fb99406edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac007fd0 tx=0x7fb9ac009040 comp rx=0 tx=0).stop 2024-04-20T21:17:25.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.312+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b4163220 msgr2=0x7fb9b41fdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.312+0000 7fb9ba200640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b4163220 0x7fb9b41fdfe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4032000 tx=0x7fb9a40046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.312+0000 7f13b9200640 1 -- 172.21.15.102:0/2386627974 wait complete. 2024-04-20T21:17:25.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.312+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1d940cd740 con 0x7f1d9415b2c0 2024-04-20T21:17:25.314 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:25.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.312+0000 7f7566000640 1 -- 172.21.15.102:0/3886159223 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f756805d750 con 0x7f75700cc300 2024-04-20T21:17:25.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 shutdown_connections 2024-04-20T21:17:25.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7fb9ba200640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4163660 0x7fb9b40c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7fb9ba200640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb99406c8f0 0x7fb99406edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7fb9ba200640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b4163220 0x7fb9b41fdfe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7fb9ba200640 1 --2- 172.21.15.102:0/2237511363 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b4162e40 0x7fb9b415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 >> 172.21.15.102:0/2237511363 conn(0x7fb9b40c9390 msgr2=0x7fb9b406e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7f1d91600640 1 -- 172.21.15.102:0/3599169512 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d940cd740 con 0x7f1d9415b2c0 2024-04-20T21:17:25.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 shutdown_connections 2024-04-20T21:17:25.316 INFO:teuthology.orchestra.run.smithi102.stdout:55834574859 2024-04-20T21:17:25.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.313+0000 7fb9ba200640 1 -- 172.21.15.102:0/2237511363 wait complete. 2024-04-20T21:17:25.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f754c06c8f0 msgr2=0x7f754c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f754c06c8f0 0x7f754c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f75700b3880 tx=0x7f7560009040 comp rx=0 tx=0).stop 2024-04-20T21:17:25.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75700cc300 msgr2=0x7f75700b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75700cc300 0x7f75700b54c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f756800e7e0 tx=0x7f756800eca0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7406c8f0 msgr2=0x7f1d7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f1d9c600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7406c8f0 0x7f1d7406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1d88036000 tx=0x7f1d8803b040 comp rx=0 tx=0).stop 2024-04-20T21:17:25.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9415b2c0 msgr2=0x7f1d9415d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f1d9c600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9415b2c0 0x7f1d9415d1c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00ceb0 tx=0x7f1d8c00a710 comp rx=0 tx=0).stop 2024-04-20T21:17:25.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 shutdown_connections 2024-04-20T21:17:25.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f754c06c8f0 0x7f754c06edb0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f75700b3880 tx=0x7f7560009040 comp rx=0 tx=0).stop 2024-04-20T21:17:25.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f757015b750 0x7f757014f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75701555e0 0x7f75700b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 --2- 172.21.15.102:0/3886159223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75700cc300 0x7f75700b54c0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f756800e7e0 tx=0x7f756800eca0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 >> 172.21.15.102:0/3886159223 conn(0x7f75700754d0 msgr2=0x7f75700c5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 shutdown_connections 2024-04-20T21:17:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.316+0000 7f7574e00640 1 -- 172.21.15.102:0/3886159223 wait complete. 2024-04-20T21:17:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.317+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 shutdown_connections 2024-04-20T21:17:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.317+0000 7f1d9c600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d7406c8f0 0x7f1d7406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.317+0000 7f1d9c600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9415b2c0 0x7f1d9415d1c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.317+0000 7f1d9c600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d9415a930 0x7f1d9415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.317+0000 7f1d9c600640 1 --2- 172.21.15.102:0/3599169512 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d940c9eb0 0x7f1d9415bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.317+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 >> 172.21.15.102:0/3599169512 conn(0x7f1d940c9390 msgr2=0x7f1d940cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.317+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 shutdown_connections 2024-04-20T21:17:25.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.319+0000 7f1d9c600640 1 -- 172.21.15.102:0/3599169512 wait complete. 2024-04-20T21:17:25.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2024-04-20T21:17:25.321 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:25.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.321+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8330005580 con 0x7f835815ee00 2024-04-20T21:17:25.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.322+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f79380c7830 con 0x7f79380cdb30 2024-04-20T21:17:25.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.322+0000 7f834f400640 1 -- 172.21.15.102:0/1028278993 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f834805d000 con 0x7f835815ee00 2024-04-20T21:17:25.322 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:25.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.324+0000 7f7935600640 1 -- 172.21.15.102:0/2309558529 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f792c05db80 con 0x7f79380cdb30 2024-04-20T21:17:25.323 INFO:teuthology.orchestra.run.smithi102.stdout:55834574860 2024-04-20T21:17:25.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f791806c920 msgr2=0x7f791806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f791806c920 0x7f791806ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7930009e50 tx=0x7f7930030000 comp rx=0 tx=0).stop 2024-04-20T21:17:25.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cdb30 msgr2=0x7f793814ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cdb30 0x7f793814ee10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f792c00e7d0 tx=0x7f792c00ec90 comp rx=0 tx=0).stop 2024-04-20T21:17:25.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 shutdown_connections 2024-04-20T21:17:25.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79380cdb30 0x7f793814ee10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79380cbd60 0x7f79381514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f791806c920 0x7f791806ede0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 --2- 172.21.15.102:0/2309558529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79380cb980 0x7f79381f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.328+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 >> 172.21.15.102:0/2309558529 conn(0x7f7938075080 msgr2=0x7f7938153210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.329+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 shutdown_connections 2024-04-20T21:17:25.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.329+0000 7f7940600640 1 -- 172.21.15.102:0/2309558529 wait complete. 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833806c920 msgr2=0x7f833806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833806c920 0x7f833806ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8344036000 tx=0x7f8344008030 comp rx=0 tx=0).stop 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835815ee00 msgr2=0x7f83580c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835815ee00 0x7f83580c64c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f834800c440 tx=0x7f8348001e70 comp rx=0 tx=0).stop 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 shutdown_connections 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833806c920 0x7f833806ede0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835815f2b0 0x7f83581598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835815ee00 0x7f83580c64c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 --2- 172.21.15.102:0/1028278993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f835815ea20 0x7f8358153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.331+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 >> 172.21.15.102:0/1028278993 conn(0x7f83580c9390 msgr2=0x7f835814a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.332+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 shutdown_connections 2024-04-20T21:17:25.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.332+0000 7f835e000640 1 -- 172.21.15.102:0/1028278993 wait complete. 2024-04-20T21:17:25.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2024-04-20T21:17:25.337 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:25.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.1 2024-04-20T21:17:25.342 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:25.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.5 2024-04-20T21:17:25.344 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:25.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-04-20T21:17:25.345 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:25.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.3 2024-04-20T21:17:25.346 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:25.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.7 2024-04-20T21:17:25.353 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:25.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2024-04-20T21:17:25.355 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:25.355 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T21:17:25.356 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:25.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.581+0000 7f8f0be00640 1 Processor -- start 2024-04-20T21:17:25.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.581+0000 7f8f0be00640 1 -- start start 2024-04-20T21:17:25.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8f0c0cd310 0x7f8f0c163660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:25.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0be00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f0c15c220 con 0x7f8f0c152290 2024-04-20T21:17:25.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0be00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f0c0cd720 con 0x7f8f0c0c3700 2024-04-20T21:17:25.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0be00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8f0c0c9180 con 0x7f8f0c0cd310 2024-04-20T21:17:25.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0aa00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8f0c152290 0x7f8f0c06e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38722/0 (socket says 172.21.15.102:38722) 2024-04-20T21:17:25.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0aa00640 1 -- 172.21.15.102:0/1075250328 learned_addr learned my addr 172.21.15.102:0/1075250328 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:25.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0b400640 1 --2- 172.21.15.102:0/1075250328 >> v2:172.21.15.102:3300/0 conn(0x7f8f0c0cd310 0x7f8f0c163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:25.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0a000640 1 -- 172.21.15.102:0/1075250328 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3885306191 0 0) 0x7f8f0c15c220 con 0x7f8f0c152290 2024-04-20T21:17:25.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0a000640 1 -- 172.21.15.102:0/1075250328 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ef0003520 con 0x7f8f0c152290 2024-04-20T21:17:25.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0b400640 1 -- 172.21.15.102:0/1075250328 >> v1:172.21.15.193:6789/0 conn(0x7f8f0c0c3700 legacy=0x7f8f0c0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0a000640 1 -- 172.21.15.102:0/1075250328 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971881774 0 0) 0x7f8f0c0cd720 con 0x7f8f0c0c3700 2024-04-20T21:17:25.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.583+0000 7f8f0b400640 1 -- 172.21.15.102:0/1075250328 >> v1:172.21.15.102:6789/0 conn(0x7f8f0c152290 legacy=0x7f8f0c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.584+0000 7f8f0b400640 1 -- 172.21.15.102:0/1075250328 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c0c4560 con 0x7f8f0c0cd310 2024-04-20T21:17:25.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0b400640 1 --2- 172.21.15.102:0/1075250328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0cd310 0x7f8f0c163660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009d90 tx=0x7f8f0402e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64325f963d7b1261 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:25.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0a000640 1 -- 172.21.15.102:0/1075250328 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0403b070 con 0x7f8f0c0cd310 2024-04-20T21:17:25.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0a000640 1 -- 172.21.15.102:0/1075250328 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f04036440 con 0x7f8f0c0cd310 2024-04-20T21:17:25.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0a000640 1 -- 172.21.15.102:0/1075250328 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04036810 con 0x7f8f0c0cd310 2024-04-20T21:17:25.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0be00640 1 -- 172.21.15.102:0/1075250328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0cd310 msgr2=0x7f8f0c163660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0be00640 1 --2- 172.21.15.102:0/1075250328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0cd310 0x7f8f0c163660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009d90 tx=0x7f8f0402e1e0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0be00640 1 -- 172.21.15.102:0/1075250328 shutdown_connections 2024-04-20T21:17:25.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0be00640 1 --2- 172.21.15.102:0/1075250328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0cd310 0x7f8f0c163660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.585+0000 7f8f0be00640 1 -- 172.21.15.102:0/1075250328 >> 172.21.15.102:0/1075250328 conn(0x7f8f0c0cc820 msgr2=0x7f8f0c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.586+0000 7f8f0be00640 1 -- 172.21.15.102:0/1075250328 shutdown_connections 2024-04-20T21:17:25.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.586+0000 7f8f0be00640 1 -- 172.21.15.102:0/1075250328 wait complete. 2024-04-20T21:17:25.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.587+0000 7f8f0be00640 1 Processor -- start 2024-04-20T21:17:25.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.587+0000 7f8f0be00640 1 -- start start 2024-04-20T21:17:25.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0c3700 0x7f8f0c0c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c0cd310 0x7f8f0c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c152290 0x7f8f0c0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c15b740 con 0x7f8f0c0c3700 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c0c4840 con 0x7f8f0c152290 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0c15ba20 con 0x7f8f0c0cd310 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0c3700 0x7f8f0c0c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0c3700 0x7f8f0c0c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51620/0 (socket says 172.21.15.102:51620) 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.588+0000 7f8f0aa00640 1 -- 172.21.15.102:0/37676569 learned_addr learned my addr 172.21.15.102:0/37676569 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f0b400640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c152290 0x7f8f0c0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f02000640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c0cd310 0x7f8f0c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f0aa00640 1 -- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c0cd310 msgr2=0x7f8f0c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f0aa00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c0cd310 0x7f8f0c0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f0aa00640 1 -- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c152290 msgr2=0x7f8f0c0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f0aa00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c152290 0x7f8f0c0cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f0aa00640 1 -- 172.21.15.102:0/37676569 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c1f1090 con 0x7f8f0c0c3700 2024-04-20T21:17:25.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f0aa00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0c3700 0x7f8f0c0c61a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00c430 tx=0x7f8efc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:25.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f03400640 1 -- 172.21.15.102:0/37676569 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc017070 con 0x7f8f0c0c3700 2024-04-20T21:17:25.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f03400640 1 -- 172.21.15.102:0/37676569 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8efc005ce0 con 0x7f8f0c0c3700 2024-04-20T21:17:25.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.589+0000 7f8f03400640 1 -- 172.21.15.102:0/37676569 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0072f0 con 0x7f8f0c0c3700 2024-04-20T21:17:25.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.590+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c1f4710 con 0x7f8f0c0c3700 2024-04-20T21:17:25.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.590+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c1f4cd0 con 0x7f8f0c0c3700 2024-04-20T21:17:25.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.591+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0c04fd60 con 0x7f8f0c0c3700 2024-04-20T21:17:25.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.592+0000 7f8f03400640 1 -- 172.21.15.102:0/37676569 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8efc01a070 con 0x7f8f0c0c3700 2024-04-20T21:17:25.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.592+0000 7f8f03400640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:25.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.593+0000 7f8f03400640 1 -- 172.21.15.102:0/37676569 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8efc08b3d0 con 0x7f8f0c0c3700 2024-04-20T21:17:25.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.593+0000 7f8f02000640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:25.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.593+0000 7f8f02000640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ee806c8f0 0x7f8ee806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c073320 tx=0x7f8ef4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:25.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.597+0000 7f8f03400640 1 -- 172.21.15.102:0/37676569 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8efc058d40 con 0x7f8f0c0c3700 2024-04-20T21:17:25.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.778+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 --> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f0c1f54d0 con 0x7f8ee806c8f0 2024-04-20T21:17:25.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.787+0000 7f8f03400640 1 -- 172.21.15.102:0/37676569 <== mgr.4112 v2:172.21.15.193:6832/2803621564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7f8f0c1f54d0 con 0x7f8ee806c8f0 2024-04-20T21:17:25.791 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:25.793 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-20T21:17:25.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ee806c8f0 msgr2=0x7f8ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ee806c8f0 0x7f8ee806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c073320 tx=0x7f8ef4009040 comp rx=0 tx=0).stop 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0c3700 msgr2=0x7f8f0c0c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0c3700 0x7f8f0c0c61a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00c430 tx=0x7f8efc00c8f0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 shutdown_connections 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c152290 0x7f8f0c0cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c0cd310 0x7f8f0c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 --2- 172.21.15.102:0/37676569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f0c0c3700 0x7f8f0c0c61a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 >> 172.21.15.102:0/37676569 conn(0x7f8f0c0cc820 msgr2=0x7f8f0c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.792+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 shutdown_connections 2024-04-20T21:17:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:25.793+0000 7f8f0be00640 1 -- 172.21.15.102:0/37676569 wait complete. 2024-04-20T21:17:25.809 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-20T21:17:24.668301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003187"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918498+0000","last_change":"2024-04-20T21:16:53.564922+0000","last_active":"2024-04-20T21:16:59.918498+0000","last_peered":"2024-04-20T21:16:59.918498+0000","last_clean":"2024-04-20T21:16:59.918498+0000","last_became_active":"2024-04-20T21:16:53.564589+0000","last_became_peered":"2024-04-20T21:16:53.564589+0000","last_unstale":"2024-04-20T21:16:59.918498+0000","last_undegraded":"2024-04-20T21:16:59.918498+0000","last_fullsized":"2024-04-20T21:16:59.918498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:47.284664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916138+0000","last_change":"2024-04-20T21:16:54.580132+0000","last_active":"2024-04-20T21:16:59.916138+0000","last_peered":"2024-04-20T21:16:59.916138+0000","last_clean":"2024-04-20T21:16:59.916138+0000","last_became_active":"2024-04-20T21:16:54.579769+0000","last_became_peered":"2024-04-20T21:16:54.579769+0000","last_unstale":"2024-04-20T21:16:59.916138+0000","last_undegraded":"2024-04-20T21:16:59.916138+0000","last_fullsized":"2024-04-20T21:16:59.916138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:00.705947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913935+0000","last_change":"2024-04-20T21:16:58.781707+0000","last_active":"2024-04-20T21:16:59.913935+0000","last_peered":"2024-04-20T21:16:59.913935+0000","last_clean":"2024-04-20T21:16:59.913935+0000","last_became_active":"2024-04-20T21:16:58.780685+0000","last_became_peered":"2024-04-20T21:16:58.780685+0000","last_unstale":"2024-04-20T21:16:59.913935+0000","last_undegraded":"2024-04-20T21:16:59.913935+0000","last_fullsized":"2024-04-20T21:16:59.913935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:50.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916578+0000","last_change":"2024-04-20T21:16:53.572137+0000","last_active":"2024-04-20T21:16:59.916578+0000","last_peered":"2024-04-20T21:16:59.916578+0000","last_clean":"2024-04-20T21:16:59.916578+0000","last_became_active":"2024-04-20T21:16:53.567907+0000","last_became_peered":"2024-04-20T21:16:53.567907+0000","last_unstale":"2024-04-20T21:16:59.916578+0000","last_undegraded":"2024-04-20T21:16:59.916578+0000","last_fullsized":"2024-04-20T21:16:59.916578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:23:28.367174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918240+0000","last_change":"2024-04-20T21:16:54.567096+0000","last_active":"2024-04-20T21:16:59.918240+0000","last_peered":"2024-04-20T21:16:59.918240+0000","last_clean":"2024-04-20T21:16:59.918240+0000","last_became_active":"2024-04-20T21:16:54.566803+0000","last_became_peered":"2024-04-20T21:16:54.566803+0000","last_unstale":"2024-04-20T21:16:59.918240+0000","last_undegraded":"2024-04-20T21:16:59.918240+0000","last_fullsized":"2024-04-20T21:16:59.918240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:13:44.730056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604352+0000","last_change":"2024-04-20T21:16:58.619283+0000","last_active":"2024-04-20T21:16:59.604352+0000","last_peered":"2024-04-20T21:16:59.604352+0000","last_clean":"2024-04-20T21:16:59.604352+0000","last_became_active":"2024-04-20T21:16:58.618881+0000","last_became_peered":"2024-04-20T21:16:58.618881+0000","last_unstale":"2024-04-20T21:16:59.604352+0000","last_undegraded":"2024-04-20T21:16:59.604352+0000","last_fullsized":"2024-04-20T21:16:59.604352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:01:40.143389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617802+0000","last_change":"2024-04-20T21:16:53.554847+0000","last_active":"2024-04-20T21:16:59.617802+0000","last_peered":"2024-04-20T21:16:59.617802+0000","last_clean":"2024-04-20T21:16:59.617802+0000","last_became_active":"2024-04-20T21:16:53.554333+0000","last_became_peered":"2024-04-20T21:16:53.554333+0000","last_unstale":"2024-04-20T21:16:59.617802+0000","last_undegraded":"2024-04-20T21:16:59.617802+0000","last_fullsized":"2024-04-20T21:16:59.617802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:33.255945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912613+0000","last_change":"2024-04-20T21:16:54.556407+0000","last_active":"2024-04-20T21:16:59.912613+0000","last_peered":"2024-04-20T21:16:59.912613+0000","last_clean":"2024-04-20T21:16:59.912613+0000","last_became_active":"2024-04-20T21:16:54.556161+0000","last_became_peered":"2024-04-20T21:16:54.556161+0000","last_unstale":"2024-04-20T21:16:59.912613+0000","last_undegraded":"2024-04-20T21:16:59.912613+0000","last_fullsized":"2024-04-20T21:16:59.912613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:44:53.586714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604519+0000","last_change":"2024-04-20T21:16:58.612063+0000","last_active":"2024-04-20T21:16:59.604519+0000","last_peered":"2024-04-20T21:16:59.604519+0000","last_clean":"2024-04-20T21:16:59.604519+0000","last_became_active":"2024-04-20T21:16:58.611487+0000","last_became_peered":"2024-04-20T21:16:58.611487+0000","last_unstale":"2024-04-20T21:16:59.604519+0000","last_undegraded":"2024-04-20T21:16:59.604519+0000","last_fullsized":"2024-04-20T21:16:59.604519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:15.097731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617458+0000","last_change":"2024-04-20T21:16:53.551212+0000","last_active":"2024-04-20T21:16:59.617458+0000","last_peered":"2024-04-20T21:16:59.617458+0000","last_clean":"2024-04-20T21:16:59.617458+0000","last_became_active":"2024-04-20T21:16:53.550899+0000","last_became_peered":"2024-04-20T21:16:53.550899+0000","last_unstale":"2024-04-20T21:16:59.617458+0000","last_undegraded":"2024-04-20T21:16:59.617458+0000","last_fullsized":"2024-04-20T21:16:59.617458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:07:18.283243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912476+0000","last_change":"2024-04-20T21:16:54.557285+0000","last_active":"2024-04-20T21:16:59.912476+0000","last_peered":"2024-04-20T21:16:59.912476+0000","last_clean":"2024-04-20T21:16:59.912476+0000","last_became_active":"2024-04-20T21:16:54.556921+0000","last_became_peered":"2024-04-20T21:16:54.556921+0000","last_unstale":"2024-04-20T21:16:59.912476+0000","last_undegraded":"2024-04-20T21:16:59.912476+0000","last_fullsized":"2024-04-20T21:16:59.912476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:33.989679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604784+0000","last_change":"2024-04-20T21:16:58.617360+0000","last_active":"2024-04-20T21:16:59.604784+0000","last_peered":"2024-04-20T21:16:59.604784+0000","last_clean":"2024-04-20T21:16:59.604784+0000","last_became_active":"2024-04-20T21:16:58.616947+0000","last_became_peered":"2024-04-20T21:16:58.616947+0000","last_unstale":"2024-04-20T21:16:59.604784+0000","last_undegraded":"2024-04-20T21:16:59.604784+0000","last_fullsized":"2024-04-20T21:16:59.604784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.202955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911434+0000","last_change":"2024-04-20T21:16:54.558067+0000","last_active":"2024-04-20T21:16:59.911434+0000","last_peered":"2024-04-20T21:16:59.911434+0000","last_clean":"2024-04-20T21:16:59.911434+0000","last_became_active":"2024-04-20T21:16:54.557781+0000","last_became_peered":"2024-04-20T21:16:54.557781+0000","last_unstale":"2024-04-20T21:16:59.911434+0000","last_undegraded":"2024-04-20T21:16:59.911434+0000","last_fullsized":"2024-04-20T21:16:59.911434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:56.576081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617756+0000","last_change":"2024-04-20T21:16:53.551615+0000","last_active":"2024-04-20T21:16:59.617756+0000","last_peered":"2024-04-20T21:16:59.617756+0000","last_clean":"2024-04-20T21:16:59.617756+0000","last_became_active":"2024-04-20T21:16:53.551073+0000","last_became_peered":"2024-04-20T21:16:53.551073+0000","last_unstale":"2024-04-20T21:16:59.617756+0000","last_undegraded":"2024-04-20T21:16:59.617756+0000","last_fullsized":"2024-04-20T21:16:59.617756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:26:29.994289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605362+0000","last_change":"2024-04-20T21:16:58.617434+0000","last_active":"2024-04-20T21:16:59.605362+0000","last_peered":"2024-04-20T21:16:59.605362+0000","last_clean":"2024-04-20T21:16:59.605362+0000","last_became_active":"2024-04-20T21:16:58.617027+0000","last_became_peered":"2024-04-20T21:16:58.617027+0000","last_unstale":"2024-04-20T21:16:59.605362+0000","last_undegraded":"2024-04-20T21:16:59.605362+0000","last_fullsized":"2024-04-20T21:16:59.605362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:44:20.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913331+0000","last_change":"2024-04-20T21:16:54.585333+0000","last_active":"2024-04-20T21:16:59.913331+0000","last_peered":"2024-04-20T21:16:59.913331+0000","last_clean":"2024-04-20T21:16:59.913331+0000","last_became_active":"2024-04-20T21:16:54.584978+0000","last_became_peered":"2024-04-20T21:16:54.584978+0000","last_unstale":"2024-04-20T21:16:59.913331+0000","last_undegraded":"2024-04-20T21:16:59.913331+0000","last_fullsized":"2024-04-20T21:16:59.913331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:23.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608374+0000","last_change":"2024-04-20T21:16:53.567775+0000","last_active":"2024-04-20T21:16:59.608374+0000","last_peered":"2024-04-20T21:16:59.608374+0000","last_clean":"2024-04-20T21:16:59.608374+0000","last_became_active":"2024-04-20T21:16:53.567498+0000","last_became_peered":"2024-04-20T21:16:53.567498+0000","last_unstale":"2024-04-20T21:16:59.608374+0000","last_undegraded":"2024-04-20T21:16:59.608374+0000","last_fullsized":"2024-04-20T21:16:59.608374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:10.695659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604469+0000","last_change":"2024-04-20T21:16:58.619302+0000","last_active":"2024-04-20T21:16:59.604469+0000","last_peered":"2024-04-20T21:16:59.604469+0000","last_clean":"2024-04-20T21:16:59.604469+0000","last_became_active":"2024-04-20T21:16:58.618843+0000","last_became_peered":"2024-04-20T21:16:58.618843+0000","last_unstale":"2024-04-20T21:16:59.604469+0000","last_undegraded":"2024-04-20T21:16:59.604469+0000","last_fullsized":"2024-04-20T21:16:59.604469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:14.927199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914247+0000","last_change":"2024-04-20T21:16:54.575862+0000","last_active":"2024-04-20T21:16:59.914247+0000","last_peered":"2024-04-20T21:16:59.914247+0000","last_clean":"2024-04-20T21:16:59.914247+0000","last_became_active":"2024-04-20T21:16:54.575523+0000","last_became_peered":"2024-04-20T21:16:54.575523+0000","last_unstale":"2024-04-20T21:16:59.914247+0000","last_undegraded":"2024-04-20T21:16:59.914247+0000","last_fullsized":"2024-04-20T21:16:59.914247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:51.041414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606833+0000","last_change":"2024-04-20T21:16:53.551995+0000","last_active":"2024-04-20T21:16:59.606833+0000","last_peered":"2024-04-20T21:16:59.606833+0000","last_clean":"2024-04-20T21:16:59.606833+0000","last_became_active":"2024-04-20T21:16:53.551603+0000","last_became_peered":"2024-04-20T21:16:53.551603+0000","last_unstale":"2024-04-20T21:16:59.606833+0000","last_undegraded":"2024-04-20T21:16:59.606833+0000","last_fullsized":"2024-04-20T21:16:59.606833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:56:21.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608819+0000","last_change":"2024-04-20T21:16:58.776463+0000","last_active":"2024-04-20T21:16:59.608819+0000","last_peered":"2024-04-20T21:16:59.608819+0000","last_clean":"2024-04-20T21:16:59.608819+0000","last_became_active":"2024-04-20T21:16:58.775746+0000","last_became_peered":"2024-04-20T21:16:58.775746+0000","last_unstale":"2024-04-20T21:16:59.608819+0000","last_undegraded":"2024-04-20T21:16:59.608819+0000","last_fullsized":"2024-04-20T21:16:59.608819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:28.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605047+0000","last_change":"2024-04-20T21:16:54.566449+0000","last_active":"2024-04-20T21:16:59.605047+0000","last_peered":"2024-04-20T21:16:59.605047+0000","last_clean":"2024-04-20T21:16:59.605047+0000","last_became_active":"2024-04-20T21:16:54.566251+0000","last_became_peered":"2024-04-20T21:16:54.566251+0000","last_unstale":"2024-04-20T21:16:59.605047+0000","last_undegraded":"2024-04-20T21:16:59.605047+0000","last_fullsized":"2024-04-20T21:16:59.605047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:53.403420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617047+0000","last_change":"2024-04-20T21:16:53.564029+0000","last_active":"2024-04-20T21:16:59.617047+0000","last_peered":"2024-04-20T21:16:59.617047+0000","last_clean":"2024-04-20T21:16:59.617047+0000","last_became_active":"2024-04-20T21:16:53.563711+0000","last_became_peered":"2024-04-20T21:16:53.563711+0000","last_unstale":"2024-04-20T21:16:59.617047+0000","last_undegraded":"2024-04-20T21:16:59.617047+0000","last_fullsized":"2024-04-20T21:16:59.617047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.634644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604995+0000","last_change":"2024-04-20T21:16:58.615080+0000","last_active":"2024-04-20T21:16:59.604995+0000","last_peered":"2024-04-20T21:16:59.604995+0000","last_clean":"2024-04-20T21:16:59.604995+0000","last_became_active":"2024-04-20T21:16:58.614611+0000","last_became_peered":"2024-04-20T21:16:58.614611+0000","last_unstale":"2024-04-20T21:16:59.604995+0000","last_undegraded":"2024-04-20T21:16:59.604995+0000","last_fullsized":"2024-04-20T21:16:59.604995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:08.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917639+0000","last_change":"2024-04-20T21:16:54.578272+0000","last_active":"2024-04-20T21:16:59.917639+0000","last_peered":"2024-04-20T21:16:59.917639+0000","last_clean":"2024-04-20T21:16:59.917639+0000","last_became_active":"2024-04-20T21:16:54.577949+0000","last_became_peered":"2024-04-20T21:16:54.577949+0000","last_unstale":"2024-04-20T21:16:59.917639+0000","last_undegraded":"2024-04-20T21:16:59.917639+0000","last_fullsized":"2024-04-20T21:16:59.917639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:35:33.807091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912813+0000","last_change":"2024-04-20T21:16:53.554600+0000","last_active":"2024-04-20T21:16:59.912813+0000","last_peered":"2024-04-20T21:16:59.912813+0000","last_clean":"2024-04-20T21:16:59.912813+0000","last_became_active":"2024-04-20T21:16:53.554347+0000","last_became_peered":"2024-04-20T21:16:53.554347+0000","last_unstale":"2024-04-20T21:16:59.912813+0000","last_undegraded":"2024-04-20T21:16:59.912813+0000","last_fullsized":"2024-04-20T21:16:59.912813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:40:58.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604315+0000","last_change":"2024-04-20T21:16:58.774474+0000","last_active":"2024-04-20T21:16:59.604315+0000","last_peered":"2024-04-20T21:16:59.604315+0000","last_clean":"2024-04-20T21:16:59.604315+0000","last_became_active":"2024-04-20T21:16:58.774037+0000","last_became_peered":"2024-04-20T21:16:58.774037+0000","last_unstale":"2024-04-20T21:16:59.604315+0000","last_undegraded":"2024-04-20T21:16:59.604315+0000","last_fullsized":"2024-04-20T21:16:59.604315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:46.992667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916707+0000","last_change":"2024-04-20T21:16:54.579435+0000","last_active":"2024-04-20T21:16:59.916707+0000","last_peered":"2024-04-20T21:16:59.916707+0000","last_clean":"2024-04-20T21:16:59.916707+0000","last_became_active":"2024-04-20T21:16:54.574617+0000","last_became_peered":"2024-04-20T21:16:54.574617+0000","last_unstale":"2024-04-20T21:16:59.916707+0000","last_undegraded":"2024-04-20T21:16:59.916707+0000","last_fullsized":"2024-04-20T21:16:59.916707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:51.500741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608822+0000","last_change":"2024-04-20T21:16:53.561144+0000","last_active":"2024-04-20T21:16:59.608822+0000","last_peered":"2024-04-20T21:16:59.608822+0000","last_clean":"2024-04-20T21:16:59.608822+0000","last_became_active":"2024-04-20T21:16:53.560742+0000","last_became_peered":"2024-04-20T21:16:53.560742+0000","last_unstale":"2024-04-20T21:16:59.608822+0000","last_undegraded":"2024-04-20T21:16:59.608822+0000","last_fullsized":"2024-04-20T21:16:59.608822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:39.212951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913482+0000","last_change":"2024-04-20T21:16:58.780124+0000","last_active":"2024-04-20T21:16:59.913482+0000","last_peered":"2024-04-20T21:16:59.913482+0000","last_clean":"2024-04-20T21:16:59.913482+0000","last_became_active":"2024-04-20T21:16:58.777449+0000","last_became_peered":"2024-04-20T21:16:58.777449+0000","last_unstale":"2024-04-20T21:16:59.913482+0000","last_undegraded":"2024-04-20T21:16:59.913482+0000","last_fullsized":"2024-04-20T21:16:59.913482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:07.622146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608726+0000","last_change":"2024-04-20T21:16:54.575331+0000","last_active":"2024-04-20T21:16:59.608726+0000","last_peered":"2024-04-20T21:16:59.608726+0000","last_clean":"2024-04-20T21:16:59.608726+0000","last_became_active":"2024-04-20T21:16:54.575061+0000","last_became_peered":"2024-04-20T21:16:54.575061+0000","last_unstale":"2024-04-20T21:16:59.608726+0000","last_undegraded":"2024-04-20T21:16:59.608726+0000","last_fullsized":"2024-04-20T21:16:59.608726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:47:20.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617871+0000","last_change":"2024-04-20T21:16:53.565561+0000","last_active":"2024-04-20T21:16:59.617871+0000","last_peered":"2024-04-20T21:16:59.617871+0000","last_clean":"2024-04-20T21:16:59.617871+0000","last_became_active":"2024-04-20T21:16:53.565207+0000","last_became_peered":"2024-04-20T21:16:53.565207+0000","last_unstale":"2024-04-20T21:16:59.617871+0000","last_undegraded":"2024-04-20T21:16:59.617871+0000","last_fullsized":"2024-04-20T21:16:59.617871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:07.512132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915133+0000","last_change":"2024-04-20T21:16:58.776337+0000","last_active":"2024-04-20T21:16:59.915133+0000","last_peered":"2024-04-20T21:16:59.915133+0000","last_clean":"2024-04-20T21:16:59.915133+0000","last_became_active":"2024-04-20T21:16:58.772404+0000","last_became_peered":"2024-04-20T21:16:58.772404+0000","last_unstale":"2024-04-20T21:16:59.915133+0000","last_undegraded":"2024-04-20T21:16:59.915133+0000","last_fullsized":"2024-04-20T21:16:59.915133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:11.554173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912522+0000","last_change":"2024-04-20T21:16:54.571199+0000","last_active":"2024-04-20T21:16:59.912522+0000","last_peered":"2024-04-20T21:16:59.912522+0000","last_clean":"2024-04-20T21:16:59.912522+0000","last_became_active":"2024-04-20T21:16:54.570882+0000","last_became_peered":"2024-04-20T21:16:54.570882+0000","last_unstale":"2024-04-20T21:16:59.912522+0000","last_undegraded":"2024-04-20T21:16:59.912522+0000","last_fullsized":"2024-04-20T21:16:59.912522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:52.046176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608915+0000","last_change":"2024-04-20T21:16:53.566253+0000","last_active":"2024-04-20T21:16:59.608915+0000","last_peered":"2024-04-20T21:16:59.608915+0000","last_clean":"2024-04-20T21:16:59.608915+0000","last_became_active":"2024-04-20T21:16:53.565893+0000","last_became_peered":"2024-04-20T21:16:53.565893+0000","last_unstale":"2024-04-20T21:16:59.608915+0000","last_undegraded":"2024-04-20T21:16:59.608915+0000","last_fullsized":"2024-04-20T21:16:59.608915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:16.302801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.607352+0000","last_change":"2024-04-20T21:16:58.611470+0000","last_active":"2024-04-20T21:16:59.607352+0000","last_peered":"2024-04-20T21:16:59.607352+0000","last_clean":"2024-04-20T21:16:59.607352+0000","last_became_active":"2024-04-20T21:16:58.611038+0000","last_became_peered":"2024-04-20T21:16:58.611038+0000","last_unstale":"2024-04-20T21:16:59.607352+0000","last_undegraded":"2024-04-20T21:16:59.607352+0000","last_fullsized":"2024-04-20T21:16:59.607352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:58:51.267480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604848+0000","last_change":"2024-04-20T21:16:54.577608+0000","last_active":"2024-04-20T21:16:59.604848+0000","last_peered":"2024-04-20T21:16:59.604848+0000","last_clean":"2024-04-20T21:16:59.604848+0000","last_became_active":"2024-04-20T21:16:54.577339+0000","last_became_peered":"2024-04-20T21:16:54.577339+0000","last_unstale":"2024-04-20T21:16:59.604848+0000","last_undegraded":"2024-04-20T21:16:59.604848+0000","last_fullsized":"2024-04-20T21:16:59.604848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:03.028585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912331+0000","last_change":"2024-04-20T21:16:53.547751+0000","last_active":"2024-04-20T21:16:59.912331+0000","last_peered":"2024-04-20T21:16:59.912331+0000","last_clean":"2024-04-20T21:16:59.912331+0000","last_became_active":"2024-04-20T21:16:53.547371+0000","last_became_peered":"2024-04-20T21:16:53.547371+0000","last_unstale":"2024-04-20T21:16:59.912331+0000","last_undegraded":"2024-04-20T21:16:59.912331+0000","last_fullsized":"2024-04-20T21:16:59.912331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:50.728500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608358+0000","last_change":"2024-04-20T21:16:58.622369+0000","last_active":"2024-04-20T21:16:59.608358+0000","last_peered":"2024-04-20T21:16:59.608358+0000","last_clean":"2024-04-20T21:16:59.608358+0000","last_became_active":"2024-04-20T21:16:58.621911+0000","last_became_peered":"2024-04-20T21:16:58.621911+0000","last_unstale":"2024-04-20T21:16:59.608358+0000","last_undegraded":"2024-04-20T21:16:59.608358+0000","last_fullsized":"2024-04-20T21:16:59.608358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:41.519583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606736+0000","last_change":"2024-04-20T21:16:54.579577+0000","last_active":"2024-04-20T21:16:59.606736+0000","last_peered":"2024-04-20T21:16:59.606736+0000","last_clean":"2024-04-20T21:16:59.606736+0000","last_became_active":"2024-04-20T21:16:54.579156+0000","last_became_peered":"2024-04-20T21:16:54.579156+0000","last_unstale":"2024-04-20T21:16:59.606736+0000","last_undegraded":"2024-04-20T21:16:59.606736+0000","last_fullsized":"2024-04-20T21:16:59.606736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:28:14.743667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605543+0000","last_change":"2024-04-20T21:16:53.567995+0000","last_active":"2024-04-20T21:16:59.605543+0000","last_peered":"2024-04-20T21:16:59.605543+0000","last_clean":"2024-04-20T21:16:59.605543+0000","last_became_active":"2024-04-20T21:16:53.567601+0000","last_became_peered":"2024-04-20T21:16:53.567601+0000","last_unstale":"2024-04-20T21:16:59.605543+0000","last_undegraded":"2024-04-20T21:16:59.605543+0000","last_fullsized":"2024-04-20T21:16:59.605543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:30.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609728+0000","last_change":"2024-04-20T21:16:58.626438+0000","last_active":"2024-04-20T21:16:59.609728+0000","last_peered":"2024-04-20T21:16:59.609728+0000","last_clean":"2024-04-20T21:16:59.609728+0000","last_became_active":"2024-04-20T21:16:58.625326+0000","last_became_peered":"2024-04-20T21:16:58.625326+0000","last_unstale":"2024-04-20T21:16:59.609728+0000","last_undegraded":"2024-04-20T21:16:59.609728+0000","last_fullsized":"2024-04-20T21:16:59.609728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:06.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914171+0000","last_change":"2024-04-20T21:16:54.560925+0000","last_active":"2024-04-20T21:16:59.914171+0000","last_peered":"2024-04-20T21:16:59.914171+0000","last_clean":"2024-04-20T21:16:59.914171+0000","last_became_active":"2024-04-20T21:16:54.560615+0000","last_became_peered":"2024-04-20T21:16:54.560615+0000","last_unstale":"2024-04-20T21:16:59.914171+0000","last_undegraded":"2024-04-20T21:16:59.914171+0000","last_fullsized":"2024-04-20T21:16:59.914171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:12.815692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617155+0000","last_change":"2024-04-20T21:16:53.559359+0000","last_active":"2024-04-20T21:16:59.617155+0000","last_peered":"2024-04-20T21:16:59.617155+0000","last_clean":"2024-04-20T21:16:59.617155+0000","last_became_active":"2024-04-20T21:16:53.558955+0000","last_became_peered":"2024-04-20T21:16:53.558955+0000","last_unstale":"2024-04-20T21:16:59.617155+0000","last_undegraded":"2024-04-20T21:16:59.617155+0000","last_fullsized":"2024-04-20T21:16:59.617155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:28.664404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917377+0000","last_change":"2024-04-20T21:16:58.772175+0000","last_active":"2024-04-20T21:16:59.917377+0000","last_peered":"2024-04-20T21:16:59.917377+0000","last_clean":"2024-04-20T21:16:59.917377+0000","last_became_active":"2024-04-20T21:16:58.771732+0000","last_became_peered":"2024-04-20T21:16:58.771732+0000","last_unstale":"2024-04-20T21:16:59.917377+0000","last_undegraded":"2024-04-20T21:16:59.917377+0000","last_fullsized":"2024-04-20T21:16:59.917377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:13.155547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912641+0000","last_change":"2024-04-20T21:16:54.575375+0000","last_active":"2024-04-20T21:16:59.912641+0000","last_peered":"2024-04-20T21:16:59.912641+0000","last_clean":"2024-04-20T21:16:59.912641+0000","last_became_active":"2024-04-20T21:16:54.574194+0000","last_became_peered":"2024-04-20T21:16:54.574194+0000","last_unstale":"2024-04-20T21:16:59.912641+0000","last_undegraded":"2024-04-20T21:16:59.912641+0000","last_fullsized":"2024-04-20T21:16:59.912641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:07.948431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911980+0000","last_change":"2024-04-20T21:16:53.559374+0000","last_active":"2024-04-20T21:16:59.911980+0000","last_peered":"2024-04-20T21:16:59.911980+0000","last_clean":"2024-04-20T21:16:59.911980+0000","last_became_active":"2024-04-20T21:16:53.559133+0000","last_became_peered":"2024-04-20T21:16:53.559133+0000","last_unstale":"2024-04-20T21:16:59.911980+0000","last_undegraded":"2024-04-20T21:16:59.911980+0000","last_fullsized":"2024-04-20T21:16:59.911980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:38.576103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617268+0000","last_change":"2024-04-20T21:16:44.100529+0000","last_active":"2024-04-20T21:16:59.617268+0000","last_peered":"2024-04-20T21:16:59.617268+0000","last_clean":"2024-04-20T21:16:59.617268+0000","last_became_active":"2024-04-20T21:16:44.099590+0000","last_became_peered":"2024-04-20T21:16:44.099590+0000","last_unstale":"2024-04-20T21:16:59.617268+0000","last_undegraded":"2024-04-20T21:16:59.617268+0000","last_fullsized":"2024-04-20T21:16:59.617268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_clean_scrub_stamp":"2024-04-20T21:16:42.782844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:08.054837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911874+0000","last_change":"2024-04-20T21:16:58.772894+0000","last_active":"2024-04-20T21:16:59.911874+0000","last_peered":"2024-04-20T21:16:59.911874+0000","last_clean":"2024-04-20T21:16:59.911874+0000","last_became_active":"2024-04-20T21:16:58.772451+0000","last_became_peered":"2024-04-20T21:16:58.772451+0000","last_unstale":"2024-04-20T21:16:59.911874+0000","last_undegraded":"2024-04-20T21:16:59.911874+0000","last_fullsized":"2024-04-20T21:16:59.911874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:29.914096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617716+0000","last_change":"2024-04-20T21:16:54.562592+0000","last_active":"2024-04-20T21:16:59.617716+0000","last_peered":"2024-04-20T21:16:59.617716+0000","last_clean":"2024-04-20T21:16:59.617716+0000","last_became_active":"2024-04-20T21:16:54.562243+0000","last_became_peered":"2024-04-20T21:16:54.562243+0000","last_unstale":"2024-04-20T21:16:59.617716+0000","last_undegraded":"2024-04-20T21:16:59.617716+0000","last_fullsized":"2024-04-20T21:16:59.617716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:52.832159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606010+0000","last_change":"2024-04-20T21:16:53.561132+0000","last_active":"2024-04-20T21:16:59.606010+0000","last_peered":"2024-04-20T21:16:59.606010+0000","last_clean":"2024-04-20T21:16:59.606010+0000","last_became_active":"2024-04-20T21:16:53.559901+0000","last_became_peered":"2024-04-20T21:16:53.559901+0000","last_unstale":"2024-04-20T21:16:59.606010+0000","last_undegraded":"2024-04-20T21:16:59.606010+0000","last_fullsized":"2024-04-20T21:16:59.606010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:20.229573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911503+0000","last_change":"2024-04-20T21:16:58.771204+0000","last_active":"2024-04-20T21:16:59.911503+0000","last_peered":"2024-04-20T21:16:59.911503+0000","last_clean":"2024-04-20T21:16:59.911503+0000","last_became_active":"2024-04-20T21:16:58.770627+0000","last_became_peered":"2024-04-20T21:16:58.770627+0000","last_unstale":"2024-04-20T21:16:59.911503+0000","last_undegraded":"2024-04-20T21:16:59.911503+0000","last_fullsized":"2024-04-20T21:16:59.911503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:39.472931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912685+0000","last_change":"2024-04-20T21:16:54.571417+0000","last_active":"2024-04-20T21:16:59.912685+0000","last_peered":"2024-04-20T21:16:59.912685+0000","last_clean":"2024-04-20T21:16:59.912685+0000","last_became_active":"2024-04-20T21:16:54.570434+0000","last_became_peered":"2024-04-20T21:16:54.570434+0000","last_unstale":"2024-04-20T21:16:59.912685+0000","last_undegraded":"2024-04-20T21:16:59.912685+0000","last_fullsized":"2024-04-20T21:16:59.912685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:24:41.026138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609282+0000","last_change":"2024-04-20T21:16:53.558554+0000","last_active":"2024-04-20T21:16:59.609282+0000","last_peered":"2024-04-20T21:16:59.609282+0000","last_clean":"2024-04-20T21:16:59.609282+0000","last_became_active":"2024-04-20T21:16:53.556934+0000","last_became_peered":"2024-04-20T21:16:53.556934+0000","last_unstale":"2024-04-20T21:16:59.609282+0000","last_undegraded":"2024-04-20T21:16:59.609282+0000","last_fullsized":"2024-04-20T21:16:59.609282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:17:52.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915951+0000","last_change":"2024-04-20T21:16:58.624410+0000","last_active":"2024-04-20T21:16:59.915951+0000","last_peered":"2024-04-20T21:16:59.915951+0000","last_clean":"2024-04-20T21:16:59.915951+0000","last_became_active":"2024-04-20T21:16:58.623677+0000","last_became_peered":"2024-04-20T21:16:58.623677+0000","last_unstale":"2024-04-20T21:16:59.915951+0000","last_undegraded":"2024-04-20T21:16:59.915951+0000","last_fullsized":"2024-04-20T21:16:59.915951+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:34.465594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617870+0000","last_change":"2024-04-20T21:16:54.560217+0000","last_active":"2024-04-20T21:16:59.617870+0000","last_peered":"2024-04-20T21:16:59.617870+0000","last_clean":"2024-04-20T21:16:59.617870+0000","last_became_active":"2024-04-20T21:16:54.559947+0000","last_became_peered":"2024-04-20T21:16:54.559947+0000","last_unstale":"2024-04-20T21:16:59.617870+0000","last_undegraded":"2024-04-20T21:16:59.617870+0000","last_fullsized":"2024-04-20T21:16:59.617870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:54.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.607164+0000","last_change":"2024-04-20T21:16:53.564779+0000","last_active":"2024-04-20T21:16:59.607164+0000","last_peered":"2024-04-20T21:16:59.607164+0000","last_clean":"2024-04-20T21:16:59.607164+0000","last_became_active":"2024-04-20T21:16:53.564500+0000","last_became_peered":"2024-04-20T21:16:53.564500+0000","last_unstale":"2024-04-20T21:16:59.607164+0000","last_undegraded":"2024-04-20T21:16:59.607164+0000","last_fullsized":"2024-04-20T21:16:59.607164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:00:58.673145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912319+0000","last_change":"2024-04-20T21:16:58.771524+0000","last_active":"2024-04-20T21:16:59.912319+0000","last_peered":"2024-04-20T21:16:59.912319+0000","last_clean":"2024-04-20T21:16:59.912319+0000","last_became_active":"2024-04-20T21:16:58.770672+0000","last_became_peered":"2024-04-20T21:16:58.770672+0000","last_unstale":"2024-04-20T21:16:59.912319+0000","last_undegraded":"2024-04-20T21:16:59.912319+0000","last_fullsized":"2024-04-20T21:16:59.912319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:52:27.037256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617704+0000","last_change":"2024-04-20T21:16:54.559570+0000","last_active":"2024-04-20T21:16:59.617704+0000","last_peered":"2024-04-20T21:16:59.617704+0000","last_clean":"2024-04-20T21:16:59.617704+0000","last_became_active":"2024-04-20T21:16:54.559307+0000","last_became_peered":"2024-04-20T21:16:54.559307+0000","last_unstale":"2024-04-20T21:16:59.617704+0000","last_undegraded":"2024-04-20T21:16:59.617704+0000","last_fullsized":"2024-04-20T21:16:59.617704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:04.417348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605512+0000","last_change":"2024-04-20T21:16:53.565289+0000","last_active":"2024-04-20T21:16:59.605512+0000","last_peered":"2024-04-20T21:16:59.605512+0000","last_clean":"2024-04-20T21:16:59.605512+0000","last_became_active":"2024-04-20T21:16:53.565014+0000","last_became_peered":"2024-04-20T21:16:53.565014+0000","last_unstale":"2024-04-20T21:16:59.605512+0000","last_undegraded":"2024-04-20T21:16:59.605512+0000","last_fullsized":"2024-04-20T21:16:59.605512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:22.125301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604480+0000","last_change":"2024-04-20T21:16:58.615103+0000","last_active":"2024-04-20T21:16:59.604480+0000","last_peered":"2024-04-20T21:16:59.604480+0000","last_clean":"2024-04-20T21:16:59.604480+0000","last_became_active":"2024-04-20T21:16:58.614068+0000","last_became_peered":"2024-04-20T21:16:58.614068+0000","last_unstale":"2024-04-20T21:16:59.604480+0000","last_undegraded":"2024-04-20T21:16:59.604480+0000","last_fullsized":"2024-04-20T21:16:59.604480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:09.555911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617489+0000","last_change":"2024-04-20T21:16:54.557439+0000","last_active":"2024-04-20T21:16:59.617489+0000","last_peered":"2024-04-20T21:16:59.617489+0000","last_clean":"2024-04-20T21:16:59.617489+0000","last_became_active":"2024-04-20T21:16:54.557178+0000","last_became_peered":"2024-04-20T21:16:54.557178+0000","last_unstale":"2024-04-20T21:16:59.617489+0000","last_undegraded":"2024-04-20T21:16:59.617489+0000","last_fullsized":"2024-04-20T21:16:59.617489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:55:33.964948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605750+0000","last_change":"2024-04-20T21:16:53.570182+0000","last_active":"2024-04-20T21:16:59.605750+0000","last_peered":"2024-04-20T21:16:59.605750+0000","last_clean":"2024-04-20T21:16:59.605750+0000","last_became_active":"2024-04-20T21:16:53.569632+0000","last_became_peered":"2024-04-20T21:16:53.569632+0000","last_unstale":"2024-04-20T21:16:59.605750+0000","last_undegraded":"2024-04-20T21:16:59.605750+0000","last_fullsized":"2024-04-20T21:16:59.605750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:12.140497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917381+0000","last_change":"2024-04-20T21:16:58.625602+0000","last_active":"2024-04-20T21:16:59.917381+0000","last_peered":"2024-04-20T21:16:59.917381+0000","last_clean":"2024-04-20T21:16:59.917381+0000","last_became_active":"2024-04-20T21:16:58.625158+0000","last_became_peered":"2024-04-20T21:16:58.625158+0000","last_unstale":"2024-04-20T21:16:59.917381+0000","last_undegraded":"2024-04-20T21:16:59.917381+0000","last_fullsized":"2024-04-20T21:16:59.917381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:09.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911816+0000","last_change":"2024-04-20T21:16:54.578603+0000","last_active":"2024-04-20T21:16:59.911816+0000","last_peered":"2024-04-20T21:16:59.911816+0000","last_clean":"2024-04-20T21:16:59.911816+0000","last_became_active":"2024-04-20T21:16:54.578338+0000","last_became_peered":"2024-04-20T21:16:54.578338+0000","last_unstale":"2024-04-20T21:16:59.911816+0000","last_undegraded":"2024-04-20T21:16:59.911816+0000","last_fullsized":"2024-04-20T21:16:59.911816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:14:56.355343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911553+0000","last_change":"2024-04-20T21:16:53.551847+0000","last_active":"2024-04-20T21:16:59.911553+0000","last_peered":"2024-04-20T21:16:59.911553+0000","last_clean":"2024-04-20T21:16:59.911553+0000","last_became_active":"2024-04-20T21:16:53.551427+0000","last_became_peered":"2024-04-20T21:16:53.551427+0000","last_unstale":"2024-04-20T21:16:59.911553+0000","last_undegraded":"2024-04-20T21:16:59.911553+0000","last_fullsized":"2024-04-20T21:16:59.911553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:55:45.930305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916262+0000","last_change":"2024-04-20T21:16:58.777159+0000","last_active":"2024-04-20T21:16:59.916262+0000","last_peered":"2024-04-20T21:16:59.916262+0000","last_clean":"2024-04-20T21:16:59.916262+0000","last_became_active":"2024-04-20T21:16:58.776233+0000","last_became_peered":"2024-04-20T21:16:58.776233+0000","last_unstale":"2024-04-20T21:16:59.916262+0000","last_undegraded":"2024-04-20T21:16:59.916262+0000","last_fullsized":"2024-04-20T21:16:59.916262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:05.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617642+0000","last_change":"2024-04-20T21:16:54.570687+0000","last_active":"2024-04-20T21:16:59.617642+0000","last_peered":"2024-04-20T21:16:59.617642+0000","last_clean":"2024-04-20T21:16:59.617642+0000","last_became_active":"2024-04-20T21:16:54.570360+0000","last_became_peered":"2024-04-20T21:16:54.570360+0000","last_unstale":"2024-04-20T21:16:59.617642+0000","last_undegraded":"2024-04-20T21:16:59.617642+0000","last_fullsized":"2024-04-20T21:16:59.617642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:05.606229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912222+0000","last_change":"2024-04-20T21:16:53.567599+0000","last_active":"2024-04-20T21:16:59.912222+0000","last_peered":"2024-04-20T21:16:59.912222+0000","last_clean":"2024-04-20T21:16:59.912222+0000","last_became_active":"2024-04-20T21:16:53.567378+0000","last_became_peered":"2024-04-20T21:16:53.567378+0000","last_unstale":"2024-04-20T21:16:59.912222+0000","last_undegraded":"2024-04-20T21:16:59.912222+0000","last_fullsized":"2024-04-20T21:16:59.912222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:36.611887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914503+0000","last_change":"2024-04-20T21:16:58.774226+0000","last_active":"2024-04-20T21:16:59.914503+0000","last_peered":"2024-04-20T21:16:59.914503+0000","last_clean":"2024-04-20T21:16:59.914503+0000","last_became_active":"2024-04-20T21:16:58.771584+0000","last_became_peered":"2024-04-20T21:16:58.771584+0000","last_unstale":"2024-04-20T21:16:59.914503+0000","last_undegraded":"2024-04-20T21:16:59.914503+0000","last_fullsized":"2024-04-20T21:16:59.914503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:31.432077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915175+0000","last_change":"2024-04-20T21:16:54.560001+0000","last_active":"2024-04-20T21:16:59.915175+0000","last_peered":"2024-04-20T21:16:59.915175+0000","last_clean":"2024-04-20T21:16:59.915175+0000","last_became_active":"2024-04-20T21:16:54.559643+0000","last_became_peered":"2024-04-20T21:16:54.559643+0000","last_unstale":"2024-04-20T21:16:59.915175+0000","last_undegraded":"2024-04-20T21:16:59.915175+0000","last_fullsized":"2024-04-20T21:16:59.915175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:52:37.325946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916695+0000","last_change":"2024-04-20T21:16:53.570498+0000","last_active":"2024-04-20T21:16:59.916695+0000","last_peered":"2024-04-20T21:16:59.916695+0000","last_clean":"2024-04-20T21:16:59.916695+0000","last_became_active":"2024-04-20T21:16:53.570095+0000","last_became_peered":"2024-04-20T21:16:53.570095+0000","last_unstale":"2024-04-20T21:16:59.916695+0000","last_undegraded":"2024-04-20T21:16:59.916695+0000","last_fullsized":"2024-04-20T21:16:59.916695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:58:28.677814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912403+0000","last_change":"2024-04-20T21:16:58.773349+0000","last_active":"2024-04-20T21:16:59.912403+0000","last_peered":"2024-04-20T21:16:59.912403+0000","last_clean":"2024-04-20T21:16:59.912403+0000","last_became_active":"2024-04-20T21:16:58.772781+0000","last_became_peered":"2024-04-20T21:16:58.772781+0000","last_unstale":"2024-04-20T21:16:59.912403+0000","last_undegraded":"2024-04-20T21:16:59.912403+0000","last_fullsized":"2024-04-20T21:16:59.912403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:53:02.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617377+0000","last_change":"2024-04-20T21:16:54.572749+0000","last_active":"2024-04-20T21:16:59.617377+0000","last_peered":"2024-04-20T21:16:59.617377+0000","last_clean":"2024-04-20T21:16:59.617377+0000","last_became_active":"2024-04-20T21:16:54.572456+0000","last_became_peered":"2024-04-20T21:16:54.572456+0000","last_unstale":"2024-04-20T21:16:59.617377+0000","last_undegraded":"2024-04-20T21:16:59.617377+0000","last_fullsized":"2024-04-20T21:16:59.617377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.787612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:17:17.226094+0000","last_change":"2024-04-20T21:16:53.568688+0000","last_active":"2024-04-20T21:17:17.226094+0000","last_peered":"2024-04-20T21:17:17.226094+0000","last_clean":"2024-04-20T21:17:17.226094+0000","last_became_active":"2024-04-20T21:16:53.568495+0000","last_became_peered":"2024-04-20T21:16:53.568495+0000","last_unstale":"2024-04-20T21:17:17.226094+0000","last_undegraded":"2024-04-20T21:17:17.226094+0000","last_fullsized":"2024-04-20T21:17:17.226094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:00.086004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608039+0000","last_change":"2024-04-20T21:16:58.609575+0000","last_active":"2024-04-20T21:16:59.608039+0000","last_peered":"2024-04-20T21:16:59.608039+0000","last_clean":"2024-04-20T21:16:59.608039+0000","last_became_active":"2024-04-20T21:16:58.609127+0000","last_became_peered":"2024-04-20T21:16:58.609127+0000","last_unstale":"2024-04-20T21:16:59.608039+0000","last_undegraded":"2024-04-20T21:16:59.608039+0000","last_fullsized":"2024-04-20T21:16:59.608039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:02.645274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605015+0000","last_change":"2024-04-20T21:16:54.563213+0000","last_active":"2024-04-20T21:16:59.605015+0000","last_peered":"2024-04-20T21:16:59.605015+0000","last_clean":"2024-04-20T21:16:59.605015+0000","last_became_active":"2024-04-20T21:16:54.562917+0000","last_became_peered":"2024-04-20T21:16:54.562917+0000","last_unstale":"2024-04-20T21:16:59.605015+0000","last_undegraded":"2024-04-20T21:16:59.605015+0000","last_fullsized":"2024-04-20T21:16:59.605015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.155771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609082+0000","last_change":"2024-04-20T21:16:53.556988+0000","last_active":"2024-04-20T21:16:59.609082+0000","last_peered":"2024-04-20T21:16:59.609082+0000","last_clean":"2024-04-20T21:16:59.609082+0000","last_became_active":"2024-04-20T21:16:53.556672+0000","last_became_peered":"2024-04-20T21:16:53.556672+0000","last_unstale":"2024-04-20T21:16:59.609082+0000","last_undegraded":"2024-04-20T21:16:59.609082+0000","last_fullsized":"2024-04-20T21:16:59.609082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:25.219611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604713+0000","last_change":"2024-04-20T21:16:58.595827+0000","last_active":"2024-04-20T21:16:59.604713+0000","last_peered":"2024-04-20T21:16:59.604713+0000","last_clean":"2024-04-20T21:16:59.604713+0000","last_became_active":"2024-04-20T21:16:58.595403+0000","last_became_peered":"2024-04-20T21:16:58.595403+0000","last_unstale":"2024-04-20T21:16:59.604713+0000","last_undegraded":"2024-04-20T21:16:59.604713+0000","last_fullsized":"2024-04-20T21:16:59.604713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:53.227573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918376+0000","last_change":"2024-04-20T21:16:54.562128+0000","last_active":"2024-04-20T21:16:59.918376+0000","last_peered":"2024-04-20T21:16:59.918376+0000","last_clean":"2024-04-20T21:16:59.918376+0000","last_became_active":"2024-04-20T21:16:54.561495+0000","last_became_peered":"2024-04-20T21:16:54.561495+0000","last_unstale":"2024-04-20T21:16:59.918376+0000","last_undegraded":"2024-04-20T21:16:59.918376+0000","last_fullsized":"2024-04-20T21:16:59.918376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:19.582218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911795+0000","last_change":"2024-04-20T21:16:53.554881+0000","last_active":"2024-04-20T21:16:59.911795+0000","last_peered":"2024-04-20T21:16:59.911795+0000","last_clean":"2024-04-20T21:16:59.911795+0000","last_became_active":"2024-04-20T21:16:53.554590+0000","last_became_peered":"2024-04-20T21:16:53.554590+0000","last_unstale":"2024-04-20T21:16:59.911795+0000","last_undegraded":"2024-04-20T21:16:59.911795+0000","last_fullsized":"2024-04-20T21:16:59.911795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:36.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913934+0000","last_change":"2024-04-20T21:16:58.610704+0000","last_active":"2024-04-20T21:16:59.913934+0000","last_peered":"2024-04-20T21:16:59.913934+0000","last_clean":"2024-04-20T21:16:59.913934+0000","last_became_active":"2024-04-20T21:16:58.610309+0000","last_became_peered":"2024-04-20T21:16:58.610309+0000","last_unstale":"2024-04-20T21:16:59.913934+0000","last_undegraded":"2024-04-20T21:16:59.913934+0000","last_fullsized":"2024-04-20T21:16:59.913934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:04.215050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911685+0000","last_change":"2024-04-20T21:16:54.562351+0000","last_active":"2024-04-20T21:16:59.911685+0000","last_peered":"2024-04-20T21:16:59.911685+0000","last_clean":"2024-04-20T21:16:59.911685+0000","last_became_active":"2024-04-20T21:16:54.561947+0000","last_became_peered":"2024-04-20T21:16:54.561947+0000","last_unstale":"2024-04-20T21:16:59.911685+0000","last_undegraded":"2024-04-20T21:16:59.911685+0000","last_fullsized":"2024-04-20T21:16:59.911685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:52.802328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912701+0000","last_change":"2024-04-20T21:16:53.554081+0000","last_active":"2024-04-20T21:16:59.912701+0000","last_peered":"2024-04-20T21:16:59.912701+0000","last_clean":"2024-04-20T21:16:59.912701+0000","last_became_active":"2024-04-20T21:16:53.553800+0000","last_became_peered":"2024-04-20T21:16:53.553800+0000","last_unstale":"2024-04-20T21:16:59.912701+0000","last_undegraded":"2024-04-20T21:16:59.912701+0000","last_fullsized":"2024-04-20T21:16:59.912701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:52.865485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914034+0000","last_change":"2024-04-20T21:16:58.779184+0000","last_active":"2024-04-20T21:16:59.914034+0000","last_peered":"2024-04-20T21:16:59.914034+0000","last_clean":"2024-04-20T21:16:59.914034+0000","last_became_active":"2024-04-20T21:16:58.778627+0000","last_became_peered":"2024-04-20T21:16:58.778627+0000","last_unstale":"2024-04-20T21:16:59.914034+0000","last_undegraded":"2024-04-20T21:16:59.914034+0000","last_fullsized":"2024-04-20T21:16:59.914034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:50:07.247736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917097+0000","last_change":"2024-04-20T21:16:58.774438+0000","last_active":"2024-04-20T21:16:59.917097+0000","last_peered":"2024-04-20T21:16:59.917097+0000","last_clean":"2024-04-20T21:16:59.917097+0000","last_became_active":"2024-04-20T21:16:58.774030+0000","last_became_peered":"2024-04-20T21:16:58.774030+0000","last_unstale":"2024-04-20T21:16:59.917097+0000","last_undegraded":"2024-04-20T21:16:59.917097+0000","last_fullsized":"2024-04-20T21:16:59.917097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:09:22.583239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606553+0000","last_change":"2024-04-20T21:16:54.573779+0000","last_active":"2024-04-20T21:16:59.606553+0000","last_peered":"2024-04-20T21:16:59.606553+0000","last_clean":"2024-04-20T21:16:59.606553+0000","last_became_active":"2024-04-20T21:16:54.573444+0000","last_became_peered":"2024-04-20T21:16:54.573444+0000","last_unstale":"2024-04-20T21:16:59.606553+0000","last_undegraded":"2024-04-20T21:16:59.606553+0000","last_fullsized":"2024-04-20T21:16:59.606553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:27.950525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911680+0000","last_change":"2024-04-20T21:16:53.543678+0000","last_active":"2024-04-20T21:16:59.911680+0000","last_peered":"2024-04-20T21:16:59.911680+0000","last_clean":"2024-04-20T21:16:59.911680+0000","last_became_active":"2024-04-20T21:16:53.543381+0000","last_became_peered":"2024-04-20T21:16:53.543381+0000","last_unstale":"2024-04-20T21:16:59.911680+0000","last_undegraded":"2024-04-20T21:16:59.911680+0000","last_fullsized":"2024-04-20T21:16:59.911680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:02.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912679+0000","last_change":"2024-04-20T21:16:58.626882+0000","last_active":"2024-04-20T21:16:59.912679+0000","last_peered":"2024-04-20T21:16:59.912679+0000","last_clean":"2024-04-20T21:16:59.912679+0000","last_became_active":"2024-04-20T21:16:58.626489+0000","last_became_peered":"2024-04-20T21:16:58.626489+0000","last_unstale":"2024-04-20T21:16:59.912679+0000","last_undegraded":"2024-04-20T21:16:59.912679+0000","last_fullsized":"2024-04-20T21:16:59.912679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:33:22.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913977+0000","last_change":"2024-04-20T21:16:54.580893+0000","last_active":"2024-04-20T21:16:59.913977+0000","last_peered":"2024-04-20T21:16:59.913977+0000","last_clean":"2024-04-20T21:16:59.913977+0000","last_became_active":"2024-04-20T21:16:54.580528+0000","last_became_peered":"2024-04-20T21:16:54.580528+0000","last_unstale":"2024-04-20T21:16:59.913977+0000","last_undegraded":"2024-04-20T21:16:59.913977+0000","last_fullsized":"2024-04-20T21:16:59.913977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:14.699123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914639+0000","last_change":"2024-04-20T21:16:53.563930+0000","last_active":"2024-04-20T21:16:59.914639+0000","last_peered":"2024-04-20T21:16:59.914639+0000","last_clean":"2024-04-20T21:16:59.914639+0000","last_became_active":"2024-04-20T21:16:53.559991+0000","last_became_peered":"2024-04-20T21:16:53.559991+0000","last_unstale":"2024-04-20T21:16:59.914639+0000","last_undegraded":"2024-04-20T21:16:59.914639+0000","last_fullsized":"2024-04-20T21:16:59.914639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:22.529712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912173+0000","last_change":"2024-04-20T21:16:53.552509+0000","last_active":"2024-04-20T21:16:59.912173+0000","last_peered":"2024-04-20T21:16:59.912173+0000","last_clean":"2024-04-20T21:16:59.912173+0000","last_became_active":"2024-04-20T21:16:53.552173+0000","last_became_peered":"2024-04-20T21:16:53.552173+0000","last_unstale":"2024-04-20T21:16:59.912173+0000","last_undegraded":"2024-04-20T21:16:59.912173+0000","last_fullsized":"2024-04-20T21:16:59.912173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:14.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916470+0000","last_change":"2024-04-20T21:16:54.562849+0000","last_active":"2024-04-20T21:16:59.916470+0000","last_peered":"2024-04-20T21:16:59.916470+0000","last_clean":"2024-04-20T21:16:59.916470+0000","last_became_active":"2024-04-20T21:16:54.561283+0000","last_became_peered":"2024-04-20T21:16:54.561283+0000","last_unstale":"2024-04-20T21:16:59.916470+0000","last_undegraded":"2024-04-20T21:16:59.916470+0000","last_fullsized":"2024-04-20T21:16:59.916470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:52.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605952+0000","last_change":"2024-04-20T21:16:58.614752+0000","last_active":"2024-04-20T21:16:59.605952+0000","last_peered":"2024-04-20T21:16:59.605952+0000","last_clean":"2024-04-20T21:16:59.605952+0000","last_became_active":"2024-04-20T21:16:58.614201+0000","last_became_peered":"2024-04-20T21:16:58.614201+0000","last_unstale":"2024-04-20T21:16:59.605952+0000","last_undegraded":"2024-04-20T21:16:59.605952+0000","last_fullsized":"2024-04-20T21:16:59.605952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:46.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913587+0000","last_change":"2024-04-20T21:16:53.564303+0000","last_active":"2024-04-20T21:16:59.913587+0000","last_peered":"2024-04-20T21:16:59.913587+0000","last_clean":"2024-04-20T21:16:59.913587+0000","last_became_active":"2024-04-20T21:16:53.560203+0000","last_became_peered":"2024-04-20T21:16:53.560203+0000","last_unstale":"2024-04-20T21:16:59.913587+0000","last_undegraded":"2024-04-20T21:16:59.913587+0000","last_fullsized":"2024-04-20T21:16:59.913587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:42.691233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604601+0000","last_change":"2024-04-20T21:16:54.554499+0000","last_active":"2024-04-20T21:16:59.604601+0000","last_peered":"2024-04-20T21:16:59.604601+0000","last_clean":"2024-04-20T21:16:59.604601+0000","last_became_active":"2024-04-20T21:16:54.554230+0000","last_became_peered":"2024-04-20T21:16:54.554230+0000","last_unstale":"2024-04-20T21:16:59.604601+0000","last_undegraded":"2024-04-20T21:16:59.604601+0000","last_fullsized":"2024-04-20T21:16:59.604601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:24:31.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915386+0000","last_change":"2024-04-20T21:16:58.610424+0000","last_active":"2024-04-20T21:16:59.915386+0000","last_peered":"2024-04-20T21:16:59.915386+0000","last_clean":"2024-04-20T21:16:59.915386+0000","last_became_active":"2024-04-20T21:16:58.610025+0000","last_became_peered":"2024-04-20T21:16:58.610025+0000","last_unstale":"2024-04-20T21:16:59.915386+0000","last_undegraded":"2024-04-20T21:16:59.915386+0000","last_fullsized":"2024-04-20T21:16:59.915386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:19.290865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911710+0000","last_change":"2024-04-20T21:16:54.578801+0000","last_active":"2024-04-20T21:16:59.911710+0000","last_peered":"2024-04-20T21:16:59.911710+0000","last_clean":"2024-04-20T21:16:59.911710+0000","last_became_active":"2024-04-20T21:16:54.578543+0000","last_became_peered":"2024-04-20T21:16:54.578543+0000","last_unstale":"2024-04-20T21:16:59.911710+0000","last_undegraded":"2024-04-20T21:16:59.911710+0000","last_fullsized":"2024-04-20T21:16:59.911710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:15.042318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605448+0000","last_change":"2024-04-20T21:16:54.565658+0000","last_active":"2024-04-20T21:16:59.605448+0000","last_peered":"2024-04-20T21:16:59.605448+0000","last_clean":"2024-04-20T21:16:59.605448+0000","last_became_active":"2024-04-20T21:16:54.565371+0000","last_became_peered":"2024-04-20T21:16:54.565371+0000","last_unstale":"2024-04-20T21:16:59.605448+0000","last_undegraded":"2024-04-20T21:16:59.605448+0000","last_fullsized":"2024-04-20T21:16:59.605448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:21.781013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912115+0000","last_change":"2024-04-20T21:16:54.568472+0000","last_active":"2024-04-20T21:16:59.912115+0000","last_peered":"2024-04-20T21:16:59.912115+0000","last_clean":"2024-04-20T21:16:59.912115+0000","last_became_active":"2024-04-20T21:16:54.568223+0000","last_became_peered":"2024-04-20T21:16:54.568223+0000","last_unstale":"2024-04-20T21:16:59.912115+0000","last_undegraded":"2024-04-20T21:16:59.912115+0000","last_fullsized":"2024-04-20T21:16:59.912115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:42:53.921935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916243+0000","last_change":"2024-04-20T21:16:54.573747+0000","last_active":"2024-04-20T21:16:59.916243+0000","last_peered":"2024-04-20T21:16:59.916243+0000","last_clean":"2024-04-20T21:16:59.916243+0000","last_became_active":"2024-04-20T21:16:54.573476+0000","last_became_peered":"2024-04-20T21:16:54.573476+0000","last_unstale":"2024-04-20T21:16:59.916243+0000","last_undegraded":"2024-04-20T21:16:59.916243+0000","last_fullsized":"2024-04-20T21:16:59.916243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:02.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913473+0000","last_change":"2024-04-20T21:16:54.577630+0000","last_active":"2024-04-20T21:16:59.913473+0000","last_peered":"2024-04-20T21:16:59.913473+0000","last_clean":"2024-04-20T21:16:59.913473+0000","last_became_active":"2024-04-20T21:16:54.577318+0000","last_became_peered":"2024-04-20T21:16:54.577318+0000","last_unstale":"2024-04-20T21:16:59.913473+0000","last_undegraded":"2024-04-20T21:16:59.913473+0000","last_fullsized":"2024-04-20T21:16:59.913473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:45.584050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608267+0000","last_change":"2024-04-20T21:16:54.579850+0000","last_active":"2024-04-20T21:16:59.608267+0000","last_peered":"2024-04-20T21:16:59.608267+0000","last_clean":"2024-04-20T21:16:59.608267+0000","last_became_active":"2024-04-20T21:16:54.579546+0000","last_became_peered":"2024-04-20T21:16:54.579546+0000","last_unstale":"2024-04-20T21:16:59.608267+0000","last_undegraded":"2024-04-20T21:16:59.608267+0000","last_fullsized":"2024-04-20T21:16:59.608267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:49:53.577077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912501+0000","last_change":"2024-04-20T21:16:54.559174+0000","last_active":"2024-04-20T21:16:59.912501+0000","last_peered":"2024-04-20T21:16:59.912501+0000","last_clean":"2024-04-20T21:16:59.912501+0000","last_became_active":"2024-04-20T21:16:54.558904+0000","last_became_peered":"2024-04-20T21:16:54.558904+0000","last_unstale":"2024-04-20T21:16:59.912501+0000","last_undegraded":"2024-04-20T21:16:59.912501+0000","last_fullsized":"2024-04-20T21:16:59.912501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:09.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916492+0000","last_change":"2024-04-20T21:16:54.574943+0000","last_active":"2024-04-20T21:16:59.916492+0000","last_peered":"2024-04-20T21:16:59.916492+0000","last_clean":"2024-04-20T21:16:59.916492+0000","last_became_active":"2024-04-20T21:16:54.574471+0000","last_became_peered":"2024-04-20T21:16:54.574471+0000","last_unstale":"2024-04-20T21:16:59.916492+0000","last_undegraded":"2024-04-20T21:16:59.916492+0000","last_fullsized":"2024-04-20T21:16:59.916492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:51:44.741339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911545+0000","last_change":"2024-04-20T21:16:54.566222+0000","last_active":"2024-04-20T21:16:59.911545+0000","last_peered":"2024-04-20T21:16:59.911545+0000","last_clean":"2024-04-20T21:16:59.911545+0000","last_became_active":"2024-04-20T21:16:54.566011+0000","last_became_peered":"2024-04-20T21:16:54.566011+0000","last_unstale":"2024-04-20T21:16:59.911545+0000","last_undegraded":"2024-04-20T21:16:59.911545+0000","last_fullsized":"2024-04-20T21:16:59.911545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:53.369950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914398+0000","last_change":"2024-04-20T21:16:54.571179+0000","last_active":"2024-04-20T21:16:59.914398+0000","last_peered":"2024-04-20T21:16:59.914398+0000","last_clean":"2024-04-20T21:16:59.914398+0000","last_became_active":"2024-04-20T21:16:54.570855+0000","last_became_peered":"2024-04-20T21:16:54.570855+0000","last_unstale":"2024-04-20T21:16:59.914398+0000","last_undegraded":"2024-04-20T21:16:59.914398+0000","last_fullsized":"2024-04-20T21:16:59.914398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:59.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911999+0000","last_change":"2024-04-20T21:16:54.559092+0000","last_active":"2024-04-20T21:16:59.911999+0000","last_peered":"2024-04-20T21:16:59.911999+0000","last_clean":"2024-04-20T21:16:59.911999+0000","last_became_active":"2024-04-20T21:16:54.558847+0000","last_became_peered":"2024-04-20T21:16:54.558847+0000","last_unstale":"2024-04-20T21:16:59.911999+0000","last_undegraded":"2024-04-20T21:16:59.911999+0000","last_fullsized":"2024-04-20T21:16:59.911999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:02.752070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915426+0000","last_change":"2024-04-20T21:16:54.571815+0000","last_active":"2024-04-20T21:16:59.915426+0000","last_peered":"2024-04-20T21:16:59.915426+0000","last_clean":"2024-04-20T21:16:59.915426+0000","last_became_active":"2024-04-20T21:16:54.570704+0000","last_became_peered":"2024-04-20T21:16:54.570704+0000","last_unstale":"2024-04-20T21:16:59.915426+0000","last_undegraded":"2024-04-20T21:16:59.915426+0000","last_fullsized":"2024-04-20T21:16:59.915426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:56.511539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912064+0000","last_change":"2024-04-20T21:16:54.555591+0000","last_active":"2024-04-20T21:16:59.912064+0000","last_peered":"2024-04-20T21:16:59.912064+0000","last_clean":"2024-04-20T21:16:59.912064+0000","last_became_active":"2024-04-20T21:16:54.555389+0000","last_became_peered":"2024-04-20T21:16:54.555389+0000","last_unstale":"2024-04-20T21:16:59.912064+0000","last_undegraded":"2024-04-20T21:16:59.912064+0000","last_fullsized":"2024-04-20T21:16:59.912064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:16:25.976707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617803+0000","last_change":"2024-04-20T21:16:54.564088+0000","last_active":"2024-04-20T21:16:59.617803+0000","last_peered":"2024-04-20T21:16:59.617803+0000","last_clean":"2024-04-20T21:16:59.617803+0000","last_became_active":"2024-04-20T21:16:54.563796+0000","last_became_peered":"2024-04-20T21:16:54.563796+0000","last_unstale":"2024-04-20T21:16:59.617803+0000","last_undegraded":"2024-04-20T21:16:59.617803+0000","last_fullsized":"2024-04-20T21:16:59.617803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:07.986736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606878+0000","last_change":"2024-04-20T21:16:54.563578+0000","last_active":"2024-04-20T21:16:59.606878+0000","last_peered":"2024-04-20T21:16:59.606878+0000","last_clean":"2024-04-20T21:16:59.606878+0000","last_became_active":"2024-04-20T21:16:54.563317+0000","last_became_peered":"2024-04-20T21:16:54.563317+0000","last_unstale":"2024-04-20T21:16:59.606878+0000","last_undegraded":"2024-04-20T21:16:59.606878+0000","last_fullsized":"2024-04-20T21:16:59.606878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:06.093675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912217+0000","last_change":"2024-04-20T21:16:54.565168+0000","last_active":"2024-04-20T21:16:59.912217+0000","last_peered":"2024-04-20T21:16:59.912217+0000","last_clean":"2024-04-20T21:16:59.912217+0000","last_became_active":"2024-04-20T21:16:54.564712+0000","last_became_peered":"2024-04-20T21:16:54.564712+0000","last_unstale":"2024-04-20T21:16:59.912217+0000","last_undegraded":"2024-04-20T21:16:59.912217+0000","last_fullsized":"2024-04-20T21:16:59.912217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:07:30.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911433+0000","last_change":"2024-04-20T21:16:54.572211+0000","last_active":"2024-04-20T21:16:59.911433+0000","last_peered":"2024-04-20T21:16:59.911433+0000","last_clean":"2024-04-20T21:16:59.911433+0000","last_became_active":"2024-04-20T21:16:54.571894+0000","last_became_peered":"2024-04-20T21:16:54.571894+0000","last_unstale":"2024-04-20T21:16:59.911433+0000","last_undegraded":"2024-04-20T21:16:59.911433+0000","last_fullsized":"2024-04-20T21:16:59.911433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:59.376514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606682+0000","last_change":"2024-04-20T21:16:54.555195+0000","last_active":"2024-04-20T21:16:59.606682+0000","last_peered":"2024-04-20T21:16:59.606682+0000","last_clean":"2024-04-20T21:16:59.606682+0000","last_became_active":"2024-04-20T21:16:54.554284+0000","last_became_peered":"2024-04-20T21:16:54.554284+0000","last_unstale":"2024-04-20T21:16:59.606682+0000","last_undegraded":"2024-04-20T21:16:59.606682+0000","last_fullsized":"2024-04-20T21:16:59.606682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:42:21.500952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915085+0000","last_change":"2024-04-20T21:16:54.565527+0000","last_active":"2024-04-20T21:16:59.915085+0000","last_peered":"2024-04-20T21:16:59.915085+0000","last_clean":"2024-04-20T21:16:59.915085+0000","last_became_active":"2024-04-20T21:16:54.565211+0000","last_became_peered":"2024-04-20T21:16:54.565211+0000","last_unstale":"2024-04-20T21:16:59.915085+0000","last_undegraded":"2024-04-20T21:16:59.915085+0000","last_fullsized":"2024-04-20T21:16:59.915085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:17:40.462670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913206+0000","last_change":"2024-04-20T21:16:54.576464+0000","last_active":"2024-04-20T21:16:59.913206+0000","last_peered":"2024-04-20T21:16:59.913206+0000","last_clean":"2024-04-20T21:16:59.913206+0000","last_became_active":"2024-04-20T21:16:54.575947+0000","last_became_peered":"2024-04-20T21:16:54.575947+0000","last_unstale":"2024-04-20T21:16:59.913206+0000","last_undegraded":"2024-04-20T21:16:59.913206+0000","last_fullsized":"2024-04-20T21:16:59.913206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:50:29.581938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604732+0000","last_change":"2024-04-20T21:16:54.558162+0000","last_active":"2024-04-20T21:16:59.604732+0000","last_peered":"2024-04-20T21:16:59.604732+0000","last_clean":"2024-04-20T21:16:59.604732+0000","last_became_active":"2024-04-20T21:16:54.557875+0000","last_became_peered":"2024-04-20T21:16:54.557875+0000","last_unstale":"2024-04-20T21:16:59.604732+0000","last_undegraded":"2024-04-20T21:16:59.604732+0000","last_fullsized":"2024-04-20T21:16:59.604732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:42:46.886405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911994+0000","last_change":"2024-04-20T21:16:54.560566+0000","last_active":"2024-04-20T21:16:59.911994+0000","last_peered":"2024-04-20T21:16:59.911994+0000","last_clean":"2024-04-20T21:16:59.911994+0000","last_became_active":"2024-04-20T21:16:54.559409+0000","last_became_peered":"2024-04-20T21:16:54.559409+0000","last_unstale":"2024-04-20T21:16:59.911994+0000","last_undegraded":"2024-04-20T21:16:59.911994+0000","last_fullsized":"2024-04-20T21:16:59.911994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:00.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912405+0000","last_change":"2024-04-20T21:16:54.559491+0000","last_active":"2024-04-20T21:16:59.912405+0000","last_peered":"2024-04-20T21:16:59.912405+0000","last_clean":"2024-04-20T21:16:59.912405+0000","last_became_active":"2024-04-20T21:16:54.558187+0000","last_became_peered":"2024-04-20T21:16:54.558187+0000","last_unstale":"2024-04-20T21:16:59.912405+0000","last_undegraded":"2024-04-20T21:16:59.912405+0000","last_fullsized":"2024-04-20T21:16:59.912405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:33.686248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617814+0000","last_change":"2024-04-20T21:16:54.564258+0000","last_active":"2024-04-20T21:16:59.617814+0000","last_peered":"2024-04-20T21:16:59.617814+0000","last_clean":"2024-04-20T21:16:59.617814+0000","last_became_active":"2024-04-20T21:16:54.563966+0000","last_became_peered":"2024-04-20T21:16:54.563966+0000","last_unstale":"2024-04-20T21:16:59.617814+0000","last_undegraded":"2024-04-20T21:16:59.617814+0000","last_fullsized":"2024-04-20T21:16:59.617814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:13:34.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912591+0000","last_change":"2024-04-20T21:16:54.573554+0000","last_active":"2024-04-20T21:16:59.912591+0000","last_peered":"2024-04-20T21:16:59.912591+0000","last_clean":"2024-04-20T21:16:59.912591+0000","last_became_active":"2024-04-20T21:16:54.571849+0000","last_became_peered":"2024-04-20T21:16:54.571849+0000","last_unstale":"2024-04-20T21:16:59.912591+0000","last_undegraded":"2024-04-20T21:16:59.912591+0000","last_fullsized":"2024-04-20T21:16:59.912591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:09.481779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617599+0000","last_change":"2024-04-20T21:16:54.571471+0000","last_active":"2024-04-20T21:16:59.617599+0000","last_peered":"2024-04-20T21:16:59.617599+0000","last_clean":"2024-04-20T21:16:59.617599+0000","last_became_active":"2024-04-20T21:16:54.571209+0000","last_became_peered":"2024-04-20T21:16:54.571209+0000","last_unstale":"2024-04-20T21:16:59.617599+0000","last_undegraded":"2024-04-20T21:16:59.617599+0000","last_fullsized":"2024-04-20T21:16:59.617599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:35.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911811+0000","last_change":"2024-04-20T21:16:54.572168+0000","last_active":"2024-04-20T21:16:59.911811+0000","last_peered":"2024-04-20T21:16:59.911811+0000","last_clean":"2024-04-20T21:16:59.911811+0000","last_became_active":"2024-04-20T21:16:54.571740+0000","last_became_peered":"2024-04-20T21:16:54.571740+0000","last_unstale":"2024-04-20T21:16:59.911811+0000","last_undegraded":"2024-04-20T21:16:59.911811+0000","last_fullsized":"2024-04-20T21:16:59.911811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:11:35.728824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617393+0000","last_change":"2024-04-20T21:16:54.569081+0000","last_active":"2024-04-20T21:16:59.617393+0000","last_peered":"2024-04-20T21:16:59.617393+0000","last_clean":"2024-04-20T21:16:59.617393+0000","last_became_active":"2024-04-20T21:16:54.568847+0000","last_became_peered":"2024-04-20T21:16:54.568847+0000","last_unstale":"2024-04-20T21:16:59.617393+0000","last_undegraded":"2024-04-20T21:16:59.617393+0000","last_fullsized":"2024-04-20T21:16:59.617393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:35.737237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912610+0000","last_change":"2024-04-20T21:16:54.571525+0000","last_active":"2024-04-20T21:16:59.912610+0000","last_peered":"2024-04-20T21:16:59.912610+0000","last_clean":"2024-04-20T21:16:59.912610+0000","last_became_active":"2024-04-20T21:16:54.570776+0000","last_became_peered":"2024-04-20T21:16:54.570776+0000","last_unstale":"2024-04-20T21:16:59.912610+0000","last_undegraded":"2024-04-20T21:16:59.912610+0000","last_fullsized":"2024-04-20T21:16:59.912610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:03.693476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916192+0000","last_change":"2024-04-20T21:16:54.570895+0000","last_active":"2024-04-20T21:16:59.916192+0000","last_peered":"2024-04-20T21:16:59.916192+0000","last_clean":"2024-04-20T21:16:59.916192+0000","last_became_active":"2024-04-20T21:16:54.570577+0000","last_became_peered":"2024-04-20T21:16:54.570577+0000","last_unstale":"2024-04-20T21:16:59.916192+0000","last_undegraded":"2024-04-20T21:16:59.916192+0000","last_fullsized":"2024-04-20T21:16:59.916192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:26.787487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608644+0000","last_change":"2024-04-20T21:16:54.581648+0000","last_active":"2024-04-20T21:16:59.608644+0000","last_peered":"2024-04-20T21:16:59.608644+0000","last_clean":"2024-04-20T21:16:59.608644+0000","last_became_active":"2024-04-20T21:16:54.581382+0000","last_became_peered":"2024-04-20T21:16:54.581382+0000","last_unstale":"2024-04-20T21:16:59.608644+0000","last_undegraded":"2024-04-20T21:16:59.608644+0000","last_fullsized":"2024-04-20T21:16:59.608644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:21.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617750+0000","last_change":"2024-04-20T21:16:54.563949+0000","last_active":"2024-04-20T21:16:59.617750+0000","last_peered":"2024-04-20T21:16:59.617750+0000","last_clean":"2024-04-20T21:16:59.617750+0000","last_became_active":"2024-04-20T21:16:54.563606+0000","last_became_peered":"2024-04-20T21:16:54.563606+0000","last_unstale":"2024-04-20T21:16:59.617750+0000","last_undegraded":"2024-04-20T21:16:59.617750+0000","last_fullsized":"2024-04-20T21:16:59.617750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:26:05.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:25.814 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:26.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.047+0000 7faa44e00640 1 Processor -- start 2024-04-20T21:17:26.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.047+0000 7faa44e00640 1 -- start start 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa44e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faa401554f0 0x7faa4015c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa44e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa4015bfd0 0x7faa400ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa44e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa40073a20 con 0x7faa4006c550 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa44e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7faa4015b510 con 0x7faa401554f0 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa44e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faa40073cc0 con 0x7faa4015bfd0 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa3d400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7faa4006c550 0x7faa400c98f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38730/0 (socket says 172.21.15.102:38730) 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa3d400640 1 -- 172.21.15.102:0/808430543 learned_addr learned my addr 172.21.15.102:0/808430543 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa3e800640 1 --2- 172.21.15.102:0/808430543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa4015bfd0 0x7faa400ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.049+0000 7faa3de00640 1 --2- 172.21.15.102:0/808430543 >> v2:172.21.15.193:3300/0 conn(0x7faa401554f0 0x7faa4015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.050+0000 7faa3ca00640 1 -- 172.21.15.102:0/808430543 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299120802 0 0) 0x7faa40073a20 con 0x7faa4006c550 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.050+0000 7faa3ca00640 1 -- 172.21.15.102:0/808430543 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa24003520 con 0x7faa4006c550 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.050+0000 7faa3e800640 1 -- 172.21.15.102:0/808430543 >> v2:172.21.15.193:3300/0 conn(0x7faa401554f0 msgr2=0x7faa4015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.050+0000 7faa3e800640 1 --2- 172.21.15.102:0/808430543 >> v2:172.21.15.193:3300/0 conn(0x7faa401554f0 0x7faa4015c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.050+0000 7faa3e800640 1 -- 172.21.15.102:0/808430543 >> v1:172.21.15.102:6789/0 conn(0x7faa4006c550 legacy=0x7faa400c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.050+0000 7faa3e800640 1 -- 172.21.15.102:0/808430543 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4015b7d0 con 0x7faa4015bfd0 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.050+0000 7faa3de00640 1 --2- 172.21.15.102:0/808430543 >> v2:172.21.15.193:3300/0 conn(0x7faa401554f0 0x7faa4015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.051+0000 7faa3e800640 1 --2- 172.21.15.102:0/808430543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa4015bfd0 0x7faa400ceb90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa38009d90 tx=0x7faa3802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f9ef2779a06f7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.051+0000 7faa3ca00640 1 -- 172.21.15.102:0/808430543 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38039070 con 0x7faa4015bfd0 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.051+0000 7faa3ca00640 1 -- 172.21.15.102:0/808430543 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3802cc60 con 0x7faa4015bfd0 2024-04-20T21:17:26.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.051+0000 7faa3ca00640 1 -- 172.21.15.102:0/808430543 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38034660 con 0x7faa4015bfd0 2024-04-20T21:17:26.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.052+0000 7faa44e00640 1 -- 172.21.15.102:0/808430543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa4015bfd0 msgr2=0x7faa400ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.052+0000 7faa44e00640 1 --2- 172.21.15.102:0/808430543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa4015bfd0 0x7faa400ceb90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa38009d90 tx=0x7faa3802beb0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.052+0000 7faa44e00640 1 -- 172.21.15.102:0/808430543 shutdown_connections 2024-04-20T21:17:26.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.052+0000 7faa44e00640 1 --2- 172.21.15.102:0/808430543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa4015bfd0 0x7faa400ceb90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.052+0000 7faa44e00640 1 --2- 172.21.15.102:0/808430543 >> v2:172.21.15.193:3300/0 conn(0x7faa401554f0 0x7faa4015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.052+0000 7faa44e00640 1 -- 172.21.15.102:0/808430543 >> 172.21.15.102:0/808430543 conn(0x7faa4006ebf0 msgr2=0x7faa4006efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:26.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.052+0000 7faa44e00640 1 -- 172.21.15.102:0/808430543 shutdown_connections 2024-04-20T21:17:26.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.052+0000 7faa44e00640 1 -- 172.21.15.102:0/808430543 wait complete. 2024-04-20T21:17:26.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.053+0000 7faa44e00640 1 Processor -- start 2024-04-20T21:17:26.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.054+0000 7faa44e00640 1 -- start start 2024-04-20T21:17:26.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.054+0000 7faa44e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4006c550 0x7faa400cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.054+0000 7faa44e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa401554f0 0x7faa401532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.054+0000 7faa44e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa4015bfd0 0x7faa40153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.054+0000 7faa44e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa400ca000 con 0x7faa4015bfd0 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.054+0000 7faa44e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa400736e0 con 0x7faa4006c550 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.054+0000 7faa44e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faa401558d0 con 0x7faa401554f0 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa4015bfd0 0x7faa40153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa401554f0 0x7faa401532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4006c550 0x7faa400cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa4015bfd0 0x7faa40153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51628/0 (socket says 172.21.15.102:51628) 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3e800640 1 -- 172.21.15.102:0/2968227699 learned_addr learned my addr 172.21.15.102:0/2968227699 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3e800640 1 -- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa401554f0 msgr2=0x7faa401532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3e800640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa401554f0 0x7faa401532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3e800640 1 -- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4006c550 msgr2=0x7faa400cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3e800640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4006c550 0x7faa400cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3e800640 1 -- 172.21.15.102:0/2968227699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa401e75e0 con 0x7faa4015bfd0 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.055+0000 7faa3de00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4006c550 0x7faa400cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.056+0000 7faa3e800640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa4015bfd0 0x7faa40153810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faa38009eb0 tx=0x7faa38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.056+0000 7faa36000640 1 -- 172.21.15.102:0/2968227699 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38039050 con 0x7faa4015bfd0 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.056+0000 7faa36000640 1 -- 172.21.15.102:0/2968227699 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa38045070 con 0x7faa4015bfd0 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.056+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa401e6d40 con 0x7faa4015bfd0 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.056+0000 7faa36000640 1 -- 172.21.15.102:0/2968227699 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38009450 con 0x7faa4015bfd0 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.056+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa401e7300 con 0x7faa4015bfd0 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.057+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa40154530 con 0x7faa4015bfd0 2024-04-20T21:17:26.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.058+0000 7faa36000640 1 -- 172.21.15.102:0/2968227699 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa3803e850 con 0x7faa4015bfd0 2024-04-20T21:17:26.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.059+0000 7faa36000640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa1c06c920 0x7faa1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.059+0000 7faa36000640 1 -- 172.21.15.102:0/2968227699 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faa380ae2d0 con 0x7faa4015bfd0 2024-04-20T21:17:26.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.059+0000 7faa3de00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa1c06c920 0x7faa1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.060+0000 7faa3de00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa1c06c920 0x7faa1c06ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa28009560 tx=0x7faa28009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.063+0000 7faa36000640 1 -- 172.21.15.102:0/2968227699 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faa3804e020 con 0x7faa4015bfd0 2024-04-20T21:17:26.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.243+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 --> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa4006dc30 con 0x7faa1c06c920 2024-04-20T21:17:26.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.252+0000 7faa36000640 1 -- 172.21.15.102:0/2968227699 <== mgr.4112 v2:172.21.15.193:6832/2803621564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7faa4006dc30 con 0x7faa1c06c920 2024-04-20T21:17:26.255 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:26.258 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa1c06c920 msgr2=0x7faa1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa1c06c920 0x7faa1c06ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa28009560 tx=0x7faa28009190 comp rx=0 tx=0).stop 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa4015bfd0 msgr2=0x7faa40153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa4015bfd0 0x7faa40153810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faa38009eb0 tx=0x7faa38004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 shutdown_connections 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa4015bfd0 0x7faa40153810 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa401554f0 0x7faa401532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa1c06c920 0x7faa1c06ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 --2- 172.21.15.102:0/2968227699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4006c550 0x7faa400cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.259+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 >> 172.21.15.102:0/2968227699 conn(0x7faa4006ebf0 msgr2=0x7faa400c98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.260+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 shutdown_connections 2024-04-20T21:17:26.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.260+0000 7faa44e00640 1 -- 172.21.15.102:0/2968227699 wait complete. 2024-04-20T21:17:26.277 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-20T21:17:24.668301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003187"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918498+0000","last_change":"2024-04-20T21:16:53.564922+0000","last_active":"2024-04-20T21:16:59.918498+0000","last_peered":"2024-04-20T21:16:59.918498+0000","last_clean":"2024-04-20T21:16:59.918498+0000","last_became_active":"2024-04-20T21:16:53.564589+0000","last_became_peered":"2024-04-20T21:16:53.564589+0000","last_unstale":"2024-04-20T21:16:59.918498+0000","last_undegraded":"2024-04-20T21:16:59.918498+0000","last_fullsized":"2024-04-20T21:16:59.918498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:47.284664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916138+0000","last_change":"2024-04-20T21:16:54.580132+0000","last_active":"2024-04-20T21:16:59.916138+0000","last_peered":"2024-04-20T21:16:59.916138+0000","last_clean":"2024-04-20T21:16:59.916138+0000","last_became_active":"2024-04-20T21:16:54.579769+0000","last_became_peered":"2024-04-20T21:16:54.579769+0000","last_unstale":"2024-04-20T21:16:59.916138+0000","last_undegraded":"2024-04-20T21:16:59.916138+0000","last_fullsized":"2024-04-20T21:16:59.916138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:00.705947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913935+0000","last_change":"2024-04-20T21:16:58.781707+0000","last_active":"2024-04-20T21:16:59.913935+0000","last_peered":"2024-04-20T21:16:59.913935+0000","last_clean":"2024-04-20T21:16:59.913935+0000","last_became_active":"2024-04-20T21:16:58.780685+0000","last_became_peered":"2024-04-20T21:16:58.780685+0000","last_unstale":"2024-04-20T21:16:59.913935+0000","last_undegraded":"2024-04-20T21:16:59.913935+0000","last_fullsized":"2024-04-20T21:16:59.913935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:50.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916578+0000","last_change":"2024-04-20T21:16:53.572137+0000","last_active":"2024-04-20T21:16:59.916578+0000","last_peered":"2024-04-20T21:16:59.916578+0000","last_clean":"2024-04-20T21:16:59.916578+0000","last_became_active":"2024-04-20T21:16:53.567907+0000","last_became_peered":"2024-04-20T21:16:53.567907+0000","last_unstale":"2024-04-20T21:16:59.916578+0000","last_undegraded":"2024-04-20T21:16:59.916578+0000","last_fullsized":"2024-04-20T21:16:59.916578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:23:28.367174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918240+0000","last_change":"2024-04-20T21:16:54.567096+0000","last_active":"2024-04-20T21:16:59.918240+0000","last_peered":"2024-04-20T21:16:59.918240+0000","last_clean":"2024-04-20T21:16:59.918240+0000","last_became_active":"2024-04-20T21:16:54.566803+0000","last_became_peered":"2024-04-20T21:16:54.566803+0000","last_unstale":"2024-04-20T21:16:59.918240+0000","last_undegraded":"2024-04-20T21:16:59.918240+0000","last_fullsized":"2024-04-20T21:16:59.918240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:13:44.730056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604352+0000","last_change":"2024-04-20T21:16:58.619283+0000","last_active":"2024-04-20T21:16:59.604352+0000","last_peered":"2024-04-20T21:16:59.604352+0000","last_clean":"2024-04-20T21:16:59.604352+0000","last_became_active":"2024-04-20T21:16:58.618881+0000","last_became_peered":"2024-04-20T21:16:58.618881+0000","last_unstale":"2024-04-20T21:16:59.604352+0000","last_undegraded":"2024-04-20T21:16:59.604352+0000","last_fullsized":"2024-04-20T21:16:59.604352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:01:40.143389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617802+0000","last_change":"2024-04-20T21:16:53.554847+0000","last_active":"2024-04-20T21:16:59.617802+0000","last_peered":"2024-04-20T21:16:59.617802+0000","last_clean":"2024-04-20T21:16:59.617802+0000","last_became_active":"2024-04-20T21:16:53.554333+0000","last_became_peered":"2024-04-20T21:16:53.554333+0000","last_unstale":"2024-04-20T21:16:59.617802+0000","last_undegraded":"2024-04-20T21:16:59.617802+0000","last_fullsized":"2024-04-20T21:16:59.617802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:33.255945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912613+0000","last_change":"2024-04-20T21:16:54.556407+0000","last_active":"2024-04-20T21:16:59.912613+0000","last_peered":"2024-04-20T21:16:59.912613+0000","last_clean":"2024-04-20T21:16:59.912613+0000","last_became_active":"2024-04-20T21:16:54.556161+0000","last_became_peered":"2024-04-20T21:16:54.556161+0000","last_unstale":"2024-04-20T21:16:59.912613+0000","last_undegraded":"2024-04-20T21:16:59.912613+0000","last_fullsized":"2024-04-20T21:16:59.912613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:44:53.586714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604519+0000","last_change":"2024-04-20T21:16:58.612063+0000","last_active":"2024-04-20T21:16:59.604519+0000","last_peered":"2024-04-20T21:16:59.604519+0000","last_clean":"2024-04-20T21:16:59.604519+0000","last_became_active":"2024-04-20T21:16:58.611487+0000","last_became_peered":"2024-04-20T21:16:58.611487+0000","last_unstale":"2024-04-20T21:16:59.604519+0000","last_undegraded":"2024-04-20T21:16:59.604519+0000","last_fullsized":"2024-04-20T21:16:59.604519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:15.097731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617458+0000","last_change":"2024-04-20T21:16:53.551212+0000","last_active":"2024-04-20T21:16:59.617458+0000","last_peered":"2024-04-20T21:16:59.617458+0000","last_clean":"2024-04-20T21:16:59.617458+0000","last_became_active":"2024-04-20T21:16:53.550899+0000","last_became_peered":"2024-04-20T21:16:53.550899+0000","last_unstale":"2024-04-20T21:16:59.617458+0000","last_undegraded":"2024-04-20T21:16:59.617458+0000","last_fullsized":"2024-04-20T21:16:59.617458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:07:18.283243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912476+0000","last_change":"2024-04-20T21:16:54.557285+0000","last_active":"2024-04-20T21:16:59.912476+0000","last_peered":"2024-04-20T21:16:59.912476+0000","last_clean":"2024-04-20T21:16:59.912476+0000","last_became_active":"2024-04-20T21:16:54.556921+0000","last_became_peered":"2024-04-20T21:16:54.556921+0000","last_unstale":"2024-04-20T21:16:59.912476+0000","last_undegraded":"2024-04-20T21:16:59.912476+0000","last_fullsized":"2024-04-20T21:16:59.912476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:33.989679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604784+0000","last_change":"2024-04-20T21:16:58.617360+0000","last_active":"2024-04-20T21:16:59.604784+0000","last_peered":"2024-04-20T21:16:59.604784+0000","last_clean":"2024-04-20T21:16:59.604784+0000","last_became_active":"2024-04-20T21:16:58.616947+0000","last_became_peered":"2024-04-20T21:16:58.616947+0000","last_unstale":"2024-04-20T21:16:59.604784+0000","last_undegraded":"2024-04-20T21:16:59.604784+0000","last_fullsized":"2024-04-20T21:16:59.604784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.202955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911434+0000","last_change":"2024-04-20T21:16:54.558067+0000","last_active":"2024-04-20T21:16:59.911434+0000","last_peered":"2024-04-20T21:16:59.911434+0000","last_clean":"2024-04-20T21:16:59.911434+0000","last_became_active":"2024-04-20T21:16:54.557781+0000","last_became_peered":"2024-04-20T21:16:54.557781+0000","last_unstale":"2024-04-20T21:16:59.911434+0000","last_undegraded":"2024-04-20T21:16:59.911434+0000","last_fullsized":"2024-04-20T21:16:59.911434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:56.576081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617756+0000","last_change":"2024-04-20T21:16:53.551615+0000","last_active":"2024-04-20T21:16:59.617756+0000","last_peered":"2024-04-20T21:16:59.617756+0000","last_clean":"2024-04-20T21:16:59.617756+0000","last_became_active":"2024-04-20T21:16:53.551073+0000","last_became_peered":"2024-04-20T21:16:53.551073+0000","last_unstale":"2024-04-20T21:16:59.617756+0000","last_undegraded":"2024-04-20T21:16:59.617756+0000","last_fullsized":"2024-04-20T21:16:59.617756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:26:29.994289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605362+0000","last_change":"2024-04-20T21:16:58.617434+0000","last_active":"2024-04-20T21:16:59.605362+0000","last_peered":"2024-04-20T21:16:59.605362+0000","last_clean":"2024-04-20T21:16:59.605362+0000","last_became_active":"2024-04-20T21:16:58.617027+0000","last_became_peered":"2024-04-20T21:16:58.617027+0000","last_unstale":"2024-04-20T21:16:59.605362+0000","last_undegraded":"2024-04-20T21:16:59.605362+0000","last_fullsized":"2024-04-20T21:16:59.605362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:44:20.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913331+0000","last_change":"2024-04-20T21:16:54.585333+0000","last_active":"2024-04-20T21:16:59.913331+0000","last_peered":"2024-04-20T21:16:59.913331+0000","last_clean":"2024-04-20T21:16:59.913331+0000","last_became_active":"2024-04-20T21:16:54.584978+0000","last_became_peered":"2024-04-20T21:16:54.584978+0000","last_unstale":"2024-04-20T21:16:59.913331+0000","last_undegraded":"2024-04-20T21:16:59.913331+0000","last_fullsized":"2024-04-20T21:16:59.913331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:23.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608374+0000","last_change":"2024-04-20T21:16:53.567775+0000","last_active":"2024-04-20T21:16:59.608374+0000","last_peered":"2024-04-20T21:16:59.608374+0000","last_clean":"2024-04-20T21:16:59.608374+0000","last_became_active":"2024-04-20T21:16:53.567498+0000","last_became_peered":"2024-04-20T21:16:53.567498+0000","last_unstale":"2024-04-20T21:16:59.608374+0000","last_undegraded":"2024-04-20T21:16:59.608374+0000","last_fullsized":"2024-04-20T21:16:59.608374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:10.695659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604469+0000","last_change":"2024-04-20T21:16:58.619302+0000","last_active":"2024-04-20T21:16:59.604469+0000","last_peered":"2024-04-20T21:16:59.604469+0000","last_clean":"2024-04-20T21:16:59.604469+0000","last_became_active":"2024-04-20T21:16:58.618843+0000","last_became_peered":"2024-04-20T21:16:58.618843+0000","last_unstale":"2024-04-20T21:16:59.604469+0000","last_undegraded":"2024-04-20T21:16:59.604469+0000","last_fullsized":"2024-04-20T21:16:59.604469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:14.927199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914247+0000","last_change":"2024-04-20T21:16:54.575862+0000","last_active":"2024-04-20T21:16:59.914247+0000","last_peered":"2024-04-20T21:16:59.914247+0000","last_clean":"2024-04-20T21:16:59.914247+0000","last_became_active":"2024-04-20T21:16:54.575523+0000","last_became_peered":"2024-04-20T21:16:54.575523+0000","last_unstale":"2024-04-20T21:16:59.914247+0000","last_undegraded":"2024-04-20T21:16:59.914247+0000","last_fullsized":"2024-04-20T21:16:59.914247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:51.041414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606833+0000","last_change":"2024-04-20T21:16:53.551995+0000","last_active":"2024-04-20T21:16:59.606833+0000","last_peered":"2024-04-20T21:16:59.606833+0000","last_clean":"2024-04-20T21:16:59.606833+0000","last_became_active":"2024-04-20T21:16:53.551603+0000","last_became_peered":"2024-04-20T21:16:53.551603+0000","last_unstale":"2024-04-20T21:16:59.606833+0000","last_undegraded":"2024-04-20T21:16:59.606833+0000","last_fullsized":"2024-04-20T21:16:59.606833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:56:21.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608819+0000","last_change":"2024-04-20T21:16:58.776463+0000","last_active":"2024-04-20T21:16:59.608819+0000","last_peered":"2024-04-20T21:16:59.608819+0000","last_clean":"2024-04-20T21:16:59.608819+0000","last_became_active":"2024-04-20T21:16:58.775746+0000","last_became_peered":"2024-04-20T21:16:58.775746+0000","last_unstale":"2024-04-20T21:16:59.608819+0000","last_undegraded":"2024-04-20T21:16:59.608819+0000","last_fullsized":"2024-04-20T21:16:59.608819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:28.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605047+0000","last_change":"2024-04-20T21:16:54.566449+0000","last_active":"2024-04-20T21:16:59.605047+0000","last_peered":"2024-04-20T21:16:59.605047+0000","last_clean":"2024-04-20T21:16:59.605047+0000","last_became_active":"2024-04-20T21:16:54.566251+0000","last_became_peered":"2024-04-20T21:16:54.566251+0000","last_unstale":"2024-04-20T21:16:59.605047+0000","last_undegraded":"2024-04-20T21:16:59.605047+0000","last_fullsized":"2024-04-20T21:16:59.605047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:53.403420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617047+0000","last_change":"2024-04-20T21:16:53.564029+0000","last_active":"2024-04-20T21:16:59.617047+0000","last_peered":"2024-04-20T21:16:59.617047+0000","last_clean":"2024-04-20T21:16:59.617047+0000","last_became_active":"2024-04-20T21:16:53.563711+0000","last_became_peered":"2024-04-20T21:16:53.563711+0000","last_unstale":"2024-04-20T21:16:59.617047+0000","last_undegraded":"2024-04-20T21:16:59.617047+0000","last_fullsized":"2024-04-20T21:16:59.617047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.634644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604995+0000","last_change":"2024-04-20T21:16:58.615080+0000","last_active":"2024-04-20T21:16:59.604995+0000","last_peered":"2024-04-20T21:16:59.604995+0000","last_clean":"2024-04-20T21:16:59.604995+0000","last_became_active":"2024-04-20T21:16:58.614611+0000","last_became_peered":"2024-04-20T21:16:58.614611+0000","last_unstale":"2024-04-20T21:16:59.604995+0000","last_undegraded":"2024-04-20T21:16:59.604995+0000","last_fullsized":"2024-04-20T21:16:59.604995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:08.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917639+0000","last_change":"2024-04-20T21:16:54.578272+0000","last_active":"2024-04-20T21:16:59.917639+0000","last_peered":"2024-04-20T21:16:59.917639+0000","last_clean":"2024-04-20T21:16:59.917639+0000","last_became_active":"2024-04-20T21:16:54.577949+0000","last_became_peered":"2024-04-20T21:16:54.577949+0000","last_unstale":"2024-04-20T21:16:59.917639+0000","last_undegraded":"2024-04-20T21:16:59.917639+0000","last_fullsized":"2024-04-20T21:16:59.917639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:35:33.807091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912813+0000","last_change":"2024-04-20T21:16:53.554600+0000","last_active":"2024-04-20T21:16:59.912813+0000","last_peered":"2024-04-20T21:16:59.912813+0000","last_clean":"2024-04-20T21:16:59.912813+0000","last_became_active":"2024-04-20T21:16:53.554347+0000","last_became_peered":"2024-04-20T21:16:53.554347+0000","last_unstale":"2024-04-20T21:16:59.912813+0000","last_undegraded":"2024-04-20T21:16:59.912813+0000","last_fullsized":"2024-04-20T21:16:59.912813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:40:58.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604315+0000","last_change":"2024-04-20T21:16:58.774474+0000","last_active":"2024-04-20T21:16:59.604315+0000","last_peered":"2024-04-20T21:16:59.604315+0000","last_clean":"2024-04-20T21:16:59.604315+0000","last_became_active":"2024-04-20T21:16:58.774037+0000","last_became_peered":"2024-04-20T21:16:58.774037+0000","last_unstale":"2024-04-20T21:16:59.604315+0000","last_undegraded":"2024-04-20T21:16:59.604315+0000","last_fullsized":"2024-04-20T21:16:59.604315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:46.992667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916707+0000","last_change":"2024-04-20T21:16:54.579435+0000","last_active":"2024-04-20T21:16:59.916707+0000","last_peered":"2024-04-20T21:16:59.916707+0000","last_clean":"2024-04-20T21:16:59.916707+0000","last_became_active":"2024-04-20T21:16:54.574617+0000","last_became_peered":"2024-04-20T21:16:54.574617+0000","last_unstale":"2024-04-20T21:16:59.916707+0000","last_undegraded":"2024-04-20T21:16:59.916707+0000","last_fullsized":"2024-04-20T21:16:59.916707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:51.500741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608822+0000","last_change":"2024-04-20T21:16:53.561144+0000","last_active":"2024-04-20T21:16:59.608822+0000","last_peered":"2024-04-20T21:16:59.608822+0000","last_clean":"2024-04-20T21:16:59.608822+0000","last_became_active":"2024-04-20T21:16:53.560742+0000","last_became_peered":"2024-04-20T21:16:53.560742+0000","last_unstale":"2024-04-20T21:16:59.608822+0000","last_undegraded":"2024-04-20T21:16:59.608822+0000","last_fullsized":"2024-04-20T21:16:59.608822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:39.212951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913482+0000","last_change":"2024-04-20T21:16:58.780124+0000","last_active":"2024-04-20T21:16:59.913482+0000","last_peered":"2024-04-20T21:16:59.913482+0000","last_clean":"2024-04-20T21:16:59.913482+0000","last_became_active":"2024-04-20T21:16:58.777449+0000","last_became_peered":"2024-04-20T21:16:58.777449+0000","last_unstale":"2024-04-20T21:16:59.913482+0000","last_undegraded":"2024-04-20T21:16:59.913482+0000","last_fullsized":"2024-04-20T21:16:59.913482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:07.622146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608726+0000","last_change":"2024-04-20T21:16:54.575331+0000","last_active":"2024-04-20T21:16:59.608726+0000","last_peered":"2024-04-20T21:16:59.608726+0000","last_clean":"2024-04-20T21:16:59.608726+0000","last_became_active":"2024-04-20T21:16:54.575061+0000","last_became_peered":"2024-04-20T21:16:54.575061+0000","last_unstale":"2024-04-20T21:16:59.608726+0000","last_undegraded":"2024-04-20T21:16:59.608726+0000","last_fullsized":"2024-04-20T21:16:59.608726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:47:20.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617871+0000","last_change":"2024-04-20T21:16:53.565561+0000","last_active":"2024-04-20T21:16:59.617871+0000","last_peered":"2024-04-20T21:16:59.617871+0000","last_clean":"2024-04-20T21:16:59.617871+0000","last_became_active":"2024-04-20T21:16:53.565207+0000","last_became_peered":"2024-04-20T21:16:53.565207+0000","last_unstale":"2024-04-20T21:16:59.617871+0000","last_undegraded":"2024-04-20T21:16:59.617871+0000","last_fullsized":"2024-04-20T21:16:59.617871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:07.512132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915133+0000","last_change":"2024-04-20T21:16:58.776337+0000","last_active":"2024-04-20T21:16:59.915133+0000","last_peered":"2024-04-20T21:16:59.915133+0000","last_clean":"2024-04-20T21:16:59.915133+0000","last_became_active":"2024-04-20T21:16:58.772404+0000","last_became_peered":"2024-04-20T21:16:58.772404+0000","last_unstale":"2024-04-20T21:16:59.915133+0000","last_undegraded":"2024-04-20T21:16:59.915133+0000","last_fullsized":"2024-04-20T21:16:59.915133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:11.554173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912522+0000","last_change":"2024-04-20T21:16:54.571199+0000","last_active":"2024-04-20T21:16:59.912522+0000","last_peered":"2024-04-20T21:16:59.912522+0000","last_clean":"2024-04-20T21:16:59.912522+0000","last_became_active":"2024-04-20T21:16:54.570882+0000","last_became_peered":"2024-04-20T21:16:54.570882+0000","last_unstale":"2024-04-20T21:16:59.912522+0000","last_undegraded":"2024-04-20T21:16:59.912522+0000","last_fullsized":"2024-04-20T21:16:59.912522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:52.046176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608915+0000","last_change":"2024-04-20T21:16:53.566253+0000","last_active":"2024-04-20T21:16:59.608915+0000","last_peered":"2024-04-20T21:16:59.608915+0000","last_clean":"2024-04-20T21:16:59.608915+0000","last_became_active":"2024-04-20T21:16:53.565893+0000","last_became_peered":"2024-04-20T21:16:53.565893+0000","last_unstale":"2024-04-20T21:16:59.608915+0000","last_undegraded":"2024-04-20T21:16:59.608915+0000","last_fullsized":"2024-04-20T21:16:59.608915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:16.302801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.607352+0000","last_change":"2024-04-20T21:16:58.611470+0000","last_active":"2024-04-20T21:16:59.607352+0000","last_peered":"2024-04-20T21:16:59.607352+0000","last_clean":"2024-04-20T21:16:59.607352+0000","last_became_active":"2024-04-20T21:16:58.611038+0000","last_became_peered":"2024-04-20T21:16:58.611038+0000","last_unstale":"2024-04-20T21:16:59.607352+0000","last_undegraded":"2024-04-20T21:16:59.607352+0000","last_fullsized":"2024-04-20T21:16:59.607352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:58:51.267480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604848+0000","last_change":"2024-04-20T21:16:54.577608+0000","last_active":"2024-04-20T21:16:59.604848+0000","last_peered":"2024-04-20T21:16:59.604848+0000","last_clean":"2024-04-20T21:16:59.604848+0000","last_became_active":"2024-04-20T21:16:54.577339+0000","last_became_peered":"2024-04-20T21:16:54.577339+0000","last_unstale":"2024-04-20T21:16:59.604848+0000","last_undegraded":"2024-04-20T21:16:59.604848+0000","last_fullsized":"2024-04-20T21:16:59.604848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:03.028585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912331+0000","last_change":"2024-04-20T21:16:53.547751+0000","last_active":"2024-04-20T21:16:59.912331+0000","last_peered":"2024-04-20T21:16:59.912331+0000","last_clean":"2024-04-20T21:16:59.912331+0000","last_became_active":"2024-04-20T21:16:53.547371+0000","last_became_peered":"2024-04-20T21:16:53.547371+0000","last_unstale":"2024-04-20T21:16:59.912331+0000","last_undegraded":"2024-04-20T21:16:59.912331+0000","last_fullsized":"2024-04-20T21:16:59.912331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:50.728500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608358+0000","last_change":"2024-04-20T21:16:58.622369+0000","last_active":"2024-04-20T21:16:59.608358+0000","last_peered":"2024-04-20T21:16:59.608358+0000","last_clean":"2024-04-20T21:16:59.608358+0000","last_became_active":"2024-04-20T21:16:58.621911+0000","last_became_peered":"2024-04-20T21:16:58.621911+0000","last_unstale":"2024-04-20T21:16:59.608358+0000","last_undegraded":"2024-04-20T21:16:59.608358+0000","last_fullsized":"2024-04-20T21:16:59.608358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:41.519583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606736+0000","last_change":"2024-04-20T21:16:54.579577+0000","last_active":"2024-04-20T21:16:59.606736+0000","last_peered":"2024-04-20T21:16:59.606736+0000","last_clean":"2024-04-20T21:16:59.606736+0000","last_became_active":"2024-04-20T21:16:54.579156+0000","last_became_peered":"2024-04-20T21:16:54.579156+0000","last_unstale":"2024-04-20T21:16:59.606736+0000","last_undegraded":"2024-04-20T21:16:59.606736+0000","last_fullsized":"2024-04-20T21:16:59.606736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:28:14.743667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605543+0000","last_change":"2024-04-20T21:16:53.567995+0000","last_active":"2024-04-20T21:16:59.605543+0000","last_peered":"2024-04-20T21:16:59.605543+0000","last_clean":"2024-04-20T21:16:59.605543+0000","last_became_active":"2024-04-20T21:16:53.567601+0000","last_became_peered":"2024-04-20T21:16:53.567601+0000","last_unstale":"2024-04-20T21:16:59.605543+0000","last_undegraded":"2024-04-20T21:16:59.605543+0000","last_fullsized":"2024-04-20T21:16:59.605543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:30.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609728+0000","last_change":"2024-04-20T21:16:58.626438+0000","last_active":"2024-04-20T21:16:59.609728+0000","last_peered":"2024-04-20T21:16:59.609728+0000","last_clean":"2024-04-20T21:16:59.609728+0000","last_became_active":"2024-04-20T21:16:58.625326+0000","last_became_peered":"2024-04-20T21:16:58.625326+0000","last_unstale":"2024-04-20T21:16:59.609728+0000","last_undegraded":"2024-04-20T21:16:59.609728+0000","last_fullsized":"2024-04-20T21:16:59.609728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:06.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914171+0000","last_change":"2024-04-20T21:16:54.560925+0000","last_active":"2024-04-20T21:16:59.914171+0000","last_peered":"2024-04-20T21:16:59.914171+0000","last_clean":"2024-04-20T21:16:59.914171+0000","last_became_active":"2024-04-20T21:16:54.560615+0000","last_became_peered":"2024-04-20T21:16:54.560615+0000","last_unstale":"2024-04-20T21:16:59.914171+0000","last_undegraded":"2024-04-20T21:16:59.914171+0000","last_fullsized":"2024-04-20T21:16:59.914171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:12.815692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617155+0000","last_change":"2024-04-20T21:16:53.559359+0000","last_active":"2024-04-20T21:16:59.617155+0000","last_peered":"2024-04-20T21:16:59.617155+0000","last_clean":"2024-04-20T21:16:59.617155+0000","last_became_active":"2024-04-20T21:16:53.558955+0000","last_became_peered":"2024-04-20T21:16:53.558955+0000","last_unstale":"2024-04-20T21:16:59.617155+0000","last_undegraded":"2024-04-20T21:16:59.617155+0000","last_fullsized":"2024-04-20T21:16:59.617155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:28.664404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917377+0000","last_change":"2024-04-20T21:16:58.772175+0000","last_active":"2024-04-20T21:16:59.917377+0000","last_peered":"2024-04-20T21:16:59.917377+0000","last_clean":"2024-04-20T21:16:59.917377+0000","last_became_active":"2024-04-20T21:16:58.771732+0000","last_became_peered":"2024-04-20T21:16:58.771732+0000","last_unstale":"2024-04-20T21:16:59.917377+0000","last_undegraded":"2024-04-20T21:16:59.917377+0000","last_fullsized":"2024-04-20T21:16:59.917377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:13.155547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912641+0000","last_change":"2024-04-20T21:16:54.575375+0000","last_active":"2024-04-20T21:16:59.912641+0000","last_peered":"2024-04-20T21:16:59.912641+0000","last_clean":"2024-04-20T21:16:59.912641+0000","last_became_active":"2024-04-20T21:16:54.574194+0000","last_became_peered":"2024-04-20T21:16:54.574194+0000","last_unstale":"2024-04-20T21:16:59.912641+0000","last_undegraded":"2024-04-20T21:16:59.912641+0000","last_fullsized":"2024-04-20T21:16:59.912641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:07.948431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911980+0000","last_change":"2024-04-20T21:16:53.559374+0000","last_active":"2024-04-20T21:16:59.911980+0000","last_peered":"2024-04-20T21:16:59.911980+0000","last_clean":"2024-04-20T21:16:59.911980+0000","last_became_active":"2024-04-20T21:16:53.559133+0000","last_became_peered":"2024-04-20T21:16:53.559133+0000","last_unstale":"2024-04-20T21:16:59.911980+0000","last_undegraded":"2024-04-20T21:16:59.911980+0000","last_fullsized":"2024-04-20T21:16:59.911980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:38.576103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617268+0000","last_change":"2024-04-20T21:16:44.100529+0000","last_active":"2024-04-20T21:16:59.617268+0000","last_peered":"2024-04-20T21:16:59.617268+0000","last_clean":"2024-04-20T21:16:59.617268+0000","last_became_active":"2024-04-20T21:16:44.099590+0000","last_became_peered":"2024-04-20T21:16:44.099590+0000","last_unstale":"2024-04-20T21:16:59.617268+0000","last_undegraded":"2024-04-20T21:16:59.617268+0000","last_fullsized":"2024-04-20T21:16:59.617268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_clean_scrub_stamp":"2024-04-20T21:16:42.782844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:08.054837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911874+0000","last_change":"2024-04-20T21:16:58.772894+0000","last_active":"2024-04-20T21:16:59.911874+0000","last_peered":"2024-04-20T21:16:59.911874+0000","last_clean":"2024-04-20T21:16:59.911874+0000","last_became_active":"2024-04-20T21:16:58.772451+0000","last_became_peered":"2024-04-20T21:16:58.772451+0000","last_unstale":"2024-04-20T21:16:59.911874+0000","last_undegraded":"2024-04-20T21:16:59.911874+0000","last_fullsized":"2024-04-20T21:16:59.911874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:29.914096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617716+0000","last_change":"2024-04-20T21:16:54.562592+0000","last_active":"2024-04-20T21:16:59.617716+0000","last_peered":"2024-04-20T21:16:59.617716+0000","last_clean":"2024-04-20T21:16:59.617716+0000","last_became_active":"2024-04-20T21:16:54.562243+0000","last_became_peered":"2024-04-20T21:16:54.562243+0000","last_unstale":"2024-04-20T21:16:59.617716+0000","last_undegraded":"2024-04-20T21:16:59.617716+0000","last_fullsized":"2024-04-20T21:16:59.617716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:52.832159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606010+0000","last_change":"2024-04-20T21:16:53.561132+0000","last_active":"2024-04-20T21:16:59.606010+0000","last_peered":"2024-04-20T21:16:59.606010+0000","last_clean":"2024-04-20T21:16:59.606010+0000","last_became_active":"2024-04-20T21:16:53.559901+0000","last_became_peered":"2024-04-20T21:16:53.559901+0000","last_unstale":"2024-04-20T21:16:59.606010+0000","last_undegraded":"2024-04-20T21:16:59.606010+0000","last_fullsized":"2024-04-20T21:16:59.606010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:20.229573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911503+0000","last_change":"2024-04-20T21:16:58.771204+0000","last_active":"2024-04-20T21:16:59.911503+0000","last_peered":"2024-04-20T21:16:59.911503+0000","last_clean":"2024-04-20T21:16:59.911503+0000","last_became_active":"2024-04-20T21:16:58.770627+0000","last_became_peered":"2024-04-20T21:16:58.770627+0000","last_unstale":"2024-04-20T21:16:59.911503+0000","last_undegraded":"2024-04-20T21:16:59.911503+0000","last_fullsized":"2024-04-20T21:16:59.911503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:39.472931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912685+0000","last_change":"2024-04-20T21:16:54.571417+0000","last_active":"2024-04-20T21:16:59.912685+0000","last_peered":"2024-04-20T21:16:59.912685+0000","last_clean":"2024-04-20T21:16:59.912685+0000","last_became_active":"2024-04-20T21:16:54.570434+0000","last_became_peered":"2024-04-20T21:16:54.570434+0000","last_unstale":"2024-04-20T21:16:59.912685+0000","last_undegraded":"2024-04-20T21:16:59.912685+0000","last_fullsized":"2024-04-20T21:16:59.912685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:24:41.026138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609282+0000","last_change":"2024-04-20T21:16:53.558554+0000","last_active":"2024-04-20T21:16:59.609282+0000","last_peered":"2024-04-20T21:16:59.609282+0000","last_clean":"2024-04-20T21:16:59.609282+0000","last_became_active":"2024-04-20T21:16:53.556934+0000","last_became_peered":"2024-04-20T21:16:53.556934+0000","last_unstale":"2024-04-20T21:16:59.609282+0000","last_undegraded":"2024-04-20T21:16:59.609282+0000","last_fullsized":"2024-04-20T21:16:59.609282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:17:52.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915951+0000","last_change":"2024-04-20T21:16:58.624410+0000","last_active":"2024-04-20T21:16:59.915951+0000","last_peered":"2024-04-20T21:16:59.915951+0000","last_clean":"2024-04-20T21:16:59.915951+0000","last_became_active":"2024-04-20T21:16:58.623677+0000","last_became_peered":"2024-04-20T21:16:58.623677+0000","last_unstale":"2024-04-20T21:16:59.915951+0000","last_undegraded":"2024-04-20T21:16:59.915951+0000","last_fullsized":"2024-04-20T21:16:59.915951+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:34.465594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617870+0000","last_change":"2024-04-20T21:16:54.560217+0000","last_active":"2024-04-20T21:16:59.617870+0000","last_peered":"2024-04-20T21:16:59.617870+0000","last_clean":"2024-04-20T21:16:59.617870+0000","last_became_active":"2024-04-20T21:16:54.559947+0000","last_became_peered":"2024-04-20T21:16:54.559947+0000","last_unstale":"2024-04-20T21:16:59.617870+0000","last_undegraded":"2024-04-20T21:16:59.617870+0000","last_fullsized":"2024-04-20T21:16:59.617870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:54.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.607164+0000","last_change":"2024-04-20T21:16:53.564779+0000","last_active":"2024-04-20T21:16:59.607164+0000","last_peered":"2024-04-20T21:16:59.607164+0000","last_clean":"2024-04-20T21:16:59.607164+0000","last_became_active":"2024-04-20T21:16:53.564500+0000","last_became_peered":"2024-04-20T21:16:53.564500+0000","last_unstale":"2024-04-20T21:16:59.607164+0000","last_undegraded":"2024-04-20T21:16:59.607164+0000","last_fullsized":"2024-04-20T21:16:59.607164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:00:58.673145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912319+0000","last_change":"2024-04-20T21:16:58.771524+0000","last_active":"2024-04-20T21:16:59.912319+0000","last_peered":"2024-04-20T21:16:59.912319+0000","last_clean":"2024-04-20T21:16:59.912319+0000","last_became_active":"2024-04-20T21:16:58.770672+0000","last_became_peered":"2024-04-20T21:16:58.770672+0000","last_unstale":"2024-04-20T21:16:59.912319+0000","last_undegraded":"2024-04-20T21:16:59.912319+0000","last_fullsized":"2024-04-20T21:16:59.912319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:52:27.037256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617704+0000","last_change":"2024-04-20T21:16:54.559570+0000","last_active":"2024-04-20T21:16:59.617704+0000","last_peered":"2024-04-20T21:16:59.617704+0000","last_clean":"2024-04-20T21:16:59.617704+0000","last_became_active":"2024-04-20T21:16:54.559307+0000","last_became_peered":"2024-04-20T21:16:54.559307+0000","last_unstale":"2024-04-20T21:16:59.617704+0000","last_undegraded":"2024-04-20T21:16:59.617704+0000","last_fullsized":"2024-04-20T21:16:59.617704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:04.417348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605512+0000","last_change":"2024-04-20T21:16:53.565289+0000","last_active":"2024-04-20T21:16:59.605512+0000","last_peered":"2024-04-20T21:16:59.605512+0000","last_clean":"2024-04-20T21:16:59.605512+0000","last_became_active":"2024-04-20T21:16:53.565014+0000","last_became_peered":"2024-04-20T21:16:53.565014+0000","last_unstale":"2024-04-20T21:16:59.605512+0000","last_undegraded":"2024-04-20T21:16:59.605512+0000","last_fullsized":"2024-04-20T21:16:59.605512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:22.125301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604480+0000","last_change":"2024-04-20T21:16:58.615103+0000","last_active":"2024-04-20T21:16:59.604480+0000","last_peered":"2024-04-20T21:16:59.604480+0000","last_clean":"2024-04-20T21:16:59.604480+0000","last_became_active":"2024-04-20T21:16:58.614068+0000","last_became_peered":"2024-04-20T21:16:58.614068+0000","last_unstale":"2024-04-20T21:16:59.604480+0000","last_undegraded":"2024-04-20T21:16:59.604480+0000","last_fullsized":"2024-04-20T21:16:59.604480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:09.555911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617489+0000","last_change":"2024-04-20T21:16:54.557439+0000","last_active":"2024-04-20T21:16:59.617489+0000","last_peered":"2024-04-20T21:16:59.617489+0000","last_clean":"2024-04-20T21:16:59.617489+0000","last_became_active":"2024-04-20T21:16:54.557178+0000","last_became_peered":"2024-04-20T21:16:54.557178+0000","last_unstale":"2024-04-20T21:16:59.617489+0000","last_undegraded":"2024-04-20T21:16:59.617489+0000","last_fullsized":"2024-04-20T21:16:59.617489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:55:33.964948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605750+0000","last_change":"2024-04-20T21:16:53.570182+0000","last_active":"2024-04-20T21:16:59.605750+0000","last_peered":"2024-04-20T21:16:59.605750+0000","last_clean":"2024-04-20T21:16:59.605750+0000","last_became_active":"2024-04-20T21:16:53.569632+0000","last_became_peered":"2024-04-20T21:16:53.569632+0000","last_unstale":"2024-04-20T21:16:59.605750+0000","last_undegraded":"2024-04-20T21:16:59.605750+0000","last_fullsized":"2024-04-20T21:16:59.605750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:12.140497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917381+0000","last_change":"2024-04-20T21:16:58.625602+0000","last_active":"2024-04-20T21:16:59.917381+0000","last_peered":"2024-04-20T21:16:59.917381+0000","last_clean":"2024-04-20T21:16:59.917381+0000","last_became_active":"2024-04-20T21:16:58.625158+0000","last_became_peered":"2024-04-20T21:16:58.625158+0000","last_unstale":"2024-04-20T21:16:59.917381+0000","last_undegraded":"2024-04-20T21:16:59.917381+0000","last_fullsized":"2024-04-20T21:16:59.917381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:09.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911816+0000","last_change":"2024-04-20T21:16:54.578603+0000","last_active":"2024-04-20T21:16:59.911816+0000","last_peered":"2024-04-20T21:16:59.911816+0000","last_clean":"2024-04-20T21:16:59.911816+0000","last_became_active":"2024-04-20T21:16:54.578338+0000","last_became_peered":"2024-04-20T21:16:54.578338+0000","last_unstale":"2024-04-20T21:16:59.911816+0000","last_undegraded":"2024-04-20T21:16:59.911816+0000","last_fullsized":"2024-04-20T21:16:59.911816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:14:56.355343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911553+0000","last_change":"2024-04-20T21:16:53.551847+0000","last_active":"2024-04-20T21:16:59.911553+0000","last_peered":"2024-04-20T21:16:59.911553+0000","last_clean":"2024-04-20T21:16:59.911553+0000","last_became_active":"2024-04-20T21:16:53.551427+0000","last_became_peered":"2024-04-20T21:16:53.551427+0000","last_unstale":"2024-04-20T21:16:59.911553+0000","last_undegraded":"2024-04-20T21:16:59.911553+0000","last_fullsized":"2024-04-20T21:16:59.911553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:55:45.930305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916262+0000","last_change":"2024-04-20T21:16:58.777159+0000","last_active":"2024-04-20T21:16:59.916262+0000","last_peered":"2024-04-20T21:16:59.916262+0000","last_clean":"2024-04-20T21:16:59.916262+0000","last_became_active":"2024-04-20T21:16:58.776233+0000","last_became_peered":"2024-04-20T21:16:58.776233+0000","last_unstale":"2024-04-20T21:16:59.916262+0000","last_undegraded":"2024-04-20T21:16:59.916262+0000","last_fullsized":"2024-04-20T21:16:59.916262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:05.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617642+0000","last_change":"2024-04-20T21:16:54.570687+0000","last_active":"2024-04-20T21:16:59.617642+0000","last_peered":"2024-04-20T21:16:59.617642+0000","last_clean":"2024-04-20T21:16:59.617642+0000","last_became_active":"2024-04-20T21:16:54.570360+0000","last_became_peered":"2024-04-20T21:16:54.570360+0000","last_unstale":"2024-04-20T21:16:59.617642+0000","last_undegraded":"2024-04-20T21:16:59.617642+0000","last_fullsized":"2024-04-20T21:16:59.617642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:05.606229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912222+0000","last_change":"2024-04-20T21:16:53.567599+0000","last_active":"2024-04-20T21:16:59.912222+0000","last_peered":"2024-04-20T21:16:59.912222+0000","last_clean":"2024-04-20T21:16:59.912222+0000","last_became_active":"2024-04-20T21:16:53.567378+0000","last_became_peered":"2024-04-20T21:16:53.567378+0000","last_unstale":"2024-04-20T21:16:59.912222+0000","last_undegraded":"2024-04-20T21:16:59.912222+0000","last_fullsized":"2024-04-20T21:16:59.912222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:36.611887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914503+0000","last_change":"2024-04-20T21:16:58.774226+0000","last_active":"2024-04-20T21:16:59.914503+0000","last_peered":"2024-04-20T21:16:59.914503+0000","last_clean":"2024-04-20T21:16:59.914503+0000","last_became_active":"2024-04-20T21:16:58.771584+0000","last_became_peered":"2024-04-20T21:16:58.771584+0000","last_unstale":"2024-04-20T21:16:59.914503+0000","last_undegraded":"2024-04-20T21:16:59.914503+0000","last_fullsized":"2024-04-20T21:16:59.914503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:31.432077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915175+0000","last_change":"2024-04-20T21:16:54.560001+0000","last_active":"2024-04-20T21:16:59.915175+0000","last_peered":"2024-04-20T21:16:59.915175+0000","last_clean":"2024-04-20T21:16:59.915175+0000","last_became_active":"2024-04-20T21:16:54.559643+0000","last_became_peered":"2024-04-20T21:16:54.559643+0000","last_unstale":"2024-04-20T21:16:59.915175+0000","last_undegraded":"2024-04-20T21:16:59.915175+0000","last_fullsized":"2024-04-20T21:16:59.915175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:52:37.325946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916695+0000","last_change":"2024-04-20T21:16:53.570498+0000","last_active":"2024-04-20T21:16:59.916695+0000","last_peered":"2024-04-20T21:16:59.916695+0000","last_clean":"2024-04-20T21:16:59.916695+0000","last_became_active":"2024-04-20T21:16:53.570095+0000","last_became_peered":"2024-04-20T21:16:53.570095+0000","last_unstale":"2024-04-20T21:16:59.916695+0000","last_undegraded":"2024-04-20T21:16:59.916695+0000","last_fullsized":"2024-04-20T21:16:59.916695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:58:28.677814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912403+0000","last_change":"2024-04-20T21:16:58.773349+0000","last_active":"2024-04-20T21:16:59.912403+0000","last_peered":"2024-04-20T21:16:59.912403+0000","last_clean":"2024-04-20T21:16:59.912403+0000","last_became_active":"2024-04-20T21:16:58.772781+0000","last_became_peered":"2024-04-20T21:16:58.772781+0000","last_unstale":"2024-04-20T21:16:59.912403+0000","last_undegraded":"2024-04-20T21:16:59.912403+0000","last_fullsized":"2024-04-20T21:16:59.912403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:53:02.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617377+0000","last_change":"2024-04-20T21:16:54.572749+0000","last_active":"2024-04-20T21:16:59.617377+0000","last_peered":"2024-04-20T21:16:59.617377+0000","last_clean":"2024-04-20T21:16:59.617377+0000","last_became_active":"2024-04-20T21:16:54.572456+0000","last_became_peered":"2024-04-20T21:16:54.572456+0000","last_unstale":"2024-04-20T21:16:59.617377+0000","last_undegraded":"2024-04-20T21:16:59.617377+0000","last_fullsized":"2024-04-20T21:16:59.617377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.787612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:17:17.226094+0000","last_change":"2024-04-20T21:16:53.568688+0000","last_active":"2024-04-20T21:17:17.226094+0000","last_peered":"2024-04-20T21:17:17.226094+0000","last_clean":"2024-04-20T21:17:17.226094+0000","last_became_active":"2024-04-20T21:16:53.568495+0000","last_became_peered":"2024-04-20T21:16:53.568495+0000","last_unstale":"2024-04-20T21:17:17.226094+0000","last_undegraded":"2024-04-20T21:17:17.226094+0000","last_fullsized":"2024-04-20T21:17:17.226094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:00.086004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608039+0000","last_change":"2024-04-20T21:16:58.609575+0000","last_active":"2024-04-20T21:16:59.608039+0000","last_peered":"2024-04-20T21:16:59.608039+0000","last_clean":"2024-04-20T21:16:59.608039+0000","last_became_active":"2024-04-20T21:16:58.609127+0000","last_became_peered":"2024-04-20T21:16:58.609127+0000","last_unstale":"2024-04-20T21:16:59.608039+0000","last_undegraded":"2024-04-20T21:16:59.608039+0000","last_fullsized":"2024-04-20T21:16:59.608039+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:02.645274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605015+0000","last_change":"2024-04-20T21:16:54.563213+0000","last_active":"2024-04-20T21:16:59.605015+0000","last_peered":"2024-04-20T21:16:59.605015+0000","last_clean":"2024-04-20T21:16:59.605015+0000","last_became_active":"2024-04-20T21:16:54.562917+0000","last_became_peered":"2024-04-20T21:16:54.562917+0000","last_unstale":"2024-04-20T21:16:59.605015+0000","last_undegraded":"2024-04-20T21:16:59.605015+0000","last_fullsized":"2024-04-20T21:16:59.605015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.155771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.609082+0000","last_change":"2024-04-20T21:16:53.556988+0000","last_active":"2024-04-20T21:16:59.609082+0000","last_peered":"2024-04-20T21:16:59.609082+0000","last_clean":"2024-04-20T21:16:59.609082+0000","last_became_active":"2024-04-20T21:16:53.556672+0000","last_became_peered":"2024-04-20T21:16:53.556672+0000","last_unstale":"2024-04-20T21:16:59.609082+0000","last_undegraded":"2024-04-20T21:16:59.609082+0000","last_fullsized":"2024-04-20T21:16:59.609082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:25.219611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604713+0000","last_change":"2024-04-20T21:16:58.595827+0000","last_active":"2024-04-20T21:16:59.604713+0000","last_peered":"2024-04-20T21:16:59.604713+0000","last_clean":"2024-04-20T21:16:59.604713+0000","last_became_active":"2024-04-20T21:16:58.595403+0000","last_became_peered":"2024-04-20T21:16:58.595403+0000","last_unstale":"2024-04-20T21:16:59.604713+0000","last_undegraded":"2024-04-20T21:16:59.604713+0000","last_fullsized":"2024-04-20T21:16:59.604713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:53.227573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.918376+0000","last_change":"2024-04-20T21:16:54.562128+0000","last_active":"2024-04-20T21:16:59.918376+0000","last_peered":"2024-04-20T21:16:59.918376+0000","last_clean":"2024-04-20T21:16:59.918376+0000","last_became_active":"2024-04-20T21:16:54.561495+0000","last_became_peered":"2024-04-20T21:16:54.561495+0000","last_unstale":"2024-04-20T21:16:59.918376+0000","last_undegraded":"2024-04-20T21:16:59.918376+0000","last_fullsized":"2024-04-20T21:16:59.918376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:19.582218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911795+0000","last_change":"2024-04-20T21:16:53.554881+0000","last_active":"2024-04-20T21:16:59.911795+0000","last_peered":"2024-04-20T21:16:59.911795+0000","last_clean":"2024-04-20T21:16:59.911795+0000","last_became_active":"2024-04-20T21:16:53.554590+0000","last_became_peered":"2024-04-20T21:16:53.554590+0000","last_unstale":"2024-04-20T21:16:59.911795+0000","last_undegraded":"2024-04-20T21:16:59.911795+0000","last_fullsized":"2024-04-20T21:16:59.911795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:36.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913934+0000","last_change":"2024-04-20T21:16:58.610704+0000","last_active":"2024-04-20T21:16:59.913934+0000","last_peered":"2024-04-20T21:16:59.913934+0000","last_clean":"2024-04-20T21:16:59.913934+0000","last_became_active":"2024-04-20T21:16:58.610309+0000","last_became_peered":"2024-04-20T21:16:58.610309+0000","last_unstale":"2024-04-20T21:16:59.913934+0000","last_undegraded":"2024-04-20T21:16:59.913934+0000","last_fullsized":"2024-04-20T21:16:59.913934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:04.215050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911685+0000","last_change":"2024-04-20T21:16:54.562351+0000","last_active":"2024-04-20T21:16:59.911685+0000","last_peered":"2024-04-20T21:16:59.911685+0000","last_clean":"2024-04-20T21:16:59.911685+0000","last_became_active":"2024-04-20T21:16:54.561947+0000","last_became_peered":"2024-04-20T21:16:54.561947+0000","last_unstale":"2024-04-20T21:16:59.911685+0000","last_undegraded":"2024-04-20T21:16:59.911685+0000","last_fullsized":"2024-04-20T21:16:59.911685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:52.802328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912701+0000","last_change":"2024-04-20T21:16:53.554081+0000","last_active":"2024-04-20T21:16:59.912701+0000","last_peered":"2024-04-20T21:16:59.912701+0000","last_clean":"2024-04-20T21:16:59.912701+0000","last_became_active":"2024-04-20T21:16:53.553800+0000","last_became_peered":"2024-04-20T21:16:53.553800+0000","last_unstale":"2024-04-20T21:16:59.912701+0000","last_undegraded":"2024-04-20T21:16:59.912701+0000","last_fullsized":"2024-04-20T21:16:59.912701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:52.865485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914034+0000","last_change":"2024-04-20T21:16:58.779184+0000","last_active":"2024-04-20T21:16:59.914034+0000","last_peered":"2024-04-20T21:16:59.914034+0000","last_clean":"2024-04-20T21:16:59.914034+0000","last_became_active":"2024-04-20T21:16:58.778627+0000","last_became_peered":"2024-04-20T21:16:58.778627+0000","last_unstale":"2024-04-20T21:16:59.914034+0000","last_undegraded":"2024-04-20T21:16:59.914034+0000","last_fullsized":"2024-04-20T21:16:59.914034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:50:07.247736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.917097+0000","last_change":"2024-04-20T21:16:58.774438+0000","last_active":"2024-04-20T21:16:59.917097+0000","last_peered":"2024-04-20T21:16:59.917097+0000","last_clean":"2024-04-20T21:16:59.917097+0000","last_became_active":"2024-04-20T21:16:58.774030+0000","last_became_peered":"2024-04-20T21:16:58.774030+0000","last_unstale":"2024-04-20T21:16:59.917097+0000","last_undegraded":"2024-04-20T21:16:59.917097+0000","last_fullsized":"2024-04-20T21:16:59.917097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:09:22.583239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606553+0000","last_change":"2024-04-20T21:16:54.573779+0000","last_active":"2024-04-20T21:16:59.606553+0000","last_peered":"2024-04-20T21:16:59.606553+0000","last_clean":"2024-04-20T21:16:59.606553+0000","last_became_active":"2024-04-20T21:16:54.573444+0000","last_became_peered":"2024-04-20T21:16:54.573444+0000","last_unstale":"2024-04-20T21:16:59.606553+0000","last_undegraded":"2024-04-20T21:16:59.606553+0000","last_fullsized":"2024-04-20T21:16:59.606553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:27.950525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911680+0000","last_change":"2024-04-20T21:16:53.543678+0000","last_active":"2024-04-20T21:16:59.911680+0000","last_peered":"2024-04-20T21:16:59.911680+0000","last_clean":"2024-04-20T21:16:59.911680+0000","last_became_active":"2024-04-20T21:16:53.543381+0000","last_became_peered":"2024-04-20T21:16:53.543381+0000","last_unstale":"2024-04-20T21:16:59.911680+0000","last_undegraded":"2024-04-20T21:16:59.911680+0000","last_fullsized":"2024-04-20T21:16:59.911680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:02.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912679+0000","last_change":"2024-04-20T21:16:58.626882+0000","last_active":"2024-04-20T21:16:59.912679+0000","last_peered":"2024-04-20T21:16:59.912679+0000","last_clean":"2024-04-20T21:16:59.912679+0000","last_became_active":"2024-04-20T21:16:58.626489+0000","last_became_peered":"2024-04-20T21:16:58.626489+0000","last_unstale":"2024-04-20T21:16:59.912679+0000","last_undegraded":"2024-04-20T21:16:59.912679+0000","last_fullsized":"2024-04-20T21:16:59.912679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:33:22.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913977+0000","last_change":"2024-04-20T21:16:54.580893+0000","last_active":"2024-04-20T21:16:59.913977+0000","last_peered":"2024-04-20T21:16:59.913977+0000","last_clean":"2024-04-20T21:16:59.913977+0000","last_became_active":"2024-04-20T21:16:54.580528+0000","last_became_peered":"2024-04-20T21:16:54.580528+0000","last_unstale":"2024-04-20T21:16:59.913977+0000","last_undegraded":"2024-04-20T21:16:59.913977+0000","last_fullsized":"2024-04-20T21:16:59.913977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:14.699123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914639+0000","last_change":"2024-04-20T21:16:53.563930+0000","last_active":"2024-04-20T21:16:59.914639+0000","last_peered":"2024-04-20T21:16:59.914639+0000","last_clean":"2024-04-20T21:16:59.914639+0000","last_became_active":"2024-04-20T21:16:53.559991+0000","last_became_peered":"2024-04-20T21:16:53.559991+0000","last_unstale":"2024-04-20T21:16:59.914639+0000","last_undegraded":"2024-04-20T21:16:59.914639+0000","last_fullsized":"2024-04-20T21:16:59.914639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:22.529712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912173+0000","last_change":"2024-04-20T21:16:53.552509+0000","last_active":"2024-04-20T21:16:59.912173+0000","last_peered":"2024-04-20T21:16:59.912173+0000","last_clean":"2024-04-20T21:16:59.912173+0000","last_became_active":"2024-04-20T21:16:53.552173+0000","last_became_peered":"2024-04-20T21:16:53.552173+0000","last_unstale":"2024-04-20T21:16:59.912173+0000","last_undegraded":"2024-04-20T21:16:59.912173+0000","last_fullsized":"2024-04-20T21:16:59.912173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:14.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916470+0000","last_change":"2024-04-20T21:16:54.562849+0000","last_active":"2024-04-20T21:16:59.916470+0000","last_peered":"2024-04-20T21:16:59.916470+0000","last_clean":"2024-04-20T21:16:59.916470+0000","last_became_active":"2024-04-20T21:16:54.561283+0000","last_became_peered":"2024-04-20T21:16:54.561283+0000","last_unstale":"2024-04-20T21:16:59.916470+0000","last_undegraded":"2024-04-20T21:16:59.916470+0000","last_fullsized":"2024-04-20T21:16:59.916470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:52.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605952+0000","last_change":"2024-04-20T21:16:58.614752+0000","last_active":"2024-04-20T21:16:59.605952+0000","last_peered":"2024-04-20T21:16:59.605952+0000","last_clean":"2024-04-20T21:16:59.605952+0000","last_became_active":"2024-04-20T21:16:58.614201+0000","last_became_peered":"2024-04-20T21:16:58.614201+0000","last_unstale":"2024-04-20T21:16:59.605952+0000","last_undegraded":"2024-04-20T21:16:59.605952+0000","last_fullsized":"2024-04-20T21:16:59.605952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:46.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913587+0000","last_change":"2024-04-20T21:16:53.564303+0000","last_active":"2024-04-20T21:16:59.913587+0000","last_peered":"2024-04-20T21:16:59.913587+0000","last_clean":"2024-04-20T21:16:59.913587+0000","last_became_active":"2024-04-20T21:16:53.560203+0000","last_became_peered":"2024-04-20T21:16:53.560203+0000","last_unstale":"2024-04-20T21:16:59.913587+0000","last_undegraded":"2024-04-20T21:16:59.913587+0000","last_fullsized":"2024-04-20T21:16:59.913587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:42.691233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604601+0000","last_change":"2024-04-20T21:16:54.554499+0000","last_active":"2024-04-20T21:16:59.604601+0000","last_peered":"2024-04-20T21:16:59.604601+0000","last_clean":"2024-04-20T21:16:59.604601+0000","last_became_active":"2024-04-20T21:16:54.554230+0000","last_became_peered":"2024-04-20T21:16:54.554230+0000","last_unstale":"2024-04-20T21:16:59.604601+0000","last_undegraded":"2024-04-20T21:16:59.604601+0000","last_fullsized":"2024-04-20T21:16:59.604601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:24:31.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915386+0000","last_change":"2024-04-20T21:16:58.610424+0000","last_active":"2024-04-20T21:16:59.915386+0000","last_peered":"2024-04-20T21:16:59.915386+0000","last_clean":"2024-04-20T21:16:59.915386+0000","last_became_active":"2024-04-20T21:16:58.610025+0000","last_became_peered":"2024-04-20T21:16:58.610025+0000","last_unstale":"2024-04-20T21:16:59.915386+0000","last_undegraded":"2024-04-20T21:16:59.915386+0000","last_fullsized":"2024-04-20T21:16:59.915386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:19.290865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911710+0000","last_change":"2024-04-20T21:16:54.578801+0000","last_active":"2024-04-20T21:16:59.911710+0000","last_peered":"2024-04-20T21:16:59.911710+0000","last_clean":"2024-04-20T21:16:59.911710+0000","last_became_active":"2024-04-20T21:16:54.578543+0000","last_became_peered":"2024-04-20T21:16:54.578543+0000","last_unstale":"2024-04-20T21:16:59.911710+0000","last_undegraded":"2024-04-20T21:16:59.911710+0000","last_fullsized":"2024-04-20T21:16:59.911710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:15.042318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.605448+0000","last_change":"2024-04-20T21:16:54.565658+0000","last_active":"2024-04-20T21:16:59.605448+0000","last_peered":"2024-04-20T21:16:59.605448+0000","last_clean":"2024-04-20T21:16:59.605448+0000","last_became_active":"2024-04-20T21:16:54.565371+0000","last_became_peered":"2024-04-20T21:16:54.565371+0000","last_unstale":"2024-04-20T21:16:59.605448+0000","last_undegraded":"2024-04-20T21:16:59.605448+0000","last_fullsized":"2024-04-20T21:16:59.605448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:21.781013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912115+0000","last_change":"2024-04-20T21:16:54.568472+0000","last_active":"2024-04-20T21:16:59.912115+0000","last_peered":"2024-04-20T21:16:59.912115+0000","last_clean":"2024-04-20T21:16:59.912115+0000","last_became_active":"2024-04-20T21:16:54.568223+0000","last_became_peered":"2024-04-20T21:16:54.568223+0000","last_unstale":"2024-04-20T21:16:59.912115+0000","last_undegraded":"2024-04-20T21:16:59.912115+0000","last_fullsized":"2024-04-20T21:16:59.912115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:42:53.921935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916243+0000","last_change":"2024-04-20T21:16:54.573747+0000","last_active":"2024-04-20T21:16:59.916243+0000","last_peered":"2024-04-20T21:16:59.916243+0000","last_clean":"2024-04-20T21:16:59.916243+0000","last_became_active":"2024-04-20T21:16:54.573476+0000","last_became_peered":"2024-04-20T21:16:54.573476+0000","last_unstale":"2024-04-20T21:16:59.916243+0000","last_undegraded":"2024-04-20T21:16:59.916243+0000","last_fullsized":"2024-04-20T21:16:59.916243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:02.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913473+0000","last_change":"2024-04-20T21:16:54.577630+0000","last_active":"2024-04-20T21:16:59.913473+0000","last_peered":"2024-04-20T21:16:59.913473+0000","last_clean":"2024-04-20T21:16:59.913473+0000","last_became_active":"2024-04-20T21:16:54.577318+0000","last_became_peered":"2024-04-20T21:16:54.577318+0000","last_unstale":"2024-04-20T21:16:59.913473+0000","last_undegraded":"2024-04-20T21:16:59.913473+0000","last_fullsized":"2024-04-20T21:16:59.913473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:45.584050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608267+0000","last_change":"2024-04-20T21:16:54.579850+0000","last_active":"2024-04-20T21:16:59.608267+0000","last_peered":"2024-04-20T21:16:59.608267+0000","last_clean":"2024-04-20T21:16:59.608267+0000","last_became_active":"2024-04-20T21:16:54.579546+0000","last_became_peered":"2024-04-20T21:16:54.579546+0000","last_unstale":"2024-04-20T21:16:59.608267+0000","last_undegraded":"2024-04-20T21:16:59.608267+0000","last_fullsized":"2024-04-20T21:16:59.608267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:49:53.577077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912501+0000","last_change":"2024-04-20T21:16:54.559174+0000","last_active":"2024-04-20T21:16:59.912501+0000","last_peered":"2024-04-20T21:16:59.912501+0000","last_clean":"2024-04-20T21:16:59.912501+0000","last_became_active":"2024-04-20T21:16:54.558904+0000","last_became_peered":"2024-04-20T21:16:54.558904+0000","last_unstale":"2024-04-20T21:16:59.912501+0000","last_undegraded":"2024-04-20T21:16:59.912501+0000","last_fullsized":"2024-04-20T21:16:59.912501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:09.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916492+0000","last_change":"2024-04-20T21:16:54.574943+0000","last_active":"2024-04-20T21:16:59.916492+0000","last_peered":"2024-04-20T21:16:59.916492+0000","last_clean":"2024-04-20T21:16:59.916492+0000","last_became_active":"2024-04-20T21:16:54.574471+0000","last_became_peered":"2024-04-20T21:16:54.574471+0000","last_unstale":"2024-04-20T21:16:59.916492+0000","last_undegraded":"2024-04-20T21:16:59.916492+0000","last_fullsized":"2024-04-20T21:16:59.916492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:51:44.741339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911545+0000","last_change":"2024-04-20T21:16:54.566222+0000","last_active":"2024-04-20T21:16:59.911545+0000","last_peered":"2024-04-20T21:16:59.911545+0000","last_clean":"2024-04-20T21:16:59.911545+0000","last_became_active":"2024-04-20T21:16:54.566011+0000","last_became_peered":"2024-04-20T21:16:54.566011+0000","last_unstale":"2024-04-20T21:16:59.911545+0000","last_undegraded":"2024-04-20T21:16:59.911545+0000","last_fullsized":"2024-04-20T21:16:59.911545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:53.369950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.914398+0000","last_change":"2024-04-20T21:16:54.571179+0000","last_active":"2024-04-20T21:16:59.914398+0000","last_peered":"2024-04-20T21:16:59.914398+0000","last_clean":"2024-04-20T21:16:59.914398+0000","last_became_active":"2024-04-20T21:16:54.570855+0000","last_became_peered":"2024-04-20T21:16:54.570855+0000","last_unstale":"2024-04-20T21:16:59.914398+0000","last_undegraded":"2024-04-20T21:16:59.914398+0000","last_fullsized":"2024-04-20T21:16:59.914398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:59.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911999+0000","last_change":"2024-04-20T21:16:54.559092+0000","last_active":"2024-04-20T21:16:59.911999+0000","last_peered":"2024-04-20T21:16:59.911999+0000","last_clean":"2024-04-20T21:16:59.911999+0000","last_became_active":"2024-04-20T21:16:54.558847+0000","last_became_peered":"2024-04-20T21:16:54.558847+0000","last_unstale":"2024-04-20T21:16:59.911999+0000","last_undegraded":"2024-04-20T21:16:59.911999+0000","last_fullsized":"2024-04-20T21:16:59.911999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:02.752070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915426+0000","last_change":"2024-04-20T21:16:54.571815+0000","last_active":"2024-04-20T21:16:59.915426+0000","last_peered":"2024-04-20T21:16:59.915426+0000","last_clean":"2024-04-20T21:16:59.915426+0000","last_became_active":"2024-04-20T21:16:54.570704+0000","last_became_peered":"2024-04-20T21:16:54.570704+0000","last_unstale":"2024-04-20T21:16:59.915426+0000","last_undegraded":"2024-04-20T21:16:59.915426+0000","last_fullsized":"2024-04-20T21:16:59.915426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:56.511539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912064+0000","last_change":"2024-04-20T21:16:54.555591+0000","last_active":"2024-04-20T21:16:59.912064+0000","last_peered":"2024-04-20T21:16:59.912064+0000","last_clean":"2024-04-20T21:16:59.912064+0000","last_became_active":"2024-04-20T21:16:54.555389+0000","last_became_peered":"2024-04-20T21:16:54.555389+0000","last_unstale":"2024-04-20T21:16:59.912064+0000","last_undegraded":"2024-04-20T21:16:59.912064+0000","last_fullsized":"2024-04-20T21:16:59.912064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:16:25.976707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617803+0000","last_change":"2024-04-20T21:16:54.564088+0000","last_active":"2024-04-20T21:16:59.617803+0000","last_peered":"2024-04-20T21:16:59.617803+0000","last_clean":"2024-04-20T21:16:59.617803+0000","last_became_active":"2024-04-20T21:16:54.563796+0000","last_became_peered":"2024-04-20T21:16:54.563796+0000","last_unstale":"2024-04-20T21:16:59.617803+0000","last_undegraded":"2024-04-20T21:16:59.617803+0000","last_fullsized":"2024-04-20T21:16:59.617803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:07.986736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606878+0000","last_change":"2024-04-20T21:16:54.563578+0000","last_active":"2024-04-20T21:16:59.606878+0000","last_peered":"2024-04-20T21:16:59.606878+0000","last_clean":"2024-04-20T21:16:59.606878+0000","last_became_active":"2024-04-20T21:16:54.563317+0000","last_became_peered":"2024-04-20T21:16:54.563317+0000","last_unstale":"2024-04-20T21:16:59.606878+0000","last_undegraded":"2024-04-20T21:16:59.606878+0000","last_fullsized":"2024-04-20T21:16:59.606878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:06.093675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912217+0000","last_change":"2024-04-20T21:16:54.565168+0000","last_active":"2024-04-20T21:16:59.912217+0000","last_peered":"2024-04-20T21:16:59.912217+0000","last_clean":"2024-04-20T21:16:59.912217+0000","last_became_active":"2024-04-20T21:16:54.564712+0000","last_became_peered":"2024-04-20T21:16:54.564712+0000","last_unstale":"2024-04-20T21:16:59.912217+0000","last_undegraded":"2024-04-20T21:16:59.912217+0000","last_fullsized":"2024-04-20T21:16:59.912217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:07:30.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911433+0000","last_change":"2024-04-20T21:16:54.572211+0000","last_active":"2024-04-20T21:16:59.911433+0000","last_peered":"2024-04-20T21:16:59.911433+0000","last_clean":"2024-04-20T21:16:59.911433+0000","last_became_active":"2024-04-20T21:16:54.571894+0000","last_became_peered":"2024-04-20T21:16:54.571894+0000","last_unstale":"2024-04-20T21:16:59.911433+0000","last_undegraded":"2024-04-20T21:16:59.911433+0000","last_fullsized":"2024-04-20T21:16:59.911433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:59.376514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.606682+0000","last_change":"2024-04-20T21:16:54.555195+0000","last_active":"2024-04-20T21:16:59.606682+0000","last_peered":"2024-04-20T21:16:59.606682+0000","last_clean":"2024-04-20T21:16:59.606682+0000","last_became_active":"2024-04-20T21:16:54.554284+0000","last_became_peered":"2024-04-20T21:16:54.554284+0000","last_unstale":"2024-04-20T21:16:59.606682+0000","last_undegraded":"2024-04-20T21:16:59.606682+0000","last_fullsized":"2024-04-20T21:16:59.606682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:42:21.500952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.915085+0000","last_change":"2024-04-20T21:16:54.565527+0000","last_active":"2024-04-20T21:16:59.915085+0000","last_peered":"2024-04-20T21:16:59.915085+0000","last_clean":"2024-04-20T21:16:59.915085+0000","last_became_active":"2024-04-20T21:16:54.565211+0000","last_became_peered":"2024-04-20T21:16:54.565211+0000","last_unstale":"2024-04-20T21:16:59.915085+0000","last_undegraded":"2024-04-20T21:16:59.915085+0000","last_fullsized":"2024-04-20T21:16:59.915085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:17:40.462670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.913206+0000","last_change":"2024-04-20T21:16:54.576464+0000","last_active":"2024-04-20T21:16:59.913206+0000","last_peered":"2024-04-20T21:16:59.913206+0000","last_clean":"2024-04-20T21:16:59.913206+0000","last_became_active":"2024-04-20T21:16:54.575947+0000","last_became_peered":"2024-04-20T21:16:54.575947+0000","last_unstale":"2024-04-20T21:16:59.913206+0000","last_undegraded":"2024-04-20T21:16:59.913206+0000","last_fullsized":"2024-04-20T21:16:59.913206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:50:29.581938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.604732+0000","last_change":"2024-04-20T21:16:54.558162+0000","last_active":"2024-04-20T21:16:59.604732+0000","last_peered":"2024-04-20T21:16:59.604732+0000","last_clean":"2024-04-20T21:16:59.604732+0000","last_became_active":"2024-04-20T21:16:54.557875+0000","last_became_peered":"2024-04-20T21:16:54.557875+0000","last_unstale":"2024-04-20T21:16:59.604732+0000","last_undegraded":"2024-04-20T21:16:59.604732+0000","last_fullsized":"2024-04-20T21:16:59.604732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:42:46.886405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911994+0000","last_change":"2024-04-20T21:16:54.560566+0000","last_active":"2024-04-20T21:16:59.911994+0000","last_peered":"2024-04-20T21:16:59.911994+0000","last_clean":"2024-04-20T21:16:59.911994+0000","last_became_active":"2024-04-20T21:16:54.559409+0000","last_became_peered":"2024-04-20T21:16:54.559409+0000","last_unstale":"2024-04-20T21:16:59.911994+0000","last_undegraded":"2024-04-20T21:16:59.911994+0000","last_fullsized":"2024-04-20T21:16:59.911994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:00.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912405+0000","last_change":"2024-04-20T21:16:54.559491+0000","last_active":"2024-04-20T21:16:59.912405+0000","last_peered":"2024-04-20T21:16:59.912405+0000","last_clean":"2024-04-20T21:16:59.912405+0000","last_became_active":"2024-04-20T21:16:54.558187+0000","last_became_peered":"2024-04-20T21:16:54.558187+0000","last_unstale":"2024-04-20T21:16:59.912405+0000","last_undegraded":"2024-04-20T21:16:59.912405+0000","last_fullsized":"2024-04-20T21:16:59.912405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:33.686248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617814+0000","last_change":"2024-04-20T21:16:54.564258+0000","last_active":"2024-04-20T21:16:59.617814+0000","last_peered":"2024-04-20T21:16:59.617814+0000","last_clean":"2024-04-20T21:16:59.617814+0000","last_became_active":"2024-04-20T21:16:54.563966+0000","last_became_peered":"2024-04-20T21:16:54.563966+0000","last_unstale":"2024-04-20T21:16:59.617814+0000","last_undegraded":"2024-04-20T21:16:59.617814+0000","last_fullsized":"2024-04-20T21:16:59.617814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:13:34.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912591+0000","last_change":"2024-04-20T21:16:54.573554+0000","last_active":"2024-04-20T21:16:59.912591+0000","last_peered":"2024-04-20T21:16:59.912591+0000","last_clean":"2024-04-20T21:16:59.912591+0000","last_became_active":"2024-04-20T21:16:54.571849+0000","last_became_peered":"2024-04-20T21:16:54.571849+0000","last_unstale":"2024-04-20T21:16:59.912591+0000","last_undegraded":"2024-04-20T21:16:59.912591+0000","last_fullsized":"2024-04-20T21:16:59.912591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:09.481779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617599+0000","last_change":"2024-04-20T21:16:54.571471+0000","last_active":"2024-04-20T21:16:59.617599+0000","last_peered":"2024-04-20T21:16:59.617599+0000","last_clean":"2024-04-20T21:16:59.617599+0000","last_became_active":"2024-04-20T21:16:54.571209+0000","last_became_peered":"2024-04-20T21:16:54.571209+0000","last_unstale":"2024-04-20T21:16:59.617599+0000","last_undegraded":"2024-04-20T21:16:59.617599+0000","last_fullsized":"2024-04-20T21:16:59.617599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:35.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.911811+0000","last_change":"2024-04-20T21:16:54.572168+0000","last_active":"2024-04-20T21:16:59.911811+0000","last_peered":"2024-04-20T21:16:59.911811+0000","last_clean":"2024-04-20T21:16:59.911811+0000","last_became_active":"2024-04-20T21:16:54.571740+0000","last_became_peered":"2024-04-20T21:16:54.571740+0000","last_unstale":"2024-04-20T21:16:59.911811+0000","last_undegraded":"2024-04-20T21:16:59.911811+0000","last_fullsized":"2024-04-20T21:16:59.911811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:11:35.728824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617393+0000","last_change":"2024-04-20T21:16:54.569081+0000","last_active":"2024-04-20T21:16:59.617393+0000","last_peered":"2024-04-20T21:16:59.617393+0000","last_clean":"2024-04-20T21:16:59.617393+0000","last_became_active":"2024-04-20T21:16:54.568847+0000","last_became_peered":"2024-04-20T21:16:54.568847+0000","last_unstale":"2024-04-20T21:16:59.617393+0000","last_undegraded":"2024-04-20T21:16:59.617393+0000","last_fullsized":"2024-04-20T21:16:59.617393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:35.737237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.912610+0000","last_change":"2024-04-20T21:16:54.571525+0000","last_active":"2024-04-20T21:16:59.912610+0000","last_peered":"2024-04-20T21:16:59.912610+0000","last_clean":"2024-04-20T21:16:59.912610+0000","last_became_active":"2024-04-20T21:16:54.570776+0000","last_became_peered":"2024-04-20T21:16:54.570776+0000","last_unstale":"2024-04-20T21:16:59.912610+0000","last_undegraded":"2024-04-20T21:16:59.912610+0000","last_fullsized":"2024-04-20T21:16:59.912610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:03.693476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.916192+0000","last_change":"2024-04-20T21:16:54.570895+0000","last_active":"2024-04-20T21:16:59.916192+0000","last_peered":"2024-04-20T21:16:59.916192+0000","last_clean":"2024-04-20T21:16:59.916192+0000","last_became_active":"2024-04-20T21:16:54.570577+0000","last_became_peered":"2024-04-20T21:16:54.570577+0000","last_unstale":"2024-04-20T21:16:59.916192+0000","last_undegraded":"2024-04-20T21:16:59.916192+0000","last_fullsized":"2024-04-20T21:16:59.916192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:26.787487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.608644+0000","last_change":"2024-04-20T21:16:54.581648+0000","last_active":"2024-04-20T21:16:59.608644+0000","last_peered":"2024-04-20T21:16:59.608644+0000","last_clean":"2024-04-20T21:16:59.608644+0000","last_became_active":"2024-04-20T21:16:54.581382+0000","last_became_peered":"2024-04-20T21:16:54.581382+0000","last_unstale":"2024-04-20T21:16:59.608644+0000","last_undegraded":"2024-04-20T21:16:59.608644+0000","last_fullsized":"2024-04-20T21:16:59.608644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:21.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-20T21:16:59.617750+0000","last_change":"2024-04-20T21:16:54.563949+0000","last_active":"2024-04-20T21:16:59.617750+0000","last_peered":"2024-04-20T21:16:59.617750+0000","last_clean":"2024-04-20T21:16:59.617750+0000","last_became_active":"2024-04-20T21:16:54.563606+0000","last_became_peered":"2024-04-20T21:16:54.563606+0000","last_unstale":"2024-04-20T21:16:59.617750+0000","last_undegraded":"2024-04-20T21:16:59.617750+0000","last_fullsized":"2024-04-20T21:16:59.617750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:26:05.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:26.282 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T21:17:26.282 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T21:17:26.282 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-20T21:17:26.282 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T21:17:26.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.516+0000 7f6983200640 1 Processor -- start 2024-04-20T21:17:26.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.516+0000 7f6983200640 1 -- start start 2024-04-20T21:17:26.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.518+0000 7f6983200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f69840c9630 0x7f69840c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.518+0000 7f6983200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6984158e80 0x7f6984152920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.518+0000 7f6983200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6984073b30 con 0x7f69840c9a10 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.518+0000 7f6983200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f69840c81c0 con 0x7f69840c9630 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.518+0000 7f6983200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f69840c5ec0 con 0x7f6984158e80 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.519+0000 7f6981400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f69840c9a10 0x7f6984073420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38738/0 (socket says 172.21.15.102:38738) 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.519+0000 7f6982800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6984158e80 0x7f6984152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.519+0000 7f6981400640 1 -- 172.21.15.102:0/3516982754 learned_addr learned my addr 172.21.15.102:0/3516982754 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.519+0000 7f6981e00640 1 --2- 172.21.15.102:0/3516982754 >> v2:172.21.15.102:3300/0 conn(0x7f69840c9630 0x7f69840c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.519+0000 7f6982800640 1 -- 172.21.15.102:0/3516982754 >> v2:172.21.15.102:3300/0 conn(0x7f69840c9630 msgr2=0x7f69840c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.519+0000 7f6982800640 1 --2- 172.21.15.102:0/3516982754 >> v2:172.21.15.102:3300/0 conn(0x7f69840c9630 0x7f69840c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.519+0000 7f6982800640 1 -- 172.21.15.102:0/3516982754 >> v1:172.21.15.102:6789/0 conn(0x7f69840c9a10 legacy=0x7f6984073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.519+0000 7f6982800640 1 -- 172.21.15.102:0/3516982754 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69840c83f0 con 0x7f6984158e80 2024-04-20T21:17:26.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.520+0000 7f6982800640 1 --2- 172.21.15.102:0/3516982754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 0x7f6984152920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f697c0097a0 tx=0x7f697c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b6abf3eaceecda server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.520+0000 7f6980a00640 1 -- 172.21.15.102:0/3516982754 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c035020 con 0x7f6984158e80 2024-04-20T21:17:26.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.520+0000 7f6980a00640 1 -- 172.21.15.102:0/3516982754 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697c031830 con 0x7f6984158e80 2024-04-20T21:17:26.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.520+0000 7f6980a00640 1 -- 172.21.15.102:0/3516982754 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c031c00 con 0x7f6984158e80 2024-04-20T21:17:26.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.521+0000 7f6983200640 1 -- 172.21.15.102:0/3516982754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 msgr2=0x7f6984152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.521+0000 7f6983200640 1 --2- 172.21.15.102:0/3516982754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 0x7f6984152920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f697c0097a0 tx=0x7f697c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.521+0000 7f6983200640 1 -- 172.21.15.102:0/3516982754 shutdown_connections 2024-04-20T21:17:26.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.521+0000 7f6983200640 1 --2- 172.21.15.102:0/3516982754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 0x7f6984152920 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.521+0000 7f6983200640 1 --2- 172.21.15.102:0/3516982754 >> v2:172.21.15.102:3300/0 conn(0x7f69840c9630 0x7f69840c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.521+0000 7f6983200640 1 -- 172.21.15.102:0/3516982754 >> 172.21.15.102:0/3516982754 conn(0x7f6984074ea0 msgr2=0x7f6984075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:26.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.521+0000 7f6983200640 1 -- 172.21.15.102:0/3516982754 shutdown_connections 2024-04-20T21:17:26.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.521+0000 7f6983200640 1 -- 172.21.15.102:0/3516982754 wait complete. 2024-04-20T21:17:26.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.523+0000 7f6983200640 1 Processor -- start 2024-04-20T21:17:26.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.523+0000 7f6983200640 1 -- start start 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.524+0000 7f6983200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840c9630 0x7f69841f0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.524+0000 7f6983200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69840c9a10 0x7f69841ef980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.524+0000 7f6983200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 0x7f69841f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.524+0000 7f6983200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f69840c8c90 con 0x7f69840c9630 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.524+0000 7f6983200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f69840c89b0 con 0x7f6984158e80 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.524+0000 7f6983200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f69840c6110 con 0x7f69840c9a10 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840c9630 0x7f69841f0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840c9630 0x7f69841f0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51648/0 (socket says 172.21.15.102:51648) 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 -- 172.21.15.102:0/2030881105 learned_addr learned my addr 172.21.15.102:0/2030881105 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6982800640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 0x7f69841f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981400640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69840c9a10 0x7f69841ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 -- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69840c9a10 msgr2=0x7f69841ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69840c9a10 0x7f69841ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 -- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 msgr2=0x7f69841f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 0x7f69841f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 -- 172.21.15.102:0/2030881105 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69841eeec0 con 0x7f69840c9630 2024-04-20T21:17:26.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6982800640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 0x7f69841f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:26.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.525+0000 7f6981e00640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840c9630 0x7f69841f0330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f697400e9b0 tx=0x7f697400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.526+0000 7f697a000640 1 -- 172.21.15.102:0/2030881105 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974017070 con 0x7f69840c9630 2024-04-20T21:17:26.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.526+0000 7f697a000640 1 -- 172.21.15.102:0/2030881105 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697400c9a0 con 0x7f69840c9630 2024-04-20T21:17:26.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.526+0000 7f697a000640 1 -- 172.21.15.102:0/2030881105 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400cd90 con 0x7f69840c9630 2024-04-20T21:17:26.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.526+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69841f2c00 con 0x7f69840c9630 2024-04-20T21:17:26.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.526+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841f31c0 con 0x7f69840c9630 2024-04-20T21:17:26.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.527+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698404fd60 con 0x7f69840c9630 2024-04-20T21:17:26.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.528+0000 7f697a000640 1 -- 172.21.15.102:0/2030881105 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f697401a070 con 0x7f69840c9630 2024-04-20T21:17:26.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.529+0000 7f697a000640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f696406c8f0 0x7f696406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.529+0000 7f697a000640 1 -- 172.21.15.102:0/2030881105 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f697408be70 con 0x7f69840c9630 2024-04-20T21:17:26.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.529+0000 7f6981400640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f696406c8f0 0x7f696406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.530+0000 7f6981400640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f696406c8f0 0x7f696406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f69841560b0 tx=0x7f6970001a30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.534+0000 7f697a000640 1 -- 172.21.15.102:0/2030881105 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6974059880 con 0x7f69840c9630 2024-04-20T21:17:26.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.762+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f69841587e0 con 0x7f69840c9630 2024-04-20T21:17:26.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.763+0000 7f697a000640 1 -- 172.21.15.102:0/2030881105 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f697405db80 con 0x7f69840c9630 2024-04-20T21:17:26.763 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:26.763 INFO:teuthology.orchestra.run.smithi102.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T21:17:26.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f696406c8f0 msgr2=0x7f696406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f696406c8f0 0x7f696406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f69841560b0 tx=0x7f6970001a30 comp rx=0 tx=0).stop 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840c9630 msgr2=0x7f69841f0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840c9630 0x7f69841f0330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f697400e9b0 tx=0x7f697400ee70 comp rx=0 tx=0).stop 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 shutdown_connections 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f696406c8f0 0x7f696406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6984158e80 0x7f69841f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69840c9a10 0x7f69841ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 --2- 172.21.15.102:0/2030881105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840c9630 0x7f69841f0330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 >> 172.21.15.102:0/2030881105 conn(0x7f6984074ea0 msgr2=0x7f6984070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 shutdown_connections 2024-04-20T21:17:26.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.766+0000 7f6983200640 1 -- 172.21.15.102:0/2030881105 wait complete. 2024-04-20T21:17:26.783 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-20T21:17:26.783 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:26.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.973+0000 7fa84aa00640 1 Processor -- start 2024-04-20T21:17:26.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.973+0000 7fa84aa00640 1 -- start start 2024-04-20T21:17:26.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.974+0000 7fa84aa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa8440c75a0 0x7fa8440c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.974+0000 7fa84aa00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa844153560 con 0x7fa844153180 2024-04-20T21:17:26.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa84aa00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8440c7a70 con 0x7fa844073c70 2024-04-20T21:17:26.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa84aa00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa8440c7d30 con 0x7fa8440c75a0 2024-04-20T21:17:26.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa849600640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa844153180 0x7fa844158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38752/0 (socket says 172.21.15.102:38752) 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa849600640 1 -- 172.21.15.102:0/2087324885 learned_addr learned my addr 172.21.15.102:0/2087324885 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa848c00640 1 --2- 172.21.15.102:0/2087324885 >> v2:172.21.15.102:3300/0 conn(0x7fa8440c75a0 0x7fa8440c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa843e00640 1 -- 172.21.15.102:0/2087324885 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2825074939 0 0) 0x7fa844153560 con 0x7fa844153180 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa843e00640 1 -- 172.21.15.102:0/2087324885 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa82c003520 con 0x7fa844153180 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa843e00640 1 -- 172.21.15.102:0/2087324885 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462127661 0 0) 0x7fa8440c7a70 con 0x7fa844073c70 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa843e00640 1 -- 172.21.15.102:0/2087324885 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa844153560 con 0x7fa844073c70 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa848c00640 1 -- 172.21.15.102:0/2087324885 >> v1:172.21.15.193:6789/0 conn(0x7fa844073c70 legacy=0x7fa8441637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa848c00640 1 -- 172.21.15.102:0/2087324885 >> v1:172.21.15.102:6789/0 conn(0x7fa844153180 legacy=0x7fa844158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.975+0000 7fa848c00640 1 -- 172.21.15.102:0/2087324885 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84415c4d0 con 0x7fa8440c75a0 2024-04-20T21:17:26.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.976+0000 7fa848c00640 1 --2- 172.21.15.102:0/2087324885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8440c75a0 0x7fa8440c5f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097a0 tx=0x7fa83402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7577f3415628962c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.977+0000 7fa843e00640 1 -- 172.21.15.102:0/2087324885 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834034070 con 0x7fa8440c75a0 2024-04-20T21:17:26.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.977+0000 7fa843e00640 1 -- 172.21.15.102:0/2087324885 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa834030610 con 0x7fa8440c75a0 2024-04-20T21:17:26.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.977+0000 7fa843e00640 1 -- 172.21.15.102:0/2087324885 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340309c0 con 0x7fa8440c75a0 2024-04-20T21:17:26.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.977+0000 7fa84aa00640 1 -- 172.21.15.102:0/2087324885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8440c75a0 msgr2=0x7fa8440c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.977+0000 7fa84aa00640 1 --2- 172.21.15.102:0/2087324885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8440c75a0 0x7fa8440c5f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097a0 tx=0x7fa83402b960 comp rx=0 tx=0).stop 2024-04-20T21:17:26.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.977+0000 7fa84aa00640 1 -- 172.21.15.102:0/2087324885 shutdown_connections 2024-04-20T21:17:26.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.977+0000 7fa84aa00640 1 --2- 172.21.15.102:0/2087324885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8440c75a0 0x7fa8440c5f30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.977+0000 7fa84aa00640 1 -- 172.21.15.102:0/2087324885 >> 172.21.15.102:0/2087324885 conn(0x7fa8440cc820 msgr2=0x7fa8440ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:26.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.978+0000 7fa84aa00640 1 -- 172.21.15.102:0/2087324885 shutdown_connections 2024-04-20T21:17:26.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.978+0000 7fa84aa00640 1 -- 172.21.15.102:0/2087324885 wait complete. 2024-04-20T21:17:26.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.979+0000 7fa84aa00640 1 Processor -- start 2024-04-20T21:17:26.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.979+0000 7fa84aa00640 1 -- start start 2024-04-20T21:17:26.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.980+0000 7fa84aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa844073c70 0x7fa844147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.980+0000 7fa84aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8440c75a0 0x7fa84415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.980+0000 7fa84aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa844153180 0x7fa84415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.980+0000 7fa84aa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa844159550 con 0x7fa844153180 2024-04-20T21:17:26.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.980+0000 7fa84aa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa84415ca90 con 0x7fa8440c75a0 2024-04-20T21:17:26.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.980+0000 7fa84aa00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa84415d290 con 0x7fa844073c70 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa844153180 0x7fa84415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa849600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa844073c70 0x7fa844147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa844153180 0x7fa84415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51670/0 (socket says 172.21.15.102:51670) 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa848c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8440c75a0 0x7fa84415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 -- 172.21.15.102:0/224159709 learned_addr learned my addr 172.21.15.102:0/224159709 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 -- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa844073c70 msgr2=0x7fa844147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa844073c70 0x7fa844147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 -- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8440c75a0 msgr2=0x7fa84415a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8440c75a0 0x7fa84415a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 -- 172.21.15.102:0/224159709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8441f1010 con 0x7fa844153180 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa84a000640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa844153180 0x7fa84415a540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c6e0 tx=0x7fa83c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.981+0000 7fa841600640 1 -- 172.21.15.102:0/224159709 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c003b70 con 0x7fa844153180 2024-04-20T21:17:26.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.982+0000 7fa841600640 1 -- 172.21.15.102:0/224159709 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83c003d10 con 0x7fa844153180 2024-04-20T21:17:26.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.982+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa844157840 con 0x7fa844153180 2024-04-20T21:17:26.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.982+0000 7fa841600640 1 -- 172.21.15.102:0/224159709 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0052d0 con 0x7fa844153180 2024-04-20T21:17:26.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.982+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441f15b0 con 0x7fa844153180 2024-04-20T21:17:26.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.983+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84404fd60 con 0x7fa844153180 2024-04-20T21:17:26.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.984+0000 7fa841600640 1 -- 172.21.15.102:0/224159709 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa83c005470 con 0x7fa844153180 2024-04-20T21:17:26.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.985+0000 7fa841600640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82806c920 0x7fa82806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:26.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.985+0000 7fa841600640 1 -- 172.21.15.102:0/224159709 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa83c08bc60 con 0x7fa844153180 2024-04-20T21:17:26.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.985+0000 7fa849600640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82806c920 0x7fa82806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:26.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.986+0000 7fa849600640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82806c920 0x7fa82806ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa838000f60 tx=0x7fa838007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:26.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:26.989+0000 7fa841600640 1 -- 172.21.15.102:0/224159709 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa83c059660 con 0x7fa844153180 2024-04-20T21:17:27.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.175+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa844148900 con 0x7fa844153180 2024-04-20T21:17:27.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.178+0000 7fa841600640 1 -- 172.21.15.102:0/224159709 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7fa83c05d960 con 0x7fa844153180 2024-04-20T21:17:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:27.178 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:27.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82806c920 msgr2=0x7fa82806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82806c920 0x7fa82806ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa838000f60 tx=0x7fa838007d50 comp rx=0 tx=0).stop 2024-04-20T21:17:27.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa844153180 msgr2=0x7fa84415a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa844153180 0x7fa84415a540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c6e0 tx=0x7fa83c00cba0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 shutdown_connections 2024-04-20T21:17:27.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa844153180 0x7fa84415a540 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82806c920 0x7fa82806ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8440c75a0 0x7fa84415a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 --2- 172.21.15.102:0/224159709 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa844073c70 0x7fa844147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 >> 172.21.15.102:0/224159709 conn(0x7fa8440cc820 msgr2=0x7fa8440c6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:27.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 shutdown_connections 2024-04-20T21:17:27.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.180+0000 7fa84aa00640 1 -- 172.21.15.102:0/224159709 wait complete. 2024-04-20T21:17:27.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:16:51.795761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:16:52.983434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T21:16:56.582297+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T21:17:27.200 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:27.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.398+0000 7fc997200640 1 Processor -- start 2024-04-20T21:17:27.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.399+0000 7fc997200640 1 -- start start 2024-04-20T21:17:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc997200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9980d0ec0 0x7fc99815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc997200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc9980c81c0 0x7fc998154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc997200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9980cd4d0 con 0x7fc9980d0ae0 2024-04-20T21:17:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc997200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc99815c1e0 con 0x7fc9980d0ec0 2024-04-20T21:17:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc997200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc9980c86a0 con 0x7fc9980c81c0 2024-04-20T21:17:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc98fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9980d0ec0 0x7fc99815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc98fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9980d0ec0 0x7fc99815bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51680/0 (socket says 172.21.15.102:51680) 2024-04-20T21:17:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc98fe00640 1 -- 172.21.15.102:0/321722005 learned_addr learned my addr 172.21.15.102:0/321722005 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.400+0000 7fc996800640 1 --2- 172.21.15.102:0/321722005 >> v2:172.21.15.193:3300/0 conn(0x7fc9980c81c0 0x7fc998154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.401+0000 7fc98fe00640 1 -- 172.21.15.102:0/321722005 >> v2:172.21.15.193:3300/0 conn(0x7fc9980c81c0 msgr2=0x7fc998154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.401+0000 7fc98fe00640 1 --2- 172.21.15.102:0/321722005 >> v2:172.21.15.193:3300/0 conn(0x7fc9980c81c0 0x7fc998154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.401+0000 7fc98fe00640 1 -- 172.21.15.102:0/321722005 >> v1:172.21.15.193:6789/0 conn(0x7fc9980d0ae0 legacy=0x7fc9980c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.401+0000 7fc98fe00640 1 -- 172.21.15.102:0/321722005 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9980cd7b0 con 0x7fc9980d0ec0 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.401+0000 7fc995400640 1 -- 172.21.15.102:0/321722005 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413582178 0 0) 0x7fc9980cd4d0 con 0x7fc9980d0ae0 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.402+0000 7fc98fe00640 1 --2- 172.21.15.102:0/321722005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc99815bca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc9980ce790 tx=0x7fc98002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd9bcf38ca4ee96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.402+0000 7fc995400640 1 -- 172.21.15.102:0/321722005 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98002f850 con 0x7fc9980d0ec0 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.402+0000 7fc995400640 1 -- 172.21.15.102:0/321722005 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98002f9f0 con 0x7fc9980d0ec0 2024-04-20T21:17:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.402+0000 7fc995400640 1 -- 172.21.15.102:0/321722005 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98002fdc0 con 0x7fc9980d0ec0 2024-04-20T21:17:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.403+0000 7fc997200640 1 -- 172.21.15.102:0/321722005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 msgr2=0x7fc99815bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.403+0000 7fc997200640 1 --2- 172.21.15.102:0/321722005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc99815bca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc9980ce790 tx=0x7fc98002ba30 comp rx=0 tx=0).stop 2024-04-20T21:17:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.403+0000 7fc997200640 1 -- 172.21.15.102:0/321722005 shutdown_connections 2024-04-20T21:17:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.403+0000 7fc997200640 1 --2- 172.21.15.102:0/321722005 >> v2:172.21.15.193:3300/0 conn(0x7fc9980c81c0 0x7fc998154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.403+0000 7fc997200640 1 --2- 172.21.15.102:0/321722005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc99815bca0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.403+0000 7fc997200640 1 -- 172.21.15.102:0/321722005 >> 172.21.15.102:0/321722005 conn(0x7fc998074ea0 msgr2=0x7fc9980752b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.403+0000 7fc997200640 1 -- 172.21.15.102:0/321722005 shutdown_connections 2024-04-20T21:17:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.403+0000 7fc997200640 1 -- 172.21.15.102:0/321722005 wait complete. 2024-04-20T21:17:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.405+0000 7fc997200640 1 Processor -- start 2024-04-20T21:17:27.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.405+0000 7fc997200640 1 -- start start 2024-04-20T21:17:27.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc997200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9980c81c0 0x7fc9981e2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc997200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9980d0ae0 0x7fc9980b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc997200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc9981e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc997200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc9980ce050 con 0x7fc9980d0ec0 2024-04-20T21:17:27.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc997200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc9980cdd70 con 0x7fc9980d0ae0 2024-04-20T21:17:27.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc997200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc9980c9240 con 0x7fc9980c81c0 2024-04-20T21:17:27.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc996800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc9981e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc996800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc9981e2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51690/0 (socket says 172.21.15.102:51690) 2024-04-20T21:17:27.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc996800640 1 -- 172.21.15.102:0/907071127 learned_addr learned my addr 172.21.15.102:0/907071127 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:27.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc98fe00640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9980d0ae0 0x7fc9980b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.406+0000 7fc995e00640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9980c81c0 0x7fc9981e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc996800640 1 -- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9980c81c0 msgr2=0x7fc9981e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc996800640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9980c81c0 0x7fc9981e2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc996800640 1 -- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9980d0ae0 msgr2=0x7fc9980b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc996800640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9980d0ae0 0x7fc9980b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc996800640 1 -- 172.21.15.102:0/907071127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9980c73c0 con 0x7fc9980d0ec0 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc98fe00640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9980d0ae0 0x7fc9980b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc996800640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc9981e2c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc990009d20 tx=0x7fc99000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc98e000640 1 -- 172.21.15.102:0/907071127 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990018020 con 0x7fc9980d0ec0 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc98e000640 1 -- 172.21.15.102:0/907071127 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc990009b10 con 0x7fc9980d0ec0 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc98e000640 1 -- 172.21.15.102:0/907071127 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9900146c0 con 0x7fc9980d0ec0 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9981f1660 con 0x7fc9980d0ec0 2024-04-20T21:17:27.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.407+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981f42f0 con 0x7fc9980d0ec0 2024-04-20T21:17:27.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.409+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99804fcf0 con 0x7fc9980d0ec0 2024-04-20T21:17:27.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.410+0000 7fc98e000640 1 -- 172.21.15.102:0/907071127 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc990005d10 con 0x7fc9980d0ec0 2024-04-20T21:17:27.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.411+0000 7fc98e000640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc97406c8f0 0x7fc97406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.411+0000 7fc98e000640 1 -- 172.21.15.102:0/907071127 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc99008d140 con 0x7fc9980d0ec0 2024-04-20T21:17:27.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.411+0000 7fc995e00640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc97406c8f0 0x7fc97406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.412+0000 7fc995e00640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc97406c8f0 0x7fc97406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc988000f60 tx=0x7fc988009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:27.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.414+0000 7fc98e000640 1 -- 172.21.15.102:0/907071127 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc990022030 con 0x7fc9980d0ec0 2024-04-20T21:17:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.615+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc998075560 con 0x7fc9980d0ec0 2024-04-20T21:17:27.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.616+0000 7fc98e000640 1 -- 172.21.15.102:0/907071127 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fc990010040 con 0x7fc9980d0ec0 2024-04-20T21:17:27.617 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:17:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc97406c8f0 msgr2=0x7fc97406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc97406c8f0 0x7fc97406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc988000f60 tx=0x7fc988009040 comp rx=0 tx=0).stop 2024-04-20T21:17:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 msgr2=0x7fc9981e2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc9981e2c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc990009d20 tx=0x7fc99000c490 comp rx=0 tx=0).stop 2024-04-20T21:17:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 shutdown_connections 2024-04-20T21:17:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9980d0ec0 0x7fc9981e2c40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc97406c8f0 0x7fc97406edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9980d0ae0 0x7fc9980b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 --2- 172.21.15.102:0/907071127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9980c81c0 0x7fc9981e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 >> 172.21.15.102:0/907071127 conn(0x7fc998074ea0 msgr2=0x7fc998152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 shutdown_connections 2024-04-20T21:17:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.619+0000 7fc997200640 1 -- 172.21.15.102:0/907071127 wait complete. 2024-04-20T21:17:27.637 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:17:27.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.809+0000 7f7183000640 1 Processor -- start 2024-04-20T21:17:27.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.809+0000 7f7183000640 1 -- start start 2024-04-20T21:17:27.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.810+0000 7f7183000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f717c155620 0x7f717c0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.810+0000 7f7183000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f717c0c66e0 0x7f717c073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f7183000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717c0c6b20 con 0x7f717c0cb470 2024-04-20T21:17:27.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f7183000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f717c0c59b0 con 0x7f717c0c66e0 2024-04-20T21:17:27.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f7183000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f717c0cb900 con 0x7f717c155620 2024-04-20T21:17:27.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f7173400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f717c0c66e0 0x7f717c073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f717be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f717c155620 0x7f717c0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f7173400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f717c0c66e0 0x7f717c073740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34928/0 (socket says 172.21.15.102:34928) 2024-04-20T21:17:27.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f7173400640 1 -- 172.21.15.102:0/1687009875 learned_addr learned my addr 172.21.15.102:0/1687009875 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:27.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f717be00640 1 -- 172.21.15.102:0/1687009875 >> v2:172.21.15.102:3300/0 conn(0x7f717c0c66e0 msgr2=0x7f717c073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:27.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f717be00640 1 --2- 172.21.15.102:0/1687009875 >> v2:172.21.15.102:3300/0 conn(0x7f717c0c66e0 0x7f717c073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f717be00640 1 -- 172.21.15.102:0/1687009875 >> v1:172.21.15.102:6789/0 conn(0x7f717c0cb470 legacy=0x7f717c06c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f717be00640 1 -- 172.21.15.102:0/1687009875 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c15bd00 con 0x7f717c155620 2024-04-20T21:17:27.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.811+0000 7f7173400640 1 --2- 172.21.15.102:0/1687009875 >> v2:172.21.15.102:3300/0 conn(0x7f717c0c66e0 0x7f717c073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:27.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.812+0000 7f717be00640 1 --2- 172.21.15.102:0/1687009875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c155620 0x7f717c0cc6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f717c0c7dc0 tx=0x7f716c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d129d3505d3cadcf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:27.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.813+0000 7f717b400640 1 -- 172.21.15.102:0/1687009875 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c035020 con 0x7f717c155620 2024-04-20T21:17:27.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.813+0000 7f717b400640 1 -- 172.21.15.102:0/1687009875 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c031830 con 0x7f717c155620 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.813+0000 7f717b400640 1 -- 172.21.15.102:0/1687009875 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c031c00 con 0x7f717c155620 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.813+0000 7f7183000640 1 -- 172.21.15.102:0/1687009875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c155620 msgr2=0x7f717c0cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.813+0000 7f7183000640 1 --2- 172.21.15.102:0/1687009875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c155620 0x7f717c0cc6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f717c0c7dc0 tx=0x7f716c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.813+0000 7f7183000640 1 -- 172.21.15.102:0/1687009875 shutdown_connections 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.813+0000 7f7183000640 1 --2- 172.21.15.102:0/1687009875 >> v2:172.21.15.102:3300/0 conn(0x7f717c0c66e0 0x7f717c073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.813+0000 7f7183000640 1 --2- 172.21.15.102:0/1687009875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c155620 0x7f717c0cc6a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.814+0000 7f7183000640 1 -- 172.21.15.102:0/1687009875 >> 172.21.15.102:0/1687009875 conn(0x7f717c0754d0 msgr2=0x7f717c072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.814+0000 7f7183000640 1 -- 172.21.15.102:0/1687009875 shutdown_connections 2024-04-20T21:17:27.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.814+0000 7f7183000640 1 -- 172.21.15.102:0/1687009875 wait complete. 2024-04-20T21:17:27.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.815+0000 7f7183000640 1 Processor -- start 2024-04-20T21:17:27.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.815+0000 7f7183000640 1 -- start start 2024-04-20T21:17:27.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f7183000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f717c0c66e0 0x7f717c1520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f7183000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c0cb470 0x7f717c150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f7183000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717c155620 0x7f717c1f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f7183000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f717c0c5ee0 con 0x7f717c0c66e0 2024-04-20T21:17:27.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f7183000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f717c0c5c00 con 0x7f717c0cb470 2024-04-20T21:17:27.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f7183000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f717c0c61c0 con 0x7f717c155620 2024-04-20T21:17:27.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f717be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f717c0c66e0 0x7f717c1520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f717be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f717c0c66e0 0x7f717c1520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34944/0 (socket says 172.21.15.102:34944) 2024-04-20T21:17:27.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.816+0000 7f717be00640 1 -- 172.21.15.102:0/3509776598 learned_addr learned my addr 172.21.15.102:0/3509776598 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:27.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f7173400640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c0cb470 0x7f717c150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f7181000640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717c155620 0x7f717c1f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f717be00640 1 -- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717c155620 msgr2=0x7f717c1f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f717be00640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717c155620 0x7f717c1f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f717be00640 1 -- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c0cb470 msgr2=0x7f717c150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f717be00640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c0cb470 0x7f717c150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f717be00640 1 -- 172.21.15.102:0/3509776598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c1f1d10 con 0x7f717c0c66e0 2024-04-20T21:17:27.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f717be00640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f717c0c66e0 0x7f717c1520a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f716c009890 tx=0x7f716c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:27.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f7178c00640 1 -- 172.21.15.102:0/3509776598 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c041070 con 0x7f717c0c66e0 2024-04-20T21:17:27.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f7178c00640 1 -- 172.21.15.102:0/3509776598 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c03f070 con 0x7f717c0c66e0 2024-04-20T21:17:27.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.817+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c15aac0 con 0x7f717c0c66e0 2024-04-20T21:17:27.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.818+0000 7f7178c00640 1 -- 172.21.15.102:0/3509776598 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c035040 con 0x7f717c0c66e0 2024-04-20T21:17:27.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.818+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1f0fd0 con 0x7f717c0c66e0 2024-04-20T21:17:27.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.819+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c1f0de0 con 0x7f717c0c66e0 2024-04-20T21:17:27.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.820+0000 7f7178c00640 1 -- 172.21.15.102:0/3509776598 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f716c038020 con 0x7f717c0c66e0 2024-04-20T21:17:27.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.821+0000 7f7178c00640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.821+0000 7f7178c00640 1 -- 172.21.15.102:0/3509776598 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f716c0ad6c0 con 0x7f717c0c66e0 2024-04-20T21:17:27.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.821+0000 7f7173400640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.822+0000 7f7173400640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f715806c8f0 0x7f715806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7164009630 tx=0x7f7164009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:27.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:27.826+0000 7f7178c00640 1 -- 172.21.15.102:0/3509776598 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f716c031800 con 0x7f717c0c66e0 2024-04-20T21:17:28.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.015+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f717c0b3670 con 0x7f717c0c66e0 2024-04-20T21:17:28.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.016+0000 7f7178c00640 1 -- 172.21.15.102:0/3509776598 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f716c07b150 con 0x7f717c0c66e0 2024-04-20T21:17:28.015 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:28.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f715806c8f0 msgr2=0x7f715806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f715806c8f0 0x7f715806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7164009630 tx=0x7f7164009240 comp rx=0 tx=0).stop 2024-04-20T21:17:28.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f717c0c66e0 msgr2=0x7f717c1520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f717c0c66e0 0x7f717c1520a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f716c009890 tx=0x7f716c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 shutdown_connections 2024-04-20T21:17:28.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717c155620 0x7f717c1f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f717c0cb470 0x7f717c150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 --2- 172.21.15.102:0/3509776598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f717c0c66e0 0x7f717c1520a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 >> 172.21.15.102:0/3509776598 conn(0x7f717c0754d0 msgr2=0x7f717c14b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 shutdown_connections 2024-04-20T21:17:28.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.018+0000 7f7183000640 1 -- 172.21.15.102:0/3509776598 wait complete. 2024-04-20T21:17:28.035 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:17:28.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.227+0000 7fe05be00640 1 Processor -- start 2024-04-20T21:17:28.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.227+0000 7fe05be00640 1 -- start start 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.228+0000 7fe05be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe05c1554f0 0x7fe05c15b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.228+0000 7fe05be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe05c0c59c0 0x7fe05c0d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.228+0000 7fe05be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c5e10 0x7fe05c0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.228+0000 7fe05be00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe05c1558d0 con 0x7fe05c1554f0 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.228+0000 7fe05be00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fe05c0cce30 con 0x7fe05c0c59c0 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.228+0000 7fe05be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe05c0cd110 con 0x7fe05c0c5e10 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05aa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe05c1554f0 0x7fe05c15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05aa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe05c1554f0 0x7fe05c15b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34960/0 (socket says 172.21.15.102:34960) 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05a000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe05c0c59c0 0x7fe05c0d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05aa00640 1 -- 172.21.15.102:0/75358680 learned_addr learned my addr 172.21.15.102:0/75358680 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:28.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05b400640 1 --2- 172.21.15.102:0/75358680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c5e10 0x7fe05c0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05aa00640 1 -- 172.21.15.102:0/75358680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c5e10 msgr2=0x7fe05c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:28.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05aa00640 1 --2- 172.21.15.102:0/75358680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c5e10 0x7fe05c0d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05aa00640 1 -- 172.21.15.102:0/75358680 >> v2:172.21.15.193:3300/0 conn(0x7fe05c0c59c0 msgr2=0x7fe05c0d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:28.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05aa00640 1 --2- 172.21.15.102:0/75358680 >> v2:172.21.15.193:3300/0 conn(0x7fe05c0c59c0 0x7fe05c0d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.229+0000 7fe05aa00640 1 -- 172.21.15.102:0/75358680 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c0cd3f0 con 0x7fe05c1554f0 2024-04-20T21:17:28.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.230+0000 7fe05aa00640 1 --2- 172.21.15.102:0/75358680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c1554f0 0x7fe05c15b120 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0d1bc0 tx=0x7fe04c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a02a618bfdd425 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.230+0000 7fe059600640 1 -- 172.21.15.102:0/75358680 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c034070 con 0x7fe05c1554f0 2024-04-20T21:17:28.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe059600640 1 -- 172.21.15.102:0/75358680 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04c030850 con 0x7fe05c1554f0 2024-04-20T21:17:28.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe059600640 1 -- 172.21.15.102:0/75358680 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c030c20 con 0x7fe05c1554f0 2024-04-20T21:17:28.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 -- 172.21.15.102:0/75358680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c1554f0 msgr2=0x7fe05c15b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 --2- 172.21.15.102:0/75358680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c1554f0 0x7fe05c15b120 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0d1bc0 tx=0x7fe04c02b990 comp rx=0 tx=0).stop 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 -- 172.21.15.102:0/75358680 shutdown_connections 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 --2- 172.21.15.102:0/75358680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c5e10 0x7fe05c0d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 --2- 172.21.15.102:0/75358680 >> v2:172.21.15.193:3300/0 conn(0x7fe05c0c59c0 0x7fe05c0d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 --2- 172.21.15.102:0/75358680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c1554f0 0x7fe05c15b120 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 -- 172.21.15.102:0/75358680 >> 172.21.15.102:0/75358680 conn(0x7fe05c06d150 msgr2=0x7fe05c074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 -- 172.21.15.102:0/75358680 shutdown_connections 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.231+0000 7fe05be00640 1 -- 172.21.15.102:0/75358680 wait complete. 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.233+0000 7fe05be00640 1 Processor -- start 2024-04-20T21:17:28.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.233+0000 7fe05be00640 1 -- start start 2024-04-20T21:17:28.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c59c0 0x7fe05c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c0c5e10 0x7fe05c145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe05c1554f0 0x7fe05c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0cdc90 con 0x7fe05c0c5e10 2024-04-20T21:17:28.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0cd9b0 con 0x7fe05c1554f0 2024-04-20T21:17:28.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe05c0ccad0 con 0x7fe05c0c59c0 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c0c5e10 0x7fe05c145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c0c5e10 0x7fe05c145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34964/0 (socket says 172.21.15.102:34964) 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05a000640 1 -- 172.21.15.102:0/221444001 learned_addr learned my addr 172.21.15.102:0/221444001 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05b400640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe05c1554f0 0x7fe05c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05aa00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c59c0 0x7fe05c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.234+0000 7fe05a000640 1 -- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c59c0 msgr2=0x7fe05c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe05a000640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c59c0 0x7fe05c0c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe05a000640 1 -- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe05c1554f0 msgr2=0x7fe05c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe05a000640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe05c1554f0 0x7fe05c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe05a000640 1 -- 172.21.15.102:0/221444001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c1f1d70 con 0x7fe05c0c5e10 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe05b400640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe05c1554f0 0x7fe05c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe05a000640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c0c5e10 0x7fe05c145f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe04800e8c0 tx=0x7fe04800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe052a00640 1 -- 172.21.15.102:0/221444001 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048017070 con 0x7fe05c0c5e10 2024-04-20T21:17:28.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe052a00640 1 -- 172.21.15.102:0/221444001 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04800c940 con 0x7fe05c0c5e10 2024-04-20T21:17:28.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe052a00640 1 -- 172.21.15.102:0/221444001 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04800cd10 con 0x7fe05c0c5e10 2024-04-20T21:17:28.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05c143750 con 0x7fe05c0c5e10 2024-04-20T21:17:28.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.235+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05c143d10 con 0x7fe05c0c5e10 2024-04-20T21:17:28.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.237+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05c04fcf0 con 0x7fe05c0c5e10 2024-04-20T21:17:28.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.237+0000 7fe052a00640 1 -- 172.21.15.102:0/221444001 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe04801a070 con 0x7fe05c0c5e10 2024-04-20T21:17:28.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.238+0000 7fe052a00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe03c06c920 0x7fe03c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.238+0000 7fe052a00640 1 -- 172.21.15.102:0/221444001 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe04808c200 con 0x7fe05c0c5e10 2024-04-20T21:17:28.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.239+0000 7fe05aa00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe03c06c920 0x7fe03c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.239+0000 7fe05aa00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe03c06c920 0x7fe03c06ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe04c036000 tx=0x7fe04c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.243+0000 7fe052a00640 1 -- 172.21.15.102:0/221444001 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe04805a260 con 0x7fe05c0c5e10 2024-04-20T21:17:28.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.439+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe05c0b3570 con 0x7fe05c0c5e10 2024-04-20T21:17:28.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.440+0000 7fe052a00640 1 -- 172.21.15.102:0/221444001 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fe048059b60 con 0x7fe05c0c5e10 2024-04-20T21:17:28.440 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.442+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe03c06c920 msgr2=0x7fe03c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.442+0000 7fe05be00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe03c06c920 0x7fe03c06ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe04c036000 tx=0x7fe04c03b040 comp rx=0 tx=0).stop 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.442+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c0c5e10 msgr2=0x7fe05c145f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c0c5e10 0x7fe05c145f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe04800e8c0 tx=0x7fe04800ed80 comp rx=0 tx=0).stop 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 shutdown_connections 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe03c06c920 0x7fe03c06ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe05c1554f0 0x7fe05c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe05c0c5e10 0x7fe05c145f50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 --2- 172.21.15.102:0/221444001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe05c0c59c0 0x7fe05c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 >> 172.21.15.102:0/221444001 conn(0x7fe05c06d150 msgr2=0x7fe05c14b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 shutdown_connections 2024-04-20T21:17:28.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.443+0000 7fe05be00640 1 -- 172.21.15.102:0/221444001 wait complete. 2024-04-20T21:17:28.459 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T21:17:28.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.642+0000 7fd2dfe00640 1 Processor -- start 2024-04-20T21:17:28.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.642+0000 7fd2dfe00640 1 -- start start 2024-04-20T21:17:28.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.644+0000 7fd2dfe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd2e01555f0 0x7fd2e00c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.644+0000 7fd2dfe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd2e015f070 0x7fd2e00c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.644+0000 7fd2dfe00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2e006c420 con 0x7fd2e00ccd90 2024-04-20T21:17:28.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.644+0000 7fd2dfe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fd2e006c700 con 0x7fd2e015f070 2024-04-20T21:17:28.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.644+0000 7fd2dfe00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fd2e006c9e0 con 0x7fd2e01555f0 2024-04-20T21:17:28.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.644+0000 7fd2d7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd2e015f070 0x7fd2e00c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.644+0000 7fd2df400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd2e00ccd90 0x7fd2e015b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34612/0 (socket says 172.21.15.102:34612) 2024-04-20T21:17:28.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.644+0000 7fd2d7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd2e015f070 0x7fd2e00c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34974/0 (socket says 172.21.15.102:34974) 2024-04-20T21:17:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.645+0000 7fd2dea00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd2e01555f0 0x7fd2e00c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.645+0000 7fd2df400640 1 -- 172.21.15.102:0/3054697199 learned_addr learned my addr 172.21.15.102:0/3054697199 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.645+0000 7fd2dea00640 1 -- 172.21.15.102:0/3054697199 >> v2:172.21.15.102:3300/0 conn(0x7fd2e015f070 msgr2=0x7fd2e00c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.645+0000 7fd2dea00640 1 --2- 172.21.15.102:0/3054697199 >> v2:172.21.15.102:3300/0 conn(0x7fd2e015f070 0x7fd2e00c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.645+0000 7fd2dea00640 1 -- 172.21.15.102:0/3054697199 >> v1:172.21.15.102:6789/0 conn(0x7fd2e00ccd90 legacy=0x7fd2e015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.645+0000 7fd2dea00640 1 -- 172.21.15.102:0/3054697199 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e006ccc0 con 0x7fd2e01555f0 2024-04-20T21:17:28.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.646+0000 7fd2dea00640 1 --2- 172.21.15.102:0/3054697199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 0x7fd2e00c5800 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0050d80 tx=0x7fd2d002b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6422e20b4bc2a723 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.646+0000 7fd2de000640 1 -- 172.21.15.102:0/3054697199 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0038070 con 0x7fd2e01555f0 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.646+0000 7fd2de000640 1 -- 172.21.15.102:0/3054697199 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d0031710 con 0x7fd2e01555f0 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.646+0000 7fd2de000640 1 -- 172.21.15.102:0/3054697199 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0031ae0 con 0x7fd2e01555f0 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.647+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3054697199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 msgr2=0x7fd2e00c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.647+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3054697199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 0x7fd2e00c5800 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0050d80 tx=0x7fd2d002b7b0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.647+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3054697199 shutdown_connections 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.647+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3054697199 >> v2:172.21.15.102:3300/0 conn(0x7fd2e015f070 0x7fd2e00c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.647+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3054697199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 0x7fd2e00c5800 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.647+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3054697199 >> 172.21.15.102:0/3054697199 conn(0x7fd2e00d0900 msgr2=0x7fd2e00d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.647+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3054697199 shutdown_connections 2024-04-20T21:17:28.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.647+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3054697199 wait complete. 2024-04-20T21:17:28.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.649+0000 7fd2dfe00640 1 Processor -- start 2024-04-20T21:17:28.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.649+0000 7fd2dfe00640 1 -- start start 2024-04-20T21:17:28.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.649+0000 7fd2dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e00ccd90 0x7fd2e00c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.649+0000 7fd2dfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 0x7fd2e014e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.649+0000 7fd2dfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2e015f070 0x7fd2e014d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.649+0000 7fd2dfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e015b830 con 0x7fd2e00ccd90 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.649+0000 7fd2dfe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e015f570 con 0x7fd2e01555f0 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.649+0000 7fd2dfe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e015b9b0 con 0x7fd2e015f070 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2dea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e00ccd90 0x7fd2e00c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2dea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e00ccd90 0x7fd2e00c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34986/0 (socket says 172.21.15.102:34986) 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2dea00640 1 -- 172.21.15.102:0/3996316694 learned_addr learned my addr 172.21.15.102:0/3996316694 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2d7e00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 0x7fd2e014e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2df400640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2e015f070 0x7fd2e014d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2dea00640 1 -- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2e015f070 msgr2=0x7fd2e014d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2dea00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2e015f070 0x7fd2e014d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2dea00640 1 -- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 msgr2=0x7fd2e014e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2dea00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 0x7fd2e014e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.650+0000 7fd2dea00640 1 -- 172.21.15.102:0/3996316694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e014c310 con 0x7fd2e00ccd90 2024-04-20T21:17:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.651+0000 7fd2dea00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e00ccd90 0x7fd2e00c6b60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2d002bdc0 tx=0x7fd2d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.651+0000 7fd2d6a00640 1 -- 172.21.15.102:0/3996316694 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0042030 con 0x7fd2e00ccd90 2024-04-20T21:17:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.651+0000 7fd2d6a00640 1 -- 172.21.15.102:0/3996316694 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d003d670 con 0x7fd2e00ccd90 2024-04-20T21:17:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.651+0000 7fd2d6a00640 1 -- 172.21.15.102:0/3996316694 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0038040 con 0x7fd2e00ccd90 2024-04-20T21:17:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.651+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e014c5d0 con 0x7fd2e00ccd90 2024-04-20T21:17:28.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.651+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e00ca110 con 0x7fd2e00ccd90 2024-04-20T21:17:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.652+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e004fcf0 con 0x7fd2e00ccd90 2024-04-20T21:17:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.653+0000 7fd2d6a00640 1 -- 172.21.15.102:0/3996316694 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2d0045070 con 0x7fd2e00ccd90 2024-04-20T21:17:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.654+0000 7fd2d6a00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd2bc06c8f0 0x7fd2bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.654+0000 7fd2d6a00640 1 -- 172.21.15.102:0/3996316694 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd2d00ad7f0 con 0x7fd2e00ccd90 2024-04-20T21:17:28.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.654+0000 7fd2d7e00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd2bc06c8f0 0x7fd2bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.655+0000 7fd2d7e00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd2bc06c8f0 0x7fd2bc06edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2e01f55c0 tx=0x7fd2c8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.659+0000 7fd2d6a00640 1 -- 172.21.15.102:0/3996316694 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd2d007b200 con 0x7fd2e00ccd90 2024-04-20T21:17:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.857+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd2e00cc960 con 0x7fd2e00ccd90 2024-04-20T21:17:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.858+0000 7fd2d6a00640 1 -- 172.21.15.102:0/3996316694 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fd2d007f500 con 0x7fd2e00ccd90 2024-04-20T21:17:28.858 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.860+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd2bc06c8f0 msgr2=0x7fd2bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.860+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd2bc06c8f0 0x7fd2bc06edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2e01f55c0 tx=0x7fd2c8009190 comp rx=0 tx=0).stop 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.860+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e00ccd90 msgr2=0x7fd2e00c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.860+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e00ccd90 0x7fd2e00c6b60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2d002bdc0 tx=0x7fd2d00046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.861+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 shutdown_connections 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.861+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2e015f070 0x7fd2e014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.861+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd2bc06c8f0 0x7fd2bc06edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.861+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2e01555f0 0x7fd2e014e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.861+0000 7fd2dfe00640 1 --2- 172.21.15.102:0/3996316694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2e00ccd90 0x7fd2e00c6b60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.861+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 >> 172.21.15.102:0/3996316694 conn(0x7fd2e00d0900 msgr2=0x7fd2e0153c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.861+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 shutdown_connections 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:28.861+0000 7fd2dfe00640 1 -- 172.21.15.102:0/3996316694 wait complete. 2024-04-20T21:17:28.877 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:29.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.073+0000 7f1c93e00640 1 Processor -- start 2024-04-20T21:17:29.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.074+0000 7f1c93e00640 1 -- start start 2024-04-20T21:17:29.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.075+0000 7f1c93e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c94155110 0x7f1c9406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.076+0000 7f1c93e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9415f3c0 con 0x7f1c940c4040 2024-04-20T21:17:29.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.076+0000 7f1c93e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9406e6d0 con 0x7f1c9406f3d0 2024-04-20T21:17:29.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.076+0000 7f1c93e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1c940c7c30 con 0x7f1c94155110 2024-04-20T21:17:29.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.076+0000 7f1c8a000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f1c9406f3d0 0x7f1c9415e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51340/0 (socket says 172.21.15.102:51340) 2024-04-20T21:17:29.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.076+0000 7f1c92a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c94155110 0x7f1c9406ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.076+0000 7f1c8a000640 1 -- 172.21.15.102:0/1729250451 learned_addr learned my addr 172.21.15.102:0/1729250451 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:29.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.076+0000 7f1c92000640 1 -- 172.21.15.102:0/1729250451 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993804457 0 0) 0x7f1c9406e6d0 con 0x7f1c9406f3d0 2024-04-20T21:17:29.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.077+0000 7f1c92000640 1 -- 172.21.15.102:0/1729250451 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c78003520 con 0x7f1c9406f3d0 2024-04-20T21:17:29.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.077+0000 7f1c92a00640 1 -- 172.21.15.102:0/1729250451 >> v1:172.21.15.193:6789/0 conn(0x7f1c9406f3d0 legacy=0x7f1c9415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.077+0000 7f1c92000640 1 -- 172.21.15.102:0/1729250451 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508929830 0 0) 0x7f1c9415f3c0 con 0x7f1c940c4040 2024-04-20T21:17:29.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.077+0000 7f1c92a00640 1 -- 172.21.15.102:0/1729250451 >> v1:172.21.15.102:6789/0 conn(0x7f1c940c4040 legacy=0x7f1c94165910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.077+0000 7f1c92a00640 1 -- 172.21.15.102:0/1729250451 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c940c7e80 con 0x7f1c94155110 2024-04-20T21:17:29.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.078+0000 7f1c92a00640 1 --2- 172.21.15.102:0/1729250451 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c94155110 0x7f1c9406ee90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c840097d0 tx=0x7f1c8402b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172844f24cec5f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.078+0000 7f1c92000640 1 -- 172.21.15.102:0/1729250451 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84034070 con 0x7f1c94155110 2024-04-20T21:17:29.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.078+0000 7f1c92000640 1 -- 172.21.15.102:0/1729250451 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c84030890 con 0x7f1c94155110 2024-04-20T21:17:29.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.078+0000 7f1c92000640 1 -- 172.21.15.102:0/1729250451 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84030c60 con 0x7f1c94155110 2024-04-20T21:17:29.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.079+0000 7f1c93e00640 1 -- 172.21.15.102:0/1729250451 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c94155110 msgr2=0x7f1c9406ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.079+0000 7f1c93e00640 1 --2- 172.21.15.102:0/1729250451 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c94155110 0x7f1c9406ee90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c840097d0 tx=0x7f1c8402b9d0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.079+0000 7f1c93e00640 1 -- 172.21.15.102:0/1729250451 shutdown_connections 2024-04-20T21:17:29.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.079+0000 7f1c93e00640 1 --2- 172.21.15.102:0/1729250451 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c94155110 0x7f1c9406ee90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.079+0000 7f1c93e00640 1 -- 172.21.15.102:0/1729250451 >> 172.21.15.102:0/1729250451 conn(0x7f1c940cc820 msgr2=0x7f1c940ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.079+0000 7f1c93e00640 1 -- 172.21.15.102:0/1729250451 shutdown_connections 2024-04-20T21:17:29.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.079+0000 7f1c93e00640 1 -- 172.21.15.102:0/1729250451 wait complete. 2024-04-20T21:17:29.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.080+0000 7f1c93e00640 1 Processor -- start 2024-04-20T21:17:29.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.081+0000 7f1c93e00640 1 -- start start 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.081+0000 7f1c93e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c9406f3d0 0x7f1c941f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c93e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c940c4040 0x7f1c941f0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c93e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c94155110 0x7f1c9415e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c93e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941554f0 con 0x7f1c94155110 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c93e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9406ea00 con 0x7f1c940c4040 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c93e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1c94155670 con 0x7f1c9406f3d0 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c92a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c9406f3d0 0x7f1c941f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c93400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c94155110 0x7f1c9415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c92a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c9406f3d0 0x7f1c941f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35850/0 (socket says 172.21.15.102:35850) 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c92a00640 1 -- 172.21.15.102:0/4020310534 learned_addr learned my addr 172.21.15.102:0/4020310534 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c8a000640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c940c4040 0x7f1c941f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c92a00640 1 -- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c940c4040 msgr2=0x7f1c941f0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c92a00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c940c4040 0x7f1c941f0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c92a00640 1 -- 172.21.15.102:0/4020310534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c94155110 msgr2=0x7f1c9415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c92a00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c94155110 0x7f1c9415e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.082+0000 7f1c92a00640 1 -- 172.21.15.102:0/4020310534 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9415e780 con 0x7f1c9406f3d0 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.083+0000 7f1c92a00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c9406f3d0 0x7f1c941f49c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c840098f0 tx=0x7f1c840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.083+0000 7f1c8b400640 1 -- 172.21.15.102:0/4020310534 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84034040 con 0x7f1c9406f3d0 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.083+0000 7f1c8b400640 1 -- 172.21.15.102:0/4020310534 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c8403ab10 con 0x7f1c9406f3d0 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.083+0000 7f1c8b400640 1 -- 172.21.15.102:0/4020310534 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84004030 con 0x7f1c9406f3d0 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.083+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c940cc0f0 con 0x7f1c9406f3d0 2024-04-20T21:17:29.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.083+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c941f41c0 con 0x7f1c9406f3d0 2024-04-20T21:17:29.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.084+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9404fd60 con 0x7f1c9406f3d0 2024-04-20T21:17:29.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.085+0000 7f1c8b400640 1 -- 172.21.15.102:0/4020310534 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c8404b060 con 0x7f1c9406f3d0 2024-04-20T21:17:29.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.086+0000 7f1c8b400640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c7006c8f0 0x7f1c7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.086+0000 7f1c8b400640 1 -- 172.21.15.102:0/4020310534 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1c840add20 con 0x7f1c9406f3d0 2024-04-20T21:17:29.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.086+0000 7f1c8a000640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c7006c8f0 0x7f1c7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.087+0000 7f1c8a000640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c7006c8f0 0x7f1c7006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c940cb950 tx=0x7f1c7c000bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.091+0000 7f1c8b400640 1 -- 172.21.15.102:0/4020310534 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1c8403f070 con 0x7f1c9406f3d0 2024-04-20T21:17:29.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.310+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c941589b0 con 0x7f1c9406f3d0 2024-04-20T21:17:29.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.311+0000 7f1c8b400640 1 -- 172.21.15.102:0/4020310534 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f1c8407b7b0 con 0x7f1c9406f3d0 2024-04-20T21:17:29.311 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:29.311 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:17:29.311 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:17:29.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c7006c8f0 msgr2=0x7f1c7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c7006c8f0 0x7f1c7006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c940cb950 tx=0x7f1c7c000bc0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c9406f3d0 msgr2=0x7f1c941f49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c9406f3d0 0x7f1c941f49c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c840098f0 tx=0x7f1c840046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 shutdown_connections 2024-04-20T21:17:29.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c94155110 0x7f1c9415e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c7006c8f0 0x7f1c7006edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c940c4040 0x7f1c941f0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 --2- 172.21.15.102:0/4020310534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c9406f3d0 0x7f1c941f49c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 >> 172.21.15.102:0/4020310534 conn(0x7f1c940cc820 msgr2=0x7f1c94152930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 shutdown_connections 2024-04-20T21:17:29.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.314+0000 7f1c93e00640 1 -- 172.21.15.102:0/4020310534 wait complete. 2024-04-20T21:17:29.331 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:29.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.532+0000 7f9465c00640 1 Processor -- start 2024-04-20T21:17:29.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.532+0000 7f9465c00640 1 -- start start 2024-04-20T21:17:29.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.533+0000 7f9465c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f94600c9eb0 0x7f94600c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.533+0000 7f9465c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94601536d0 con 0x7f9460073a10 2024-04-20T21:17:29.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.533+0000 7f9465c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94600c76c0 con 0x7f946014fa50 2024-04-20T21:17:29.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.533+0000 7f9465c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f94600c7960 con 0x7f94600c9eb0 2024-04-20T21:17:29.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.534+0000 7f945f400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9460073a10 0x7f9460163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34632/0 (socket says 172.21.15.102:34632) 2024-04-20T21:17:29.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.534+0000 7f945f400640 1 -- 172.21.15.102:0/1125405482 learned_addr learned my addr 172.21.15.102:0/1125405482 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:29.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.534+0000 7f9456000640 1 --2- 172.21.15.102:0/1125405482 >> v2:172.21.15.193:3300/0 conn(0x7f94600c9eb0 0x7f94600c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.534+0000 7f945ea00640 1 --1- 172.21.15.102:0/1125405482 >> v1:172.21.15.193:6789/0 conn(0x7f946014fa50 0x7f94600c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51348/0 (socket says 172.21.15.102:51348) 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.534+0000 7f9456000640 1 -- 172.21.15.102:0/1125405482 >> v1:172.21.15.193:6789/0 conn(0x7f946014fa50 legacy=0x7f94600c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.534+0000 7f945e000640 1 -- 172.21.15.102:0/1125405482 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284166553 0 0) 0x7f9458001490 con 0x7f9460073a10 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.534+0000 7f9456000640 1 -- 172.21.15.102:0/1125405482 >> v1:172.21.15.102:6789/0 conn(0x7f9460073a10 legacy=0x7f9460163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.534+0000 7f9456000640 1 -- 172.21.15.102:0/1125405482 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94600c3d90 con 0x7f94600c9eb0 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.535+0000 7f9456000640 1 --2- 172.21.15.102:0/1125405482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94600c4830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f94600c7890 tx=0x7f944802b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73219aa5f9db6453 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.535+0000 7f945e000640 1 -- 172.21.15.102:0/1125405482 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448034070 con 0x7f94600c9eb0 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.535+0000 7f945e000640 1 -- 172.21.15.102:0/1125405482 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9448030890 con 0x7f94600c9eb0 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.536+0000 7f945e000640 1 -- 172.21.15.102:0/1125405482 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448030c60 con 0x7f94600c9eb0 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.536+0000 7f9465c00640 1 -- 172.21.15.102:0/1125405482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 msgr2=0x7f94600c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.536+0000 7f9465c00640 1 --2- 172.21.15.102:0/1125405482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94600c4830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f94600c7890 tx=0x7f944802b9d0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.536+0000 7f9465c00640 1 -- 172.21.15.102:0/1125405482 shutdown_connections 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.536+0000 7f9465c00640 1 --2- 172.21.15.102:0/1125405482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94600c4830 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.536+0000 7f9465c00640 1 -- 172.21.15.102:0/1125405482 >> 172.21.15.102:0/1125405482 conn(0x7f94600c9390 msgr2=0x7f94600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.536+0000 7f9465c00640 1 -- 172.21.15.102:0/1125405482 shutdown_connections 2024-04-20T21:17:29.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.536+0000 7f9465c00640 1 -- 172.21.15.102:0/1125405482 wait complete. 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.538+0000 7f9465c00640 1 Processor -- start 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.538+0000 7f9465c00640 1 -- start start 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f9465c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 0x7f94601f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f9465c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94601f0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f9465c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f94601efd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f9465c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94600c4030 con 0x7f946014fa50 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f9465c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94600c7c00 con 0x7f94600c9eb0 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f9465c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f94600c4310 con 0x7f9460073a10 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f94601efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f94601efd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35014/0 (socket says 172.21.15.102:35014) 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945f400640 1 -- 172.21.15.102:0/1202270226 learned_addr learned my addr 172.21.15.102:0/1202270226 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945ea00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 0x7f94601f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f9456000640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94601f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945f400640 1 -- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 msgr2=0x7f94601f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945f400640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 0x7f94601f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945f400640 1 -- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 msgr2=0x7f94601f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945f400640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94601f0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.539+0000 7f945f400640 1 -- 172.21.15.102:0/1202270226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601f02d0 con 0x7f946014fa50 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.540+0000 7f945f400640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f94601efd90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f945800c7f0 tx=0x7f945800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.540+0000 7f9457400640 1 -- 172.21.15.102:0/1202270226 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458018070 con 0x7f946014fa50 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.540+0000 7f9457400640 1 -- 172.21.15.102:0/1202270226 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945800a5e0 con 0x7f946014fa50 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.540+0000 7f9457400640 1 -- 172.21.15.102:0/1202270226 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945800aa10 con 0x7f946014fa50 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.540+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9460157f00 con 0x7f946014fa50 2024-04-20T21:17:29.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.540+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601f2a30 con 0x7f946014fa50 2024-04-20T21:17:29.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.541+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9460074340 con 0x7f946014fa50 2024-04-20T21:17:29.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.542+0000 7f9457400640 1 -- 172.21.15.102:0/1202270226 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f945800abb0 con 0x7f946014fa50 2024-04-20T21:17:29.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.543+0000 7f9457400640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f943c06c8f0 0x7f943c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.543+0000 7f9457400640 1 -- 172.21.15.102:0/1202270226 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f945808b4c0 con 0x7f946014fa50 2024-04-20T21:17:29.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.543+0000 7f945ea00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f943c06c8f0 0x7f943c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.544+0000 7f945ea00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f943c06c8f0 0x7f943c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9450000f60 tx=0x7f9450000ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.547+0000 7f9457400640 1 -- 172.21.15.102:0/1202270226 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f94601f02d0 con 0x7f946014fa50 2024-04-20T21:17:29.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.748+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f946004fd90 con 0x7f946014fa50 2024-04-20T21:17:29.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.750+0000 7f9457400640 1 -- 172.21.15.102:0/1202270226 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f94601536d0 con 0x7f946014fa50 2024-04-20T21:17:29.750 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:29.751 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f943c06c8f0 msgr2=0x7f943c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f943c06c8f0 0x7f943c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9450000f60 tx=0x7f9450000ed0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 msgr2=0x7f94601efd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f94601efd90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f945800c7f0 tx=0x7f945800ccb0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 shutdown_connections 2024-04-20T21:17:29.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f94601efd90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f943c06c8f0 0x7f943c06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94601f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 --2- 172.21.15.102:0/1202270226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 0x7f94601f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.753+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 >> 172.21.15.102:0/1202270226 conn(0x7f94600c9390 msgr2=0x7f946006f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.754+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 shutdown_connections 2024-04-20T21:17:29.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.754+0000 7f9465c00640 1 -- 172.21.15.102:0/1202270226 wait complete. 2024-04-20T21:17:29.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:16:51.795761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:16:52.983434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T21:16:56.582297+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T21:17:29.772 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:29.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.967+0000 7fa429200640 1 Processor -- start 2024-04-20T21:17:29.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.967+0000 7fa429200640 1 -- start start 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa429200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa424152d80 0x7fa424073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa429200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa4240cd060 0x7fa4240c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa429200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa424158f80 con 0x7fa4240c5c50 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa429200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa424159de0 con 0x7fa4240cd060 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa429200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa424159260 con 0x7fa424152d80 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa423400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa424152d80 0x7fa424073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa41be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa4240cd060 0x7fa4240c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa423400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa424152d80 0x7fa424073ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35874/0 (socket says 172.21.15.102:35874) 2024-04-20T21:17:29.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.969+0000 7fa423400640 1 -- 172.21.15.102:0/3951948054 learned_addr learned my addr 172.21.15.102:0/3951948054 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:29.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.970+0000 7fa423400640 1 -- 172.21.15.102:0/3951948054 >> v2:172.21.15.193:3300/0 conn(0x7fa4240cd060 msgr2=0x7fa4240c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:29.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.970+0000 7fa423400640 1 --2- 172.21.15.102:0/3951948054 >> v2:172.21.15.193:3300/0 conn(0x7fa4240cd060 0x7fa4240c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.970+0000 7fa423400640 1 -- 172.21.15.102:0/3951948054 >> v1:172.21.15.102:6789/0 conn(0x7fa4240c5c50 legacy=0x7fa42415c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.970+0000 7fa423400640 1 -- 172.21.15.102:0/3951948054 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa424159540 con 0x7fa424152d80 2024-04-20T21:17:29.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.970+0000 7fa41be00640 1 --2- 172.21.15.102:0/3951948054 >> v2:172.21.15.193:3300/0 conn(0x7fa4240cd060 0x7fa4240c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:29.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.971+0000 7fa423400640 1 --2- 172.21.15.102:0/3951948054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa424152d80 0x7fa424073ba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa414009870 tx=0x7fa41402ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dddc3edfd5fc9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.971+0000 7fa422a00640 1 -- 172.21.15.102:0/3951948054 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41402f830 con 0x7fa424152d80 2024-04-20T21:17:29.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.971+0000 7fa422a00640 1 -- 172.21.15.102:0/3951948054 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41402f9d0 con 0x7fa424152d80 2024-04-20T21:17:29.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.971+0000 7fa422a00640 1 -- 172.21.15.102:0/3951948054 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41402fda0 con 0x7fa424152d80 2024-04-20T21:17:29.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.971+0000 7fa429200640 1 -- 172.21.15.102:0/3951948054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa424152d80 msgr2=0x7fa424073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.972+0000 7fa429200640 1 --2- 172.21.15.102:0/3951948054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa424152d80 0x7fa424073ba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa414009870 tx=0x7fa41402ba10 comp rx=0 tx=0).stop 2024-04-20T21:17:29.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.972+0000 7fa429200640 1 -- 172.21.15.102:0/3951948054 shutdown_connections 2024-04-20T21:17:29.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.972+0000 7fa429200640 1 --2- 172.21.15.102:0/3951948054 >> v2:172.21.15.193:3300/0 conn(0x7fa4240cd060 0x7fa4240c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.972+0000 7fa429200640 1 --2- 172.21.15.102:0/3951948054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa424152d80 0x7fa424073ba0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.972+0000 7fa429200640 1 -- 172.21.15.102:0/3951948054 >> 172.21.15.102:0/3951948054 conn(0x7fa4240cc540 msgr2=0x7fa4240cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.972+0000 7fa429200640 1 -- 172.21.15.102:0/3951948054 shutdown_connections 2024-04-20T21:17:29.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.972+0000 7fa429200640 1 -- 172.21.15.102:0/3951948054 wait complete. 2024-04-20T21:17:29.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.973+0000 7fa429200640 1 Processor -- start 2024-04-20T21:17:29.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.974+0000 7fa429200640 1 -- start start 2024-04-20T21:17:29.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.974+0000 7fa429200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4240c5c50 0x7fa4240cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.974+0000 7fa429200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4240cd060 0x7fa4241539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.974+0000 7fa429200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa424152d80 0x7fa4241471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.974+0000 7fa429200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa424153160 con 0x7fa4240cd060 2024-04-20T21:17:29.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.974+0000 7fa429200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa424159b00 con 0x7fa424152d80 2024-04-20T21:17:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.974+0000 7fa429200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa4241532e0 con 0x7fa4240c5c50 2024-04-20T21:17:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4240cd060 0x7fa4241539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4240cd060 0x7fa4241539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35026/0 (socket says 172.21.15.102:35026) 2024-04-20T21:17:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 -- 172.21.15.102:0/4002566457 learned_addr learned my addr 172.21.15.102:0/4002566457 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa423400640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4240c5c50 0x7fa4240cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa423e00640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa424152d80 0x7fa4241471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 -- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4240c5c50 msgr2=0x7fa4240cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4240c5c50 0x7fa4240cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 -- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa424152d80 msgr2=0x7fa4241471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa424152d80 0x7fa4241471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 -- 172.21.15.102:0/4002566457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa424151c80 con 0x7fa4240cd060 2024-04-20T21:17:29.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.975+0000 7fa41be00640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4240cd060 0x7fa4241539b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00ec10 tx=0x7fa40c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.976+0000 7fa41b400640 1 -- 172.21.15.102:0/4002566457 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c017070 con 0x7fa4240cd060 2024-04-20T21:17:29.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.976+0000 7fa41b400640 1 -- 172.21.15.102:0/4002566457 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c00cbd0 con 0x7fa4240cd060 2024-04-20T21:17:29.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.976+0000 7fa41b400640 1 -- 172.21.15.102:0/4002566457 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c013470 con 0x7fa4240cd060 2024-04-20T21:17:29.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.976+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42415bfc0 con 0x7fa4240cd060 2024-04-20T21:17:29.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.976+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa424155ad0 con 0x7fa4240cd060 2024-04-20T21:17:29.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.977+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa424069340 con 0x7fa4240cd060 2024-04-20T21:17:29.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.978+0000 7fa41b400640 1 -- 172.21.15.102:0/4002566457 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa40c01a070 con 0x7fa4240cd060 2024-04-20T21:17:29.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.979+0000 7fa41b400640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa40006c920 0x7fa40006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.979+0000 7fa41b400640 1 -- 172.21.15.102:0/4002566457 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa40c08bfd0 con 0x7fa4240cd060 2024-04-20T21:17:29.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.979+0000 7fa423400640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa40006c920 0x7fa40006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.980+0000 7fa423400640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa40006c920 0x7fa40006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa414036000 tx=0x7fa414008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:29.983+0000 7fa41b400640 1 -- 172.21.15.102:0/4002566457 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa424069340 con 0x7fa4240cd060 2024-04-20T21:17:30.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.179+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa42404fcf0 con 0x7fa4240cd060 2024-04-20T21:17:30.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.180+0000 7fa41b400640 1 -- 172.21.15.102:0/4002566457 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fa40c059a60 con 0x7fa4240cd060 2024-04-20T21:17:30.180 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:17:30.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.182+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa40006c920 msgr2=0x7fa40006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.182+0000 7fa429200640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa40006c920 0x7fa40006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa414036000 tx=0x7fa414008030 comp rx=0 tx=0).stop 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.182+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4240cd060 msgr2=0x7fa4241539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.182+0000 7fa429200640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4240cd060 0x7fa4241539b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00ec10 tx=0x7fa40c00c520 comp rx=0 tx=0).stop 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.182+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 shutdown_connections 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.182+0000 7fa429200640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa40006c920 0x7fa40006ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.182+0000 7fa429200640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa424152d80 0x7fa4241471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.182+0000 7fa429200640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4240cd060 0x7fa4241539b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.183+0000 7fa429200640 1 --2- 172.21.15.102:0/4002566457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4240c5c50 0x7fa4240cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.183+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 >> 172.21.15.102:0/4002566457 conn(0x7fa4240cc540 msgr2=0x7fa4240cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.183+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 shutdown_connections 2024-04-20T21:17:30.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.183+0000 7fa429200640 1 -- 172.21.15.102:0/4002566457 wait complete. 2024-04-20T21:17:30.201 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:17:30.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.407+0000 7f95a2400640 1 Processor -- start 2024-04-20T21:17:30.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.407+0000 7f95a2400640 1 -- start start 2024-04-20T21:17:30.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.408+0000 7f95a2400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f959c153110 0x7f959c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.408+0000 7f95a2400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c1534f0 0x7f959c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f95a2400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959c0c89f0 con 0x7f959c158f70 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f95a2400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f959c0c3920 con 0x7f959c153110 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f95a2400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f959c0c91b0 con 0x7f959c1534f0 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f959b400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f959c153110 0x7f959c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f959b400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f959c153110 0x7f959c0c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:46282/0 (socket says 172.21.15.102:46282) 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f959b400640 1 -- 172.21.15.102:0/445102763 learned_addr learned my addr 172.21.15.102:0/445102763 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f959aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c1534f0 0x7f959c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f959b400640 1 -- 172.21.15.102:0/445102763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c1534f0 msgr2=0x7f959c06f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:30.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f959b400640 1 --2- 172.21.15.102:0/445102763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c1534f0 0x7f959c06f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f959b400640 1 -- 172.21.15.102:0/445102763 >> v1:172.21.15.102:6789/0 conn(0x7f959c158f70 legacy=0x7f959c0c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.409+0000 7f959b400640 1 -- 172.21.15.102:0/445102763 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c0c8cd0 con 0x7f959c153110 2024-04-20T21:17:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.411+0000 7f959b400640 1 --2- 172.21.15.102:0/445102763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c153110 0x7f959c0c83d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f958c009e50 tx=0x7f958c02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9d5313c6ffabf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.411+0000 7f959a000640 1 -- 172.21.15.102:0/445102763 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c02ca80 con 0x7f959c153110 2024-04-20T21:17:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.411+0000 7f959a000640 1 -- 172.21.15.102:0/445102763 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c02cc20 con 0x7f959c153110 2024-04-20T21:17:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.411+0000 7f959a000640 1 -- 172.21.15.102:0/445102763 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c0343f0 con 0x7f959c153110 2024-04-20T21:17:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.411+0000 7f95a2400640 1 -- 172.21.15.102:0/445102763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c153110 msgr2=0x7f959c0c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.411+0000 7f95a2400640 1 --2- 172.21.15.102:0/445102763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c153110 0x7f959c0c83d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f958c009e50 tx=0x7f958c02bee0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.412+0000 7f95a2400640 1 -- 172.21.15.102:0/445102763 shutdown_connections 2024-04-20T21:17:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.412+0000 7f95a2400640 1 --2- 172.21.15.102:0/445102763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c1534f0 0x7f959c06f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.412+0000 7f95a2400640 1 --2- 172.21.15.102:0/445102763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c153110 0x7f959c0c83d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.412+0000 7f95a2400640 1 -- 172.21.15.102:0/445102763 >> 172.21.15.102:0/445102763 conn(0x7f959c074ea0 msgr2=0x7f959c075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.412+0000 7f95a2400640 1 -- 172.21.15.102:0/445102763 shutdown_connections 2024-04-20T21:17:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.412+0000 7f95a2400640 1 -- 172.21.15.102:0/445102763 wait complete. 2024-04-20T21:17:30.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.413+0000 7f95a2400640 1 Processor -- start 2024-04-20T21:17:30.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.414+0000 7f95a2400640 1 -- start start 2024-04-20T21:17:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.414+0000 7f95a2400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c153110 0x7f959c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f95a2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c158f70 0x7f959c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f95a2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c14e770 0x7f959c14ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f95a2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f959c0c99c0 con 0x7f959c14e770 2024-04-20T21:17:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f95a2400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f959c0c96e0 con 0x7f959c158f70 2024-04-20T21:17:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f95a2400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f959c06f5f0 con 0x7f959c153110 2024-04-20T21:17:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c14e770 0x7f959c14ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c14e770 0x7f959c14ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35030/0 (socket says 172.21.15.102:35030) 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959be00640 1 -- 172.21.15.102:0/242413511 learned_addr learned my addr 172.21.15.102:0/242413511 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959b400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c153110 0x7f959c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959aa00640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c158f70 0x7f959c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959be00640 1 -- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c153110 msgr2=0x7f959c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959be00640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c153110 0x7f959c0caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959be00640 1 -- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c158f70 msgr2=0x7f959c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959be00640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c158f70 0x7f959c1581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.415+0000 7f959be00640 1 -- 172.21.15.102:0/242413511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c1f0f60 con 0x7f959c14e770 2024-04-20T21:17:30.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.416+0000 7f959be00640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c14e770 0x7f959c14ec20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f959400c3c0 tx=0x7f959400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.416+0000 7f9593400640 1 -- 172.21.15.102:0/242413511 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594017070 con 0x7f959c14e770 2024-04-20T21:17:30.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.416+0000 7f9593400640 1 -- 172.21.15.102:0/242413511 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95940052d0 con 0x7f959c14e770 2024-04-20T21:17:30.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.416+0000 7f9593400640 1 -- 172.21.15.102:0/242413511 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594003810 con 0x7f959c14e770 2024-04-20T21:17:30.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.416+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c1f1200 con 0x7f959c14e770 2024-04-20T21:17:30.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.416+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c1f17c0 con 0x7f959c14e770 2024-04-20T21:17:30.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.417+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959c04fdd0 con 0x7f959c14e770 2024-04-20T21:17:30.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.418+0000 7f9593400640 1 -- 172.21.15.102:0/242413511 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f959401a070 con 0x7f959c14e770 2024-04-20T21:17:30.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.419+0000 7f9593400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957806c920 0x7f957806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.419+0000 7f9593400640 1 -- 172.21.15.102:0/242413511 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f959408b1e0 con 0x7f959c14e770 2024-04-20T21:17:30.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.419+0000 7f959b400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957806c920 0x7f957806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.420+0000 7f959b400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957806c920 0x7f957806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f958c035000 tx=0x7f958c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.423+0000 7f9593400640 1 -- 172.21.15.102:0/242413511 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9594058b50 con 0x7f959c14e770 2024-04-20T21:17:30.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.625+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f959c148a20 con 0x7f959c14e770 2024-04-20T21:17:30.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.626+0000 7f9593400640 1 -- 172.21.15.102:0/242413511 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f959405ce50 con 0x7f959c14e770 2024-04-20T21:17:30.625 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:30.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.628+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957806c920 msgr2=0x7f957806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.628+0000 7f95a2400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957806c920 0x7f957806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f958c035000 tx=0x7f958c030000 comp rx=0 tx=0).stop 2024-04-20T21:17:30.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.628+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c14e770 msgr2=0x7f959c14ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.628+0000 7f95a2400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c14e770 0x7f959c14ec20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f959400c3c0 tx=0x7f959400c880 comp rx=0 tx=0).stop 2024-04-20T21:17:30.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.628+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 shutdown_connections 2024-04-20T21:17:30.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.629+0000 7f95a2400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f959c14e770 0x7f959c14ec20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.629+0000 7f95a2400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f957806c920 0x7f957806ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.629+0000 7f95a2400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f959c158f70 0x7f959c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.629+0000 7f95a2400640 1 --2- 172.21.15.102:0/242413511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f959c153110 0x7f959c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.629+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 >> 172.21.15.102:0/242413511 conn(0x7f959c074ea0 msgr2=0x7f959c075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.629+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 shutdown_connections 2024-04-20T21:17:30.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.629+0000 7f95a2400640 1 -- 172.21.15.102:0/242413511 wait complete. 2024-04-20T21:17:30.646 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:17:30.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.842+0000 7f2a2e800640 1 Processor -- start 2024-04-20T21:17:30.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.842+0000 7f2a2e800640 1 -- start start 2024-04-20T21:17:30.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.843+0000 7f2a2e800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a28073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2e800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a280c7190 0x7f2a2815c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2e800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a28158f80 con 0x7f2a280cd060 2024-04-20T21:17:30.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2e800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f2a28159de0 con 0x7f2a280c7190 2024-04-20T21:17:30.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2e800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2a28159260 con 0x7f2a28152d80 2024-04-20T21:17:30.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2ca00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f2a280cd060 0x7f2a280c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51364/0 (socket says 172.21.15.102:51364) 2024-04-20T21:17:30.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2ca00640 1 -- 172.21.15.102:0/3480948360 learned_addr learned my addr 172.21.15.102:0/3480948360 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:30.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a28073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2de00640 1 --2- 172.21.15.102:0/3480948360 >> v2:172.21.15.102:3300/0 conn(0x7f2a280c7190 0x7f2a2815c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2d400640 1 -- 172.21.15.102:0/3480948360 >> v2:172.21.15.102:3300/0 conn(0x7f2a280c7190 msgr2=0x7f2a2815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:30.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.844+0000 7f2a2d400640 1 --2- 172.21.15.102:0/3480948360 >> v2:172.21.15.102:3300/0 conn(0x7f2a280c7190 0x7f2a2815c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.845+0000 7f2a2d400640 1 -- 172.21.15.102:0/3480948360 >> v1:172.21.15.193:6789/0 conn(0x7f2a280cd060 legacy=0x7f2a280c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.845+0000 7f2a2d400640 1 -- 172.21.15.102:0/3480948360 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a28159540 con 0x7f2a28152d80 2024-04-20T21:17:30.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.845+0000 7f2a2de00640 1 --2- 172.21.15.102:0/3480948360 >> v2:172.21.15.102:3300/0 conn(0x7f2a280c7190 0x7f2a2815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:30.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.846+0000 7f2a2d400640 1 --2- 172.21.15.102:0/3480948360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a28073ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0097a0 tx=0x7f2a1c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e848ab52552c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.847+0000 7f2a27e00640 1 -- 172.21.15.102:0/3480948360 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c034070 con 0x7f2a28152d80 2024-04-20T21:17:30.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.847+0000 7f2a27e00640 1 -- 172.21.15.102:0/3480948360 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c030800 con 0x7f2a28152d80 2024-04-20T21:17:30.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.847+0000 7f2a27e00640 1 -- 172.21.15.102:0/3480948360 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c030bd0 con 0x7f2a28152d80 2024-04-20T21:17:30.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.848+0000 7f2a2e800640 1 -- 172.21.15.102:0/3480948360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 msgr2=0x7f2a28073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.848+0000 7f2a2e800640 1 --2- 172.21.15.102:0/3480948360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a28073ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0097a0 tx=0x7f2a1c02b940 comp rx=0 tx=0).stop 2024-04-20T21:17:30.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.848+0000 7f2a2e800640 1 -- 172.21.15.102:0/3480948360 shutdown_connections 2024-04-20T21:17:30.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.848+0000 7f2a2e800640 1 --2- 172.21.15.102:0/3480948360 >> v2:172.21.15.102:3300/0 conn(0x7f2a280c7190 0x7f2a2815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.848+0000 7f2a2e800640 1 --2- 172.21.15.102:0/3480948360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a28073ba0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.848+0000 7f2a2e800640 1 -- 172.21.15.102:0/3480948360 >> 172.21.15.102:0/3480948360 conn(0x7f2a280cc540 msgr2=0x7f2a280cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.848+0000 7f2a2e800640 1 -- 172.21.15.102:0/3480948360 shutdown_connections 2024-04-20T21:17:30.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.848+0000 7f2a2e800640 1 -- 172.21.15.102:0/3480948360 wait complete. 2024-04-20T21:17:30.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.850+0000 7f2a2e800640 1 Processor -- start 2024-04-20T21:17:30.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.850+0000 7f2a2e800640 1 -- start start 2024-04-20T21:17:30.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.851+0000 7f2a2e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a280c7190 0x7f2a28157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.851+0000 7f2a2e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a280cd060 0x7f2a2814ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2e800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a281ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2e800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a280c6360 con 0x7f2a280c7190 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2e800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28159b00 con 0x7f2a280cd060 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2e800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2a280c64e0 con 0x7f2a28152d80 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a280c7190 0x7f2a28157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a281ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a280cd060 0x7f2a2814ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a280c7190 0x7f2a28157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35050/0 (socket says 172.21.15.102:35050) 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2d400640 1 -- 172.21.15.102:0/1697401081 learned_addr learned my addr 172.21.15.102:0/1697401081 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:30.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a281ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35892/0 (socket says 172.21.15.102:35892) 2024-04-20T21:17:30.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2d400640 1 -- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 msgr2=0x7f2a281ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2d400640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a281ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2d400640 1 -- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a280cd060 msgr2=0x7f2a2814ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2d400640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a280cd060 0x7f2a2814ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.852+0000 7f2a2d400640 1 -- 172.21.15.102:0/1697401081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a281587b0 con 0x7f2a280c7190 2024-04-20T21:17:30.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.853+0000 7f2a2d400640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a280c7190 0x7f2a28157680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c02bf50 tx=0x7f2a1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.853+0000 7f2a25600640 1 -- 172.21.15.102:0/1697401081 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c034040 con 0x7f2a280c7190 2024-04-20T21:17:30.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.853+0000 7f2a25600640 1 -- 172.21.15.102:0/1697401081 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c03ada0 con 0x7f2a280c7190 2024-04-20T21:17:30.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.853+0000 7f2a25600640 1 -- 172.21.15.102:0/1697401081 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c006e10 con 0x7f2a280c7190 2024-04-20T21:17:30.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.853+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a28158a50 con 0x7f2a280c7190 2024-04-20T21:17:30.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.853+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a28156e80 con 0x7f2a280c7190 2024-04-20T21:17:30.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.854+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a280c3760 con 0x7f2a280c7190 2024-04-20T21:17:30.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.855+0000 7f2a25600640 1 -- 172.21.15.102:0/1697401081 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a1c04b060 con 0x7f2a280c7190 2024-04-20T21:17:30.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.857+0000 7f2a25600640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a0806c8f0 0x7f2a0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.857+0000 7f2a25600640 1 -- 172.21.15.102:0/1697401081 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2a1c0add60 con 0x7f2a280c7190 2024-04-20T21:17:30.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.857+0000 7f2a2ca00640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a0806c8f0 0x7f2a0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.858+0000 7f2a2ca00640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a0806c8f0 0x7f2a0806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a281e9620 tx=0x7f2a14007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:30.862+0000 7f2a25600640 1 -- 172.21.15.102:0/1697401081 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a280c3760 con 0x7f2a280c7190 2024-04-20T21:17:31.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.064+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a2804fcf0 con 0x7f2a280c7190 2024-04-20T21:17:31.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.065+0000 7f2a25600640 1 -- 172.21.15.102:0/1697401081 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f2a1c03f070 con 0x7f2a280c7190 2024-04-20T21:17:31.064 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-20T21:17:31.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a0806c8f0 msgr2=0x7f2a0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a0806c8f0 0x7f2a0806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a281e9620 tx=0x7f2a14007e80 comp rx=0 tx=0).stop 2024-04-20T21:17:31.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a280c7190 msgr2=0x7f2a28157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a280c7190 0x7f2a28157680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c02bf50 tx=0x7f2a1c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 shutdown_connections 2024-04-20T21:17:31.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a28152d80 0x7f2a281ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a0806c8f0 0x7f2a0806edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a280cd060 0x7f2a2814ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 --2- 172.21.15.102:0/1697401081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a280c7190 0x7f2a28157680 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.067+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 >> 172.21.15.102:0/1697401081 conn(0x7f2a280cc540 msgr2=0x7f2a28150db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:31.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.068+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 shutdown_connections 2024-04-20T21:17:31.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.068+0000 7f2a2e800640 1 -- 172.21.15.102:0/1697401081 wait complete. 2024-04-20T21:17:31.086 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T21:17:31.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.292+0000 7f8fdb200640 1 Processor -- start 2024-04-20T21:17:31.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.292+0000 7f8fdb200640 1 -- start start 2024-04-20T21:17:31.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.294+0000 7f8fdb200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8fdc155250 0x7f8fdc0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.294+0000 7f8fdb200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc0cccf0 0x7f8fdc0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.294+0000 7f8fdb200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fdc0cd3b0 con 0x7f8fdc0cc590 2024-04-20T21:17:31.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.294+0000 7f8fdb200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8fdc0cb770 con 0x7f8fdc155250 2024-04-20T21:17:31.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.294+0000 7f8fdb200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8fdc15b180 con 0x7f8fdc0cccf0 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.294+0000 7f8fd9e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8fdc155250 0x7f8fdc0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.295+0000 7f8fd9e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8fdc155250 0x7f8fdc0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35056/0 (socket says 172.21.15.102:35056) 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.295+0000 7f8fd9e00640 1 -- 172.21.15.102:0/4155296437 learned_addr learned my addr 172.21.15.102:0/4155296437 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.295+0000 7f8fd9400640 1 --2- 172.21.15.102:0/4155296437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc0cccf0 0x7f8fdc0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.295+0000 7f8fd9e00640 1 -- 172.21.15.102:0/4155296437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc0cccf0 msgr2=0x7f8fdc0cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.295+0000 7f8fd9e00640 1 --2- 172.21.15.102:0/4155296437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc0cccf0 0x7f8fdc0cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.295+0000 7f8fd9e00640 1 -- 172.21.15.102:0/4155296437 >> v1:172.21.15.193:6789/0 conn(0x7f8fdc0cc590 legacy=0x7f8fdc15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.295+0000 7f8fd8a00640 1 -- 172.21.15.102:0/4155296437 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560199387 0 0) 0x7f8fdc0cd3b0 con 0x7f8fdc0cc590 2024-04-20T21:17:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.295+0000 7f8fd9e00640 1 -- 172.21.15.102:0/4155296437 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc15b420 con 0x7f8fdc155250 2024-04-20T21:17:31.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.297+0000 7f8fd9e00640 1 --2- 172.21.15.102:0/4155296437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc155250 0x7f8fdc0ce030 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009870 tx=0x7f8fcc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee1e639061d610c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:31.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.297+0000 7f8fd8a00640 1 -- 172.21.15.102:0/4155296437 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc034070 con 0x7f8fdc155250 2024-04-20T21:17:31.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.297+0000 7f8fd8a00640 1 -- 172.21.15.102:0/4155296437 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fcc030840 con 0x7f8fdc155250 2024-04-20T21:17:31.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.297+0000 7f8fd8a00640 1 -- 172.21.15.102:0/4155296437 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc030c10 con 0x7f8fdc155250 2024-04-20T21:17:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.298+0000 7f8fdb200640 1 -- 172.21.15.102:0/4155296437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc155250 msgr2=0x7f8fdc0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.298+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4155296437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc155250 0x7f8fdc0ce030 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009870 tx=0x7f8fcc02b980 comp rx=0 tx=0).stop 2024-04-20T21:17:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.298+0000 7f8fdb200640 1 -- 172.21.15.102:0/4155296437 shutdown_connections 2024-04-20T21:17:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.298+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4155296437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc0cccf0 0x7f8fdc0cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.298+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4155296437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc155250 0x7f8fdc0ce030 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.298+0000 7f8fdb200640 1 -- 172.21.15.102:0/4155296437 >> 172.21.15.102:0/4155296437 conn(0x7f8fdc06cb40 msgr2=0x7f8fdc06cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.298+0000 7f8fdb200640 1 -- 172.21.15.102:0/4155296437 shutdown_connections 2024-04-20T21:17:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.298+0000 7f8fdb200640 1 -- 172.21.15.102:0/4155296437 wait complete. 2024-04-20T21:17:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.300+0000 7f8fdb200640 1 Processor -- start 2024-04-20T21:17:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.300+0000 7f8fdb200640 1 -- start start 2024-04-20T21:17:31.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fdb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc0cc590 0x7f8fdc153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fdb200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fdc0cccf0 0x7f8fdc0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fdb200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc155250 0x7f8fdc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fdb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc15bcc0 con 0x7f8fdc0cc590 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fdb200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc15b9e0 con 0x7f8fdc0cccf0 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fdb200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8fdc155630 con 0x7f8fdc155250 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc0cc590 0x7f8fdc153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc0cc590 0x7f8fdc153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35058/0 (socket says 172.21.15.102:35058) 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9e00640 1 -- 172.21.15.102:0/4243240901 learned_addr learned my addr 172.21.15.102:0/4243240901 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9400640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fdc0cccf0 0x7f8fdc0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fda800640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc155250 0x7f8fdc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9e00640 1 -- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc155250 msgr2=0x7f8fdc0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9e00640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc155250 0x7f8fdc0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9e00640 1 -- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fdc0cccf0 msgr2=0x7f8fdc0c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9e00640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fdc0cccf0 0x7f8fdc0c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.301+0000 7f8fd9e00640 1 -- 172.21.15.102:0/4243240901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc1f0ce0 con 0x7f8fdc0cc590 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.302+0000 7f8fd9e00640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc0cc590 0x7f8fdc153e60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009990 tx=0x7f8fcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.302+0000 7f8fd2000640 1 -- 172.21.15.102:0/4243240901 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc034040 con 0x7f8fdc0cc590 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.302+0000 7f8fd2000640 1 -- 172.21.15.102:0/4243240901 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fcc039d90 con 0x7f8fdc0cc590 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.302+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fdc1f0f50 con 0x7f8fdc0cc590 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.302+0000 7f8fd2000640 1 -- 172.21.15.102:0/4243240901 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc043610 con 0x7f8fdc0cc590 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.302+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fdc1f1510 con 0x7f8fdc0cc590 2024-04-20T21:17:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.303+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fdc0cb8f0 con 0x7f8fdc0cc590 2024-04-20T21:17:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.304+0000 7f8fd2000640 1 -- 172.21.15.102:0/4243240901 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fcc04a060 con 0x7f8fdc0cc590 2024-04-20T21:17:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.305+0000 7f8fd2000640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8fb806c8f0 0x7f8fb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.305+0000 7f8fd2000640 1 -- 172.21.15.102:0/4243240901 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8fcc0adcd0 con 0x7f8fdc0cc590 2024-04-20T21:17:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.305+0000 7f8fd9400640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8fb806c8f0 0x7f8fb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.306+0000 7f8fd9400640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8fb806c8f0 0x7f8fb806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0cef60 tx=0x7f8fc400b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:31.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.310+0000 7f8fd2000640 1 -- 172.21.15.102:0/4243240901 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8fdc0cb8f0 con 0x7f8fdc0cc590 2024-04-20T21:17:31.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.511+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8fdc04fcf0 con 0x7f8fdc0cc590 2024-04-20T21:17:31.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.512+0000 7f8fd2000640 1 -- 172.21.15.102:0/4243240901 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f8fcc03e070 con 0x7f8fdc0cc590 2024-04-20T21:17:31.511 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:31.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8fb806c8f0 msgr2=0x7f8fb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8fb806c8f0 0x7f8fb806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0cef60 tx=0x7f8fc400b040 comp rx=0 tx=0).stop 2024-04-20T21:17:31.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc0cc590 msgr2=0x7f8fdc153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc0cc590 0x7f8fdc153e60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009990 tx=0x7f8fcc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 shutdown_connections 2024-04-20T21:17:31.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fdc155250 0x7f8fdc0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8fb806c8f0 0x7f8fb806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fdc0cccf0 0x7f8fdc0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 --2- 172.21.15.102:0/4243240901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc0cc590 0x7f8fdc153e60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 >> 172.21.15.102:0/4243240901 conn(0x7f8fdc06cb40 msgr2=0x7f8fdc0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:31.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 shutdown_connections 2024-04-20T21:17:31.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.514+0000 7f8fdb200640 1 -- 172.21.15.102:0/4243240901 wait complete. 2024-04-20T21:17:31.533 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:31.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.730+0000 7fc112000640 1 Processor -- start 2024-04-20T21:17:31.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.730+0000 7fc112000640 1 -- start start 2024-04-20T21:17:31.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.732+0000 7fc112000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.732+0000 7fc112000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10c1536d0 con 0x7fc10c073a10 2024-04-20T21:17:31.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.732+0000 7fc112000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10c0c76c0 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.732+0000 7fc112000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c0c7960 con 0x7fc10c14fa50 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.732+0000 7fc10be00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc10c073a10 0x7fc10c163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34656/0 (socket says 172.21.15.102:34656) 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.732+0000 7fc10be00640 1 -- 172.21.15.102:0/3487168844 learned_addr learned my addr 172.21.15.102:0/3487168844 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.733+0000 7fc10b400640 1 --2- 172.21.15.102:0/3487168844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.733+0000 7fc10aa00640 1 -- 172.21.15.102:0/3487168844 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771114678 0 0) 0x7fc10c0c76c0 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.733+0000 7fc10aa00640 1 -- 172.21.15.102:0/3487168844 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0f0003520 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.733+0000 7fc10aa00640 1 -- 172.21.15.102:0/3487168844 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4212994176 0 0) 0x7fc104001490 con 0x7fc10c073a10 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.733+0000 7fc10b400640 1 -- 172.21.15.102:0/3487168844 >> v1:172.21.15.193:6789/0 conn(0x7fc10c0c9eb0 legacy=0x7fc10c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.733+0000 7fc10b400640 1 -- 172.21.15.102:0/3487168844 >> v1:172.21.15.102:6789/0 conn(0x7fc10c073a10 legacy=0x7fc10c163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.733+0000 7fc10b400640 1 -- 172.21.15.102:0/3487168844 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c0c3d90 con 0x7fc10c14fa50 2024-04-20T21:17:31.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.734+0000 7fc10b400640 1 --2- 172.21.15.102:0/3487168844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0c4830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0c3f60 tx=0x7fc0fc02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc25cc44faa2e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:31.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.734+0000 7fc10aa00640 1 -- 172.21.15.102:0/3487168844 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc034070 con 0x7fc10c14fa50 2024-04-20T21:17:31.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.734+0000 7fc10aa00640 1 -- 172.21.15.102:0/3487168844 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc030880 con 0x7fc10c14fa50 2024-04-20T21:17:31.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.734+0000 7fc10aa00640 1 -- 172.21.15.102:0/3487168844 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc030c50 con 0x7fc10c14fa50 2024-04-20T21:17:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.735+0000 7fc112000640 1 -- 172.21.15.102:0/3487168844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 msgr2=0x7fc10c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.735+0000 7fc112000640 1 --2- 172.21.15.102:0/3487168844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0c4830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0c3f60 tx=0x7fc0fc02b9c0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.735+0000 7fc112000640 1 -- 172.21.15.102:0/3487168844 shutdown_connections 2024-04-20T21:17:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.735+0000 7fc112000640 1 --2- 172.21.15.102:0/3487168844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0c4830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.735+0000 7fc112000640 1 -- 172.21.15.102:0/3487168844 >> 172.21.15.102:0/3487168844 conn(0x7fc10c0c9390 msgr2=0x7fc10c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.735+0000 7fc112000640 1 -- 172.21.15.102:0/3487168844 shutdown_connections 2024-04-20T21:17:31.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.735+0000 7fc112000640 1 -- 172.21.15.102:0/3487168844 wait complete. 2024-04-20T21:17:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.737+0000 7fc112000640 1 Processor -- start 2024-04-20T21:17:31.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.737+0000 7fc112000640 1 -- start start 2024-04-20T21:17:31.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc112000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc10c073a10 0x7fc10c158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc112000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0c9eb0 0x7fc10c1f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc112000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc112000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0c4030 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc112000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0c7c00 con 0x7fc10c073a10 2024-04-20T21:17:31.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc112000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c0c4310 con 0x7fc10c14fa50 2024-04-20T21:17:31.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc103e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0c9eb0 0x7fc10c1f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc103e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0c9eb0 0x7fc10c1f0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35060/0 (socket says 172.21.15.102:35060) 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc103e00640 1 -- 172.21.15.102:0/3023422736 learned_addr learned my addr 172.21.15.102:0/3023422736 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc10be00640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc10b400640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc10c073a10 0x7fc10c158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc103e00640 1 -- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 msgr2=0x7fc10c0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc103e00640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc103e00640 1 -- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc10c073a10 msgr2=0x7fc10c158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc103e00640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc10c073a10 0x7fc10c158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.738+0000 7fc103e00640 1 -- 172.21.15.102:0/3023422736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c148b50 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.739+0000 7fc103e00640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0c9eb0 0x7fc10c1f0090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c490 tx=0x7fc0f400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:31.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.739+0000 7fc103400640 1 -- 172.21.15.102:0/3023422736 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40052d0 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.739+0000 7fc103400640 1 -- 172.21.15.102:0/3023422736 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f4005470 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.739+0000 7fc103400640 1 -- 172.21.15.102:0/3023422736 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4012850 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.739+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c148e30 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.739+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c158290 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.740+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10c074340 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.741+0000 7fc103400640 1 -- 172.21.15.102:0/3023422736 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0f40129f0 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.742+0000 7fc103400640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc0e806c8f0 0x7fc0e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.742+0000 7fc103400640 1 -- 172.21.15.102:0/3023422736 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc0f408bb00 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.742+0000 7fc10b400640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc0e806c8f0 0x7fc0e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.743+0000 7fc10b400640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc0e806c8f0 0x7fc0e806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc036000 tx=0x7fc0fc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:31.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.746+0000 7fc103400640 1 -- 172.21.15.102:0/3023422736 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc0f4059be0 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.987+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc10c1533b0 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.989+0000 7fc103400640 1 -- 172.21.15.102:0/3023422736 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7fc0f408be20 con 0x7fc10c0c9eb0 2024-04-20T21:17:31.988 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:31.989 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:17:31.989 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:17:31.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.991+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc0e806c8f0 msgr2=0x7fc0e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.991+0000 7fc112000640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc0e806c8f0 0x7fc0e806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc036000 tx=0x7fc0fc008030 comp rx=0 tx=0).stop 2024-04-20T21:17:31.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.991+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0c9eb0 msgr2=0x7fc10c1f0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.991+0000 7fc112000640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0c9eb0 0x7fc10c1f0090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c490 tx=0x7fc0f400c950 comp rx=0 tx=0).stop 2024-04-20T21:17:31.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.991+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 shutdown_connections 2024-04-20T21:17:31.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.991+0000 7fc112000640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc10c14fa50 0x7fc10c0cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.992+0000 7fc112000640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0c9eb0 0x7fc10c1f0090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.992+0000 7fc112000640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc0e806c8f0 0x7fc0e806edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.992+0000 7fc112000640 1 --2- 172.21.15.102:0/3023422736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc10c073a10 0x7fc10c158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.992+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 >> 172.21.15.102:0/3023422736 conn(0x7fc10c0c9390 msgr2=0x7fc10c14dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:31.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.992+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 shutdown_connections 2024-04-20T21:17:31.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:31.992+0000 7fc112000640 1 -- 172.21.15.102:0/3023422736 wait complete. 2024-04-20T21:17:32.010 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:32.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.201+0000 7f2cb0600640 1 Processor -- start 2024-04-20T21:17:32.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.201+0000 7f2cb0600640 1 -- start start 2024-04-20T21:17:32.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.202+0000 7f2cb0600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2ca80c9eb0 0x7f2ca80c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.202+0000 7f2cb0600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca8158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.202+0000 7f2cb0600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca81536d0 con 0x7f2ca814fa50 2024-04-20T21:17:32.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.202+0000 7f2cb0600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca8159500 con 0x7f2ca80c9eb0 2024-04-20T21:17:32.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.202+0000 7f2cb0600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca80c76c0 con 0x7f2ca8073a10 2024-04-20T21:17:32.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cad200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2ca80c9eb0 0x7f2ca80c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cadc00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f2ca814fa50 0x7f2ca80c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51388/0 (socket says 172.21.15.102:51388) 2024-04-20T21:17:32.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cadc00640 1 -- 172.21.15.102:0/1544587282 learned_addr learned my addr 172.21.15.102:0/1544587282 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:32.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cad200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2ca80c9eb0 0x7f2ca80c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:46342/0 (socket says 172.21.15.102:46342) 2024-04-20T21:17:32.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cae600640 1 --2- 172.21.15.102:0/1544587282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca8158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cad200640 1 -- 172.21.15.102:0/1544587282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 msgr2=0x7f2ca8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:32.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cad200640 1 --2- 172.21.15.102:0/1544587282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca8158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cad200640 1 -- 172.21.15.102:0/1544587282 >> v1:172.21.15.193:6789/0 conn(0x7f2ca814fa50 legacy=0x7f2ca80c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2cad200640 1 -- 172.21.15.102:0/1544587282 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca80c7960 con 0x7f2ca80c9eb0 2024-04-20T21:17:32.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.203+0000 7f2ca7e00640 1 -- 172.21.15.102:0/1544587282 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2953759649 0 0) 0x7f2c9c001110 con 0x7f2ca814fa50 2024-04-20T21:17:32.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.204+0000 7f2cad200640 1 --2- 172.21.15.102:0/1544587282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca80c9eb0 0x7f2ca80c4830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c940097a0 tx=0x7f2c9402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70277c662013e2c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:32.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.205+0000 7f2ca7e00640 1 -- 172.21.15.102:0/1544587282 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94034070 con 0x7f2ca80c9eb0 2024-04-20T21:17:32.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.205+0000 7f2ca7e00640 1 -- 172.21.15.102:0/1544587282 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c94030610 con 0x7f2ca80c9eb0 2024-04-20T21:17:32.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.205+0000 7f2ca7e00640 1 -- 172.21.15.102:0/1544587282 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c940309e0 con 0x7f2ca80c9eb0 2024-04-20T21:17:32.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.206+0000 7f2cb0600640 1 -- 172.21.15.102:0/1544587282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca80c9eb0 msgr2=0x7f2ca80c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.206+0000 7f2cb0600640 1 --2- 172.21.15.102:0/1544587282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca80c9eb0 0x7f2ca80c4830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c940097a0 tx=0x7f2c9402b960 comp rx=0 tx=0).stop 2024-04-20T21:17:32.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.206+0000 7f2cb0600640 1 -- 172.21.15.102:0/1544587282 shutdown_connections 2024-04-20T21:17:32.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.206+0000 7f2cb0600640 1 --2- 172.21.15.102:0/1544587282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca8158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.206+0000 7f2cb0600640 1 --2- 172.21.15.102:0/1544587282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca80c9eb0 0x7f2ca80c4830 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.206+0000 7f2cb0600640 1 -- 172.21.15.102:0/1544587282 >> 172.21.15.102:0/1544587282 conn(0x7f2ca80c9390 msgr2=0x7f2ca80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:32.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.206+0000 7f2cb0600640 1 -- 172.21.15.102:0/1544587282 shutdown_connections 2024-04-20T21:17:32.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.206+0000 7f2cb0600640 1 -- 172.21.15.102:0/1544587282 wait complete. 2024-04-20T21:17:32.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.208+0000 7f2cb0600640 1 Processor -- start 2024-04-20T21:17:32.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.208+0000 7f2cb0600640 1 -- start start 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cb0600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca81580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cb0600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca80c9eb0 0x7f2ca8145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cb0600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca814fa50 0x7f2ca8145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cb0600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80c7c00 con 0x7f2ca80c9eb0 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cb0600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80c3d90 con 0x7f2ca814fa50 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cb0600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca80c4030 con 0x7f2ca8073a10 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cadc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca81580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cae600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca814fa50 0x7f2ca8145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca80c9eb0 0x7f2ca8145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cadc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca81580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35962/0 (socket says 172.21.15.102:35962) 2024-04-20T21:17:32.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cadc00640 1 -- 172.21.15.102:0/2830184648 learned_addr learned my addr 172.21.15.102:0/2830184648 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:32.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cae600640 1 -- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 msgr2=0x7f2ca81580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cae600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca81580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cae600640 1 -- 172.21.15.102:0/2830184648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca80c9eb0 msgr2=0x7f2ca8145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cae600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca80c9eb0 0x7f2ca8145580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.209+0000 7f2cae600640 1 -- 172.21.15.102:0/2830184648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca81472a0 con 0x7f2ca814fa50 2024-04-20T21:17:32.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.210+0000 7f2cae600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca814fa50 0x7f2ca8145ac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000ebd0 tx=0x7f2ca000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:32.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.210+0000 7f2ca5600640 1 -- 172.21.15.102:0/2830184648 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0018020 con 0x7f2ca814fa50 2024-04-20T21:17:32.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.210+0000 7f2ca5600640 1 -- 172.21.15.102:0/2830184648 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca000ccc0 con 0x7f2ca814fa50 2024-04-20T21:17:32.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.210+0000 7f2ca5600640 1 -- 172.21.15.102:0/2830184648 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00146c0 con 0x7f2ca814fa50 2024-04-20T21:17:32.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.210+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca81ef2c0 con 0x7f2ca814fa50 2024-04-20T21:17:32.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.210+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81ef880 con 0x7f2ca814fa50 2024-04-20T21:17:32.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.211+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca8074340 con 0x7f2ca814fa50 2024-04-20T21:17:32.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.212+0000 7f2ca5600640 1 -- 172.21.15.102:0/2830184648 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ca0005d10 con 0x7f2ca814fa50 2024-04-20T21:17:32.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.213+0000 7f2ca5600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.213+0000 7f2ca5600640 1 -- 172.21.15.102:0/2830184648 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ca008ca50 con 0x7f2ca814fa50 2024-04-20T21:17:32.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.214+0000 7f2cadc00640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.214+0000 7f2cadc00640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c8806c8f0 0x7f2c8806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c007e30 tx=0x7f2c9c007dc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:32.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.218+0000 7f2ca5600640 1 -- 172.21.15.102:0/2830184648 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ca0022030 con 0x7f2ca814fa50 2024-04-20T21:17:32.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.420+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ca80c7320 con 0x7f2ca814fa50 2024-04-20T21:17:32.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.422+0000 7f2ca5600640 1 -- 172.21.15.102:0/2830184648 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f2ca0010040 con 0x7f2ca814fa50 2024-04-20T21:17:32.422 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:32.422 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:16:59.052879+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c8806c8f0 msgr2=0x7f2c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c8806c8f0 0x7f2c8806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c007e30 tx=0x7f2c9c007dc0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca814fa50 msgr2=0x7f2ca8145ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca814fa50 0x7f2ca8145ac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000ebd0 tx=0x7f2ca000c490 comp rx=0 tx=0).stop 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 shutdown_connections 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ca814fa50 0x7f2ca8145ac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca80c9eb0 0x7f2ca8145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 --2- 172.21.15.102:0/2830184648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ca8073a10 0x7f2ca81580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.424+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 >> 172.21.15.102:0/2830184648 conn(0x7f2ca80c9390 msgr2=0x7f2ca806eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.425+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 shutdown_connections 2024-04-20T21:17:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.425+0000 7f2cb0600640 1 -- 172.21.15.102:0/2830184648 wait complete. 2024-04-20T21:17:32.445 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:32.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.646+0000 7f9b15200640 1 Processor -- start 2024-04-20T21:17:32.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.646+0000 7f9b15200640 1 -- start start 2024-04-20T21:17:32.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.648+0000 7f9b15200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b10155530 0x7f9b1015e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.648+0000 7f9b15200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1015ef60 con 0x7f9b100cc470 2024-04-20T21:17:32.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.648+0000 7f9b15200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1015f240 con 0x7f9b100cb6a0 2024-04-20T21:17:32.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.648+0000 7f9b15200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9b100c5ca0 con 0x7f9b10155530 2024-04-20T21:17:32.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.648+0000 7f9b06a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9b100cc470 0x7f9b100cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34660/0 (socket says 172.21.15.102:34660) 2024-04-20T21:17:32.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.648+0000 7f9b06a00640 1 -- 172.21.15.102:0/490397160 learned_addr learned my addr 172.21.15.102:0/490397160 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:32.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.648+0000 7f9b0f400640 1 --2- 172.21.15.102:0/490397160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b10155530 0x7f9b1015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.649+0000 7f9b0f400640 1 -- 172.21.15.102:0/490397160 >> v1:172.21.15.193:6789/0 conn(0x7f9b100cb6a0 legacy=0x7f9b101659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.649+0000 7f9b0ea00640 1 -- 172.21.15.102:0/490397160 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 17508786 0 0) 0x7f9b1015ef60 con 0x7f9b100cc470 2024-04-20T21:17:32.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.649+0000 7f9b0f400640 1 -- 172.21.15.102:0/490397160 >> v1:172.21.15.102:6789/0 conn(0x7f9b100cc470 legacy=0x7f9b100cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.649+0000 7f9b0f400640 1 -- 172.21.15.102:0/490397160 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1015f520 con 0x7f9b10155530 2024-04-20T21:17:32.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.650+0000 7f9b0f400640 1 --2- 172.21.15.102:0/490397160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b10155530 0x7f9b1015e640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009750 tx=0x7f9b0002b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584f9d4fc484ca4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:32.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.650+0000 7f9b0ea00640 1 -- 172.21.15.102:0/490397160 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00038070 con 0x7f9b10155530 2024-04-20T21:17:32.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.650+0000 7f9b0ea00640 1 -- 172.21.15.102:0/490397160 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b00031710 con 0x7f9b10155530 2024-04-20T21:17:32.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.650+0000 7f9b0ea00640 1 -- 172.21.15.102:0/490397160 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00031ae0 con 0x7f9b10155530 2024-04-20T21:17:32.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.651+0000 7f9b15200640 1 -- 172.21.15.102:0/490397160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b10155530 msgr2=0x7f9b1015e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.651+0000 7f9b15200640 1 --2- 172.21.15.102:0/490397160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b10155530 0x7f9b1015e640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009750 tx=0x7f9b0002b7b0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.651+0000 7f9b15200640 1 -- 172.21.15.102:0/490397160 shutdown_connections 2024-04-20T21:17:32.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.651+0000 7f9b15200640 1 --2- 172.21.15.102:0/490397160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b10155530 0x7f9b1015e640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.651+0000 7f9b15200640 1 -- 172.21.15.102:0/490397160 >> 172.21.15.102:0/490397160 conn(0x7f9b10075080 msgr2=0x7f9b10075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:32.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.651+0000 7f9b15200640 1 -- 172.21.15.102:0/490397160 shutdown_connections 2024-04-20T21:17:32.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.651+0000 7f9b15200640 1 -- 172.21.15.102:0/490397160 wait complete. 2024-04-20T21:17:32.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.653+0000 7f9b15200640 1 Processor -- start 2024-04-20T21:17:32.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.653+0000 7f9b15200640 1 -- start start 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b15200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b100cb6a0 0x7f9b101e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b15200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b100cc470 0x7f9b1014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b15200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10155530 0x7f9b1014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b15200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b100cd280 con 0x7f9b10155530 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b15200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b100cd5f0 con 0x7f9b100cb6a0 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b15200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9b100ccbe0 con 0x7f9b100cc470 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10155530 0x7f9b1014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10155530 0x7f9b1014b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35074/0 (socket says 172.21.15.102:35074) 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b0fe00640 1 -- 172.21.15.102:0/3332244646 learned_addr learned my addr 172.21.15.102:0/3332244646 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b0f400640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b100cb6a0 0x7f9b101e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.654+0000 7f9b06a00640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b100cc470 0x7f9b1014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.655+0000 7f9b0f400640 1 -- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b100cc470 msgr2=0x7f9b1014b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.655+0000 7f9b0f400640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b100cc470 0x7f9b1014b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.655+0000 7f9b0f400640 1 -- 172.21.15.102:0/3332244646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10155530 msgr2=0x7f9b1014b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.655+0000 7f9b0f400640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10155530 0x7f9b1014b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.655+0000 7f9b0f400640 1 -- 172.21.15.102:0/3332244646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b101f1180 con 0x7f9b100cb6a0 2024-04-20T21:17:32.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.655+0000 7f9b0fe00640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10155530 0x7f9b1014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:32.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.655+0000 7f9b0f400640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b100cb6a0 0x7f9b101e5d50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b00033000 tx=0x7f9b000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:32.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.656+0000 7f9b07e00640 1 -- 172.21.15.102:0/3332244646 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00043070 con 0x7f9b100cb6a0 2024-04-20T21:17:32.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.656+0000 7f9b07e00640 1 -- 172.21.15.102:0/3332244646 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b000339c0 con 0x7f9b100cb6a0 2024-04-20T21:17:32.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.656+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b101f1460 con 0x7f9b100cb6a0 2024-04-20T21:17:32.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.656+0000 7f9b07e00640 1 -- 172.21.15.102:0/3332244646 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00038040 con 0x7f9b100cb6a0 2024-04-20T21:17:32.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.656+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101f1a20 con 0x7f9b100cb6a0 2024-04-20T21:17:32.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.657+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1004fd60 con 0x7f9b100cb6a0 2024-04-20T21:17:32.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.658+0000 7f9b07e00640 1 -- 172.21.15.102:0/3332244646 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b00046070 con 0x7f9b100cb6a0 2024-04-20T21:17:32.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.659+0000 7f9b07e00640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9aec06c8f0 0x7f9aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.659+0000 7f9b07e00640 1 -- 172.21.15.102:0/3332244646 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9b000addf0 con 0x7f9b100cb6a0 2024-04-20T21:17:32.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.659+0000 7f9b06a00640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9aec06c8f0 0x7f9aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.660+0000 7f9b06a00640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9aec06c8f0 0x7f9aec06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9af8007f70 tx=0x7f9af8007f00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:32.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.664+0000 7f9b07e00640 1 -- 172.21.15.102:0/3332244646 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b0007b880 con 0x7f9b100cb6a0 2024-04-20T21:17:32.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.904+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b100b6c50 con 0x7f9b100cb6a0 2024-04-20T21:17:32.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.905+0000 7f9b07e00640 1 -- 172.21.15.102:0/3332244646 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f9b0007fb80 con 0x7f9b100cb6a0 2024-04-20T21:17:32.905 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:32.905 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:17:32.905 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:17:32.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.907+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9aec06c8f0 msgr2=0x7f9aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.907+0000 7f9b15200640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9aec06c8f0 0x7f9aec06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9af8007f70 tx=0x7f9af8007f00 comp rx=0 tx=0).stop 2024-04-20T21:17:32.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b100cb6a0 msgr2=0x7f9b101e5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b100cb6a0 0x7f9b101e5d50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b00033000 tx=0x7f9b000046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 shutdown_connections 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10155530 0x7f9b1014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b100cc470 0x7f9b1014b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9aec06c8f0 0x7f9aec06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 --2- 172.21.15.102:0/3332244646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b100cb6a0 0x7f9b101e5d50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 >> 172.21.15.102:0/3332244646 conn(0x7f9b10075080 msgr2=0x7f9b100d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 shutdown_connections 2024-04-20T21:17:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:32.908+0000 7f9b15200640 1 -- 172.21.15.102:0/3332244646 wait complete. 2024-04-20T21:17:33.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:16:54.017441+0000', 'modified': '2024-04-20T21:17:20.270633+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.102:6835/2873333699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 2873333699}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 2873333699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4300, 'qdb_cluster': [4300]} 2024-04-20T21:17:33.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:17:33.530 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T21:17:33.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.736+0000 7fa73a800640 1 Processor -- start 2024-04-20T21:17:33.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.736+0000 7fa73a800640 1 -- start start 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa73a800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734073b20 0x7fa73406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa73a800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa7340caec0 0x7fa73415cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa73a800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa73406def0 con 0x7fa734153180 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa73a800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa73415cae0 con 0x7fa7340caec0 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa73a800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa73406e1d0 con 0x7fa734073b20 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa739e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa7340caec0 0x7fa73415cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa739e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa7340caec0 0x7fa73415cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35090/0 (socket says 172.21.15.102:35090) 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa739e00640 1 -- 172.21.15.102:0/1493503811 learned_addr learned my addr 172.21.15.102:0/1493503811 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:33.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.738+0000 7fa738a00640 1 --2- 172.21.15.102:0/1493503811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734073b20 0x7fa73406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.739+0000 7fa739e00640 1 -- 172.21.15.102:0/1493503811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734073b20 msgr2=0x7fa73406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.739+0000 7fa739e00640 1 --2- 172.21.15.102:0/1493503811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734073b20 0x7fa73406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.739+0000 7fa739e00640 1 -- 172.21.15.102:0/1493503811 >> v1:172.21.15.193:6789/0 conn(0x7fa734153180 legacy=0x7fa7340c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.739+0000 7fa739e00640 1 -- 172.21.15.102:0/1493503811 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73406e4b0 con 0x7fa7340caec0 2024-04-20T21:17:33.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.739+0000 7fa733e00640 1 -- 172.21.15.102:0/1493503811 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554057156 0 0) 0x7fa73406def0 con 0x7fa734153180 2024-04-20T21:17:33.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.740+0000 7fa739e00640 1 --2- 172.21.15.102:0/1493503811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7340caec0 0x7fa73415cf70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009e50 tx=0x7fa72c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ee33c96f5dce31 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:33.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.740+0000 7fa733e00640 1 -- 172.21.15.102:0/1493503811 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c02cb00 con 0x7fa7340caec0 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.740+0000 7fa733e00640 1 -- 172.21.15.102:0/1493503811 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c02cca0 con 0x7fa7340caec0 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.740+0000 7fa733e00640 1 -- 172.21.15.102:0/1493503811 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c034610 con 0x7fa7340caec0 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.741+0000 7fa73a800640 1 -- 172.21.15.102:0/1493503811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7340caec0 msgr2=0x7fa73415cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.741+0000 7fa73a800640 1 --2- 172.21.15.102:0/1493503811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7340caec0 0x7fa73415cf70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009e50 tx=0x7fa72c02bf60 comp rx=0 tx=0).stop 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.741+0000 7fa73a800640 1 -- 172.21.15.102:0/1493503811 shutdown_connections 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.741+0000 7fa73a800640 1 --2- 172.21.15.102:0/1493503811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7340caec0 0x7fa73415cf70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.741+0000 7fa73a800640 1 --2- 172.21.15.102:0/1493503811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734073b20 0x7fa73406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.741+0000 7fa73a800640 1 -- 172.21.15.102:0/1493503811 >> 172.21.15.102:0/1493503811 conn(0x7fa7340ca3a0 msgr2=0x7fa7340ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.741+0000 7fa73a800640 1 -- 172.21.15.102:0/1493503811 shutdown_connections 2024-04-20T21:17:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.741+0000 7fa73a800640 1 -- 172.21.15.102:0/1493503811 wait complete. 2024-04-20T21:17:33.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.743+0000 7fa73a800640 1 Processor -- start 2024-04-20T21:17:33.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.743+0000 7fa73a800640 1 -- start start 2024-04-20T21:17:33.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa73a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa734073b20 0x7fa734075110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa73a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7340caec0 0x7fa73414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa73a800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734153180 0x7fa73414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa73a800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa734153560 con 0x7fa734073b20 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa73a800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa73415c770 con 0x7fa7340caec0 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa73a800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa7341536e0 con 0x7fa734153180 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa734073b20 0x7fa734075110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa734073b20 0x7fa734075110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35092/0 (socket says 172.21.15.102:35092) 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739400640 1 -- 172.21.15.102:0/2144276322 learned_addr learned my addr 172.21.15.102:0/2144276322 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739e00640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734153180 0x7fa73414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa738a00640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7340caec0 0x7fa73414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739400640 1 -- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734153180 msgr2=0x7fa73414eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739400640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734153180 0x7fa73414eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739400640 1 -- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7340caec0 msgr2=0x7fa73414e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739400640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7340caec0 0x7fa73414e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa739400640 1 -- 172.21.15.102:0/2144276322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7341f15c0 con 0x7fa734073b20 2024-04-20T21:17:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.744+0000 7fa738a00640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7340caec0 0x7fa73414e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.745+0000 7fa739400640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa734073b20 0x7fa734075110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c4b0 tx=0x7fa72800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.745+0000 7fa731600640 1 -- 172.21.15.102:0/2144276322 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728017070 con 0x7fa734073b20 2024-04-20T21:17:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.745+0000 7fa731600640 1 -- 172.21.15.102:0/2144276322 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa728005480 con 0x7fa734073b20 2024-04-20T21:17:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.745+0000 7fa731600640 1 -- 172.21.15.102:0/2144276322 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728003830 con 0x7fa734073b20 2024-04-20T21:17:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.745+0000 7fa73a800640 1 -- 172.21.15.102:0/2144276322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7341f1860 con 0x7fa734073b20 2024-04-20T21:17:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.745+0000 7fa73a800640 1 -- 172.21.15.102:0/2144276322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7341f1e20 con 0x7fa734073b20 2024-04-20T21:17:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.747+0000 7fa731600640 1 -- 172.21.15.102:0/2144276322 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa72801a070 con 0x7fa734073b20 2024-04-20T21:17:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.748+0000 7fa731600640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa71406c8f0 0x7fa71406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.748+0000 7fa731600640 1 -- 172.21.15.102:0/2144276322 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa72808b220 con 0x7fa734073b20 2024-04-20T21:17:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.748+0000 7fa738a00640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa71406c8f0 0x7fa71406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.749+0000 7fa73eb69740 1 Processor -- start 2024-04-20T21:17:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.749+0000 7fa73eb69740 1 -- start start 2024-04-20T21:17:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.749+0000 7fa738a00640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa71406c8f0 0x7fa71406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa734150370 tx=0x7fa720008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633467135f0 0x563346715ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467139d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x563346741270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa73eb69740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56334667e9d0 con 0x563346763d70 2024-04-20T21:17:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa73eb69740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563346625270 con 0x5633467135f0 2024-04-20T21:17:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa73eb69740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x56334667a990 con 0x563346766880 2024-04-20T21:17:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x563346741270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x563346741270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35094/0 (socket says 172.21.15.102:35094) 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739e00640 1 -- 172.21.15.102:0/2284232486 learned_addr learned my addr 172.21.15.102:0/2284232486 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa738a00640 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739400640 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633467135f0 0x563346715ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739e00640 1 -- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 msgr2=0x5633467139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739e00640 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467139d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739e00640 1 -- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633467135f0 msgr2=0x563346715ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739e00640 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633467135f0 0x563346715ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa739e00640 1 -- 172.21.15.102:0/2284232486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633463912d0 con 0x563346763d70 2024-04-20T21:17:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.750+0000 7fa738a00640 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:33.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.751+0000 7fa739e00640 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x563346741270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009e50 tx=0x7fa72c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fde04c452b5a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.751+0000 7fa725c00640 1 -- 172.21.15.102:0/2284232486 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c02c900 con 0x563346763d70 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.751+0000 7fa725c00640 1 -- 172.21.15.102:0/2284232486 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c02ce50 con 0x563346763d70 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.751+0000 7fa725c00640 1 -- 172.21.15.102:0/2284232486 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c03f440 con 0x563346763d70 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 -- 172.21.15.102:0/2284232486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 msgr2=0x563346741270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x563346741270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009e50 tx=0x7fa72c02e000 comp rx=0 tx=0).stop 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 -- 172.21.15.102:0/2284232486 shutdown_connections 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x563346741270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2284232486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633467135f0 0x563346715ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 -- 172.21.15.102:0/2284232486 >> 172.21.15.102:0/2284232486 conn(0x563346714d60 msgr2=0x5633467176a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 -- 172.21.15.102:0/2284232486 shutdown_connections 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.752+0000 7fa73eb69740 1 -- 172.21.15.102:0/2284232486 wait complete. 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.753+0000 7fa73eb69740 1 -- start start 2024-04-20T21:17:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.753+0000 7fa73eb69740 1 Processor -- start 2024-04-20T21:17:33.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x56334675b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346761cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa73eb69740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5633466385e0 con 0x563346762200 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa73eb69740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563346766d50 con 0x563346763d70 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa73eb69740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x56334675bcf0 con 0x563346766880 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa73eb69740 10 client.0 _ll_register_callbacks cb 0x563346390ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa739e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa739400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x56334675b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa738a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346761cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa739e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35110/0 (socket says 172.21.15.102:35110) 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa739e00640 1 -- 172.21.15.102:0/2187769996 learned_addr learned my addr 172.21.15.102:0/2187769996 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa738a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346761cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:36006/0 (socket says 172.21.15.102:36006) 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.754+0000 7fa739400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x56334675b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:46406/0 (socket says 172.21.15.102:46406) 2024-04-20T21:17:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa739e00640 1 -- 172.21.15.102:0/2187769996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 msgr2=0x563346761cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa739e00640 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346761cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa739e00640 1 -- 172.21.15.102:0/2187769996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 msgr2=0x56334675b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa739e00640 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x56334675b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa739e00640 1 -- 172.21.15.102:0/2187769996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334645b6b0 con 0x563346762200 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa739400640 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x56334675b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa739e00640 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675ddc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009f40 tx=0x7fa72c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a053a421254f632 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa71a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa71a000640 1 -- 172.21.15.102:0/2187769996 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c03fc30 con 0x563346762200 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa71a000640 1 -- 172.21.15.102:0/2187769996 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa72c03fdd0 con 0x563346762200 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa73eb69740 1 -- 172.21.15.102:0/2187769996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa72c02ce50 con 0x563346762200 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa71a000640 1 -- 172.21.15.102:0/2187769996 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fa72c056050 con 0x563346762200 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa73eb69740 10 client.4722 fetch_fsmap learned FSMap version 9 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.755+0000 7fa73eb69740 1 -- 172.21.15.102:0/2187769996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56334673fd00 con 0x563346762200 2024-04-20T21:17:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa71a000640 1 -- 172.21.15.102:0/2187769996 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c02ce50 con 0x563346762200 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa71a000640 1 -- 172.21.15.102:0/2187769996 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 9) ==== 4713+0+0 (secure 0 0 0) 0x7fa72c059020 con 0x563346762200 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa73eb69740 10 client.4722 fetch_fsmap finished waiting for FSMap version 9 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa73eb69740 10 client.4722 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa73eb69740 20 client.4722 populate_metadata read hostname 'smithi102' 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x56334673ed30 0x563346763080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa73eb69740 4 client.4722 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa73eb69740 1 -- 172.21.15.102:0/2187769996 --> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56334668c3f0 con 0x56334673ed30 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.756+0000 7fa739400640 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x56334673ed30 0x563346763080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.757+0000 7fa739400640 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x56334673ed30 0x563346763080 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4218666897765e1a server_cookie=40516f3ec3786c23 in_seq=0 out_seq=0 2024-04-20T21:17:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.757+0000 7fa71a000640 10 client.4722 ms_handle_connect on v2:172.21.15.102:6834/2873333699 2024-04-20T21:17:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.757+0000 7fa71a000640 1 -- 172.21.15.102:0/2187769996 <== mds.0 v2:172.21.15.102:6834/2873333699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56334668c3f0 con 0x56334673ed30 2024-04-20T21:17:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.757+0000 7fa71a000640 10 client.4722 handle_command_reply: tid=0 2024-04-20T21:17:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 client.4722 shutdown 2024-04-20T21:17:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 -- 172.21.15.102:0/2187769996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 msgr2=0x56334675ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675ddc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009f40 tx=0x7fa72c004ab0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 -- 172.21.15.102:0/2187769996 shutdown_connections 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x56334673ed30 0x563346763080 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675ddc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346761cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 --2- 172.21.15.102:0/2187769996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x56334675b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 -- 172.21.15.102:0/2187769996 >> 172.21.15.102:0/2187769996 conn(0x5633467135f0 msgr2=0x5633467183b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa71a000640 0 client.4722 ms_handle_reset on v2:172.21.15.102:6834/2873333699 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 -- 172.21.15.102:0/2187769996 shutdown_connections 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 1 -- 172.21.15.102:0/2187769996 wait complete. 2024-04-20T21:17:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.758+0000 7fa73eb69740 20 client.4722 trim_cache size 0 max 16384 2024-04-20T21:17:33.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.778+0000 7fa73eb69740 1 Processor -- start 2024-04-20T21:17:33.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.778+0000 7fa73eb69740 1 -- start start 2024-04-20T21:17:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.778+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346762200 0x5633467349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x5633467823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346782910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa73eb69740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa7341f92f0 con 0x563346763d70 2024-04-20T21:17:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa73eb69740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56334675ba80 con 0x563346762200 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa73eb69740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x56334673ebb0 con 0x563346766880 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa738a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x5633467823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa738a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x5633467823d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35120/0 (socket says 172.21.15.102:35120) 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa738a00640 1 -- 172.21.15.102:0/440225374 learned_addr learned my addr 172.21.15.102:0/440225374 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa739400640 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346762200 0x5633467349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa739e00640 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346782910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa739400640 1 -- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 msgr2=0x563346782910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa739400640 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346782910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa739400640 1 -- 172.21.15.102:0/440225374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 msgr2=0x5633467823d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa739400640 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x5633467823d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa739400640 1 -- 172.21.15.102:0/440225374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563346765370 con 0x563346762200 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.779+0000 7fa739400640 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346762200 0x5633467349a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa728089340 tx=0x7fa728089240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e94b88a31addd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa725200640 1 -- 172.21.15.102:0/440225374 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728056f20 con 0x563346762200 2024-04-20T21:17:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa725200640 1 -- 172.21.15.102:0/440225374 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7280612c0 con 0x563346762200 2024-04-20T21:17:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa725200640 1 -- 172.21.15.102:0/440225374 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728058070 con 0x563346762200 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 -- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346762200 msgr2=0x5633467349a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346762200 0x5633467349a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa728089340 tx=0x7fa728089240 comp rx=0 tx=0).stop 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 -- 172.21.15.102:0/440225374 shutdown_connections 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x563346782910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346763d70 0x5633467823d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 --2- 172.21.15.102:0/440225374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346762200 0x5633467349a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 -- 172.21.15.102:0/440225374 >> 172.21.15.102:0/440225374 conn(0x5633467135f0 msgr2=0x563346770da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 -- 172.21.15.102:0/440225374 shutdown_connections 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.780+0000 7fa73eb69740 1 -- 172.21.15.102:0/440225374 wait complete. 2024-04-20T21:17:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.781+0000 7fa73eb69740 1 -- start start 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.781+0000 7fa73eb69740 1 Processor -- start 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x5633467d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa73eb69740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa73eb69740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563346762f00 con 0x563346762200 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa73eb69740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5633467628b0 con 0x563346763d70 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa73eb69740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x5633467635d0 con 0x563346766880 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa73eb69740 10 client.0 _ll_register_callbacks cb 0x7fa7280589f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa739400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.782+0000 7fa738a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x5633467d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa738a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x5633467d51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:46420/0 (socket says 172.21.15.102:46420) 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa738a00640 1 -- 172.21.15.102:0/1630479335 learned_addr learned my addr 172.21.15.102:0/1630479335 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa739e00640 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa738a00640 1 -- 172.21.15.102:0/1630479335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 msgr2=0x5633467d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa738a00640 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467d5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa738a00640 1 -- 172.21.15.102:0/1630479335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 msgr2=0x56334675a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa738a00640 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa738a00640 1 -- 172.21.15.102:0/1630479335 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334670e870 con 0x563346763d70 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa739400640 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa738a00640 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x5633467d51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa73414c6e0 tx=0x7fa7200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cffd55b0c2240d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa71a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa71a000640 1 -- 172.21.15.102:0/1630479335 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720023b10 con 0x563346763d70 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.783+0000 7fa71a000640 1 -- 172.21.15.102:0/1630479335 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa72000f050 con 0x563346763d70 2024-04-20T21:17:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 1 -- 172.21.15.102:0/1630479335 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7280612c0 con 0x563346763d70 2024-04-20T21:17:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa71a000640 1 -- 172.21.15.102:0/1630479335 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fa720023cb0 con 0x563346763d70 2024-04-20T21:17:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 10 client.4831 fetch_fsmap learned FSMap version 9 2024-04-20T21:17:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 1 -- 172.21.15.102:0/1630479335 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x563346717b80 con 0x563346763d70 2024-04-20T21:17:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa71a000640 1 -- 172.21.15.102:0/1630479335 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7280612c0 con 0x563346763d70 2024-04-20T21:17:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa71a000640 1 -- 172.21.15.102:0/1630479335 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 9) ==== 4713+0+0 (secure 0 0 0) 0x7fa720035050 con 0x563346763d70 2024-04-20T21:17:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 10 client.4831 fetch_fsmap finished waiting for FSMap version 9 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 10 client.4831 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 20 client.4831 populate_metadata read hostname 'smithi102' 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x563346741340 0x5633467e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 4 client.4831 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa73eb69740 1 -- 172.21.15.102:0/1630479335 --> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] -- command(tid 0: {"prefix": "status"}) -- 0x563346759e00 con 0x563346741340 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.784+0000 7fa739400640 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x563346741340 0x5633467e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa739400640 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x563346741340 0x5633467e48d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d05b2f54e39bae3 server_cookie=637b8261756d42f9 in_seq=0 out_seq=0 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa71a000640 10 client.4831 ms_handle_connect on v2:172.21.15.102:6834/2873333699 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa71a000640 1 -- 172.21.15.102:0/1630479335 <== mds.0 v2:172.21.15.102:6834/2873333699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x563346759e00 con 0x563346741340 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa71a000640 10 client.4831 handle_command_reply: tid=0 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 client.4831 shutdown 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 -- 172.21.15.102:0/1630479335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 msgr2=0x5633467d51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x5633467d51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa73414c6e0 tx=0x7fa7200046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 -- 172.21.15.102:0/1630479335 shutdown_connections 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.102:6834/2873333699,v1:172.21.15.102:6835/2873333699] conn(0x563346741340 0x5633467e48d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563346766880 0x5633467d5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563346763d70 0x5633467d51c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 --2- 172.21.15.102:0/1630479335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563346762200 0x56334675a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa71a000640 0 client.4831 ms_handle_reset on v2:172.21.15.102:6834/2873333699 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.785+0000 7fa73eb69740 1 -- 172.21.15.102:0/1630479335 >> 172.21.15.102:0/1630479335 conn(0x5633467135f0 msgr2=0x563346719fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73eb69740 1 -- 172.21.15.102:0/1630479335 shutdown_connections 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73eb69740 1 -- 172.21.15.102:0/1630479335 wait complete. 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73eb69740 20 client.4831 trim_cache size 0 max 16384 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-20T21:17:33.789 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "1c72cacb-b320-4a89-8b72-ff890245905f", 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 0, 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 4300, 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 39.226138798000001, 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 9, 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 24, 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 24, 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 44.962178844 2024-04-20T21:17:33.790 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-20T21:17:33.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 -- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa71406c8f0 msgr2=0x7fa71406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa71406c8f0 0x7fa71406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa734150370 tx=0x7fa720008040 comp rx=0 tx=0).stop 2024-04-20T21:17:33.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 -- 172.21.15.102:0/2144276322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa734073b20 msgr2=0x7fa734075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:33.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa734073b20 0x7fa734075110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c4b0 tx=0x7fa72800c970 comp rx=0 tx=0).stop 2024-04-20T21:17:33.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 -- 172.21.15.102:0/2144276322 shutdown_connections 2024-04-20T21:17:33.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa734153180 0x7fa73414eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa71406c8f0 0x7fa71406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7340caec0 0x7fa73414e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 --2- 172.21.15.102:0/2144276322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa734073b20 0x7fa734075110 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:33.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.786+0000 7fa73a800640 1 -- 172.21.15.102:0/2144276322 >> 172.21.15.102:0/2144276322 conn(0x7fa7340ca3a0 msgr2=0x7fa7340cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:33.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.787+0000 7fa73a800640 1 -- 172.21.15.102:0/2144276322 shutdown_connections 2024-04-20T21:17:33.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:33.787+0000 7fa73a800640 1 -- 172.21.15.102:0/2144276322 wait complete. 2024-04-20T21:17:33.802 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T21:17:33.811 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T21:17:33.811 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T21:17:33.811 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T21:17:33.811 INFO:teuthology.run_tasks:Running task kclient... 2024-04-20T21:17:33.851 INFO:tasks.kclient:Mounting kernel clients... 2024-04-20T21:17:33.851 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-20T21:17:33.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:33.851 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-04-20T21:17:33.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-20T21:17:33.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:33.907 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T21:17:33.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:33.986 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-04-20T21:17:34.097 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-20T21:17:34.097 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T21:17:34.097 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:17:34.097 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:17:34.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.274+0000 7f5f74600640 1 Processor -- start 2024-04-20T21:17:34.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.274+0000 7f5f74600640 1 -- start start 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f74600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f6c0c9eb0 0x7f5f6c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f74600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c073a10 0x7f5f6c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f74600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6c1536d0 con 0x7f5f6c14fa50 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f74600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5f6c159500 con 0x7f5f6c0c9eb0 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f74600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c0c76c0 con 0x7f5f6c073a10 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f71200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f6c0c9eb0 0x7f5f6c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f71200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f6c0c9eb0 0x7f5f6c0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35146/0 (socket says 172.21.15.102:35146) 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f71200640 1 -- 172.21.15.102:0/2163577842 learned_addr learned my addr 172.21.15.102:0/2163577842 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f72600640 1 --2- 172.21.15.102:0/2163577842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c073a10 0x7f5f6c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:34.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.276+0000 7f5f71c00640 1 --1- 172.21.15.102:0/2163577842 >> v1:172.21.15.193:6789/0 conn(0x7f5f6c14fa50 0x7f5f6c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51414/0 (socket says 172.21.15.102:51414) 2024-04-20T21:17:34.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.277+0000 7f5f71200640 1 -- 172.21.15.102:0/2163577842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c073a10 msgr2=0x7f5f6c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:34.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.277+0000 7f5f71200640 1 --2- 172.21.15.102:0/2163577842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c073a10 0x7f5f6c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.277+0000 7f5f71200640 1 -- 172.21.15.102:0/2163577842 >> v1:172.21.15.193:6789/0 conn(0x7f5f6c14fa50 legacy=0x7f5f6c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:34.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.277+0000 7f5f71200640 1 -- 172.21.15.102:0/2163577842 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c0c7960 con 0x7f5f6c0c9eb0 2024-04-20T21:17:34.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.277+0000 7f5f72600640 1 --2- 172.21.15.102:0/2163577842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c073a10 0x7f5f6c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:34.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.278+0000 7f5f71200640 1 --2- 172.21.15.102:0/2163577842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c0c9eb0 0x7f5f6c0c4830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009870 tx=0x7f5f5802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37414ec4e4304fec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:34.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.278+0000 7f5f6be00640 1 -- 172.21.15.102:0/2163577842 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5802f830 con 0x7f5f6c0c9eb0 2024-04-20T21:17:34.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.278+0000 7f5f6be00640 1 -- 172.21.15.102:0/2163577842 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5802f9d0 con 0x7f5f6c0c9eb0 2024-04-20T21:17:34.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.278+0000 7f5f6be00640 1 -- 172.21.15.102:0/2163577842 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5802fda0 con 0x7f5f6c0c9eb0 2024-04-20T21:17:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.279+0000 7f5f74600640 1 -- 172.21.15.102:0/2163577842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c0c9eb0 msgr2=0x7f5f6c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.279+0000 7f5f74600640 1 --2- 172.21.15.102:0/2163577842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c0c9eb0 0x7f5f6c0c4830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009870 tx=0x7f5f5802ba10 comp rx=0 tx=0).stop 2024-04-20T21:17:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.279+0000 7f5f74600640 1 -- 172.21.15.102:0/2163577842 shutdown_connections 2024-04-20T21:17:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.279+0000 7f5f74600640 1 --2- 172.21.15.102:0/2163577842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c073a10 0x7f5f6c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.279+0000 7f5f74600640 1 --2- 172.21.15.102:0/2163577842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c0c9eb0 0x7f5f6c0c4830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.279+0000 7f5f74600640 1 -- 172.21.15.102:0/2163577842 >> 172.21.15.102:0/2163577842 conn(0x7f5f6c0c9390 msgr2=0x7f5f6c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.279+0000 7f5f74600640 1 -- 172.21.15.102:0/2163577842 shutdown_connections 2024-04-20T21:17:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.279+0000 7f5f74600640 1 -- 172.21.15.102:0/2163577842 wait complete. 2024-04-20T21:17:34.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.281+0000 7f5f74600640 1 Processor -- start 2024-04-20T21:17:34.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.281+0000 7f5f74600640 1 -- start start 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f74600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c073a10 0x7f5f6c14bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f74600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c0c9eb0 0x7f5f6c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f74600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f6c14fa50 0x7f5f6c1f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f74600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0c7c00 con 0x7f5f6c073a10 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f74600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0c3d90 con 0x7f5f6c14fa50 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f74600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c0c4030 con 0x7f5f6c0c9eb0 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c073a10 0x7f5f6c14bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c073a10 0x7f5f6c14bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35148/0 (socket says 172.21.15.102:35148) 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71c00640 1 -- 172.21.15.102:0/138222489 learned_addr learned my addr 172.21.15.102:0/138222489 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71200640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c0c9eb0 0x7f5f6c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f72600640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f6c14fa50 0x7f5f6c1f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71c00640 1 -- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c0c9eb0 msgr2=0x7f5f6c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:34.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71c00640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c0c9eb0 0x7f5f6c1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71c00640 1 -- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f6c14fa50 msgr2=0x7f5f6c1f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71c00640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f6c14fa50 0x7f5f6c1f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.282+0000 7f5f71c00640 1 -- 172.21.15.102:0/138222489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c157160 con 0x7f5f6c073a10 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.283+0000 7f5f71c00640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c073a10 0x7f5f6c14bf70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c400 tx=0x7f5f6000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.283+0000 7f5f69600640 1 -- 172.21.15.102:0/138222489 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600074e0 con 0x7f5f6c073a10 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.283+0000 7f5f69600640 1 -- 172.21.15.102:0/138222489 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f60007680 con 0x7f5f6c073a10 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.283+0000 7f5f69600640 1 -- 172.21.15.102:0/138222489 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60004a00 con 0x7f5f6c073a10 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.283+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c157440 con 0x7f5f6c073a10 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.283+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c157ff0 con 0x7f5f6c073a10 2024-04-20T21:17:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.284+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c074340 con 0x7f5f6c073a10 2024-04-20T21:17:34.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.285+0000 7f5f69600640 1 -- 172.21.15.102:0/138222489 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f6001a020 con 0x7f5f6c073a10 2024-04-20T21:17:34.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.286+0000 7f5f69600640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:34.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.286+0000 7f5f69600640 1 -- 172.21.15.102:0/138222489 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5f6008b6c0 con 0x7f5f6c073a10 2024-04-20T21:17:34.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.287+0000 7f5f71200640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.287+0000 7f5f71200640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f58036000 tx=0x7f5f58008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:34.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.291+0000 7f5f69600640 1 -- 172.21.15.102:0/138222489 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5f6c1536d0 con 0x7f5f6c073a10 2024-04-20T21:17:34.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.536+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5f6c0c3690 con 0x7f5f6c073a10 2024-04-20T21:17:34.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.537+0000 7f5f69600640 1 -- 172.21.15.102:0/138222489 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5f60059680 con 0x7f5f6c073a10 2024-04-20T21:17:34.537 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T21:17:34.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f4c06c8f0 msgr2=0x7f5f4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:34.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f58036000 tx=0x7f5f58008030 comp rx=0 tx=0).stop 2024-04-20T21:17:34.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c073a10 msgr2=0x7f5f6c14bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:34.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c073a10 0x7f5f6c14bf70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c400 tx=0x7f5f6000c800 comp rx=0 tx=0).stop 2024-04-20T21:17:34.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 shutdown_connections 2024-04-20T21:17:34.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f6c14fa50 0x7f5f6c1f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f6c0c9eb0 0x7f5f6c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 --2- 172.21.15.102:0/138222489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f6c073a10 0x7f5f6c14bf70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:34.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 >> 172.21.15.102:0/138222489 conn(0x7f5f6c0c9390 msgr2=0x7f5f6c0c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:34.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 shutdown_connections 2024-04-20T21:17:34.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34.540+0000 7f5f74600640 1 -- 172.21.15.102:0/138222489 wait complete. 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi102.front.sepia.ceph.com 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:17:34.559 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:17:34.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:34.560 DEBUG:teuthology.orchestra.run.smithi102:> ip addr 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 ::1/128 scope host 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dc brd ff:ff:ff:ff:ff:ff 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f0 2024-04-20T21:17:34.606 INFO:teuthology.orchestra.run.smithi102.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:17:34.607 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dd brd ff:ff:ff:ff:ff:ff 2024-04-20T21:17:34.607 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f1 2024-04-20T21:17:34.607 INFO:teuthology.orchestra.run.smithi102.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:17:34.607 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:68 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:17:34.608 INFO:teuthology.orchestra.run.smithi102.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:17:34.608 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:69 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:17:34.608 INFO:teuthology.orchestra.run.smithi102.stdout: inet 172.21.15.102/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:17:34.608 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft 42782sec preferred_lft 42782sec 2024-04-20T21:17:34.608 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::ec4:7aff:fe8f:d769/64 scope link 2024-04-20T21:17:34.608 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:17:34.608 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T21:17:34.608 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:34.608 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:34.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add name ceph-brx type bridge 2024-04-20T21:17:34.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr flush dev ceph-brx 2024-04-20T21:17:34.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set ceph-brx up 2024-04-20T21:17:34.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T21:17:34.609 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:34.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:34.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:34.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:34.898 DEBUG:teuthology.orchestra.run.smithi102:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:17:34.954 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-04-20T21:17:34.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:34.957 DEBUG:teuthology.orchestra.run.smithi102:> route 2024-04-20T21:17:34.999 INFO:teuthology.orchestra.run.smithi102.stdout:Kernel IP routing table 2024-04-20T21:17:34.999 INFO:teuthology.orchestra.run.smithi102.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:17:34.999 INFO:teuthology.orchestra.run.smithi102.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:17:34.999 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:17:34.999 INFO:teuthology.orchestra.run.smithi102.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:17:35.000 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:35.000 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:35.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:17:35.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:17:35.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:17:35.000 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:35.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:35.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:35.239 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-04-20T21:17:35.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:35.285 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list-id 2024-04-20T21:17:35.383 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:35.383 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:35.383 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:17:35.383 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T21:17:35.383 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:35.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:35.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:35.573 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T21:17:35.573 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:35.573 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:35.573 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T21:17:35.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T21:17:35.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T21:17:35.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T21:17:35.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T21:17:35.574 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:35.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:35.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:35.837 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:17:35.837 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-20T21:17:35.837 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set brx.0 up 2024-04-20T21:17:35.837 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T21:17:35.837 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-20T21:17:35.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:36.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:36.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:17:36.034 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:17:36.075 INFO:teuthology.orchestra.run.smithi102.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:17:36.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:17:36.075 DEBUG:teuthology.orchestra.run.smithi102:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:17:36.165 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-20T21:17:36.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:36.165 DEBUG:teuthology.orchestra.run.smithi102:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-20T21:17:36.278 INFO:teuthology.orchestra.run.smithi102.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-20T21:17:36.279 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-20T21:17:37.000 INFO:teuthology.orchestra.run.smithi102.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-20T21:17:37.000 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-20T21:17:37.000 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: resolved to: "172.21.15.102:3300,172.21.15.193:3300,172.21.15.193:3301" 2024-04-20T21:17:37.000 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: trying mount with new device syntax: 0@1c72cacb-b320-4a89-8b72-ff890245905f.cephfs=/ 2024-04-20T21:17:37.000 INFO:teuthology.orchestra.run.smithi102.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.102:3300/172.21.15.193:3300/172.21.15.193:3301" will pass to kernel 2024-04-20T21:17:37.003 INFO:teuthology.orchestra.run.smithi102.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-20T21:17:37.003 INFO:teuthology.orchestra.run.smithi102.stdout: You just mounted a file system that supports labels which does not 2024-04-20T21:17:37.003 INFO:teuthology.orchestra.run.smithi102.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-20T21:17:37.003 INFO:teuthology.orchestra.run.smithi102.stdout: applications will generate AVC messages and not be allowed access to 2024-04-20T21:17:37.003 INFO:teuthology.orchestra.run.smithi102.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-20T21:17:37.007 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-20T21:17:37.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:17:37.007 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:17:37.079 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:17:37.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:17:38.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:38.113 INFO:teuthology.orchestra.run.smithi102.stdout:client4661 2024-04-20T21:17:38.113 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:17:38.113 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:17:38.168 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:17:38.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:17:38.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:38.293 INFO:teuthology.orchestra.run.smithi102.stdout:client4661 2024-04-20T21:17:38.294 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:17:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:17:38.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:38.412 INFO:teuthology.orchestra.run.smithi102.stdout:client4661 2024-04-20T21:17:38.413 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1c72cacb-b320-4a89-8b72-ff890245905f.client4661/status') 2024-04-20T21:17:38.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:17:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:instance: client.4661 (3)172.21.15.102:0/3755986011 2024-04-20T21:17:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:blocklisted: false 2024-04-20T21:17:38.545 INFO:teuthology.orchestra.run.smithi102.stderr:0+1 records in 2024-04-20T21:17:38.545 INFO:teuthology.orchestra.run.smithi102.stderr:0+1 records out 2024-04-20T21:17:38.545 INFO:teuthology.orchestra.run.smithi102.stderr:71 bytes copied, 7.879e-05 s, 901 kB/s 2024-04-20T21:17:38.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:17:38.558 INFO:teuthology.run_tasks:Running task mds_thrash... 2024-04-20T21:17:38.589 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:38.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.789+0000 7fa564e00640 1 Processor -- start 2024-04-20T21:17:38.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.790+0000 7fa564e00640 1 -- start start 2024-04-20T21:17:38.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.791+0000 7fa564e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5600c75a0 0x7fa5600c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:38.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.791+0000 7fa564e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa560153560 con 0x7fa560073c70 2024-04-20T21:17:38.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.791+0000 7fa564e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5600c7a70 con 0x7fa560153180 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.791+0000 7fa564e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa5600c7d30 con 0x7fa5600c75a0 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.791+0000 7fa55f400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fa560153180 0x7fa560158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38106/0 (socket says 172.21.15.102:38106) 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.791+0000 7fa55f400640 1 -- 172.21.15.102:0/3825180165 learned_addr learned my addr 172.21.15.102:0/3825180165 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.791+0000 7fa556a00640 1 --2- 172.21.15.102:0/3825180165 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5600c75a0 0x7fa5600c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.791+0000 7fa55ea00640 1 -- 172.21.15.102:0/3825180165 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762481520 0 0) 0x7fa5600c7a70 con 0x7fa560153180 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.792+0000 7fa55ea00640 1 -- 172.21.15.102:0/3825180165 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa544003520 con 0x7fa560153180 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.792+0000 7fa55ea00640 1 -- 172.21.15.102:0/3825180165 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3810233059 0 0) 0x7fa560153560 con 0x7fa560073c70 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.792+0000 7fa556a00640 1 -- 172.21.15.102:0/3825180165 >> v1:172.21.15.193:6789/0 conn(0x7fa560153180 legacy=0x7fa560158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.792+0000 7fa556a00640 1 -- 172.21.15.102:0/3825180165 >> v1:172.21.15.102:6789/0 conn(0x7fa560073c70 legacy=0x7fa5601637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:38.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.792+0000 7fa556a00640 1 -- 172.21.15.102:0/3825180165 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56015c4d0 con 0x7fa5600c75a0 2024-04-20T21:17:38.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.793+0000 7fa556a00640 1 --2- 172.21.15.102:0/3825180165 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5600c75a0 0x7fa5600c5f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa5480097d0 tx=0x7fa54802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1738666692e1057 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:38.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.793+0000 7fa55ea00640 1 -- 172.21.15.102:0/3825180165 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548035020 con 0x7fa5600c75a0 2024-04-20T21:17:38.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.793+0000 7fa55ea00640 1 -- 172.21.15.102:0/3825180165 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa548031890 con 0x7fa5600c75a0 2024-04-20T21:17:38.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.793+0000 7fa55ea00640 1 -- 172.21.15.102:0/3825180165 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548031c40 con 0x7fa5600c75a0 2024-04-20T21:17:38.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.794+0000 7fa564e00640 1 -- 172.21.15.102:0/3825180165 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5600c75a0 msgr2=0x7fa5600c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:38.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.794+0000 7fa564e00640 1 --2- 172.21.15.102:0/3825180165 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5600c75a0 0x7fa5600c5f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa5480097d0 tx=0x7fa54802b930 comp rx=0 tx=0).stop 2024-04-20T21:17:38.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.794+0000 7fa564e00640 1 -- 172.21.15.102:0/3825180165 shutdown_connections 2024-04-20T21:17:38.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.794+0000 7fa564e00640 1 --2- 172.21.15.102:0/3825180165 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5600c75a0 0x7fa5600c5f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:38.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.794+0000 7fa564e00640 1 -- 172.21.15.102:0/3825180165 >> 172.21.15.102:0/3825180165 conn(0x7fa5600cc820 msgr2=0x7fa5600ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:38.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.794+0000 7fa564e00640 1 -- 172.21.15.102:0/3825180165 shutdown_connections 2024-04-20T21:17:38.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.794+0000 7fa564e00640 1 -- 172.21.15.102:0/3825180165 wait complete. 2024-04-20T21:17:38.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.796+0000 7fa564e00640 1 Processor -- start 2024-04-20T21:17:38.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.796+0000 7fa564e00640 1 -- start start 2024-04-20T21:17:38.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa564e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa560073c70 0x7fa56014ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:38.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa564e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5600c75a0 0x7fa56014e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:38.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa564e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa560153180 0x7fa56014a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:38.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa564e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa560159550 con 0x7fa5600c75a0 2024-04-20T21:17:38.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa564e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa56015ca90 con 0x7fa560073c70 2024-04-20T21:17:38.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa564e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa56015d290 con 0x7fa560153180 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa556a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5600c75a0 0x7fa56014e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa556a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5600c75a0 0x7fa56014e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48254/0 (socket says 172.21.15.102:48254) 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa556a00640 1 -- 172.21.15.102:0/3687695042 learned_addr learned my addr 172.21.15.102:0/3687695042 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa55f400640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa560073c70 0x7fa56014ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.797+0000 7fa55fe00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa560153180 0x7fa56014a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa556a00640 1 -- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa560153180 msgr2=0x7fa56014a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa556a00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa560153180 0x7fa56014a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa556a00640 1 -- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa560073c70 msgr2=0x7fa56014ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa556a00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa560073c70 0x7fa56014ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa556a00640 1 -- 172.21.15.102:0/3687695042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56014b6f0 con 0x7fa5600c75a0 2024-04-20T21:17:38.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa556a00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5600c75a0 0x7fa56014e2c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5480098f0 tx=0x7fa5480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:38.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa557e00640 1 -- 172.21.15.102:0/3687695042 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548035040 con 0x7fa5600c75a0 2024-04-20T21:17:38.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa557e00640 1 -- 172.21.15.102:0/3687695042 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa548003820 con 0x7fa5600c75a0 2024-04-20T21:17:38.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56014b9b0 con 0x7fa5600c75a0 2024-04-20T21:17:38.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.798+0000 7fa557e00640 1 -- 172.21.15.102:0/3687695042 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548003bd0 con 0x7fa5600c75a0 2024-04-20T21:17:38.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.799+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56014bf70 con 0x7fa5600c75a0 2024-04-20T21:17:38.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.800+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56004fd60 con 0x7fa5600c75a0 2024-04-20T21:17:38.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.801+0000 7fa557e00640 1 -- 172.21.15.102:0/3687695042 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa548038020 con 0x7fa5600c75a0 2024-04-20T21:17:38.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.802+0000 7fa557e00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa53c06c920 0x7fa53c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:38.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.802+0000 7fa557e00640 1 -- 172.21.15.102:0/3687695042 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa5480ae150 con 0x7fa5600c75a0 2024-04-20T21:17:38.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.803+0000 7fa55f400640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa53c06c920 0x7fa53c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:38.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.803+0000 7fa55f400640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa53c06c920 0x7fa53c06ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa550005880 tx=0x7fa550009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:38.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:38.807+0000 7fa557e00640 1 -- 172.21.15.102:0/3687695042 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa54802d040 con 0x7fa5600c75a0 2024-04-20T21:17:39.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.008+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5601528b0 con 0x7fa5600c75a0 2024-04-20T21:17:39.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.009+0000 7fa557e00640 1 -- 172.21.15.102:0/3687695042 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18894 (secure 0 0 0) 0x7fa54807baa0 con 0x7fa5600c75a0 2024-04-20T21:17:39.009 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:39.009 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:17:35.869503+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa53c06c920 msgr2=0x7fa53c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa53c06c920 0x7fa53c06ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa550005880 tx=0x7fa550009040 comp rx=0 tx=0).stop 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5600c75a0 msgr2=0x7fa56014e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5600c75a0 0x7fa56014e2c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5480098f0 tx=0x7fa5480046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 shutdown_connections 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa560153180 0x7fa56014a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5600c75a0 0x7fa56014e2c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa53c06c920 0x7fa53c06ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 --2- 172.21.15.102:0/3687695042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa560073c70 0x7fa56014ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 >> 172.21.15.102:0/3687695042 conn(0x7fa5600cc820 msgr2=0x7fa5601519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.012+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 shutdown_connections 2024-04-20T21:17:39.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.013+0000 7fa564e00640 1 -- 172.21.15.102:0/3687695042 wait complete. 2024-04-20T21:17:39.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:16:51.795761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:16:52.983434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T21:16:56.582297+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T21:17:39.032 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:39.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.239+0000 7f2a3a400640 1 Processor -- start 2024-04-20T21:17:39.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.240+0000 7f2a3a400640 1 -- start start 2024-04-20T21:17:39.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.241+0000 7f2a3a400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a341522e0 0x7f2a34158990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.241+0000 7f2a3a400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3415b520 con 0x7f2a3415bce0 2024-04-20T21:17:39.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.241+0000 7f2a3a400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a340712b0 con 0x7f2a3415c120 2024-04-20T21:17:39.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.241+0000 7f2a3a400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2a34071060 con 0x7f2a341522e0 2024-04-20T21:17:39.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.242+0000 7f2a33e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f2a3415c120 0x7f2a340c8ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38110/0 (socket says 172.21.15.102:38110) 2024-04-20T21:17:39.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.242+0000 7f2a33e00640 1 -- 172.21.15.102:0/2988299208 learned_addr learned my addr 172.21.15.102:0/2988299208 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:39.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.242+0000 7f2a33400640 1 --2- 172.21.15.102:0/2988299208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a341522e0 0x7f2a34158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.242+0000 7f2a33400640 1 -- 172.21.15.102:0/2988299208 >> v1:172.21.15.193:6789/0 conn(0x7f2a3415c120 legacy=0x7f2a340c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.242+0000 7f2a33400640 1 -- 172.21.15.102:0/2988299208 >> v1:172.21.15.102:6789/0 conn(0x7f2a3415bce0 legacy=0x7f2a340c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.242+0000 7f2a32a00640 1 -- 172.21.15.102:0/2988299208 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3784508635 0 0) 0x7f2a340712b0 con 0x7f2a3415c120 2024-04-20T21:17:39.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.242+0000 7f2a33400640 1 -- 172.21.15.102:0/2988299208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a34071590 con 0x7f2a341522e0 2024-04-20T21:17:39.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.243+0000 7f2a33400640 1 --2- 172.21.15.102:0/2988299208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a341522e0 0x7f2a34158990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a240097d0 tx=0x7f2a2402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720c6895511c78f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.243+0000 7f2a32a00640 1 -- 172.21.15.102:0/2988299208 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24034070 con 0x7f2a341522e0 2024-04-20T21:17:39.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.243+0000 7f2a32a00640 1 -- 172.21.15.102:0/2988299208 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24030800 con 0x7f2a341522e0 2024-04-20T21:17:39.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.244+0000 7f2a32a00640 1 -- 172.21.15.102:0/2988299208 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24030bd0 con 0x7f2a341522e0 2024-04-20T21:17:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.244+0000 7f2a3a400640 1 -- 172.21.15.102:0/2988299208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a341522e0 msgr2=0x7f2a34158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.244+0000 7f2a3a400640 1 --2- 172.21.15.102:0/2988299208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a341522e0 0x7f2a34158990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a240097d0 tx=0x7f2a2402b940 comp rx=0 tx=0).stop 2024-04-20T21:17:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.244+0000 7f2a3a400640 1 -- 172.21.15.102:0/2988299208 shutdown_connections 2024-04-20T21:17:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.244+0000 7f2a3a400640 1 --2- 172.21.15.102:0/2988299208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a341522e0 0x7f2a34158990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.244+0000 7f2a3a400640 1 -- 172.21.15.102:0/2988299208 >> 172.21.15.102:0/2988299208 conn(0x7f2a340c9390 msgr2=0x7f2a340c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.244+0000 7f2a3a400640 1 -- 172.21.15.102:0/2988299208 shutdown_connections 2024-04-20T21:17:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.245+0000 7f2a3a400640 1 -- 172.21.15.102:0/2988299208 wait complete. 2024-04-20T21:17:39.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.247+0000 7f2a3a400640 1 Processor -- start 2024-04-20T21:17:39.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.247+0000 7f2a3a400640 1 -- start start 2024-04-20T21:17:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a3a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a341522e0 0x7f2a3413ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a3a400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3415bce0 0x7f2a34141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a3a400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a3415c120 0x7f2a341421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a3a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34071870 con 0x7f2a341522e0 2024-04-20T21:17:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a3a400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a341526c0 con 0x7f2a3415bce0 2024-04-20T21:17:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a3a400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2a340c5060 con 0x7f2a3415c120 2024-04-20T21:17:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a33400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a341522e0 0x7f2a3413ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a33400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a341522e0 0x7f2a3413ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48256/0 (socket says 172.21.15.102:48256) 2024-04-20T21:17:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a33400640 1 -- 172.21.15.102:0/1260069015 learned_addr learned my addr 172.21.15.102:0/1260069015 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a33e00640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a3415c120 0x7f2a341421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.248+0000 7f2a2aa00640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3415bce0 0x7f2a34141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.249+0000 7f2a33400640 1 -- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a3415c120 msgr2=0x7f2a341421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.249+0000 7f2a33400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a3415c120 0x7f2a341421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.249+0000 7f2a33400640 1 -- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3415bce0 msgr2=0x7f2a34141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.249+0000 7f2a33400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3415bce0 0x7f2a34141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.249+0000 7f2a33400640 1 -- 172.21.15.102:0/1260069015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a340c8aa0 con 0x7f2a341522e0 2024-04-20T21:17:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.249+0000 7f2a33e00640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a3415c120 0x7f2a341421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.249+0000 7f2a33400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a341522e0 0x7f2a3413ea50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a240098f0 tx=0x7f2a240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.249+0000 7f2a2be00640 1 -- 172.21.15.102:0/1260069015 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24034040 con 0x7f2a341522e0 2024-04-20T21:17:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.250+0000 7f2a2be00640 1 -- 172.21.15.102:0/1260069015 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a240443f0 con 0x7f2a341522e0 2024-04-20T21:17:39.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.250+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3415ae60 con 0x7f2a341522e0 2024-04-20T21:17:39.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.250+0000 7f2a2be00640 1 -- 172.21.15.102:0/1260069015 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240447e0 con 0x7f2a341522e0 2024-04-20T21:17:39.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.250+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a341591f0 con 0x7f2a341522e0 2024-04-20T21:17:39.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.251+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a3404fd60 con 0x7f2a341522e0 2024-04-20T21:17:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.252+0000 7f2a2be00640 1 -- 172.21.15.102:0/1260069015 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a24037070 con 0x7f2a341522e0 2024-04-20T21:17:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.253+0000 7f2a2be00640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a1006c8f0 0x7f2a1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.253+0000 7f2a2be00640 1 -- 172.21.15.102:0/1260069015 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2a240ae110 con 0x7f2a341522e0 2024-04-20T21:17:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.254+0000 7f2a2aa00640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a1006c8f0 0x7f2a1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.254+0000 7f2a2aa00640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a1006c8f0 0x7f2a1006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a3413fe50 tx=0x7f2a1c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.258+0000 7f2a2be00640 1 -- 172.21.15.102:0/1260069015 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a2403f070 con 0x7f2a341522e0 2024-04-20T21:17:39.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.459+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2a34070e40 con 0x7f2a341522e0 2024-04-20T21:17:39.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.460+0000 7f2a2be00640 1 -- 172.21.15.102:0/1260069015 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2a2407bc10 con 0x7f2a341522e0 2024-04-20T21:17:39.460 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:17:39.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a1006c8f0 msgr2=0x7f2a1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a1006c8f0 0x7f2a1006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a3413fe50 tx=0x7f2a1c009040 comp rx=0 tx=0).stop 2024-04-20T21:17:39.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a341522e0 msgr2=0x7f2a3413ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a341522e0 0x7f2a3413ea50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a240098f0 tx=0x7f2a240046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 shutdown_connections 2024-04-20T21:17:39.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a3415c120 0x7f2a341421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a1006c8f0 0x7f2a1006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3415bce0 0x7f2a34141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 --2- 172.21.15.102:0/1260069015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a341522e0 0x7f2a3413ea50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 >> 172.21.15.102:0/1260069015 conn(0x7f2a340c9390 msgr2=0x7f2a34150470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:39.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.463+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 shutdown_connections 2024-04-20T21:17:39.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.464+0000 7f2a3a400640 1 -- 172.21.15.102:0/1260069015 wait complete. 2024-04-20T21:17:39.483 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:17:39.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.682+0000 7fd7a4e00640 1 Processor -- start 2024-04-20T21:17:39.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.682+0000 7fd7a4e00640 1 -- start start 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.683+0000 7fd7a4e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a01422c0 0x7fd7a01597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd7a4e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd7a00c7250 0x7fd7a00c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd7a4e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7a0073480 con 0x7fd7a00c6270 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd7a4e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a0073230 con 0x7fd7a00c7250 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd7a4e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a006e5a0 con 0x7fd7a01422c0 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd79f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a01422c0 0x7fd7a01597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd79ea00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd7a00c7250 0x7fd7a00c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd79f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a01422c0 0x7fd7a01597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:33314/0 (socket says 172.21.15.102:33314) 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd79fe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fd7a00c6270 0x7fd7a006ea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38126/0 (socket says 172.21.15.102:38126) 2024-04-20T21:17:39.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.684+0000 7fd79f400640 1 -- 172.21.15.102:0/3268619842 learned_addr learned my addr 172.21.15.102:0/3268619842 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:39.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.685+0000 7fd79f400640 1 -- 172.21.15.102:0/3268619842 >> v2:172.21.15.193:3300/0 conn(0x7fd7a00c7250 msgr2=0x7fd7a00c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:39.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.685+0000 7fd79f400640 1 --2- 172.21.15.102:0/3268619842 >> v2:172.21.15.193:3300/0 conn(0x7fd7a00c7250 0x7fd7a00c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.685+0000 7fd79f400640 1 -- 172.21.15.102:0/3268619842 >> v1:172.21.15.193:6789/0 conn(0x7fd7a00c6270 legacy=0x7fd7a006ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.685+0000 7fd79f400640 1 -- 172.21.15.102:0/3268619842 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a0073760 con 0x7fd7a01422c0 2024-04-20T21:17:39.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.685+0000 7fd79e000640 1 -- 172.21.15.102:0/3268619842 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 308227512 0 0) 0x7fd7a0073480 con 0x7fd7a00c6270 2024-04-20T21:17:39.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.686+0000 7fd79f400640 1 --2- 172.21.15.102:0/3268619842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a01422c0 0x7fd7a01597d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd790009870 tx=0x7fd79002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec61774a343b1dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.686+0000 7fd79e000640 1 -- 172.21.15.102:0/3268619842 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790034070 con 0x7fd7a01422c0 2024-04-20T21:17:39.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.686+0000 7fd79e000640 1 -- 172.21.15.102:0/3268619842 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd790030840 con 0x7fd7a01422c0 2024-04-20T21:17:39.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.686+0000 7fd79e000640 1 -- 172.21.15.102:0/3268619842 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790030c10 con 0x7fd7a01422c0 2024-04-20T21:17:39.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.686+0000 7fd7a4e00640 1 -- 172.21.15.102:0/3268619842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a01422c0 msgr2=0x7fd7a01597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.686+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/3268619842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a01422c0 0x7fd7a01597d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd790009870 tx=0x7fd79002b980 comp rx=0 tx=0).stop 2024-04-20T21:17:39.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.687+0000 7fd7a4e00640 1 -- 172.21.15.102:0/3268619842 shutdown_connections 2024-04-20T21:17:39.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.687+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/3268619842 >> v2:172.21.15.193:3300/0 conn(0x7fd7a00c7250 0x7fd7a00c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.687+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/3268619842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a01422c0 0x7fd7a01597d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.687+0000 7fd7a4e00640 1 -- 172.21.15.102:0/3268619842 >> 172.21.15.102:0/3268619842 conn(0x7fd7a00cc620 msgr2=0x7fd7a00cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:39.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.687+0000 7fd7a4e00640 1 -- 172.21.15.102:0/3268619842 shutdown_connections 2024-04-20T21:17:39.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.687+0000 7fd7a4e00640 1 -- 172.21.15.102:0/3268619842 wait complete. 2024-04-20T21:17:39.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.688+0000 7fd7a4e00640 1 Processor -- start 2024-04-20T21:17:39.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.689+0000 7fd7a4e00640 1 -- start start 2024-04-20T21:17:39.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.689+0000 7fd7a4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7a00c6270 0x7fd7a00c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.689+0000 7fd7a4e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a00c7250 0x7fd7a01e37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.689+0000 7fd7a4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7a01422c0 0x7fd7a01f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.689+0000 7fd7a4e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0073d20 con 0x7fd7a00c6270 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.689+0000 7fd7a4e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0159d10 con 0x7fd7a01422c0 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.689+0000 7fd7a4e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a01426a0 con 0x7fd7a00c7250 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7a01422c0 0x7fd7a01f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a00c7250 0x7fd7a01e37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7a00c6270 0x7fd7a00c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7a01422c0 0x7fd7a01f0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:53984/0 (socket says 172.21.15.102:53984) 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79fe00640 1 -- 172.21.15.102:0/722151424 learned_addr learned my addr 172.21.15.102:0/722151424 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:39.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79ea00640 1 -- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7a01422c0 msgr2=0x7fd7a01f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79ea00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7a01422c0 0x7fd7a01f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79ea00640 1 -- 172.21.15.102:0/722151424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7a00c6270 msgr2=0x7fd7a00c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79ea00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7a00c6270 0x7fd7a00c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79ea00640 1 -- 172.21.15.102:0/722151424 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a00c9190 con 0x7fd7a00c7250 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79fe00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7a01422c0 0x7fd7a01f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.690+0000 7fd79ea00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a00c7250 0x7fd7a01e37b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd78c012be0 tx=0x7fd78c010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.691+0000 7fd797400640 1 -- 172.21.15.102:0/722151424 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c00d2d0 con 0x7fd7a00c7250 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.691+0000 7fd797400640 1 -- 172.21.15.102:0/722151424 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c01d070 con 0x7fd7a00c7250 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.691+0000 7fd797400640 1 -- 172.21.15.102:0/722151424 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c010d60 con 0x7fd7a00c7250 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.691+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a00c9470 con 0x7fd7a00c7250 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.691+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a00c9a30 con 0x7fd7a00c7250 2024-04-20T21:17:39.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.692+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a004fcf0 con 0x7fd7a00c7250 2024-04-20T21:17:39.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.693+0000 7fd797400640 1 -- 172.21.15.102:0/722151424 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd78c016050 con 0x7fd7a00c7250 2024-04-20T21:17:39.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.694+0000 7fd797400640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd78006c8f0 0x7fd78006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.694+0000 7fd797400640 1 -- 172.21.15.102:0/722151424 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd78c0900e0 con 0x7fd7a00c7250 2024-04-20T21:17:39.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.694+0000 7fd79f400640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd78006c8f0 0x7fd78006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.695+0000 7fd79f400640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd78006c8f0 0x7fd78006edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd790036000 tx=0x7fd790008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.699+0000 7fd797400640 1 -- 172.21.15.102:0/722151424 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd78c027020 con 0x7fd7a00c7250 2024-04-20T21:17:39.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.899+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd7a00cd490 con 0x7fd7a00c7250 2024-04-20T21:17:39.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.900+0000 7fd797400640 1 -- 172.21.15.102:0/722151424 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fd78c014040 con 0x7fd7a00c7250 2024-04-20T21:17:39.900 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.902+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd78006c8f0 msgr2=0x7fd78006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.902+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd78006c8f0 0x7fd78006edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd790036000 tx=0x7fd790008030 comp rx=0 tx=0).stop 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.902+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a00c7250 msgr2=0x7fd7a01e37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.902+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a00c7250 0x7fd7a01e37b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd78c012be0 tx=0x7fd78c010490 comp rx=0 tx=0).stop 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.903+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 shutdown_connections 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.903+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd78006c8f0 0x7fd78006edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.903+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7a01422c0 0x7fd7a01f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.903+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7a00c7250 0x7fd7a01e37b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.903+0000 7fd7a4e00640 1 --2- 172.21.15.102:0/722151424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7a00c6270 0x7fd7a00c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.903+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 >> 172.21.15.102:0/722151424 conn(0x7fd7a00cc620 msgr2=0x7fd7a0158110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.903+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 shutdown_connections 2024-04-20T21:17:39.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:39.903+0000 7fd7a4e00640 1 -- 172.21.15.102:0/722151424 wait complete. 2024-04-20T21:17:39.921 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:17:40.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.124+0000 7f2c0d600640 1 Processor -- start 2024-04-20T21:17:40.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.124+0000 7f2c0d600640 1 -- start start 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c0d600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2c08158990 0x7f2c080c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c0d600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2c080c88e0 0x7f2c081627d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c0d600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c0815b520 con 0x7f2c081522e0 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c0d600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f2c0815c3f0 con 0x7f2c08158990 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c0d600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f2c08071060 con 0x7f2c080c88e0 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c06a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2c081522e0 0x7f2c0815bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57488/0 (socket says 172.21.15.102:57488) 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c06a00640 1 -- 172.21.15.102:0/236492360 learned_addr learned my addr 172.21.15.102:0/236492360 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c06000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2c08158990 0x7f2c080c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.126+0000 7f2c07400640 1 --2- 172.21.15.102:0/236492360 >> v2:172.21.15.193:3300/0 conn(0x7f2c080c88e0 0x7f2c081627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.127+0000 7f2c06000640 1 -- 172.21.15.102:0/236492360 >> v2:172.21.15.193:3300/0 conn(0x7f2c080c88e0 msgr2=0x7f2c081627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:40.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.127+0000 7f2c06000640 1 --2- 172.21.15.102:0/236492360 >> v2:172.21.15.193:3300/0 conn(0x7f2c080c88e0 0x7f2c081627d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.127+0000 7f2c06000640 1 -- 172.21.15.102:0/236492360 >> v1:172.21.15.102:6789/0 conn(0x7f2c081522e0 legacy=0x7f2c0815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.127+0000 7f2c06000640 1 -- 172.21.15.102:0/236492360 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c080712b0 con 0x7f2c08158990 2024-04-20T21:17:40.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.127+0000 7f2c05600640 1 -- 172.21.15.102:0/236492360 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017937304 0 0) 0x7f2c0815b520 con 0x7f2c081522e0 2024-04-20T21:17:40.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.128+0000 7f2c06000640 1 --2- 172.21.15.102:0/236492360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c08158990 0x7f2c080c83a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2c08074130 tx=0x7f2bf402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ea5ab4622a04d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:40.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.128+0000 7f2c05600640 1 -- 172.21.15.102:0/236492360 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4035020 con 0x7f2c08158990 2024-04-20T21:17:40.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.128+0000 7f2c05600640 1 -- 172.21.15.102:0/236492360 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf4031830 con 0x7f2c08158990 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.128+0000 7f2c05600640 1 -- 172.21.15.102:0/236492360 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4031c00 con 0x7f2c08158990 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.129+0000 7f2c0d600640 1 -- 172.21.15.102:0/236492360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c08158990 msgr2=0x7f2c080c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.129+0000 7f2c0d600640 1 --2- 172.21.15.102:0/236492360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c08158990 0x7f2c080c83a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2c08074130 tx=0x7f2bf402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.129+0000 7f2c0d600640 1 -- 172.21.15.102:0/236492360 shutdown_connections 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.129+0000 7f2c0d600640 1 --2- 172.21.15.102:0/236492360 >> v2:172.21.15.193:3300/0 conn(0x7f2c080c88e0 0x7f2c081627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.129+0000 7f2c0d600640 1 --2- 172.21.15.102:0/236492360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c08158990 0x7f2c080c83a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.129+0000 7f2c0d600640 1 -- 172.21.15.102:0/236492360 >> 172.21.15.102:0/236492360 conn(0x7f2c080c9390 msgr2=0x7f2c080c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.129+0000 7f2c0d600640 1 -- 172.21.15.102:0/236492360 shutdown_connections 2024-04-20T21:17:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.129+0000 7f2c0d600640 1 -- 172.21.15.102:0/236492360 wait complete. 2024-04-20T21:17:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.130+0000 7f2c0d600640 1 Processor -- start 2024-04-20T21:17:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.131+0000 7f2c0d600640 1 -- start start 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.131+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c080c88e0 0x7f2c0814b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.131+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c081522e0 0x7f2c080c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.131+0000 7f2c0d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c08158990 0x7f2c081f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.131+0000 7f2c0d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0815b7c0 con 0x7f2c081522e0 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.131+0000 7f2c0d600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081526c0 con 0x7f2c080c88e0 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.131+0000 7f2c0d600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2c08071870 con 0x7f2c08158990 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c080c88e0 0x7f2c0814b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c081522e0 0x7f2c080c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c07400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c08158990 0x7f2c081f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c081522e0 0x7f2c080c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48288/0 (socket says 172.21.15.102:48288) 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c07400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c08158990 0x7f2c081f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:33344/0 (socket says 172.21.15.102:33344) 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06000640 1 -- 172.21.15.102:0/780080196 learned_addr learned my addr 172.21.15.102:0/780080196 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06000640 1 -- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c08158990 msgr2=0x7f2c081f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06000640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c08158990 0x7f2c081f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06000640 1 -- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c080c88e0 msgr2=0x7f2c0814b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06000640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c080c88e0 0x7f2c0814b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.132+0000 7f2c06000640 1 -- 172.21.15.102:0/780080196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c081552b0 con 0x7f2c081522e0 2024-04-20T21:17:40.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.133+0000 7f2c06000640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c081522e0 0x7f2c080c99e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009890 tx=0x7f2bf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:40.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.133+0000 7f2bfea00640 1 -- 172.21.15.102:0/780080196 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4041070 con 0x7f2c081522e0 2024-04-20T21:17:40.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.133+0000 7f2bfea00640 1 -- 172.21.15.102:0/780080196 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf403b920 con 0x7f2c081522e0 2024-04-20T21:17:40.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.133+0000 7f2bfea00640 1 -- 172.21.15.102:0/780080196 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4035040 con 0x7f2c081522e0 2024-04-20T21:17:40.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.133+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c08155570 con 0x7f2c081522e0 2024-04-20T21:17:40.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.133+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c08151900 con 0x7f2c081522e0 2024-04-20T21:17:40.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.134+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c08151b40 con 0x7f2c081522e0 2024-04-20T21:17:40.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.135+0000 7f2bfea00640 1 -- 172.21.15.102:0/780080196 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bf4038020 con 0x7f2c081522e0 2024-04-20T21:17:40.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.136+0000 7f2bfea00640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2be806c920 0x7f2be806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.136+0000 7f2bfea00640 1 -- 172.21.15.102:0/780080196 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2bf40ad820 con 0x7f2c081522e0 2024-04-20T21:17:40.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.136+0000 7f2c06a00640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2be806c920 0x7f2be806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.137+0000 7f2c06a00640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2be806c920 0x7f2be806ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8000f60 tx=0x7f2bf8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:40.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.140+0000 7f2bfea00640 1 -- 172.21.15.102:0/780080196 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2c08151b40 con 0x7f2c081522e0 2024-04-20T21:17:40.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.339+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2c0804fd60 con 0x7f2c081522e0 2024-04-20T21:17:40.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.340+0000 7f2bfea00640 1 -- 172.21.15.102:0/780080196 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f2bf4031800 con 0x7f2c081522e0 2024-04-20T21:17:40.339 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-20T21:17:40.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2be806c920 msgr2=0x7f2be806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2be806c920 0x7f2be806ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8000f60 tx=0x7f2bf8009040 comp rx=0 tx=0).stop 2024-04-20T21:17:40.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c081522e0 msgr2=0x7f2c080c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c081522e0 0x7f2c080c99e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009890 tx=0x7f2bf40046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 shutdown_connections 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c08158990 0x7f2c081f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c081522e0 0x7f2c080c99e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2be806c920 0x7f2be806ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.342+0000 7f2c0d600640 1 --2- 172.21.15.102:0/780080196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c080c88e0 0x7f2c0814b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.343+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 >> 172.21.15.102:0/780080196 conn(0x7f2c080c9390 msgr2=0x7f2c080c5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.343+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 shutdown_connections 2024-04-20T21:17:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.343+0000 7f2c0d600640 1 -- 172.21.15.102:0/780080196 wait complete. 2024-04-20T21:17:40.361 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T21:17:40.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.569+0000 7f510ac00640 1 Processor -- start 2024-04-20T21:17:40.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.569+0000 7f510ac00640 1 -- start start 2024-04-20T21:17:40.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.570+0000 7f510ac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51041534f0 0x7f51040c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.570+0000 7f510ac00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51040c89f0 con 0x7f5104158f70 2024-04-20T21:17:40.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.570+0000 7f510ac00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51040c8cd0 con 0x7f5104153110 2024-04-20T21:17:40.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.570+0000 7f510ac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f51040c3920 con 0x7f51041534f0 2024-04-20T21:17:40.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5108c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5104158f70 0x7f51040c5e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57502/0 (socket says 172.21.15.102:57502) 2024-04-20T21:17:40.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5108c00640 1 -- 172.21.15.102:0/791170640 learned_addr learned my addr 172.21.15.102:0/791170640 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:40.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f50fb400640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51041534f0 0x7f51040c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981097711 0 0) 0x7f51040c8cd0 con 0x7f5104153110 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50e8003520 con 0x7f5104153110 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3711267103 0 0) 0x7f51040c89f0 con 0x7f5104158f70 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51040c8cd0 con 0x7f5104158f70 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1065588231 0 0) 0x7f50e8003520 con 0x7f5104153110 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51041534f0 msgr2=0x7f51040c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51041534f0 0x7f51040c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 >> v1:172.21.15.102:6789/0 conn(0x7f5104158f70 legacy=0x7f51040c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.571+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 --> v1:172.21.15.193:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51040c9400 con 0x7f5104153110 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 <== mon.1 v1:172.21.15.193:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7f50f4001670 con 0x7f5104153110 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 >> v1:172.21.15.193:6789/0 conn(0x7f5104153110 legacy=0x7f510406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103400640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f50e8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103400640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50e8004800 0x7f50e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103400640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e80051c0 0x7f50e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51040c3920 con 0x7f50e80051c0 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50e800c090 con 0x7f50e8003b40 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f50e800c290 con 0x7f50e8004800 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f50fb400640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51041534f0 0x7f51040c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103e00640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f50e8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5108c00640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e80051c0 0x7f50e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f50fb400640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50e8004800 0x7f50e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103e00640 1 -- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50e8004800 msgr2=0x7f50e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103e00640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50e8004800 0x7f50e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103e00640 1 -- 172.21.15.102:0/791170640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e80051c0 msgr2=0x7f50e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103e00640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e80051c0 0x7f50e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5103e00640 1 -- 172.21.15.102:0/791170640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e8003520 con 0x7f50e8003b40 2024-04-20T21:17:40.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f5108c00640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50e80051c0 0x7f50e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:40.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.572+0000 7f50fb400640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50e8004800 0x7f50e8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:40.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.573+0000 7f5103e00640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f50e8003f40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50f4007ef0 tx=0x7f50e8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e496162460ba8f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:40.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.574+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4039070 con 0x7f50e8003b40 2024-04-20T21:17:40.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.574+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f402c8e0 con 0x7f50e8003b40 2024-04-20T21:17:40.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.574+0000 7f5103400640 1 -- 172.21.15.102:0/791170640 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4007b10 con 0x7f50e8003b40 2024-04-20T21:17:40.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.574+0000 7f510ac00640 1 -- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 msgr2=0x7f50e8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.574+0000 7f510ac00640 1 --2- 172.21.15.102:0/791170640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f50e8003f40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50f4007ef0 tx=0x7f50e8005830 comp rx=0 tx=0).stop 2024-04-20T21:17:40.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.574+0000 7f5108c00640 1 -- 172.21.15.102:0/791170640 reap_dead start 2024-04-20T21:17:40.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.575+0000 7f510ac00640 1 -- 172.21.15.102:0/791170640 shutdown_connections 2024-04-20T21:17:40.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.575+0000 7f510ac00640 1 -- 172.21.15.102:0/791170640 >> 172.21.15.102:0/791170640 conn(0x7f5104074ea0 msgr2=0x7f5104075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:40.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.575+0000 7f510ac00640 1 -- 172.21.15.102:0/791170640 shutdown_connections 2024-04-20T21:17:40.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.575+0000 7f510ac00640 1 -- 172.21.15.102:0/791170640 wait complete. 2024-04-20T21:17:40.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.576+0000 7f510ac00640 1 Processor -- start 2024-04-20T21:17:40.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.576+0000 7f510ac00640 1 -- start start 2024-04-20T21:17:40.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.577+0000 7f510ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f51040ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.577+0000 7f510ac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f510415d220 0x7f5104148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.577+0000 7f510ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51041f1ad0 0x7f5104157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.577+0000 7f510ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51040c99c0 con 0x7f51041f1ad0 2024-04-20T21:17:40.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.577+0000 7f510ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51040c91b0 con 0x7f50e8003b40 2024-04-20T21:17:40.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.577+0000 7f510ac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f51040c9ee0 con 0x7f510415d220 2024-04-20T21:17:40.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f51040ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f50fb400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f510415d220 0x7f5104148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f51040ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54026/0 (socket says 172.21.15.102:54026) 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 -- 172.21.15.102:0/3897293505 learned_addr learned my addr 172.21.15.102:0/3897293505 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5108c00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51041f1ad0 0x7f5104157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 -- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f510415d220 msgr2=0x7f5104148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f510415d220 0x7f5104148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 -- 172.21.15.102:0/3897293505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51041f1ad0 msgr2=0x7f5104157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51041f1ad0 0x7f5104157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 -- 172.21.15.102:0/3897293505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51040caa60 con 0x7f50e8003b40 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.578+0000 7f5103e00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f51040ca520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50f4037040 tx=0x7f51040c95d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.579+0000 7f5100c00640 1 -- 172.21.15.102:0/3897293505 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4039040 con 0x7f50e8003b40 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.579+0000 7f5100c00640 1 -- 172.21.15.102:0/3897293505 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f4005270 con 0x7f50e8003b40 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.579+0000 7f5100c00640 1 -- 172.21.15.102:0/3897293505 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4001b80 con 0x7f50e8003b40 2024-04-20T21:17:40.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.579+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51040cc3a0 con 0x7f50e8003b40 2024-04-20T21:17:40.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.579+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51040cc0c0 con 0x7f50e8003b40 2024-04-20T21:17:40.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.580+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51041434a0 con 0x7f50e8003b40 2024-04-20T21:17:40.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.581+0000 7f5100c00640 1 -- 172.21.15.102:0/3897293505 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50f40035f0 con 0x7f50e8003b40 2024-04-20T21:17:40.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.582+0000 7f5100c00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f50e006c920 0x7f50e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:40.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.582+0000 7f5100c00640 1 -- 172.21.15.102:0/3897293505 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f50f40ad8c0 con 0x7f50e8003b40 2024-04-20T21:17:40.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.582+0000 7f50fb400640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f50e006c920 0x7f50e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:40.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.583+0000 7f50fb400640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f50e006c920 0x7f50e006ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5104071470 tx=0x7f50ec008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:40.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.587+0000 7f5100c00640 1 -- 172.21.15.102:0/3897293505 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f50f407b200 con 0x7f50e8003b40 2024-04-20T21:17:40.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.785+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f510404fdd0 con 0x7f50e8003b40 2024-04-20T21:17:40.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.786+0000 7f5100c00640 1 -- 172.21.15.102:0/3897293505 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f50f407f500 con 0x7f50e8003b40 2024-04-20T21:17:40.786 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:40.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f50e006c920 msgr2=0x7f50e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f50e006c920 0x7f50e006ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5104071470 tx=0x7f50ec008040 comp rx=0 tx=0).stop 2024-04-20T21:17:40.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 msgr2=0x7f51040ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:40.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f51040ca520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50f4037040 tx=0x7f51040c95d0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 shutdown_connections 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51041f1ad0 0x7f5104157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f510415d220 0x7f5104148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f50e006c920 0x7f50e006ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 --2- 172.21.15.102:0/3897293505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e8003b40 0x7f51040ca520 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.789+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 >> 172.21.15.102:0/3897293505 conn(0x7f5104074ea0 msgr2=0x7f51040c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.790+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 shutdown_connections 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:40.790+0000 7f510ac00640 1 -- 172.21.15.102:0/3897293505 wait complete. 2024-04-20T21:17:40.810 INFO:tasks.mds_thrash:mds thrasher using random seed: 1713647860 2024-04-20T21:17:40.810 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:41.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.015+0000 7f94bd600640 1 Processor -- start 2024-04-20T21:17:41.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.015+0000 7f94bd600640 1 -- start start 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94bd600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b8153180 0x7f94b8158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94bd600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f94b8159380 0x7f94b80c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94bd600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f94b80c75a0 0x7f94b80c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94bd600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f94b8153560 con 0x7f94b8159380 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94bd600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f94b80c7a70 con 0x7f94b80c75a0 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94bd600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f94b80c7d30 con 0x7f94b8153180 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94af400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f94b8159380 0x7f94b80c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94af400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f94b8159380 0x7f94b80c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48322/0 (socket says 172.21.15.102:48322) 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94bcc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f94b80c75a0 0x7f94b80c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94b7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b8153180 0x7f94b8158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.017+0000 7f94af400640 1 -- 172.21.15.102:0/2969749010 learned_addr learned my addr 172.21.15.102:0/2969749010 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:41.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.018+0000 7f94af400640 1 -- 172.21.15.102:0/2969749010 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b8153180 msgr2=0x7f94b8158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:41.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.018+0000 7f94af400640 1 --2- 172.21.15.102:0/2969749010 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b8153180 0x7f94b8158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.018+0000 7f94af400640 1 -- 172.21.15.102:0/2969749010 >> v2:172.21.15.193:3300/0 conn(0x7f94b80c75a0 msgr2=0x7f94b80c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:41.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.018+0000 7f94af400640 1 --2- 172.21.15.102:0/2969749010 >> v2:172.21.15.193:3300/0 conn(0x7f94b80c75a0 0x7f94b80c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.018+0000 7f94af400640 1 -- 172.21.15.102:0/2969749010 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b815c4d0 con 0x7f94b8159380 2024-04-20T21:17:41.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.019+0000 7f94af400640 1 --2- 172.21.15.102:0/2969749010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b80c5f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94b80c72b0 tx=0x7f94a002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9936feb66e3f6cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.019+0000 7f94b7400640 1 -- 172.21.15.102:0/2969749010 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035020 con 0x7f94b8159380 2024-04-20T21:17:41.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.019+0000 7f94b7400640 1 -- 172.21.15.102:0/2969749010 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a002d040 con 0x7f94b8159380 2024-04-20T21:17:41.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.019+0000 7f94b7400640 1 -- 172.21.15.102:0/2969749010 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0031a80 con 0x7f94b8159380 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 -- 172.21.15.102:0/2969749010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 msgr2=0x7f94b80c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 --2- 172.21.15.102:0/2969749010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b80c5f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94b80c72b0 tx=0x7f94a002b900 comp rx=0 tx=0).stop 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 -- 172.21.15.102:0/2969749010 shutdown_connections 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 --2- 172.21.15.102:0/2969749010 >> v2:172.21.15.193:3300/0 conn(0x7f94b80c75a0 0x7f94b80c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 --2- 172.21.15.102:0/2969749010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b80c5f30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 --2- 172.21.15.102:0/2969749010 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b8153180 0x7f94b8158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 -- 172.21.15.102:0/2969749010 >> 172.21.15.102:0/2969749010 conn(0x7f94b80cc820 msgr2=0x7f94b80ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 -- 172.21.15.102:0/2969749010 shutdown_connections 2024-04-20T21:17:41.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.020+0000 7f94bd600640 1 -- 172.21.15.102:0/2969749010 wait complete. 2024-04-20T21:17:41.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.022+0000 7f94bd600640 1 Processor -- start 2024-04-20T21:17:41.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.022+0000 7f94bd600640 1 -- start start 2024-04-20T21:17:41.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bd600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b80c75a0 0x7f94b81f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b8153180 0x7f94b814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b8151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bd600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94b815d290 con 0x7f94b8159380 2024-04-20T21:17:41.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bd600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94b815ca90 con 0x7f94b8153180 2024-04-20T21:17:41.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bd600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f94b80c6be0 con 0x7f94b80c75a0 2024-04-20T21:17:41.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b8151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94af400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b8153180 0x7f94b814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94b7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b80c75a0 0x7f94b81f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b8151050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48334/0 (socket says 172.21.15.102:48334) 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.023+0000 7f94bcc00640 1 -- 172.21.15.102:0/4211121897 learned_addr learned my addr 172.21.15.102:0/4211121897 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.024+0000 7f94af400640 1 -- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b80c75a0 msgr2=0x7f94b81f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.024+0000 7f94af400640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b80c75a0 0x7f94b81f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.024+0000 7f94af400640 1 -- 172.21.15.102:0/4211121897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 msgr2=0x7f94b8151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.024+0000 7f94af400640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b8151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.024+0000 7f94af400640 1 -- 172.21.15.102:0/4211121897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b81f0e40 con 0x7f94b8153180 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.024+0000 7f94bcc00640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b8151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.024+0000 7f94b7e00640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b80c75a0 0x7f94b81f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.024+0000 7f94af400640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b8153180 0x7f94b814ff60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94a0033000 tx=0x7f94a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.025+0000 7f94b4c00640 1 -- 172.21.15.102:0/4211121897 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0040070 con 0x7f94b8153180 2024-04-20T21:17:41.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.025+0000 7f94b4c00640 1 -- 172.21.15.102:0/4211121897 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a0039960 con 0x7f94b8153180 2024-04-20T21:17:41.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.025+0000 7f94b4c00640 1 -- 172.21.15.102:0/4211121897 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035040 con 0x7f94b8153180 2024-04-20T21:17:41.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.025+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b81f1120 con 0x7f94b8153180 2024-04-20T21:17:41.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.025+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b81f16e0 con 0x7f94b8153180 2024-04-20T21:17:41.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.026+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b804fd60 con 0x7f94b8153180 2024-04-20T21:17:41.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.027+0000 7f94b4c00640 1 -- 172.21.15.102:0/4211121897 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94a004b060 con 0x7f94b8153180 2024-04-20T21:17:41.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.028+0000 7f94b4c00640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.028+0000 7f94b4c00640 1 -- 172.21.15.102:0/4211121897 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f94a00ad900 con 0x7f94b8153180 2024-04-20T21:17:41.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.028+0000 7f94b7e00640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.029+0000 7f94b7e00640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f949406c8f0 0x7f949406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009f30 tx=0x7f94a8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.032+0000 7f94b4c00640 1 -- 172.21.15.102:0/4211121897 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f94a007b240 con 0x7f94b8153180 2024-04-20T21:17:41.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.234+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94b80c3690 con 0x7f94b8153180 2024-04-20T21:17:41.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.236+0000 7f94b4c00640 1 -- 172.21.15.102:0/4211121897 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18894 (secure 0 0 0) 0x7f94a007f540 con 0x7f94b8153180 2024-04-20T21:17:41.236 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:41.236 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:17:35.869503+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:41.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.238+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f949406c8f0 msgr2=0x7f949406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.238+0000 7f94bd600640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f949406c8f0 0x7f949406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009f30 tx=0x7f94a8009190 comp rx=0 tx=0).stop 2024-04-20T21:17:41.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.238+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b8153180 msgr2=0x7f94b814ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.238+0000 7f94bd600640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b8153180 0x7f94b814ff60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94a0033000 tx=0x7f94a00046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.239+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 shutdown_connections 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.239+0000 7f94bd600640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94b8159380 0x7f94b8151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.239+0000 7f94bd600640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f949406c8f0 0x7f949406edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.239+0000 7f94bd600640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b8153180 0x7f94b814ff60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.239+0000 7f94bd600640 1 --2- 172.21.15.102:0/4211121897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94b80c75a0 0x7f94b81f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.239+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 >> 172.21.15.102:0/4211121897 conn(0x7f94b80cc820 msgr2=0x7f94b806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.239+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 shutdown_connections 2024-04-20T21:17:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.239+0000 7f94bd600640 1 -- 172.21.15.102:0/4211121897 wait complete. 2024-04-20T21:17:41.256 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:16:51.795761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:16:52.983434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T21:16:56.582297+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T21:17:41.256 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:41.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.459+0000 7fb85ca00640 1 Processor -- start 2024-04-20T21:17:41.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.459+0000 7fb85ca00640 1 -- start start 2024-04-20T21:17:41.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.461+0000 7fb85ca00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb8580cd060 0x7fb858159460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.461+0000 7fb85ca00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb85815b320 con 0x7fb8580c86a0 2024-04-20T21:17:41.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.461+0000 7fb85ca00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb85815b600 con 0x7fb8580c82c0 2024-04-20T21:17:41.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.461+0000 7fb85ca00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb85815b8e0 con 0x7fb8580cd060 2024-04-20T21:17:41.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.461+0000 7fb84ea00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb8580c86a0 0x7fb858151b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57514/0 (socket says 172.21.15.102:57514) 2024-04-20T21:17:41.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.461+0000 7fb84ea00640 1 -- 172.21.15.102:0/616745974 learned_addr learned my addr 172.21.15.102:0/616745974 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:41.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.461+0000 7fb857e00640 1 --2- 172.21.15.102:0/616745974 >> v2:172.21.15.193:3300/0 conn(0x7fb8580cd060 0x7fb858159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.462+0000 7fb856a00640 1 -- 172.21.15.102:0/616745974 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307384034 0 0) 0x7fb85815b320 con 0x7fb8580c86a0 2024-04-20T21:17:41.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.462+0000 7fb856a00640 1 -- 172.21.15.102:0/616745974 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb83c003520 con 0x7fb8580c86a0 2024-04-20T21:17:41.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.462+0000 7fb856a00640 1 -- 172.21.15.102:0/616745974 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790202150 0 0) 0x7fb85815b600 con 0x7fb8580c82c0 2024-04-20T21:17:41.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.462+0000 7fb856a00640 1 -- 172.21.15.102:0/616745974 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb85815b320 con 0x7fb8580c82c0 2024-04-20T21:17:41.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.462+0000 7fb857e00640 1 -- 172.21.15.102:0/616745974 >> v1:172.21.15.193:6789/0 conn(0x7fb8580c82c0 legacy=0x7fb8581586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.462+0000 7fb857e00640 1 -- 172.21.15.102:0/616745974 >> v1:172.21.15.102:6789/0 conn(0x7fb8580c86a0 legacy=0x7fb858151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.462+0000 7fb857e00640 1 -- 172.21.15.102:0/616745974 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85815bbc0 con 0x7fb8580cd060 2024-04-20T21:17:41.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.463+0000 7fb857e00640 1 --2- 172.21.15.102:0/616745974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580cd060 0x7fb858159460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8500097a0 tx=0x7fb85002b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e126f71185b518a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.463+0000 7fb856a00640 1 -- 172.21.15.102:0/616745974 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85002f810 con 0x7fb8580cd060 2024-04-20T21:17:41.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.463+0000 7fb856a00640 1 -- 172.21.15.102:0/616745974 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85002f9b0 con 0x7fb8580cd060 2024-04-20T21:17:41.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.463+0000 7fb856a00640 1 -- 172.21.15.102:0/616745974 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85002fd80 con 0x7fb8580cd060 2024-04-20T21:17:41.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.464+0000 7fb85ca00640 1 -- 172.21.15.102:0/616745974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580cd060 msgr2=0x7fb858159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.464+0000 7fb85ca00640 1 --2- 172.21.15.102:0/616745974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580cd060 0x7fb858159460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8500097a0 tx=0x7fb85002b9f0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.464+0000 7fb85ca00640 1 -- 172.21.15.102:0/616745974 shutdown_connections 2024-04-20T21:17:41.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.464+0000 7fb85ca00640 1 --2- 172.21.15.102:0/616745974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580cd060 0x7fb858159460 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.464+0000 7fb85ca00640 1 -- 172.21.15.102:0/616745974 >> 172.21.15.102:0/616745974 conn(0x7fb8580cc540 msgr2=0x7fb8580cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:41.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.464+0000 7fb85ca00640 1 -- 172.21.15.102:0/616745974 shutdown_connections 2024-04-20T21:17:41.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.464+0000 7fb85ca00640 1 -- 172.21.15.102:0/616745974 wait complete. 2024-04-20T21:17:41.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.465+0000 7fb85ca00640 1 Processor -- start 2024-04-20T21:17:41.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.466+0000 7fb85ca00640 1 -- start start 2024-04-20T21:17:41.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.466+0000 7fb85ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8580c82c0 0x7fb85814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.466+0000 7fb85ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580c86a0 0x7fb8581f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.466+0000 7fb85ca00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8580cd060 0x7fb8581f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.466+0000 7fb85ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb858152210 con 0x7fb8580c82c0 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.466+0000 7fb85ca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb85815c180 con 0x7fb8580c86a0 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.466+0000 7fb85ca00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb858152390 con 0x7fb8580cd060 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8580c82c0 0x7fb85814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8580c82c0 0x7fb85814df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48348/0 (socket says 172.21.15.102:48348) 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8580cd060 0x7fb8581f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857400640 1 -- 172.21.15.102:0/1853552524 learned_addr learned my addr 172.21.15.102:0/1853552524 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb84ea00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580c86a0 0x7fb8581f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857400640 1 -- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8580cd060 msgr2=0x7fb8581f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857400640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8580cd060 0x7fb8581f12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857400640 1 -- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580c86a0 msgr2=0x7fb8581f0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857400640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580c86a0 0x7fb8581f0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.467+0000 7fb857400640 1 -- 172.21.15.102:0/1853552524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85814c0b0 con 0x7fb8580c82c0 2024-04-20T21:17:41.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.468+0000 7fb857400640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8580c82c0 0x7fb85814df60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c800 tx=0x7fb84800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.468+0000 7fb84fe00640 1 -- 172.21.15.102:0/1853552524 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848003b70 con 0x7fb8580c82c0 2024-04-20T21:17:41.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.468+0000 7fb84fe00640 1 -- 172.21.15.102:0/1853552524 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb848003d10 con 0x7fb8580c82c0 2024-04-20T21:17:41.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.468+0000 7fb84fe00640 1 -- 172.21.15.102:0/1853552524 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800aa90 con 0x7fb8580c82c0 2024-04-20T21:17:41.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.468+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb858074500 con 0x7fb8580c82c0 2024-04-20T21:17:41.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.468+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85814c320 con 0x7fb8580c82c0 2024-04-20T21:17:41.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.469+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb858071660 con 0x7fb8580c82c0 2024-04-20T21:17:41.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.470+0000 7fb84fe00640 1 -- 172.21.15.102:0/1853552524 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb84800ac30 con 0x7fb8580c82c0 2024-04-20T21:17:41.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.471+0000 7fb84fe00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb83406c920 0x7fb83406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.471+0000 7fb84fe00640 1 -- 172.21.15.102:0/1853552524 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb84808bf70 con 0x7fb8580c82c0 2024-04-20T21:17:41.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.471+0000 7fb84ea00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb83406c920 0x7fb83406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.472+0000 7fb84ea00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb83406c920 0x7fb83406ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8580732d0 tx=0x7fb840007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.475+0000 7fb84fe00640 1 -- 172.21.15.102:0/1853552524 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb858071660 con 0x7fb8580c82c0 2024-04-20T21:17:41.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.686+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb858151c50 con 0x7fb8580c82c0 2024-04-20T21:17:41.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.687+0000 7fb84fe00640 1 -- 172.21.15.102:0/1853552524 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fb848059820 con 0x7fb8580c82c0 2024-04-20T21:17:41.686 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:17:41.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb83406c920 msgr2=0x7fb83406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb83406c920 0x7fb83406ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8580732d0 tx=0x7fb840007d30 comp rx=0 tx=0).stop 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8580c82c0 msgr2=0x7fb85814df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8580c82c0 0x7fb85814df60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c800 tx=0x7fb84800ccc0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 shutdown_connections 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8580cd060 0x7fb8581f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb83406c920 0x7fb83406ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8580c86a0 0x7fb8581f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 --2- 172.21.15.102:0/1853552524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8580c82c0 0x7fb85814df60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 >> 172.21.15.102:0/1853552524 conn(0x7fb8580cc540 msgr2=0x7fb858072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 shutdown_connections 2024-04-20T21:17:41.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.690+0000 7fb85ca00640 1 -- 172.21.15.102:0/1853552524 wait complete. 2024-04-20T21:17:41.708 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:17:41.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.905+0000 7f97cfc00640 1 Processor -- start 2024-04-20T21:17:41.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.905+0000 7f97cfc00640 1 -- start start 2024-04-20T21:17:41.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.907+0000 7f97cfc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f97c8153110 0x7f97c80c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.907+0000 7f97cfc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c80c89f0 con 0x7f97c8158f70 2024-04-20T21:17:41.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.907+0000 7f97cfc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c80c8cd0 con 0x7f97c81534f0 2024-04-20T21:17:41.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.907+0000 7f97cfc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f97c80c3920 con 0x7f97c8153110 2024-04-20T21:17:41.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.907+0000 7f97cdc00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f97c8158f70 0x7f97c80c5e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57526/0 (socket says 172.21.15.102:57526) 2024-04-20T21:17:41.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.907+0000 7f97cdc00640 1 -- 172.21.15.102:0/2108054403 learned_addr learned my addr 172.21.15.102:0/2108054403 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:41.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.907+0000 7f97cd200640 1 --2- 172.21.15.102:0/2108054403 >> v2:172.21.15.102:3300/0 conn(0x7f97c8153110 0x7f97c80c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.908+0000 7f97c7400640 1 -- 172.21.15.102:0/2108054403 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2912047540 0 0) 0x7f97c80c8cd0 con 0x7f97c81534f0 2024-04-20T21:17:41.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.908+0000 7f97c7400640 1 -- 172.21.15.102:0/2108054403 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97b0003520 con 0x7f97c81534f0 2024-04-20T21:17:41.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.908+0000 7f97cd200640 1 -- 172.21.15.102:0/2108054403 >> v1:172.21.15.193:6789/0 conn(0x7f97c81534f0 legacy=0x7f97c806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.908+0000 7f97cd200640 1 -- 172.21.15.102:0/2108054403 >> v1:172.21.15.102:6789/0 conn(0x7f97c8158f70 legacy=0x7f97c80c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.908+0000 7f97cd200640 1 -- 172.21.15.102:0/2108054403 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c80c9400 con 0x7f97c8153110 2024-04-20T21:17:41.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.910+0000 7f97cd200640 1 --2- 172.21.15.102:0/2108054403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c80c83d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0097a0 tx=0x7f97bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be30918a5486c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.911+0000 7f97c7400640 1 -- 172.21.15.102:0/2108054403 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc035020 con 0x7f97c8153110 2024-04-20T21:17:41.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.911+0000 7f97c7400640 1 -- 172.21.15.102:0/2108054403 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc031830 con 0x7f97c8153110 2024-04-20T21:17:41.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.911+0000 7f97c7400640 1 -- 172.21.15.102:0/2108054403 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc031c00 con 0x7f97c8153110 2024-04-20T21:17:41.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.912+0000 7f97cfc00640 1 -- 172.21.15.102:0/2108054403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 msgr2=0x7f97c80c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.912+0000 7f97cfc00640 1 --2- 172.21.15.102:0/2108054403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c80c83d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0097a0 tx=0x7f97bc02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.912+0000 7f97cfc00640 1 -- 172.21.15.102:0/2108054403 shutdown_connections 2024-04-20T21:17:41.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.912+0000 7f97cfc00640 1 --2- 172.21.15.102:0/2108054403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c80c83d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.912+0000 7f97cfc00640 1 -- 172.21.15.102:0/2108054403 >> 172.21.15.102:0/2108054403 conn(0x7f97c8074ea0 msgr2=0x7f97c8075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:41.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.912+0000 7f97cfc00640 1 -- 172.21.15.102:0/2108054403 shutdown_connections 2024-04-20T21:17:41.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.912+0000 7f97cfc00640 1 -- 172.21.15.102:0/2108054403 wait complete. 2024-04-20T21:17:41.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.914+0000 7f97cfc00640 1 Processor -- start 2024-04-20T21:17:41.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.915+0000 7f97cfc00640 1 -- start start 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cfc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c8145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cfc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97c8158f70 0x7f97c8154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cfc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97c815cea0 0x7f97c81551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cfc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f97c80c99c0 con 0x7f97c8153110 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cfc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97c80c91b0 con 0x7f97c815cea0 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cfc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f97c80c9ee0 con 0x7f97c8158f70 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c8145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cdc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97c815cea0 0x7f97c81551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c8145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48360/0 (socket says 172.21.15.102:48360) 2024-04-20T21:17:41.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97cd200640 1 -- 172.21.15.102:0/250519973 learned_addr learned my addr 172.21.15.102:0/250519973 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:41.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.916+0000 7f97c7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97c8158f70 0x7f97c8154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97cd200640 1 -- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97c8158f70 msgr2=0x7f97c8154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97cd200640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97c8158f70 0x7f97c8154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97cd200640 1 -- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97c815cea0 msgr2=0x7f97c81551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97cd200640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97c815cea0 0x7f97c81551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97cd200640 1 -- 172.21.15.102:0/250519973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c8154c20 con 0x7f97c8153110 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97cd200640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c8145380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009890 tx=0x7f97bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97c4c00640 1 -- 172.21.15.102:0/250519973 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc035040 con 0x7f97c8153110 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97c4c00640 1 -- 172.21.15.102:0/250519973 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc03b920 con 0x7f97c8153110 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97c4c00640 1 -- 172.21.15.102:0/250519973 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc03bd10 con 0x7f97c8153110 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.917+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c81eef80 con 0x7f97c8153110 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.918+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c81ef540 con 0x7f97c8153110 2024-04-20T21:17:41.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.919+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c815d3f0 con 0x7f97c8153110 2024-04-20T21:17:41.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.920+0000 7f97c4c00640 1 -- 172.21.15.102:0/250519973 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97bc038020 con 0x7f97c8153110 2024-04-20T21:17:41.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.921+0000 7f97c4c00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97a806c920 0x7f97a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:41.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.921+0000 7f97c4c00640 1 -- 172.21.15.102:0/250519973 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f97bc0adfe0 con 0x7f97c8153110 2024-04-20T21:17:41.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.921+0000 7f97c7e00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97a806c920 0x7f97a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:41.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.922+0000 7f97c7e00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97a806c920 0x7f97a806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f97b4007fd0 tx=0x7f97b4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:41.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:41.925+0000 7f97c4c00640 1 -- 172.21.15.102:0/250519973 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f97bc031800 con 0x7f97c8153110 2024-04-20T21:17:42.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.125+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97c80c73f0 con 0x7f97c8153110 2024-04-20T21:17:42.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.126+0000 7f97c4c00640 1 -- 172.21.15.102:0/250519973 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f97bc07b8a0 con 0x7f97c8153110 2024-04-20T21:17:42.126 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97a806c920 msgr2=0x7f97a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97a806c920 0x7f97a806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f97b4007fd0 tx=0x7f97b4009040 comp rx=0 tx=0).stop 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 msgr2=0x7f97c8145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c8145380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009890 tx=0x7f97bc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 shutdown_connections 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97a806c920 0x7f97a806ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97c815cea0 0x7f97c81551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97c8158f70 0x7f97c8154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 --2- 172.21.15.102:0/250519973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97c8153110 0x7f97c8145380 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 >> 172.21.15.102:0/250519973 conn(0x7f97c8074ea0 msgr2=0x7f97c8148110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:42.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 shutdown_connections 2024-04-20T21:17:42.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.129+0000 7f97cfc00640 1 -- 172.21.15.102:0/250519973 wait complete. 2024-04-20T21:17:42.148 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:17:42.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.354+0000 7f02ef000640 1 Processor -- start 2024-04-20T21:17:42.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.354+0000 7f02ef000640 1 -- start start 2024-04-20T21:17:42.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.355+0000 7f02ef000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e80c6ba0 0x7f02e815c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.355+0000 7f02ef000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f02e80c9eb0 0x7f02e8159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02ef000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e80c44d0 con 0x7f02e8153210 2024-04-20T21:17:42.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02ef000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f02e81596f0 con 0x7f02e80c9eb0 2024-04-20T21:17:42.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02ef000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f02e80c47b0 con 0x7f02e80c6ba0 2024-04-20T21:17:42.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e80c6ba0 0x7f02e815c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02e7e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f02e8153210 0x7f02e8072fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38162/0 (socket says 172.21.15.102:38162) 2024-04-20T21:17:42.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02e7e00640 1 -- 172.21.15.102:0/3410254743 learned_addr learned my addr 172.21.15.102:0/3410254743 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:42.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02ed000640 1 --2- 172.21.15.102:0/3410254743 >> v2:172.21.15.193:3300/0 conn(0x7f02e80c9eb0 0x7f02e8159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02dfe00640 1 -- 172.21.15.102:0/3410254743 >> v2:172.21.15.193:3300/0 conn(0x7f02e80c9eb0 msgr2=0x7f02e8159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:42.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02dfe00640 1 --2- 172.21.15.102:0/3410254743 >> v2:172.21.15.193:3300/0 conn(0x7f02e80c9eb0 0x7f02e8159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02e7400640 1 -- 172.21.15.102:0/3410254743 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847064903 0 0) 0x7f02e80c44d0 con 0x7f02e8153210 2024-04-20T21:17:42.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02dfe00640 1 -- 172.21.15.102:0/3410254743 >> v1:172.21.15.193:6789/0 conn(0x7f02e8153210 legacy=0x7f02e8072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.356+0000 7f02dfe00640 1 -- 172.21.15.102:0/3410254743 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e80c4a90 con 0x7f02e80c6ba0 2024-04-20T21:17:42.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.357+0000 7f02dfe00640 1 --2- 172.21.15.102:0/3410254743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e80c6ba0 0x7f02e815c5e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f02d00097a0 tx=0x7f02d002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374ebcd647651589 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:42.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.358+0000 7f02e7400640 1 -- 172.21.15.102:0/3410254743 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0034070 con 0x7f02e80c6ba0 2024-04-20T21:17:42.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.358+0000 7f02e7400640 1 -- 172.21.15.102:0/3410254743 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d0030800 con 0x7f02e80c6ba0 2024-04-20T21:17:42.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.358+0000 7f02e7400640 1 -- 172.21.15.102:0/3410254743 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0030bd0 con 0x7f02e80c6ba0 2024-04-20T21:17:42.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.358+0000 7f02ef000640 1 -- 172.21.15.102:0/3410254743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e80c6ba0 msgr2=0x7f02e815c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.358+0000 7f02ef000640 1 --2- 172.21.15.102:0/3410254743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e80c6ba0 0x7f02e815c5e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f02d00097a0 tx=0x7f02d002b940 comp rx=0 tx=0).stop 2024-04-20T21:17:42.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.359+0000 7f02ef000640 1 -- 172.21.15.102:0/3410254743 shutdown_connections 2024-04-20T21:17:42.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.359+0000 7f02ef000640 1 --2- 172.21.15.102:0/3410254743 >> v2:172.21.15.193:3300/0 conn(0x7f02e80c9eb0 0x7f02e8159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.359+0000 7f02ef000640 1 --2- 172.21.15.102:0/3410254743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e80c6ba0 0x7f02e815c5e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.359+0000 7f02ef000640 1 -- 172.21.15.102:0/3410254743 >> 172.21.15.102:0/3410254743 conn(0x7f02e80c9390 msgr2=0x7f02e80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:42.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.359+0000 7f02ef000640 1 -- 172.21.15.102:0/3410254743 shutdown_connections 2024-04-20T21:17:42.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.359+0000 7f02ef000640 1 -- 172.21.15.102:0/3410254743 wait complete. 2024-04-20T21:17:42.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.361+0000 7f02ef000640 1 Processor -- start 2024-04-20T21:17:42.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.361+0000 7f02ef000640 1 -- start start 2024-04-20T21:17:42.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02ef000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02e80c6ba0 0x7f02e81485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02ef000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e80c9eb0 0x7f02e81eed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02ef000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e8153210 0x7f02e806e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02ef000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02e80736d0 con 0x7f02e80c9eb0 2024-04-20T21:17:42.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02ef000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8159030 con 0x7f02e80c6ba0 2024-04-20T21:17:42.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02ef000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f02e8073850 con 0x7f02e8153210 2024-04-20T21:17:42.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e80c9eb0 0x7f02e81eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02ed000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e8153210 0x7f02e806e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e80c9eb0 0x7f02e81eed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48368/0 (socket says 172.21.15.102:48368) 2024-04-20T21:17:42.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02e80c6ba0 0x7f02e81485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.362+0000 7f02dfe00640 1 -- 172.21.15.102:0/1207783426 learned_addr learned my addr 172.21.15.102:0/1207783426 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.363+0000 7f02ed000640 1 -- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02e80c6ba0 msgr2=0x7f02e81485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.363+0000 7f02ed000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02e80c6ba0 0x7f02e81485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.363+0000 7f02ed000640 1 -- 172.21.15.102:0/1207783426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e80c9eb0 msgr2=0x7f02e81eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.363+0000 7f02ed000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e80c9eb0 0x7f02e81eed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.363+0000 7f02ed000640 1 -- 172.21.15.102:0/1207783426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e81f31f0 con 0x7f02e8153210 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.363+0000 7f02e7e00640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02e80c6ba0 0x7f02e81485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.363+0000 7f02ed000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e8153210 0x7f02e806e500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02e000efc0 tx=0x7f02e000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.363+0000 7f02e4c00640 1 -- 172.21.15.102:0/1207783426 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0018020 con 0x7f02e8153210 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.364+0000 7f02e4c00640 1 -- 172.21.15.102:0/1207783426 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e0014480 con 0x7f02e8153210 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.364+0000 7f02e4c00640 1 -- 172.21.15.102:0/1207783426 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0014870 con 0x7f02e8153210 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.364+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e8150d50 con 0x7f02e8153210 2024-04-20T21:17:42.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.364+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e81f2710 con 0x7f02e8153210 2024-04-20T21:17:42.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.365+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e804fd60 con 0x7f02e8153210 2024-04-20T21:17:42.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.366+0000 7f02e4c00640 1 -- 172.21.15.102:0/1207783426 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02e001b020 con 0x7f02e8153210 2024-04-20T21:17:42.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.367+0000 7f02e4c00640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f02c406c940 0x7f02c406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.367+0000 7f02e4c00640 1 -- 172.21.15.102:0/1207783426 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f02e008ccd0 con 0x7f02e8153210 2024-04-20T21:17:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.367+0000 7f02e7e00640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f02c406c940 0x7f02c406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.368+0000 7f02e7e00640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f02c406c940 0x7f02c406ee00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02d8007fd0 tx=0x7f02d8007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:42.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.372+0000 7f02e4c00640 1 -- 172.21.15.102:0/1207783426 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f02e0022020 con 0x7f02e8153210 2024-04-20T21:17:42.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.581+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f02e8150450 con 0x7f02e8153210 2024-04-20T21:17:42.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.582+0000 7f02e4c00640 1 -- 172.21.15.102:0/1207783426 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f02e0010040 con 0x7f02e8153210 2024-04-20T21:17:42.582 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-20T21:17:42.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.584+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f02c406c940 msgr2=0x7f02c406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.584+0000 7f02ef000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f02c406c940 0x7f02c406ee00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02d8007fd0 tx=0x7f02d8007de0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e8153210 msgr2=0x7f02e806e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e8153210 0x7f02e806e500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02e000efc0 tx=0x7f02e000c490 comp rx=0 tx=0).stop 2024-04-20T21:17:42.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 shutdown_connections 2024-04-20T21:17:42.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02e8153210 0x7f02e806e500 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e80c9eb0 0x7f02e81eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f02c406c940 0x7f02c406ee00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 --2- 172.21.15.102:0/1207783426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02e80c6ba0 0x7f02e81485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 >> 172.21.15.102:0/1207783426 conn(0x7f02e80c9390 msgr2=0x7f02e81588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:42.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 shutdown_connections 2024-04-20T21:17:42.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.585+0000 7f02ef000640 1 -- 172.21.15.102:0/1207783426 wait complete. 2024-04-20T21:17:42.603 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T21:17:42.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.790+0000 7f859b000640 1 Processor -- start 2024-04-20T21:17:42.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.791+0000 7f859b000640 1 -- start start 2024-04-20T21:17:42.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.792+0000 7f859b000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f85941534f0 0x7f85940c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.792+0000 7f859b000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8594158f70 0x7f85940c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.792+0000 7f859b000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85940c89f0 con 0x7f8594153110 2024-04-20T21:17:42.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.792+0000 7f859b000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f85940c3920 con 0x7f85941534f0 2024-04-20T21:17:42.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.792+0000 7f859b000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f85940c91b0 con 0x7f8594158f70 2024-04-20T21:17:42.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.792+0000 7f858be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f85941534f0 0x7f85940c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.792+0000 7f8593e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8594153110 0x7f859406f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57542/0 (socket says 172.21.15.102:57542) 2024-04-20T21:17:42.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.792+0000 7f8593e00640 1 -- 172.21.15.102:0/3621787334 learned_addr learned my addr 172.21.15.102:0/3621787334 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:42.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.793+0000 7f8599000640 1 --2- 172.21.15.102:0/3621787334 >> v2:172.21.15.193:3300/0 conn(0x7f8594158f70 0x7f85940c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.793+0000 7f858be00640 1 -- 172.21.15.102:0/3621787334 >> v2:172.21.15.193:3300/0 conn(0x7f8594158f70 msgr2=0x7f85940c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:42.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.793+0000 7f858be00640 1 --2- 172.21.15.102:0/3621787334 >> v2:172.21.15.193:3300/0 conn(0x7f8594158f70 0x7f85940c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.793+0000 7f858be00640 1 -- 172.21.15.102:0/3621787334 >> v1:172.21.15.102:6789/0 conn(0x7f8594153110 legacy=0x7f859406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.793+0000 7f858be00640 1 -- 172.21.15.102:0/3621787334 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940c8cd0 con 0x7f85941534f0 2024-04-20T21:17:42.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.793+0000 7f8593400640 1 -- 172.21.15.102:0/3621787334 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352903483 0 0) 0x7f85940c89f0 con 0x7f8594153110 2024-04-20T21:17:42.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.793+0000 7f8599000640 1 --2- 172.21.15.102:0/3621787334 >> v2:172.21.15.193:3300/0 conn(0x7f8594158f70 0x7f85940c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:42.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.794+0000 7f858be00640 1 --2- 172.21.15.102:0/3621787334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85941534f0 0x7f85940c83d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85940c32d0 tx=0x7f857c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f89fe49fb0423dd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:42.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.794+0000 7f8593400640 1 -- 172.21.15.102:0/3621787334 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c0315a0 con 0x7f85941534f0 2024-04-20T21:17:42.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f8593400640 1 -- 172.21.15.102:0/3621787334 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c037070 con 0x7f85941534f0 2024-04-20T21:17:42.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f8593400640 1 -- 172.21.15.102:0/3621787334 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c031960 con 0x7f85941534f0 2024-04-20T21:17:42.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f859b000640 1 -- 172.21.15.102:0/3621787334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85941534f0 msgr2=0x7f85940c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f859b000640 1 --2- 172.21.15.102:0/3621787334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85941534f0 0x7f85940c83d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85940c32d0 tx=0x7f857c02b840 comp rx=0 tx=0).stop 2024-04-20T21:17:42.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f859b000640 1 -- 172.21.15.102:0/3621787334 shutdown_connections 2024-04-20T21:17:42.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f859b000640 1 --2- 172.21.15.102:0/3621787334 >> v2:172.21.15.193:3300/0 conn(0x7f8594158f70 0x7f85940c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f859b000640 1 --2- 172.21.15.102:0/3621787334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85941534f0 0x7f85940c83d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f859b000640 1 -- 172.21.15.102:0/3621787334 >> 172.21.15.102:0/3621787334 conn(0x7f8594074ea0 msgr2=0x7f8594075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:42.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f859b000640 1 -- 172.21.15.102:0/3621787334 shutdown_connections 2024-04-20T21:17:42.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.795+0000 7f859b000640 1 -- 172.21.15.102:0/3621787334 wait complete. 2024-04-20T21:17:42.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.797+0000 7f859b000640 1 Processor -- start 2024-04-20T21:17:42.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.797+0000 7f859b000640 1 -- start start 2024-04-20T21:17:42.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f859b000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8594153110 0x7f8594153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f859b000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8594158f70 0x7f85940cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f859b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8594154bb0 0x7f8594148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f859b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85940c99c0 con 0x7f8594154bb0 2024-04-20T21:17:42.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f859b000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85940c96e0 con 0x7f8594158f70 2024-04-20T21:17:42.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f859b000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f85940c9ee0 con 0x7f8594153110 2024-04-20T21:17:42.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f8599000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8594154bb0 0x7f8594148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f8599000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8594154bb0 0x7f8594148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48380/0 (socket says 172.21.15.102:48380) 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f8599000640 1 -- 172.21.15.102:0/1874553939 learned_addr learned my addr 172.21.15.102:0/1874553939 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f8593e00640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8594153110 0x7f8594153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.798+0000 7f858be00640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8594158f70 0x7f85940cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8599000640 1 -- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8594153110 msgr2=0x7f8594153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8599000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8594153110 0x7f8594153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8599000640 1 -- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8594158f70 msgr2=0x7f85940cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8599000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8594158f70 0x7f85940cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8599000640 1 -- 172.21.15.102:0/1874553939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8594155110 con 0x7f8594154bb0 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8599000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8594154bb0 0x7f8594148e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f858c009e60 tx=0x7f858c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8590c00640 1 -- 172.21.15.102:0/1874553939 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c018030 con 0x7f8594154bb0 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8590c00640 1 -- 172.21.15.102:0/1874553939 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858c00cd90 con 0x7f8594154bb0 2024-04-20T21:17:42.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8594156b90 con 0x7f8594154bb0 2024-04-20T21:17:42.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.799+0000 7f8590c00640 1 -- 172.21.15.102:0/1874553939 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c003bb0 con 0x7f8594154bb0 2024-04-20T21:17:42.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.800+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8594156460 con 0x7f8594154bb0 2024-04-20T21:17:42.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.801+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85940c7a80 con 0x7f8594154bb0 2024-04-20T21:17:42.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.802+0000 7f8590c00640 1 -- 172.21.15.102:0/1874553939 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f858c01b070 con 0x7f8594154bb0 2024-04-20T21:17:42.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.802+0000 7f8590c00640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857006c920 0x7f857006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:42.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.803+0000 7f8590c00640 1 -- 172.21.15.102:0/1874553939 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f858c08c800 con 0x7f8594154bb0 2024-04-20T21:17:42.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.803+0000 7f8593e00640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857006c920 0x7f857006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:42.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.803+0000 7f8593e00640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857006c920 0x7f857006ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8584000f60 tx=0x7f8584007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:42.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:42.807+0000 7f8590c00640 1 -- 172.21.15.102:0/1874553939 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f858c010040 con 0x7f8594154bb0 2024-04-20T21:17:43.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.006+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f859404fdd0 con 0x7f8594154bb0 2024-04-20T21:17:43.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.007+0000 7f8590c00640 1 -- 172.21.15.102:0/1874553939 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f858c05a0b0 con 0x7f8594154bb0 2024-04-20T21:17:43.007 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:43.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857006c920 msgr2=0x7f857006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857006c920 0x7f857006ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8584000f60 tx=0x7f8584007d50 comp rx=0 tx=0).stop 2024-04-20T21:17:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8594154bb0 msgr2=0x7f8594148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8594154bb0 0x7f8594148e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f858c009e60 tx=0x7f858c00c490 comp rx=0 tx=0).stop 2024-04-20T21:17:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 shutdown_connections 2024-04-20T21:17:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8594154bb0 0x7f8594148e90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857006c920 0x7f857006ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8594158f70 0x7f85940cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 --2- 172.21.15.102:0/1874553939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8594153110 0x7f8594153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 >> 172.21.15.102:0/1874553939 conn(0x7f8594074ea0 msgr2=0x7f85940c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 shutdown_connections 2024-04-20T21:17:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.010+0000 7f859b000640 1 -- 172.21.15.102:0/1874553939 wait complete. 2024-04-20T21:17:43.029 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2024-04-20T21:17:43.030 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:43.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.231+0000 7fedbe600640 1 Processor -- start 2024-04-20T21:17:43.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.232+0000 7fedbe600640 1 -- start start 2024-04-20T21:17:43.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.233+0000 7fedbe600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fedb8162e40 0x7fedb806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.233+0000 7fedbe600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fedb8163220 0x7fedb8159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.233+0000 7fedbe600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163660 0x7fedb8159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.233+0000 7fedbe600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fedb80c7290 con 0x7fedb8163220 2024-04-20T21:17:43.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.233+0000 7fedbe600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fedb80c7570 con 0x7fedb8162e40 2024-04-20T21:17:43.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.233+0000 7fedbe600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fedb806d2e0 con 0x7fedb8163660 2024-04-20T21:17:43.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.233+0000 7fedb6a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fedb8163220 0x7fedb8159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb6a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fedb8163220 0x7fedb8159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48386/0 (socket says 172.21.15.102:48386) 2024-04-20T21:17:43.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163660 0x7fedb8159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb7400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fedb8162e40 0x7fedb806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb6a00640 1 -- 172.21.15.102:0/2686188826 learned_addr learned my addr 172.21.15.102:0/2686188826 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:43.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb6a00640 1 -- 172.21.15.102:0/2686188826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163660 msgr2=0x7fedb8159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:43.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb6a00640 1 --2- 172.21.15.102:0/2686188826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163660 0x7fedb8159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb6a00640 1 -- 172.21.15.102:0/2686188826 >> v2:172.21.15.193:3300/0 conn(0x7fedb8162e40 msgr2=0x7fedb806c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:43.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb6a00640 1 --2- 172.21.15.102:0/2686188826 >> v2:172.21.15.193:3300/0 conn(0x7fedb8162e40 0x7fedb806c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.234+0000 7fedb6a00640 1 -- 172.21.15.102:0/2686188826 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb806d5c0 con 0x7fedb8163220 2024-04-20T21:17:43.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.235+0000 7fedb6a00640 1 --2- 172.21.15.102:0/2686188826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8163220 0x7fedb8159440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fedb80cd8d0 tx=0x7feda402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797447a29c46e2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:43.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedb6000640 1 -- 172.21.15.102:0/2686188826 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4035020 con 0x7fedb8163220 2024-04-20T21:17:43.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedb6000640 1 -- 172.21.15.102:0/2686188826 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda402d040 con 0x7fedb8163220 2024-04-20T21:17:43.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedb6000640 1 -- 172.21.15.102:0/2686188826 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4031a80 con 0x7fedb8163220 2024-04-20T21:17:43.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedbe600640 1 -- 172.21.15.102:0/2686188826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8163220 msgr2=0x7fedb8159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedbe600640 1 --2- 172.21.15.102:0/2686188826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8163220 0x7fedb8159440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fedb80cd8d0 tx=0x7feda402b900 comp rx=0 tx=0).stop 2024-04-20T21:17:43.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedbe600640 1 -- 172.21.15.102:0/2686188826 shutdown_connections 2024-04-20T21:17:43.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedbe600640 1 --2- 172.21.15.102:0/2686188826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163660 0x7fedb8159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedbe600640 1 --2- 172.21.15.102:0/2686188826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8163220 0x7fedb8159440 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedbe600640 1 --2- 172.21.15.102:0/2686188826 >> v2:172.21.15.193:3300/0 conn(0x7fedb8162e40 0x7fedb806c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.236+0000 7fedbe600640 1 -- 172.21.15.102:0/2686188826 >> 172.21.15.102:0/2686188826 conn(0x7fedb80c9390 msgr2=0x7fedb80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:43.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.237+0000 7fedbe600640 1 -- 172.21.15.102:0/2686188826 shutdown_connections 2024-04-20T21:17:43.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.237+0000 7fedbe600640 1 -- 172.21.15.102:0/2686188826 wait complete. 2024-04-20T21:17:43.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.238+0000 7fedbe600640 1 Processor -- start 2024-04-20T21:17:43.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.238+0000 7fedbe600640 1 -- start start 2024-04-20T21:17:43.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.239+0000 7fedbe600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8162e40 0x7fedb80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.239+0000 7fedbe600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163220 0x7fedb815fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.239+0000 7fedbe600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedb8163660 0x7fedb8161b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.239+0000 7fedbe600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fedb80c9080 con 0x7fedb8162e40 2024-04-20T21:17:43.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.239+0000 7fedbe600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedb806db80 con 0x7fedb8163660 2024-04-20T21:17:43.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.239+0000 7fedbe600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fedb806cf20 con 0x7fedb8163220 2024-04-20T21:17:43.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8162e40 0x7fedb80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8162e40 0x7fedb80c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48394/0 (socket says 172.21.15.102:48394) 2024-04-20T21:17:43.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 -- 172.21.15.102:0/1793615730 learned_addr learned my addr 172.21.15.102:0/1793615730 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:43.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7e00640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedb8163660 0x7fedb8161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb6a00640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163220 0x7fedb815fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 -- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163220 msgr2=0x7fedb815fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163220 0x7fedb815fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 -- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedb8163660 msgr2=0x7fedb8161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedb8163660 0x7fedb8161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 -- 172.21.15.102:0/1793615730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb80c6be0 con 0x7fedb8162e40 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.240+0000 7fedb7400640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8162e40 0x7fedb80c61d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feda800e8c0 tx=0x7feda800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.241+0000 7fedaf400640 1 -- 172.21.15.102:0/1793615730 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8017070 con 0x7fedb8162e40 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.241+0000 7fedaf400640 1 -- 172.21.15.102:0/1793615730 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda800c940 con 0x7fedb8162e40 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.241+0000 7fedaf400640 1 -- 172.21.15.102:0/1793615730 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda800cd10 con 0x7fedb8162e40 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.241+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb81f5630 con 0x7fedb8162e40 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.241+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb81f5bd0 con 0x7fedb8162e40 2024-04-20T21:17:43.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.242+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb81f6570 con 0x7fedb8162e40 2024-04-20T21:17:43.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.243+0000 7fedaf400640 1 -- 172.21.15.102:0/1793615730 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feda801a070 con 0x7fedb8162e40 2024-04-20T21:17:43.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.244+0000 7fedaf400640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fed9806c920 0x7fed9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.244+0000 7fedaf400640 1 -- 172.21.15.102:0/1793615730 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7feda808c1f0 con 0x7fedb8162e40 2024-04-20T21:17:43.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.244+0000 7fedb6a00640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fed9806c920 0x7fed9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.245+0000 7fedb6a00640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fed9806c920 0x7fed9806ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feda4036000 tx=0x7feda403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:43.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.249+0000 7fedaf400640 1 -- 172.21.15.102:0/1793615730 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fedb81f6570 con 0x7fedb8162e40 2024-04-20T21:17:43.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.494+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedb8074340 con 0x7fedb8162e40 2024-04-20T21:17:43.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.495+0000 7fedaf400640 1 -- 172.21.15.102:0/1793615730 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7feda8059bd0 con 0x7fedb8162e40 2024-04-20T21:17:43.494 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:43.495 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:17:43.495 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:17:43.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.497+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fed9806c920 msgr2=0x7fed9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.497+0000 7fedbe600640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fed9806c920 0x7fed9806ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feda4036000 tx=0x7feda403c040 comp rx=0 tx=0).stop 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.497+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8162e40 msgr2=0x7fedb80c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.497+0000 7fedbe600640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8162e40 0x7fedb80c61d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feda800e8c0 tx=0x7feda800ed80 comp rx=0 tx=0).stop 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.498+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 shutdown_connections 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.498+0000 7fedbe600640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fed9806c920 0x7fed9806ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.498+0000 7fedbe600640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedb8163660 0x7fedb8161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.498+0000 7fedbe600640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedb8163220 0x7fedb815fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.498+0000 7fedbe600640 1 --2- 172.21.15.102:0/1793615730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedb8162e40 0x7fedb80c61d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.498+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 >> 172.21.15.102:0/1793615730 conn(0x7fedb80c9390 msgr2=0x7fedb8157900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.498+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 shutdown_connections 2024-04-20T21:17:43.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.498+0000 7fedbe600640 1 -- 172.21.15.102:0/1793615730 wait complete. 2024-04-20T21:17:43.517 INFO:tasks.mds_thrash:Ready to start thrashing 2024-04-20T21:17:43.517 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2024-04-20T21:17:43.517 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:43.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.716+0000 7fe84e400640 1 Processor -- start 2024-04-20T21:17:43.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.717+0000 7fe84e400640 1 -- start start 2024-04-20T21:17:43.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.718+0000 7fe84e400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe848073a10 0x7fe8480c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.718+0000 7fe84e400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8481536d0 con 0x7fe8480c9eb0 2024-04-20T21:17:43.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.718+0000 7fe84e400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8480c76c0 con 0x7fe84814fa50 2024-04-20T21:17:43.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.718+0000 7fe84e400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe8480c7960 con 0x7fe848073a10 2024-04-20T21:17:43.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.718+0000 7fe847e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe848073a10 0x7fe8480c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.719+0000 7fe83fe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe8480c9eb0 0x7fe848161930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57544/0 (socket says 172.21.15.102:57544) 2024-04-20T21:17:43.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.719+0000 7fe83fe00640 1 -- 172.21.15.102:0/274111598 learned_addr learned my addr 172.21.15.102:0/274111598 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:43.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.719+0000 7fe847e00640 1 -- 172.21.15.102:0/274111598 >> v1:172.21.15.193:6789/0 conn(0x7fe84814fa50 legacy=0x7fe8480c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.719+0000 7fe847e00640 1 -- 172.21.15.102:0/274111598 >> v1:172.21.15.102:6789/0 conn(0x7fe8480c9eb0 legacy=0x7fe848161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.719+0000 7fe846a00640 1 -- 172.21.15.102:0/274111598 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965219089 0 0) 0x7fe8480c76c0 con 0x7fe84814fa50 2024-04-20T21:17:43.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.719+0000 7fe847e00640 1 -- 172.21.15.102:0/274111598 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8480c3d90 con 0x7fe848073a10 2024-04-20T21:17:43.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.720+0000 7fe847e00640 1 --2- 172.21.15.102:0/274111598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe8480c4830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8480c7890 tx=0x7fe84002be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc05a61ddda6c070 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:43.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.721+0000 7fe846a00640 1 -- 172.21.15.102:0/274111598 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840039070 con 0x7fe848073a10 2024-04-20T21:17:43.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.721+0000 7fe846a00640 1 -- 172.21.15.102:0/274111598 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84002cb30 con 0x7fe848073a10 2024-04-20T21:17:43.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.721+0000 7fe846a00640 1 -- 172.21.15.102:0/274111598 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840034440 con 0x7fe848073a10 2024-04-20T21:17:43.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.721+0000 7fe84e400640 1 -- 172.21.15.102:0/274111598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 msgr2=0x7fe8480c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.721+0000 7fe84e400640 1 --2- 172.21.15.102:0/274111598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe8480c4830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8480c7890 tx=0x7fe84002be00 comp rx=0 tx=0).stop 2024-04-20T21:17:43.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.721+0000 7fe84e400640 1 -- 172.21.15.102:0/274111598 shutdown_connections 2024-04-20T21:17:43.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.722+0000 7fe84e400640 1 --2- 172.21.15.102:0/274111598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe8480c4830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.722+0000 7fe84e400640 1 -- 172.21.15.102:0/274111598 >> 172.21.15.102:0/274111598 conn(0x7fe8480c9390 msgr2=0x7fe8480c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:43.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.722+0000 7fe84e400640 1 -- 172.21.15.102:0/274111598 shutdown_connections 2024-04-20T21:17:43.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.722+0000 7fe84e400640 1 -- 172.21.15.102:0/274111598 wait complete. 2024-04-20T21:17:43.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.723+0000 7fe84e400640 1 Processor -- start 2024-04-20T21:17:43.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.723+0000 7fe84e400640 1 -- start start 2024-04-20T21:17:43.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.724+0000 7fe84e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe848072a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.724+0000 7fe84e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8480c9eb0 0x7fe8481f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.724+0000 7fe84e400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe84814fa50 0x7fe8480cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.724+0000 7fe84e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8480c4030 con 0x7fe848073a10 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.724+0000 7fe84e400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8480c7c00 con 0x7fe8480c9eb0 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.724+0000 7fe84e400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe8480c4310 con 0x7fe84814fa50 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe848072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe83fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8480c9eb0 0x7fe8481f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe84814fa50 0x7fe8480cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe84814fa50 0x7fe8480cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:33458/0 (socket says 172.21.15.102:33458) 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe848072a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48412/0 (socket says 172.21.15.102:48412) 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847e00640 1 -- 172.21.15.102:0/1697700712 learned_addr learned my addr 172.21.15.102:0/1697700712 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:43.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe83fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8480c9eb0 0x7fe8481f4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54130/0 (socket says 172.21.15.102:54130) 2024-04-20T21:17:43.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847e00640 1 -- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8480c9eb0 msgr2=0x7fe8481f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847e00640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8480c9eb0 0x7fe8481f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847e00640 1 -- 172.21.15.102:0/1697700712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 msgr2=0x7fe848072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847e00640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe848072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847e00640 1 -- 172.21.15.102:0/1697700712 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8481f4ce0 con 0x7fe84814fa50 2024-04-20T21:17:43.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.725+0000 7fe847400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe848072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:43.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.726+0000 7fe847e00640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe84814fa50 0x7fe8480cb860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe840037040 tx=0x7fe840004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:43.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.726+0000 7fe83f400640 1 -- 172.21.15.102:0/1697700712 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840044020 con 0x7fe84814fa50 2024-04-20T21:17:43.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.726+0000 7fe83f400640 1 -- 172.21.15.102:0/1697700712 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84003f470 con 0x7fe84814fa50 2024-04-20T21:17:43.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.726+0000 7fe83f400640 1 -- 172.21.15.102:0/1697700712 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840039040 con 0x7fe84814fa50 2024-04-20T21:17:43.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.726+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe848071740 con 0x7fe84814fa50 2024-04-20T21:17:43.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.726+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8481f4f80 con 0x7fe84814fa50 2024-04-20T21:17:43.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.727+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe84814ccd0 con 0x7fe84814fa50 2024-04-20T21:17:43.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.728+0000 7fe83f400640 1 -- 172.21.15.102:0/1697700712 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe840005d10 con 0x7fe84814fa50 2024-04-20T21:17:43.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.729+0000 7fe83f400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe82406c8f0 0x7fe82406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:43.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.729+0000 7fe83f400640 1 -- 172.21.15.102:0/1697700712 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe8400aeb80 con 0x7fe84814fa50 2024-04-20T21:17:43.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.729+0000 7fe847400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe82406c8f0 0x7fe82406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:43.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.730+0000 7fe847400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe82406c8f0 0x7fe82406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe838000f60 tx=0x7fe838009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:43.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.734+0000 7fe83f400640 1 -- 172.21.15.102:0/1697700712 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe84814ccd0 con 0x7fe84814fa50 2024-04-20T21:17:43.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.930+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 --> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8480c7320 con 0x7fe82406c8f0 2024-04-20T21:17:43.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.939+0000 7fe83f400640 1 -- 172.21.15.102:0/1697700712 <== mgr.4112 v2:172.21.15.193:6832/2803621564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332649 (secure 0 0 0) 0x7fe8480c7320 con 0x7fe82406c8f0 2024-04-20T21:17:43.944 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:43.946 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-20T21:17:43.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe82406c8f0 msgr2=0x7fe82406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe82406c8f0 0x7fe82406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe838000f60 tx=0x7fe838009040 comp rx=0 tx=0).stop 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe84814fa50 msgr2=0x7fe8480cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe84814fa50 0x7fe8480cb860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe840037040 tx=0x7fe840004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 shutdown_connections 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe84814fa50 0x7fe8480cb860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe82406c8f0 0x7fe82406edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8480c9eb0 0x7fe8481f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 --2- 172.21.15.102:0/1697700712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe848073a10 0x7fe848072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 >> 172.21.15.102:0/1697700712 conn(0x7fe8480c9390 msgr2=0x7fe84814dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 shutdown_connections 2024-04-20T21:17:43.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:43.947+0000 7fe84e400640 1 -- 172.21.15.102:0/1697700712 wait complete. 2024-04-20T21:17:43.966 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-04-20T21:17:42.673363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595266,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":210,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":319,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3481600,"data_stored":2105162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002466"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884977+0000","last_change":"2024-04-20T21:16:53.564922+0000","last_active":"2024-04-20T21:17:35.884977+0000","last_peered":"2024-04-20T21:17:35.884977+0000","last_clean":"2024-04-20T21:17:35.884977+0000","last_became_active":"2024-04-20T21:16:53.564589+0000","last_became_peered":"2024-04-20T21:16:53.564589+0000","last_unstale":"2024-04-20T21:17:35.884977+0000","last_undegraded":"2024-04-20T21:17:35.884977+0000","last_fullsized":"2024-04-20T21:17:35.884977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:47.284664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188817+0000","last_change":"2024-04-20T21:16:54.580132+0000","last_active":"2024-04-20T21:17:36.188817+0000","last_peered":"2024-04-20T21:17:36.188817+0000","last_clean":"2024-04-20T21:17:36.188817+0000","last_became_active":"2024-04-20T21:16:54.579769+0000","last_became_peered":"2024-04-20T21:16:54.579769+0000","last_unstale":"2024-04-20T21:17:36.188817+0000","last_undegraded":"2024-04-20T21:17:36.188817+0000","last_fullsized":"2024-04-20T21:17:36.188817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:00.705947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186600+0000","last_change":"2024-04-20T21:16:58.781707+0000","last_active":"2024-04-20T21:17:36.186600+0000","last_peered":"2024-04-20T21:17:36.186600+0000","last_clean":"2024-04-20T21:17:36.186600+0000","last_became_active":"2024-04-20T21:16:58.780685+0000","last_became_peered":"2024-04-20T21:16:58.780685+0000","last_unstale":"2024-04-20T21:17:36.186600+0000","last_undegraded":"2024-04-20T21:17:36.186600+0000","last_fullsized":"2024-04-20T21:17:36.186600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:50.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189794+0000","last_change":"2024-04-20T21:16:53.572137+0000","last_active":"2024-04-20T21:17:36.189794+0000","last_peered":"2024-04-20T21:17:36.189794+0000","last_clean":"2024-04-20T21:17:36.189794+0000","last_became_active":"2024-04-20T21:16:53.567907+0000","last_became_peered":"2024-04-20T21:16:53.567907+0000","last_unstale":"2024-04-20T21:17:36.189794+0000","last_undegraded":"2024-04-20T21:17:36.189794+0000","last_fullsized":"2024-04-20T21:17:36.189794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:23:28.367174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884881+0000","last_change":"2024-04-20T21:16:54.567096+0000","last_active":"2024-04-20T21:17:35.884881+0000","last_peered":"2024-04-20T21:17:35.884881+0000","last_clean":"2024-04-20T21:17:35.884881+0000","last_became_active":"2024-04-20T21:16:54.566803+0000","last_became_peered":"2024-04-20T21:16:54.566803+0000","last_unstale":"2024-04-20T21:17:35.884881+0000","last_undegraded":"2024-04-20T21:17:35.884881+0000","last_fullsized":"2024-04-20T21:17:35.884881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:13:44.730056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.191835+0000","last_change":"2024-04-20T21:17:36.191835+0000","last_active":"2024-04-20T21:17:36.191835+0000","last_peered":"2024-04-20T21:17:36.191835+0000","last_clean":"2024-04-20T21:17:36.191835+0000","last_became_active":"2024-04-20T21:17:36.191319+0000","last_became_peered":"2024-04-20T21:17:36.191319+0000","last_unstale":"2024-04-20T21:17:36.191835+0000","last_undegraded":"2024-04-20T21:17:36.191835+0000","last_fullsized":"2024-04-20T21:17:36.191835+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:29:36.899051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2,3],"acting":[4,6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186471+0000","last_change":"2024-04-20T21:16:53.554847+0000","last_active":"2024-04-20T21:17:36.186471+0000","last_peered":"2024-04-20T21:17:36.186471+0000","last_clean":"2024-04-20T21:17:36.186471+0000","last_became_active":"2024-04-20T21:16:53.554333+0000","last_became_peered":"2024-04-20T21:16:53.554333+0000","last_unstale":"2024-04-20T21:17:36.186471+0000","last_undegraded":"2024-04-20T21:17:36.186471+0000","last_fullsized":"2024-04-20T21:17:36.186471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:33.255945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878947+0000","last_change":"2024-04-20T21:16:54.556407+0000","last_active":"2024-04-20T21:17:35.878947+0000","last_peered":"2024-04-20T21:17:35.878947+0000","last_clean":"2024-04-20T21:17:35.878947+0000","last_became_active":"2024-04-20T21:16:54.556161+0000","last_became_peered":"2024-04-20T21:16:54.556161+0000","last_unstale":"2024-04-20T21:17:35.878947+0000","last_undegraded":"2024-04-20T21:17:35.878947+0000","last_fullsized":"2024-04-20T21:17:35.878947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:44:53.586714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877433+0000","last_change":"2024-04-20T21:16:58.612063+0000","last_active":"2024-04-20T21:17:35.877433+0000","last_peered":"2024-04-20T21:17:35.877433+0000","last_clean":"2024-04-20T21:17:35.877433+0000","last_became_active":"2024-04-20T21:16:58.611487+0000","last_became_peered":"2024-04-20T21:16:58.611487+0000","last_unstale":"2024-04-20T21:17:35.877433+0000","last_undegraded":"2024-04-20T21:17:35.877433+0000","last_fullsized":"2024-04-20T21:17:35.877433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:15.097731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186246+0000","last_change":"2024-04-20T21:16:53.551212+0000","last_active":"2024-04-20T21:17:36.186246+0000","last_peered":"2024-04-20T21:17:36.186246+0000","last_clean":"2024-04-20T21:17:36.186246+0000","last_became_active":"2024-04-20T21:16:53.550899+0000","last_became_peered":"2024-04-20T21:16:53.550899+0000","last_unstale":"2024-04-20T21:17:36.186246+0000","last_undegraded":"2024-04-20T21:17:36.186246+0000","last_fullsized":"2024-04-20T21:17:36.186246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:07:18.283243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878561+0000","last_change":"2024-04-20T21:16:54.557285+0000","last_active":"2024-04-20T21:17:35.878561+0000","last_peered":"2024-04-20T21:17:35.878561+0000","last_clean":"2024-04-20T21:17:35.878561+0000","last_became_active":"2024-04-20T21:16:54.556921+0000","last_became_peered":"2024-04-20T21:16:54.556921+0000","last_unstale":"2024-04-20T21:17:35.878561+0000","last_undegraded":"2024-04-20T21:17:35.878561+0000","last_fullsized":"2024-04-20T21:17:35.878561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:33.989679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877397+0000","last_change":"2024-04-20T21:16:58.617360+0000","last_active":"2024-04-20T21:17:35.877397+0000","last_peered":"2024-04-20T21:17:35.877397+0000","last_clean":"2024-04-20T21:17:35.877397+0000","last_became_active":"2024-04-20T21:16:58.616947+0000","last_became_peered":"2024-04-20T21:16:58.616947+0000","last_unstale":"2024-04-20T21:17:35.877397+0000","last_undegraded":"2024-04-20T21:17:35.877397+0000","last_fullsized":"2024-04-20T21:17:35.877397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.202955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937723+0000","last_change":"2024-04-20T21:16:54.558067+0000","last_active":"2024-04-20T21:17:35.937723+0000","last_peered":"2024-04-20T21:17:35.937723+0000","last_clean":"2024-04-20T21:17:35.937723+0000","last_became_active":"2024-04-20T21:16:54.557781+0000","last_became_peered":"2024-04-20T21:16:54.557781+0000","last_unstale":"2024-04-20T21:17:35.937723+0000","last_undegraded":"2024-04-20T21:17:35.937723+0000","last_fullsized":"2024-04-20T21:17:35.937723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:56.576081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186328+0000","last_change":"2024-04-20T21:16:53.551615+0000","last_active":"2024-04-20T21:17:36.186328+0000","last_peered":"2024-04-20T21:17:36.186328+0000","last_clean":"2024-04-20T21:17:36.186328+0000","last_became_active":"2024-04-20T21:16:53.551073+0000","last_became_peered":"2024-04-20T21:16:53.551073+0000","last_unstale":"2024-04-20T21:17:36.186328+0000","last_undegraded":"2024-04-20T21:17:36.186328+0000","last_fullsized":"2024-04-20T21:17:36.186328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:26:29.994289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877528+0000","last_change":"2024-04-20T21:16:58.617434+0000","last_active":"2024-04-20T21:17:35.877528+0000","last_peered":"2024-04-20T21:17:35.877528+0000","last_clean":"2024-04-20T21:17:35.877528+0000","last_became_active":"2024-04-20T21:16:58.617027+0000","last_became_peered":"2024-04-20T21:16:58.617027+0000","last_unstale":"2024-04-20T21:17:35.877528+0000","last_undegraded":"2024-04-20T21:17:35.877528+0000","last_fullsized":"2024-04-20T21:17:35.877528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:44:20.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188865+0000","last_change":"2024-04-20T21:16:54.585333+0000","last_active":"2024-04-20T21:17:36.188865+0000","last_peered":"2024-04-20T21:17:36.188865+0000","last_clean":"2024-04-20T21:17:36.188865+0000","last_became_active":"2024-04-20T21:16:54.584978+0000","last_became_peered":"2024-04-20T21:16:54.584978+0000","last_unstale":"2024-04-20T21:17:36.188865+0000","last_undegraded":"2024-04-20T21:17:36.188865+0000","last_fullsized":"2024-04-20T21:17:36.188865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:23.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879889+0000","last_change":"2024-04-20T21:16:53.567775+0000","last_active":"2024-04-20T21:17:35.879889+0000","last_peered":"2024-04-20T21:17:35.879889+0000","last_clean":"2024-04-20T21:17:35.879889+0000","last_became_active":"2024-04-20T21:16:53.567498+0000","last_became_peered":"2024-04-20T21:16:53.567498+0000","last_unstale":"2024-04-20T21:17:35.879889+0000","last_undegraded":"2024-04-20T21:17:35.879889+0000","last_fullsized":"2024-04-20T21:17:35.879889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:10.695659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878180+0000","last_change":"2024-04-20T21:16:58.619302+0000","last_active":"2024-04-20T21:17:35.878180+0000","last_peered":"2024-04-20T21:17:35.878180+0000","last_clean":"2024-04-20T21:17:35.878180+0000","last_became_active":"2024-04-20T21:16:58.618843+0000","last_became_peered":"2024-04-20T21:16:58.618843+0000","last_unstale":"2024-04-20T21:17:35.878180+0000","last_undegraded":"2024-04-20T21:17:35.878180+0000","last_fullsized":"2024-04-20T21:17:35.878180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:14.927199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188147+0000","last_change":"2024-04-20T21:16:54.575862+0000","last_active":"2024-04-20T21:17:36.188147+0000","last_peered":"2024-04-20T21:17:36.188147+0000","last_clean":"2024-04-20T21:17:36.188147+0000","last_became_active":"2024-04-20T21:16:54.575523+0000","last_became_peered":"2024-04-20T21:16:54.575523+0000","last_unstale":"2024-04-20T21:17:36.188147+0000","last_undegraded":"2024-04-20T21:17:36.188147+0000","last_fullsized":"2024-04-20T21:17:36.188147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:51.041414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880494+0000","last_change":"2024-04-20T21:16:53.551995+0000","last_active":"2024-04-20T21:17:35.880494+0000","last_peered":"2024-04-20T21:17:35.880494+0000","last_clean":"2024-04-20T21:17:35.880494+0000","last_became_active":"2024-04-20T21:16:53.551603+0000","last_became_peered":"2024-04-20T21:16:53.551603+0000","last_unstale":"2024-04-20T21:17:35.880494+0000","last_undegraded":"2024-04-20T21:17:35.880494+0000","last_fullsized":"2024-04-20T21:17:35.880494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:56:21.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878943+0000","last_change":"2024-04-20T21:16:58.776463+0000","last_active":"2024-04-20T21:17:35.878943+0000","last_peered":"2024-04-20T21:17:35.878943+0000","last_clean":"2024-04-20T21:17:35.878943+0000","last_became_active":"2024-04-20T21:16:58.775746+0000","last_became_peered":"2024-04-20T21:16:58.775746+0000","last_unstale":"2024-04-20T21:17:35.878943+0000","last_undegraded":"2024-04-20T21:17:35.878943+0000","last_fullsized":"2024-04-20T21:17:35.878943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:28.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878811+0000","last_change":"2024-04-20T21:16:54.566449+0000","last_active":"2024-04-20T21:17:35.878811+0000","last_peered":"2024-04-20T21:17:35.878811+0000","last_clean":"2024-04-20T21:17:35.878811+0000","last_became_active":"2024-04-20T21:16:54.566251+0000","last_became_peered":"2024-04-20T21:16:54.566251+0000","last_unstale":"2024-04-20T21:17:35.878811+0000","last_undegraded":"2024-04-20T21:17:35.878811+0000","last_fullsized":"2024-04-20T21:17:35.878811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:53.403420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185741+0000","last_change":"2024-04-20T21:16:53.564029+0000","last_active":"2024-04-20T21:17:36.185741+0000","last_peered":"2024-04-20T21:17:36.185741+0000","last_clean":"2024-04-20T21:17:36.185741+0000","last_became_active":"2024-04-20T21:16:53.563711+0000","last_became_peered":"2024-04-20T21:16:53.563711+0000","last_unstale":"2024-04-20T21:17:36.185741+0000","last_undegraded":"2024-04-20T21:17:36.185741+0000","last_fullsized":"2024-04-20T21:17:36.185741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.634644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878711+0000","last_change":"2024-04-20T21:16:58.615080+0000","last_active":"2024-04-20T21:17:35.878711+0000","last_peered":"2024-04-20T21:17:35.878711+0000","last_clean":"2024-04-20T21:17:35.878711+0000","last_became_active":"2024-04-20T21:16:58.614611+0000","last_became_peered":"2024-04-20T21:16:58.614611+0000","last_unstale":"2024-04-20T21:17:35.878711+0000","last_undegraded":"2024-04-20T21:17:35.878711+0000","last_fullsized":"2024-04-20T21:17:35.878711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:08.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884557+0000","last_change":"2024-04-20T21:16:54.578272+0000","last_active":"2024-04-20T21:17:35.884557+0000","last_peered":"2024-04-20T21:17:35.884557+0000","last_clean":"2024-04-20T21:17:35.884557+0000","last_became_active":"2024-04-20T21:16:54.577949+0000","last_became_peered":"2024-04-20T21:16:54.577949+0000","last_unstale":"2024-04-20T21:17:35.884557+0000","last_undegraded":"2024-04-20T21:17:35.884557+0000","last_fullsized":"2024-04-20T21:17:35.884557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:35:33.807091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938557+0000","last_change":"2024-04-20T21:16:53.554600+0000","last_active":"2024-04-20T21:17:35.938557+0000","last_peered":"2024-04-20T21:17:35.938557+0000","last_clean":"2024-04-20T21:17:35.938557+0000","last_became_active":"2024-04-20T21:16:53.554347+0000","last_became_peered":"2024-04-20T21:16:53.554347+0000","last_unstale":"2024-04-20T21:17:35.938557+0000","last_undegraded":"2024-04-20T21:17:35.938557+0000","last_fullsized":"2024-04-20T21:17:35.938557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:40:58.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878403+0000","last_change":"2024-04-20T21:16:58.774474+0000","last_active":"2024-04-20T21:17:35.878403+0000","last_peered":"2024-04-20T21:17:35.878403+0000","last_clean":"2024-04-20T21:17:35.878403+0000","last_became_active":"2024-04-20T21:16:58.774037+0000","last_became_peered":"2024-04-20T21:16:58.774037+0000","last_unstale":"2024-04-20T21:17:35.878403+0000","last_undegraded":"2024-04-20T21:17:35.878403+0000","last_fullsized":"2024-04-20T21:17:35.878403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:46.992667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189865+0000","last_change":"2024-04-20T21:17:36.187769+0000","last_active":"2024-04-20T21:17:36.189865+0000","last_peered":"2024-04-20T21:17:36.189865+0000","last_clean":"2024-04-20T21:17:36.189865+0000","last_became_active":"2024-04-20T21:17:36.187283+0000","last_became_peered":"2024-04-20T21:17:36.187283+0000","last_unstale":"2024-04-20T21:17:36.189865+0000","last_undegraded":"2024-04-20T21:17:36.189865+0000","last_fullsized":"2024-04-20T21:17:36.189865+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:46:58.418395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878574+0000","last_change":"2024-04-20T21:16:53.561144+0000","last_active":"2024-04-20T21:17:35.878574+0000","last_peered":"2024-04-20T21:17:35.878574+0000","last_clean":"2024-04-20T21:17:35.878574+0000","last_became_active":"2024-04-20T21:16:53.560742+0000","last_became_peered":"2024-04-20T21:16:53.560742+0000","last_unstale":"2024-04-20T21:17:35.878574+0000","last_undegraded":"2024-04-20T21:17:35.878574+0000","last_fullsized":"2024-04-20T21:17:35.878574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:39.212951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186836+0000","last_change":"2024-04-20T21:16:58.780124+0000","last_active":"2024-04-20T21:17:36.186836+0000","last_peered":"2024-04-20T21:17:36.186836+0000","last_clean":"2024-04-20T21:17:36.186836+0000","last_became_active":"2024-04-20T21:16:58.777449+0000","last_became_peered":"2024-04-20T21:16:58.777449+0000","last_unstale":"2024-04-20T21:17:36.186836+0000","last_undegraded":"2024-04-20T21:17:36.186836+0000","last_fullsized":"2024-04-20T21:17:36.186836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:07.622146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879405+0000","last_change":"2024-04-20T21:16:54.575331+0000","last_active":"2024-04-20T21:17:35.879405+0000","last_peered":"2024-04-20T21:17:35.879405+0000","last_clean":"2024-04-20T21:17:35.879405+0000","last_became_active":"2024-04-20T21:16:54.575061+0000","last_became_peered":"2024-04-20T21:16:54.575061+0000","last_unstale":"2024-04-20T21:17:35.879405+0000","last_undegraded":"2024-04-20T21:17:35.879405+0000","last_fullsized":"2024-04-20T21:17:35.879405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:47:20.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186611+0000","last_change":"2024-04-20T21:16:53.565561+0000","last_active":"2024-04-20T21:17:36.186611+0000","last_peered":"2024-04-20T21:17:36.186611+0000","last_clean":"2024-04-20T21:17:36.186611+0000","last_became_active":"2024-04-20T21:16:53.565207+0000","last_became_peered":"2024-04-20T21:16:53.565207+0000","last_unstale":"2024-04-20T21:17:36.186611+0000","last_undegraded":"2024-04-20T21:17:36.186611+0000","last_fullsized":"2024-04-20T21:17:36.186611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:07.512132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188224+0000","last_change":"2024-04-20T21:17:36.188224+0000","last_active":"2024-04-20T21:17:36.188224+0000","last_peered":"2024-04-20T21:17:36.188224+0000","last_clean":"2024-04-20T21:17:36.188224+0000","last_became_active":"2024-04-20T21:17:36.187615+0000","last_became_peered":"2024-04-20T21:17:36.187615+0000","last_unstale":"2024-04-20T21:17:36.188224+0000","last_undegraded":"2024-04-20T21:17:36.188224+0000","last_fullsized":"2024-04-20T21:17:36.188224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:45:48.216716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,2],"acting":[0,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878608+0000","last_change":"2024-04-20T21:16:54.571199+0000","last_active":"2024-04-20T21:17:35.878608+0000","last_peered":"2024-04-20T21:17:35.878608+0000","last_clean":"2024-04-20T21:17:35.878608+0000","last_became_active":"2024-04-20T21:16:54.570882+0000","last_became_peered":"2024-04-20T21:16:54.570882+0000","last_unstale":"2024-04-20T21:17:35.878608+0000","last_undegraded":"2024-04-20T21:17:35.878608+0000","last_fullsized":"2024-04-20T21:17:35.878608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:52.046176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879661+0000","last_change":"2024-04-20T21:16:53.566253+0000","last_active":"2024-04-20T21:17:35.879661+0000","last_peered":"2024-04-20T21:17:35.879661+0000","last_clean":"2024-04-20T21:17:35.879661+0000","last_became_active":"2024-04-20T21:16:53.565893+0000","last_became_peered":"2024-04-20T21:16:53.565893+0000","last_unstale":"2024-04-20T21:17:35.879661+0000","last_undegraded":"2024-04-20T21:17:35.879661+0000","last_fullsized":"2024-04-20T21:17:35.879661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:16.302801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879509+0000","last_change":"2024-04-20T21:16:58.611470+0000","last_active":"2024-04-20T21:17:35.879509+0000","last_peered":"2024-04-20T21:17:35.879509+0000","last_clean":"2024-04-20T21:17:35.879509+0000","last_became_active":"2024-04-20T21:16:58.611038+0000","last_became_peered":"2024-04-20T21:16:58.611038+0000","last_unstale":"2024-04-20T21:17:35.879509+0000","last_undegraded":"2024-04-20T21:17:35.879509+0000","last_fullsized":"2024-04-20T21:17:35.879509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:58:51.267480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877868+0000","last_change":"2024-04-20T21:16:54.577608+0000","last_active":"2024-04-20T21:17:35.877868+0000","last_peered":"2024-04-20T21:17:35.877868+0000","last_clean":"2024-04-20T21:17:35.877868+0000","last_became_active":"2024-04-20T21:16:54.577339+0000","last_became_peered":"2024-04-20T21:16:54.577339+0000","last_unstale":"2024-04-20T21:17:35.877868+0000","last_undegraded":"2024-04-20T21:17:35.877868+0000","last_fullsized":"2024-04-20T21:17:35.877868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:03.028585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879518+0000","last_change":"2024-04-20T21:16:53.547751+0000","last_active":"2024-04-20T21:17:35.879518+0000","last_peered":"2024-04-20T21:17:35.879518+0000","last_clean":"2024-04-20T21:17:35.879518+0000","last_became_active":"2024-04-20T21:16:53.547371+0000","last_became_peered":"2024-04-20T21:16:53.547371+0000","last_unstale":"2024-04-20T21:17:35.879518+0000","last_undegraded":"2024-04-20T21:17:35.879518+0000","last_fullsized":"2024-04-20T21:17:35.879518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:50.728500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878920+0000","last_change":"2024-04-20T21:16:58.622369+0000","last_active":"2024-04-20T21:17:35.878920+0000","last_peered":"2024-04-20T21:17:35.878920+0000","last_clean":"2024-04-20T21:17:35.878920+0000","last_became_active":"2024-04-20T21:16:58.621911+0000","last_became_peered":"2024-04-20T21:16:58.621911+0000","last_unstale":"2024-04-20T21:17:35.878920+0000","last_undegraded":"2024-04-20T21:17:35.878920+0000","last_fullsized":"2024-04-20T21:17:35.878920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:41.519583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878299+0000","last_change":"2024-04-20T21:16:54.579577+0000","last_active":"2024-04-20T21:17:35.878299+0000","last_peered":"2024-04-20T21:17:35.878299+0000","last_clean":"2024-04-20T21:17:35.878299+0000","last_became_active":"2024-04-20T21:16:54.579156+0000","last_became_peered":"2024-04-20T21:16:54.579156+0000","last_unstale":"2024-04-20T21:17:35.878299+0000","last_undegraded":"2024-04-20T21:17:35.878299+0000","last_fullsized":"2024-04-20T21:17:35.878299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:28:14.743667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.886735+0000","last_change":"2024-04-20T21:16:53.567995+0000","last_active":"2024-04-20T21:17:35.886735+0000","last_peered":"2024-04-20T21:17:35.886735+0000","last_clean":"2024-04-20T21:17:35.886735+0000","last_became_active":"2024-04-20T21:16:53.567601+0000","last_became_peered":"2024-04-20T21:16:53.567601+0000","last_unstale":"2024-04-20T21:17:35.886735+0000","last_undegraded":"2024-04-20T21:17:35.886735+0000","last_fullsized":"2024-04-20T21:17:35.886735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:30.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879045+0000","last_change":"2024-04-20T21:16:58.626438+0000","last_active":"2024-04-20T21:17:35.879045+0000","last_peered":"2024-04-20T21:17:35.879045+0000","last_clean":"2024-04-20T21:17:35.879045+0000","last_became_active":"2024-04-20T21:16:58.625326+0000","last_became_peered":"2024-04-20T21:16:58.625326+0000","last_unstale":"2024-04-20T21:17:35.879045+0000","last_undegraded":"2024-04-20T21:17:35.879045+0000","last_fullsized":"2024-04-20T21:17:35.879045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:06.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883279+0000","last_change":"2024-04-20T21:16:54.560925+0000","last_active":"2024-04-20T21:17:35.883279+0000","last_peered":"2024-04-20T21:17:35.883279+0000","last_clean":"2024-04-20T21:17:35.883279+0000","last_became_active":"2024-04-20T21:16:54.560615+0000","last_became_peered":"2024-04-20T21:16:54.560615+0000","last_unstale":"2024-04-20T21:17:35.883279+0000","last_undegraded":"2024-04-20T21:17:35.883279+0000","last_fullsized":"2024-04-20T21:17:35.883279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:12.815692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185774+0000","last_change":"2024-04-20T21:16:53.559359+0000","last_active":"2024-04-20T21:17:36.185774+0000","last_peered":"2024-04-20T21:17:36.185774+0000","last_clean":"2024-04-20T21:17:36.185774+0000","last_became_active":"2024-04-20T21:16:53.558955+0000","last_became_peered":"2024-04-20T21:16:53.558955+0000","last_unstale":"2024-04-20T21:17:36.185774+0000","last_undegraded":"2024-04-20T21:17:36.185774+0000","last_fullsized":"2024-04-20T21:17:36.185774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:28.664404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884324+0000","last_change":"2024-04-20T21:16:58.772175+0000","last_active":"2024-04-20T21:17:35.884324+0000","last_peered":"2024-04-20T21:17:35.884324+0000","last_clean":"2024-04-20T21:17:35.884324+0000","last_became_active":"2024-04-20T21:16:58.771732+0000","last_became_peered":"2024-04-20T21:16:58.771732+0000","last_unstale":"2024-04-20T21:17:35.884324+0000","last_undegraded":"2024-04-20T21:17:35.884324+0000","last_fullsized":"2024-04-20T21:17:35.884324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:13.155547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186877+0000","last_change":"2024-04-20T21:16:54.575375+0000","last_active":"2024-04-20T21:17:36.186877+0000","last_peered":"2024-04-20T21:17:36.186877+0000","last_clean":"2024-04-20T21:17:36.186877+0000","last_became_active":"2024-04-20T21:16:54.574194+0000","last_became_peered":"2024-04-20T21:16:54.574194+0000","last_unstale":"2024-04-20T21:17:36.186877+0000","last_undegraded":"2024-04-20T21:17:36.186877+0000","last_fullsized":"2024-04-20T21:17:36.186877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:07.948431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878007+0000","last_change":"2024-04-20T21:16:53.559374+0000","last_active":"2024-04-20T21:17:35.878007+0000","last_peered":"2024-04-20T21:17:35.878007+0000","last_clean":"2024-04-20T21:17:35.878007+0000","last_became_active":"2024-04-20T21:16:53.559133+0000","last_became_peered":"2024-04-20T21:16:53.559133+0000","last_unstale":"2024-04-20T21:17:35.878007+0000","last_undegraded":"2024-04-20T21:17:35.878007+0000","last_fullsized":"2024-04-20T21:17:35.878007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:38.576103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185908+0000","last_change":"2024-04-20T21:16:44.100529+0000","last_active":"2024-04-20T21:17:36.185908+0000","last_peered":"2024-04-20T21:17:36.185908+0000","last_clean":"2024-04-20T21:17:36.185908+0000","last_became_active":"2024-04-20T21:16:44.099590+0000","last_became_peered":"2024-04-20T21:16:44.099590+0000","last_unstale":"2024-04-20T21:17:36.185908+0000","last_undegraded":"2024-04-20T21:17:36.185908+0000","last_fullsized":"2024-04-20T21:17:36.185908+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_clean_scrub_stamp":"2024-04-20T21:16:42.782844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:08.054837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937912+0000","last_change":"2024-04-20T21:16:58.772894+0000","last_active":"2024-04-20T21:17:35.937912+0000","last_peered":"2024-04-20T21:17:35.937912+0000","last_clean":"2024-04-20T21:17:35.937912+0000","last_became_active":"2024-04-20T21:16:58.772451+0000","last_became_peered":"2024-04-20T21:16:58.772451+0000","last_unstale":"2024-04-20T21:17:35.937912+0000","last_undegraded":"2024-04-20T21:17:35.937912+0000","last_fullsized":"2024-04-20T21:17:35.937912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:29.914096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186355+0000","last_change":"2024-04-20T21:16:54.562592+0000","last_active":"2024-04-20T21:17:36.186355+0000","last_peered":"2024-04-20T21:17:36.186355+0000","last_clean":"2024-04-20T21:17:36.186355+0000","last_became_active":"2024-04-20T21:16:54.562243+0000","last_became_peered":"2024-04-20T21:16:54.562243+0000","last_unstale":"2024-04-20T21:17:36.186355+0000","last_undegraded":"2024-04-20T21:17:36.186355+0000","last_fullsized":"2024-04-20T21:17:36.186355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:52.832159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887300+0000","last_change":"2024-04-20T21:16:53.561132+0000","last_active":"2024-04-20T21:17:35.887300+0000","last_peered":"2024-04-20T21:17:35.887300+0000","last_clean":"2024-04-20T21:17:35.887300+0000","last_became_active":"2024-04-20T21:16:53.559901+0000","last_became_peered":"2024-04-20T21:16:53.559901+0000","last_unstale":"2024-04-20T21:17:35.887300+0000","last_undegraded":"2024-04-20T21:17:35.887300+0000","last_fullsized":"2024-04-20T21:17:35.887300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:20.229573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937705+0000","last_change":"2024-04-20T21:16:58.771204+0000","last_active":"2024-04-20T21:17:35.937705+0000","last_peered":"2024-04-20T21:17:35.937705+0000","last_clean":"2024-04-20T21:17:35.937705+0000","last_became_active":"2024-04-20T21:16:58.770627+0000","last_became_peered":"2024-04-20T21:16:58.770627+0000","last_unstale":"2024-04-20T21:17:35.937705+0000","last_undegraded":"2024-04-20T21:17:35.937705+0000","last_fullsized":"2024-04-20T21:17:35.937705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:39.472931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.194007+0000","last_change":"2024-04-20T21:17:36.192480+0000","last_active":"2024-04-20T21:17:36.194007+0000","last_peered":"2024-04-20T21:17:36.194007+0000","last_clean":"2024-04-20T21:17:36.194007+0000","last_became_active":"2024-04-20T21:17:36.190832+0000","last_became_peered":"2024-04-20T21:17:36.190832+0000","last_unstale":"2024-04-20T21:17:36.194007+0000","last_undegraded":"2024-04-20T21:17:36.194007+0000","last_fullsized":"2024-04-20T21:17:36.194007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:41.520453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.881260+0000","last_change":"2024-04-20T21:16:53.558554+0000","last_active":"2024-04-20T21:17:35.881260+0000","last_peered":"2024-04-20T21:17:35.881260+0000","last_clean":"2024-04-20T21:17:35.881260+0000","last_became_active":"2024-04-20T21:16:53.556934+0000","last_became_peered":"2024-04-20T21:16:53.556934+0000","last_unstale":"2024-04-20T21:17:35.881260+0000","last_undegraded":"2024-04-20T21:17:35.881260+0000","last_fullsized":"2024-04-20T21:17:35.881260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:17:52.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884183+0000","last_change":"2024-04-20T21:16:58.624410+0000","last_active":"2024-04-20T21:17:35.884183+0000","last_peered":"2024-04-20T21:17:35.884183+0000","last_clean":"2024-04-20T21:17:35.884183+0000","last_became_active":"2024-04-20T21:16:58.623677+0000","last_became_peered":"2024-04-20T21:16:58.623677+0000","last_unstale":"2024-04-20T21:17:35.884183+0000","last_undegraded":"2024-04-20T21:17:35.884183+0000","last_fullsized":"2024-04-20T21:17:35.884183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:34.465594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186797+0000","last_change":"2024-04-20T21:16:54.560217+0000","last_active":"2024-04-20T21:17:36.186797+0000","last_peered":"2024-04-20T21:17:36.186797+0000","last_clean":"2024-04-20T21:17:36.186797+0000","last_became_active":"2024-04-20T21:16:54.559947+0000","last_became_peered":"2024-04-20T21:16:54.559947+0000","last_unstale":"2024-04-20T21:17:36.186797+0000","last_undegraded":"2024-04-20T21:17:36.186797+0000","last_fullsized":"2024-04-20T21:17:36.186797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:54.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887513+0000","last_change":"2024-04-20T21:16:53.564779+0000","last_active":"2024-04-20T21:17:35.887513+0000","last_peered":"2024-04-20T21:17:35.887513+0000","last_clean":"2024-04-20T21:17:35.887513+0000","last_became_active":"2024-04-20T21:16:53.564500+0000","last_became_peered":"2024-04-20T21:16:53.564500+0000","last_unstale":"2024-04-20T21:17:35.887513+0000","last_undegraded":"2024-04-20T21:17:35.887513+0000","last_fullsized":"2024-04-20T21:17:35.887513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:00:58.673145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877151+0000","last_change":"2024-04-20T21:16:58.771524+0000","last_active":"2024-04-20T21:17:35.877151+0000","last_peered":"2024-04-20T21:17:35.877151+0000","last_clean":"2024-04-20T21:17:35.877151+0000","last_became_active":"2024-04-20T21:16:58.770672+0000","last_became_peered":"2024-04-20T21:16:58.770672+0000","last_unstale":"2024-04-20T21:17:35.877151+0000","last_undegraded":"2024-04-20T21:17:35.877151+0000","last_fullsized":"2024-04-20T21:17:35.877151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:52:27.037256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186604+0000","last_change":"2024-04-20T21:16:54.559570+0000","last_active":"2024-04-20T21:17:36.186604+0000","last_peered":"2024-04-20T21:17:36.186604+0000","last_clean":"2024-04-20T21:17:36.186604+0000","last_became_active":"2024-04-20T21:16:54.559307+0000","last_became_peered":"2024-04-20T21:16:54.559307+0000","last_unstale":"2024-04-20T21:17:36.186604+0000","last_undegraded":"2024-04-20T21:17:36.186604+0000","last_fullsized":"2024-04-20T21:17:36.186604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:04.417348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879042+0000","last_change":"2024-04-20T21:16:53.565289+0000","last_active":"2024-04-20T21:17:35.879042+0000","last_peered":"2024-04-20T21:17:35.879042+0000","last_clean":"2024-04-20T21:17:35.879042+0000","last_became_active":"2024-04-20T21:16:53.565014+0000","last_became_peered":"2024-04-20T21:16:53.565014+0000","last_unstale":"2024-04-20T21:17:35.879042+0000","last_undegraded":"2024-04-20T21:17:35.879042+0000","last_fullsized":"2024-04-20T21:17:35.879042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:22.125301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878310+0000","last_change":"2024-04-20T21:16:58.615103+0000","last_active":"2024-04-20T21:17:35.878310+0000","last_peered":"2024-04-20T21:17:35.878310+0000","last_clean":"2024-04-20T21:17:35.878310+0000","last_became_active":"2024-04-20T21:16:58.614068+0000","last_became_peered":"2024-04-20T21:16:58.614068+0000","last_unstale":"2024-04-20T21:17:35.878310+0000","last_undegraded":"2024-04-20T21:17:35.878310+0000","last_fullsized":"2024-04-20T21:17:35.878310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:09.555911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.896249+0000","last_change":"2024-04-20T21:17:35.891448+0000","last_active":"2024-04-20T21:17:35.896249+0000","last_peered":"2024-04-20T21:17:35.896249+0000","last_clean":"2024-04-20T21:17:35.896249+0000","last_became_active":"2024-04-20T21:17:35.891149+0000","last_became_peered":"2024-04-20T21:17:35.891149+0000","last_unstale":"2024-04-20T21:17:35.896249+0000","last_undegraded":"2024-04-20T21:17:35.896249+0000","last_fullsized":"2024-04-20T21:17:35.896249+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:24:05.580789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.885970+0000","last_change":"2024-04-20T21:16:53.570182+0000","last_active":"2024-04-20T21:17:35.885970+0000","last_peered":"2024-04-20T21:17:35.885970+0000","last_clean":"2024-04-20T21:17:35.885970+0000","last_became_active":"2024-04-20T21:16:53.569632+0000","last_became_peered":"2024-04-20T21:16:53.569632+0000","last_unstale":"2024-04-20T21:17:35.885970+0000","last_undegraded":"2024-04-20T21:17:35.885970+0000","last_fullsized":"2024-04-20T21:17:35.885970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:12.140497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884216+0000","last_change":"2024-04-20T21:16:58.625602+0000","last_active":"2024-04-20T21:17:35.884216+0000","last_peered":"2024-04-20T21:17:35.884216+0000","last_clean":"2024-04-20T21:17:35.884216+0000","last_became_active":"2024-04-20T21:16:58.625158+0000","last_became_peered":"2024-04-20T21:16:58.625158+0000","last_unstale":"2024-04-20T21:17:35.884216+0000","last_undegraded":"2024-04-20T21:17:35.884216+0000","last_fullsized":"2024-04-20T21:17:35.884216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:09.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938129+0000","last_change":"2024-04-20T21:16:54.578603+0000","last_active":"2024-04-20T21:17:35.938129+0000","last_peered":"2024-04-20T21:17:35.938129+0000","last_clean":"2024-04-20T21:17:35.938129+0000","last_became_active":"2024-04-20T21:16:54.578338+0000","last_became_peered":"2024-04-20T21:16:54.578338+0000","last_unstale":"2024-04-20T21:17:35.938129+0000","last_undegraded":"2024-04-20T21:17:35.938129+0000","last_fullsized":"2024-04-20T21:17:35.938129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:14:56.355343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937825+0000","last_change":"2024-04-20T21:16:53.551847+0000","last_active":"2024-04-20T21:17:35.937825+0000","last_peered":"2024-04-20T21:17:35.937825+0000","last_clean":"2024-04-20T21:17:35.937825+0000","last_became_active":"2024-04-20T21:16:53.551427+0000","last_became_peered":"2024-04-20T21:16:53.551427+0000","last_unstale":"2024-04-20T21:17:35.937825+0000","last_undegraded":"2024-04-20T21:17:35.937825+0000","last_fullsized":"2024-04-20T21:17:35.937825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:55:45.930305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189251+0000","last_change":"2024-04-20T21:16:58.777159+0000","last_active":"2024-04-20T21:17:36.189251+0000","last_peered":"2024-04-20T21:17:36.189251+0000","last_clean":"2024-04-20T21:17:36.189251+0000","last_became_active":"2024-04-20T21:16:58.776233+0000","last_became_peered":"2024-04-20T21:16:58.776233+0000","last_unstale":"2024-04-20T21:17:36.189251+0000","last_undegraded":"2024-04-20T21:17:36.189251+0000","last_fullsized":"2024-04-20T21:17:36.189251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:05.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186236+0000","last_change":"2024-04-20T21:16:54.570687+0000","last_active":"2024-04-20T21:17:36.186236+0000","last_peered":"2024-04-20T21:17:36.186236+0000","last_clean":"2024-04-20T21:17:36.186236+0000","last_became_active":"2024-04-20T21:16:54.570360+0000","last_became_peered":"2024-04-20T21:16:54.570360+0000","last_unstale":"2024-04-20T21:17:36.186236+0000","last_undegraded":"2024-04-20T21:17:36.186236+0000","last_fullsized":"2024-04-20T21:17:36.186236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:05.606229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879415+0000","last_change":"2024-04-20T21:16:53.567599+0000","last_active":"2024-04-20T21:17:35.879415+0000","last_peered":"2024-04-20T21:17:35.879415+0000","last_clean":"2024-04-20T21:17:35.879415+0000","last_became_active":"2024-04-20T21:16:53.567378+0000","last_became_peered":"2024-04-20T21:16:53.567378+0000","last_unstale":"2024-04-20T21:17:35.879415+0000","last_undegraded":"2024-04-20T21:17:35.879415+0000","last_fullsized":"2024-04-20T21:17:35.879415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:36.611887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188405+0000","last_change":"2024-04-20T21:16:58.774226+0000","last_active":"2024-04-20T21:17:36.188405+0000","last_peered":"2024-04-20T21:17:36.188405+0000","last_clean":"2024-04-20T21:17:36.188405+0000","last_became_active":"2024-04-20T21:16:58.771584+0000","last_became_peered":"2024-04-20T21:16:58.771584+0000","last_unstale":"2024-04-20T21:17:36.188405+0000","last_undegraded":"2024-04-20T21:17:36.188405+0000","last_fullsized":"2024-04-20T21:17:36.188405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:31.432077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883628+0000","last_change":"2024-04-20T21:16:54.560001+0000","last_active":"2024-04-20T21:17:35.883628+0000","last_peered":"2024-04-20T21:17:35.883628+0000","last_clean":"2024-04-20T21:17:35.883628+0000","last_became_active":"2024-04-20T21:16:54.559643+0000","last_became_peered":"2024-04-20T21:16:54.559643+0000","last_unstale":"2024-04-20T21:17:35.883628+0000","last_undegraded":"2024-04-20T21:17:35.883628+0000","last_fullsized":"2024-04-20T21:17:35.883628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:52:37.325946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884673+0000","last_change":"2024-04-20T21:16:53.570498+0000","last_active":"2024-04-20T21:17:35.884673+0000","last_peered":"2024-04-20T21:17:35.884673+0000","last_clean":"2024-04-20T21:17:35.884673+0000","last_became_active":"2024-04-20T21:16:53.570095+0000","last_became_peered":"2024-04-20T21:16:53.570095+0000","last_unstale":"2024-04-20T21:17:35.884673+0000","last_undegraded":"2024-04-20T21:17:35.884673+0000","last_fullsized":"2024-04-20T21:17:35.884673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:58:28.677814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878493+0000","last_change":"2024-04-20T21:16:58.773349+0000","last_active":"2024-04-20T21:17:35.878493+0000","last_peered":"2024-04-20T21:17:35.878493+0000","last_clean":"2024-04-20T21:17:35.878493+0000","last_became_active":"2024-04-20T21:16:58.772781+0000","last_became_peered":"2024-04-20T21:16:58.772781+0000","last_unstale":"2024-04-20T21:17:35.878493+0000","last_undegraded":"2024-04-20T21:17:35.878493+0000","last_fullsized":"2024-04-20T21:17:35.878493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:53:02.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186216+0000","last_change":"2024-04-20T21:16:54.572749+0000","last_active":"2024-04-20T21:17:36.186216+0000","last_peered":"2024-04-20T21:17:36.186216+0000","last_clean":"2024-04-20T21:17:36.186216+0000","last_became_active":"2024-04-20T21:16:54.572456+0000","last_became_peered":"2024-04-20T21:16:54.572456+0000","last_unstale":"2024-04-20T21:17:36.186216+0000","last_undegraded":"2024-04-20T21:17:36.186216+0000","last_fullsized":"2024-04-20T21:17:36.186216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.787612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'12","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.998099+0000","last_change":"2024-04-20T21:16:53.568688+0000","last_active":"2024-04-20T21:17:36.998099+0000","last_peered":"2024-04-20T21:17:36.998099+0000","last_clean":"2024-04-20T21:17:36.998099+0000","last_became_active":"2024-04-20T21:16:53.568495+0000","last_became_peered":"2024-04-20T21:16:53.568495+0000","last_unstale":"2024-04-20T21:17:36.998099+0000","last_undegraded":"2024-04-20T21:17:36.998099+0000","last_fullsized":"2024-04-20T21:17:36.998099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:00.086004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4100,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879709+0000","last_change":"2024-04-20T21:16:58.609575+0000","last_active":"2024-04-20T21:17:35.879709+0000","last_peered":"2024-04-20T21:17:35.879709+0000","last_clean":"2024-04-20T21:17:35.879709+0000","last_became_active":"2024-04-20T21:16:58.609127+0000","last_became_peered":"2024-04-20T21:16:58.609127+0000","last_unstale":"2024-04-20T21:17:35.879709+0000","last_undegraded":"2024-04-20T21:17:35.879709+0000","last_fullsized":"2024-04-20T21:17:35.879709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:02.645274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878544+0000","last_change":"2024-04-20T21:16:54.563213+0000","last_active":"2024-04-20T21:17:35.878544+0000","last_peered":"2024-04-20T21:17:35.878544+0000","last_clean":"2024-04-20T21:17:35.878544+0000","last_became_active":"2024-04-20T21:16:54.562917+0000","last_became_peered":"2024-04-20T21:16:54.562917+0000","last_unstale":"2024-04-20T21:17:35.878544+0000","last_undegraded":"2024-04-20T21:17:35.878544+0000","last_fullsized":"2024-04-20T21:17:35.878544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.155771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880903+0000","last_change":"2024-04-20T21:16:53.556988+0000","last_active":"2024-04-20T21:17:35.880903+0000","last_peered":"2024-04-20T21:17:35.880903+0000","last_clean":"2024-04-20T21:17:35.880903+0000","last_became_active":"2024-04-20T21:16:53.556672+0000","last_became_peered":"2024-04-20T21:16:53.556672+0000","last_unstale":"2024-04-20T21:17:35.880903+0000","last_undegraded":"2024-04-20T21:17:35.880903+0000","last_fullsized":"2024-04-20T21:17:35.880903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:25.219611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.939270+0000","last_change":"2024-04-20T21:17:35.939270+0000","last_active":"2024-04-20T21:17:35.939270+0000","last_peered":"2024-04-20T21:17:35.939270+0000","last_clean":"2024-04-20T21:17:35.939270+0000","last_became_active":"2024-04-20T21:17:35.938833+0000","last_became_peered":"2024-04-20T21:17:35.938833+0000","last_unstale":"2024-04-20T21:17:35.939270+0000","last_undegraded":"2024-04-20T21:17:35.939270+0000","last_fullsized":"2024-04-20T21:17:35.939270+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.790995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884930+0000","last_change":"2024-04-20T21:16:54.562128+0000","last_active":"2024-04-20T21:17:35.884930+0000","last_peered":"2024-04-20T21:17:35.884930+0000","last_clean":"2024-04-20T21:17:35.884930+0000","last_became_active":"2024-04-20T21:16:54.561495+0000","last_became_peered":"2024-04-20T21:16:54.561495+0000","last_unstale":"2024-04-20T21:17:35.884930+0000","last_undegraded":"2024-04-20T21:17:35.884930+0000","last_fullsized":"2024-04-20T21:17:35.884930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:19.582218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878813+0000","last_change":"2024-04-20T21:16:53.554881+0000","last_active":"2024-04-20T21:17:35.878813+0000","last_peered":"2024-04-20T21:17:35.878813+0000","last_clean":"2024-04-20T21:17:35.878813+0000","last_became_active":"2024-04-20T21:16:53.554590+0000","last_became_peered":"2024-04-20T21:16:53.554590+0000","last_unstale":"2024-04-20T21:17:35.878813+0000","last_undegraded":"2024-04-20T21:17:35.878813+0000","last_fullsized":"2024-04-20T21:17:35.878813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:36.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.187143+0000","last_change":"2024-04-20T21:16:58.610704+0000","last_active":"2024-04-20T21:17:36.187143+0000","last_peered":"2024-04-20T21:17:36.187143+0000","last_clean":"2024-04-20T21:17:36.187143+0000","last_became_active":"2024-04-20T21:16:58.610309+0000","last_became_peered":"2024-04-20T21:16:58.610309+0000","last_unstale":"2024-04-20T21:17:36.187143+0000","last_undegraded":"2024-04-20T21:17:36.187143+0000","last_fullsized":"2024-04-20T21:17:36.187143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:04.215050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878430+0000","last_change":"2024-04-20T21:16:54.562351+0000","last_active":"2024-04-20T21:17:35.878430+0000","last_peered":"2024-04-20T21:17:35.878430+0000","last_clean":"2024-04-20T21:17:35.878430+0000","last_became_active":"2024-04-20T21:16:54.561947+0000","last_became_peered":"2024-04-20T21:16:54.561947+0000","last_unstale":"2024-04-20T21:17:35.878430+0000","last_undegraded":"2024-04-20T21:17:35.878430+0000","last_fullsized":"2024-04-20T21:17:35.878430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:52.802328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938343+0000","last_change":"2024-04-20T21:16:53.554081+0000","last_active":"2024-04-20T21:17:35.938343+0000","last_peered":"2024-04-20T21:17:35.938343+0000","last_clean":"2024-04-20T21:17:35.938343+0000","last_became_active":"2024-04-20T21:16:53.553800+0000","last_became_peered":"2024-04-20T21:16:53.553800+0000","last_unstale":"2024-04-20T21:17:35.938343+0000","last_undegraded":"2024-04-20T21:17:35.938343+0000","last_fullsized":"2024-04-20T21:17:35.938343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:52.865485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186668+0000","last_change":"2024-04-20T21:16:58.779184+0000","last_active":"2024-04-20T21:17:36.186668+0000","last_peered":"2024-04-20T21:17:36.186668+0000","last_clean":"2024-04-20T21:17:36.186668+0000","last_became_active":"2024-04-20T21:16:58.778627+0000","last_became_peered":"2024-04-20T21:16:58.778627+0000","last_unstale":"2024-04-20T21:17:36.186668+0000","last_undegraded":"2024-04-20T21:17:36.186668+0000","last_fullsized":"2024-04-20T21:17:36.186668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:50:07.247736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884103+0000","last_change":"2024-04-20T21:16:58.774438+0000","last_active":"2024-04-20T21:17:35.884103+0000","last_peered":"2024-04-20T21:17:35.884103+0000","last_clean":"2024-04-20T21:17:35.884103+0000","last_became_active":"2024-04-20T21:16:58.774030+0000","last_became_peered":"2024-04-20T21:16:58.774030+0000","last_unstale":"2024-04-20T21:17:35.884103+0000","last_undegraded":"2024-04-20T21:17:35.884103+0000","last_fullsized":"2024-04-20T21:17:35.884103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:09:22.583239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880247+0000","last_change":"2024-04-20T21:16:54.573779+0000","last_active":"2024-04-20T21:17:35.880247+0000","last_peered":"2024-04-20T21:17:35.880247+0000","last_clean":"2024-04-20T21:17:35.880247+0000","last_became_active":"2024-04-20T21:16:54.573444+0000","last_became_peered":"2024-04-20T21:16:54.573444+0000","last_unstale":"2024-04-20T21:17:35.880247+0000","last_undegraded":"2024-04-20T21:17:35.880247+0000","last_fullsized":"2024-04-20T21:17:35.880247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:27.950525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877644+0000","last_change":"2024-04-20T21:16:53.543678+0000","last_active":"2024-04-20T21:17:35.877644+0000","last_peered":"2024-04-20T21:17:35.877644+0000","last_clean":"2024-04-20T21:17:35.877644+0000","last_became_active":"2024-04-20T21:16:53.543381+0000","last_became_peered":"2024-04-20T21:16:53.543381+0000","last_unstale":"2024-04-20T21:17:35.877644+0000","last_undegraded":"2024-04-20T21:17:35.877644+0000","last_fullsized":"2024-04-20T21:17:35.877644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:02.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938285+0000","last_change":"2024-04-20T21:16:58.626882+0000","last_active":"2024-04-20T21:17:35.938285+0000","last_peered":"2024-04-20T21:17:35.938285+0000","last_clean":"2024-04-20T21:17:35.938285+0000","last_became_active":"2024-04-20T21:16:58.626489+0000","last_became_peered":"2024-04-20T21:16:58.626489+0000","last_unstale":"2024-04-20T21:17:35.938285+0000","last_undegraded":"2024-04-20T21:17:35.938285+0000","last_fullsized":"2024-04-20T21:17:35.938285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:33:22.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.187276+0000","last_change":"2024-04-20T21:16:54.580893+0000","last_active":"2024-04-20T21:17:36.187276+0000","last_peered":"2024-04-20T21:17:36.187276+0000","last_clean":"2024-04-20T21:17:36.187276+0000","last_became_active":"2024-04-20T21:16:54.580528+0000","last_became_peered":"2024-04-20T21:16:54.580528+0000","last_unstale":"2024-04-20T21:17:36.187276+0000","last_undegraded":"2024-04-20T21:17:36.187276+0000","last_fullsized":"2024-04-20T21:17:36.187276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:14.699123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186916+0000","last_change":"2024-04-20T21:16:53.563930+0000","last_active":"2024-04-20T21:17:36.186916+0000","last_peered":"2024-04-20T21:17:36.186916+0000","last_clean":"2024-04-20T21:17:36.186916+0000","last_became_active":"2024-04-20T21:16:53.559991+0000","last_became_peered":"2024-04-20T21:16:53.559991+0000","last_unstale":"2024-04-20T21:17:36.186916+0000","last_undegraded":"2024-04-20T21:17:36.186916+0000","last_fullsized":"2024-04-20T21:17:36.186916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:22.529712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937884+0000","last_change":"2024-04-20T21:16:53.552509+0000","last_active":"2024-04-20T21:17:35.937884+0000","last_peered":"2024-04-20T21:17:35.937884+0000","last_clean":"2024-04-20T21:17:35.937884+0000","last_became_active":"2024-04-20T21:16:53.552173+0000","last_became_peered":"2024-04-20T21:16:53.552173+0000","last_unstale":"2024-04-20T21:17:35.937884+0000","last_undegraded":"2024-04-20T21:17:35.937884+0000","last_fullsized":"2024-04-20T21:17:35.937884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:14.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884531+0000","last_change":"2024-04-20T21:16:54.562849+0000","last_active":"2024-04-20T21:17:35.884531+0000","last_peered":"2024-04-20T21:17:35.884531+0000","last_clean":"2024-04-20T21:17:35.884531+0000","last_became_active":"2024-04-20T21:16:54.561283+0000","last_became_peered":"2024-04-20T21:16:54.561283+0000","last_unstale":"2024-04-20T21:17:35.884531+0000","last_undegraded":"2024-04-20T21:17:35.884531+0000","last_fullsized":"2024-04-20T21:17:35.884531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:52.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887173+0000","last_change":"2024-04-20T21:16:58.614752+0000","last_active":"2024-04-20T21:17:35.887173+0000","last_peered":"2024-04-20T21:17:35.887173+0000","last_clean":"2024-04-20T21:17:35.887173+0000","last_became_active":"2024-04-20T21:16:58.614201+0000","last_became_peered":"2024-04-20T21:16:58.614201+0000","last_unstale":"2024-04-20T21:17:35.887173+0000","last_undegraded":"2024-04-20T21:17:35.887173+0000","last_fullsized":"2024-04-20T21:17:35.887173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:46.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188988+0000","last_change":"2024-04-20T21:16:53.564303+0000","last_active":"2024-04-20T21:17:36.188988+0000","last_peered":"2024-04-20T21:17:36.188988+0000","last_clean":"2024-04-20T21:17:36.188988+0000","last_became_active":"2024-04-20T21:16:53.560203+0000","last_became_peered":"2024-04-20T21:16:53.560203+0000","last_unstale":"2024-04-20T21:17:36.188988+0000","last_undegraded":"2024-04-20T21:17:36.188988+0000","last_fullsized":"2024-04-20T21:17:36.188988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:42.691233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878072+0000","last_change":"2024-04-20T21:16:54.554499+0000","last_active":"2024-04-20T21:17:35.878072+0000","last_peered":"2024-04-20T21:17:35.878072+0000","last_clean":"2024-04-20T21:17:35.878072+0000","last_became_active":"2024-04-20T21:16:54.554230+0000","last_became_peered":"2024-04-20T21:16:54.554230+0000","last_unstale":"2024-04-20T21:17:35.878072+0000","last_undegraded":"2024-04-20T21:17:35.878072+0000","last_fullsized":"2024-04-20T21:17:35.878072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:24:31.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188251+0000","last_change":"2024-04-20T21:16:58.610424+0000","last_active":"2024-04-20T21:17:36.188251+0000","last_peered":"2024-04-20T21:17:36.188251+0000","last_clean":"2024-04-20T21:17:36.188251+0000","last_became_active":"2024-04-20T21:16:58.610025+0000","last_became_peered":"2024-04-20T21:16:58.610025+0000","last_unstale":"2024-04-20T21:17:36.188251+0000","last_undegraded":"2024-04-20T21:17:36.188251+0000","last_fullsized":"2024-04-20T21:17:36.188251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:19.290865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937939+0000","last_change":"2024-04-20T21:16:54.578801+0000","last_active":"2024-04-20T21:17:35.937939+0000","last_peered":"2024-04-20T21:17:35.937939+0000","last_clean":"2024-04-20T21:17:35.937939+0000","last_became_active":"2024-04-20T21:16:54.578543+0000","last_became_peered":"2024-04-20T21:16:54.578543+0000","last_unstale":"2024-04-20T21:17:35.937939+0000","last_undegraded":"2024-04-20T21:17:35.937939+0000","last_fullsized":"2024-04-20T21:17:35.937939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:15.042318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.886717+0000","last_change":"2024-04-20T21:16:54.565658+0000","last_active":"2024-04-20T21:17:35.886717+0000","last_peered":"2024-04-20T21:17:35.886717+0000","last_clean":"2024-04-20T21:17:35.886717+0000","last_became_active":"2024-04-20T21:16:54.565371+0000","last_became_peered":"2024-04-20T21:16:54.565371+0000","last_unstale":"2024-04-20T21:17:35.886717+0000","last_undegraded":"2024-04-20T21:17:35.886717+0000","last_fullsized":"2024-04-20T21:17:35.886717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:21.781013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879317+0000","last_change":"2024-04-20T21:16:54.568472+0000","last_active":"2024-04-20T21:17:35.879317+0000","last_peered":"2024-04-20T21:17:35.879317+0000","last_clean":"2024-04-20T21:17:35.879317+0000","last_became_active":"2024-04-20T21:16:54.568223+0000","last_became_peered":"2024-04-20T21:16:54.568223+0000","last_unstale":"2024-04-20T21:17:35.879317+0000","last_undegraded":"2024-04-20T21:17:35.879317+0000","last_fullsized":"2024-04-20T21:17:35.879317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:42:53.921935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884279+0000","last_change":"2024-04-20T21:16:54.573747+0000","last_active":"2024-04-20T21:17:35.884279+0000","last_peered":"2024-04-20T21:17:35.884279+0000","last_clean":"2024-04-20T21:17:35.884279+0000","last_became_active":"2024-04-20T21:16:54.573476+0000","last_became_peered":"2024-04-20T21:16:54.573476+0000","last_unstale":"2024-04-20T21:17:35.884279+0000","last_undegraded":"2024-04-20T21:17:35.884279+0000","last_fullsized":"2024-04-20T21:17:35.884279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:02.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188768+0000","last_change":"2024-04-20T21:16:54.577630+0000","last_active":"2024-04-20T21:17:36.188768+0000","last_peered":"2024-04-20T21:17:36.188768+0000","last_clean":"2024-04-20T21:17:36.188768+0000","last_became_active":"2024-04-20T21:16:54.577318+0000","last_became_peered":"2024-04-20T21:16:54.577318+0000","last_unstale":"2024-04-20T21:17:36.188768+0000","last_undegraded":"2024-04-20T21:17:36.188768+0000","last_fullsized":"2024-04-20T21:17:36.188768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:45.584050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879003+0000","last_change":"2024-04-20T21:16:54.579850+0000","last_active":"2024-04-20T21:17:35.879003+0000","last_peered":"2024-04-20T21:17:35.879003+0000","last_clean":"2024-04-20T21:17:35.879003+0000","last_became_active":"2024-04-20T21:16:54.579546+0000","last_became_peered":"2024-04-20T21:16:54.579546+0000","last_unstale":"2024-04-20T21:17:35.879003+0000","last_undegraded":"2024-04-20T21:17:35.879003+0000","last_fullsized":"2024-04-20T21:17:35.879003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:49:53.577077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938400+0000","last_change":"2024-04-20T21:16:54.559174+0000","last_active":"2024-04-20T21:17:35.938400+0000","last_peered":"2024-04-20T21:17:35.938400+0000","last_clean":"2024-04-20T21:17:35.938400+0000","last_became_active":"2024-04-20T21:16:54.558904+0000","last_became_peered":"2024-04-20T21:16:54.558904+0000","last_unstale":"2024-04-20T21:17:35.938400+0000","last_undegraded":"2024-04-20T21:17:35.938400+0000","last_fullsized":"2024-04-20T21:17:35.938400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:09.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189372+0000","last_change":"2024-04-20T21:16:54.574943+0000","last_active":"2024-04-20T21:17:36.189372+0000","last_peered":"2024-04-20T21:17:36.189372+0000","last_clean":"2024-04-20T21:17:36.189372+0000","last_became_active":"2024-04-20T21:16:54.574471+0000","last_became_peered":"2024-04-20T21:16:54.574471+0000","last_unstale":"2024-04-20T21:17:36.189372+0000","last_undegraded":"2024-04-20T21:17:36.189372+0000","last_fullsized":"2024-04-20T21:17:36.189372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:51:44.741339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.193375+0000","last_change":"2024-04-20T21:17:36.191229+0000","last_active":"2024-04-20T21:17:36.193375+0000","last_peered":"2024-04-20T21:17:36.193375+0000","last_clean":"2024-04-20T21:17:36.193375+0000","last_became_active":"2024-04-20T21:17:36.190776+0000","last_became_peered":"2024-04-20T21:17:36.190776+0000","last_unstale":"2024-04-20T21:17:36.193375+0000","last_undegraded":"2024-04-20T21:17:36.193375+0000","last_fullsized":"2024-04-20T21:17:36.193375+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:49:11.938154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186859+0000","last_change":"2024-04-20T21:16:54.571179+0000","last_active":"2024-04-20T21:17:36.186859+0000","last_peered":"2024-04-20T21:17:36.186859+0000","last_clean":"2024-04-20T21:17:36.186859+0000","last_became_active":"2024-04-20T21:16:54.570855+0000","last_became_peered":"2024-04-20T21:16:54.570855+0000","last_unstale":"2024-04-20T21:17:36.186859+0000","last_undegraded":"2024-04-20T21:17:36.186859+0000","last_fullsized":"2024-04-20T21:17:36.186859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:59.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879202+0000","last_change":"2024-04-20T21:16:54.559092+0000","last_active":"2024-04-20T21:17:35.879202+0000","last_peered":"2024-04-20T21:17:35.879202+0000","last_clean":"2024-04-20T21:17:35.879202+0000","last_became_active":"2024-04-20T21:16:54.558847+0000","last_became_peered":"2024-04-20T21:16:54.558847+0000","last_unstale":"2024-04-20T21:17:35.879202+0000","last_undegraded":"2024-04-20T21:17:35.879202+0000","last_fullsized":"2024-04-20T21:17:35.879202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:02.752070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884080+0000","last_change":"2024-04-20T21:16:54.571815+0000","last_active":"2024-04-20T21:17:35.884080+0000","last_peered":"2024-04-20T21:17:35.884080+0000","last_clean":"2024-04-20T21:17:35.884080+0000","last_became_active":"2024-04-20T21:16:54.570704+0000","last_became_peered":"2024-04-20T21:16:54.570704+0000","last_unstale":"2024-04-20T21:17:35.884080+0000","last_undegraded":"2024-04-20T21:17:35.884080+0000","last_fullsized":"2024-04-20T21:17:35.884080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:56.511539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938000+0000","last_change":"2024-04-20T21:16:54.555591+0000","last_active":"2024-04-20T21:17:35.938000+0000","last_peered":"2024-04-20T21:17:35.938000+0000","last_clean":"2024-04-20T21:17:35.938000+0000","last_became_active":"2024-04-20T21:16:54.555389+0000","last_became_peered":"2024-04-20T21:16:54.555389+0000","last_unstale":"2024-04-20T21:17:35.938000+0000","last_undegraded":"2024-04-20T21:17:35.938000+0000","last_fullsized":"2024-04-20T21:17:35.938000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:16:25.976707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186528+0000","last_change":"2024-04-20T21:16:54.564088+0000","last_active":"2024-04-20T21:17:36.186528+0000","last_peered":"2024-04-20T21:17:36.186528+0000","last_clean":"2024-04-20T21:17:36.186528+0000","last_became_active":"2024-04-20T21:16:54.563796+0000","last_became_peered":"2024-04-20T21:16:54.563796+0000","last_unstale":"2024-04-20T21:17:36.186528+0000","last_undegraded":"2024-04-20T21:17:36.186528+0000","last_fullsized":"2024-04-20T21:17:36.186528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:07.986736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887460+0000","last_change":"2024-04-20T21:16:54.563578+0000","last_active":"2024-04-20T21:17:35.887460+0000","last_peered":"2024-04-20T21:17:35.887460+0000","last_clean":"2024-04-20T21:17:35.887460+0000","last_became_active":"2024-04-20T21:16:54.563317+0000","last_became_peered":"2024-04-20T21:16:54.563317+0000","last_unstale":"2024-04-20T21:17:35.887460+0000","last_undegraded":"2024-04-20T21:17:35.887460+0000","last_fullsized":"2024-04-20T21:17:35.887460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:06.093675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879169+0000","last_change":"2024-04-20T21:16:54.565168+0000","last_active":"2024-04-20T21:17:35.879169+0000","last_peered":"2024-04-20T21:17:35.879169+0000","last_clean":"2024-04-20T21:17:35.879169+0000","last_became_active":"2024-04-20T21:16:54.564712+0000","last_became_peered":"2024-04-20T21:16:54.564712+0000","last_unstale":"2024-04-20T21:17:35.879169+0000","last_undegraded":"2024-04-20T21:17:35.879169+0000","last_fullsized":"2024-04-20T21:17:35.879169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:07:30.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877246+0000","last_change":"2024-04-20T21:16:54.572211+0000","last_active":"2024-04-20T21:17:35.877246+0000","last_peered":"2024-04-20T21:17:35.877246+0000","last_clean":"2024-04-20T21:17:35.877246+0000","last_became_active":"2024-04-20T21:16:54.571894+0000","last_became_peered":"2024-04-20T21:16:54.571894+0000","last_unstale":"2024-04-20T21:17:35.877246+0000","last_undegraded":"2024-04-20T21:17:35.877246+0000","last_fullsized":"2024-04-20T21:17:35.877246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:59.376514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880303+0000","last_change":"2024-04-20T21:16:54.555195+0000","last_active":"2024-04-20T21:17:35.880303+0000","last_peered":"2024-04-20T21:17:35.880303+0000","last_clean":"2024-04-20T21:17:35.880303+0000","last_became_active":"2024-04-20T21:16:54.554284+0000","last_became_peered":"2024-04-20T21:16:54.554284+0000","last_unstale":"2024-04-20T21:17:35.880303+0000","last_undegraded":"2024-04-20T21:17:35.880303+0000","last_fullsized":"2024-04-20T21:17:35.880303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:42:21.500952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883524+0000","last_change":"2024-04-20T21:16:54.565527+0000","last_active":"2024-04-20T21:17:35.883524+0000","last_peered":"2024-04-20T21:17:35.883524+0000","last_clean":"2024-04-20T21:17:35.883524+0000","last_became_active":"2024-04-20T21:16:54.565211+0000","last_became_peered":"2024-04-20T21:16:54.565211+0000","last_unstale":"2024-04-20T21:17:35.883524+0000","last_undegraded":"2024-04-20T21:17:35.883524+0000","last_fullsized":"2024-04-20T21:17:35.883524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:17:40.462670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188717+0000","last_change":"2024-04-20T21:16:54.576464+0000","last_active":"2024-04-20T21:17:36.188717+0000","last_peered":"2024-04-20T21:17:36.188717+0000","last_clean":"2024-04-20T21:17:36.188717+0000","last_became_active":"2024-04-20T21:16:54.575947+0000","last_became_peered":"2024-04-20T21:16:54.575947+0000","last_unstale":"2024-04-20T21:17:36.188717+0000","last_undegraded":"2024-04-20T21:17:36.188717+0000","last_fullsized":"2024-04-20T21:17:36.188717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:50:29.581938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878458+0000","last_change":"2024-04-20T21:16:54.558162+0000","last_active":"2024-04-20T21:17:35.878458+0000","last_peered":"2024-04-20T21:17:35.878458+0000","last_clean":"2024-04-20T21:17:35.878458+0000","last_became_active":"2024-04-20T21:16:54.557875+0000","last_became_peered":"2024-04-20T21:16:54.557875+0000","last_unstale":"2024-04-20T21:17:35.878458+0000","last_undegraded":"2024-04-20T21:17:35.878458+0000","last_fullsized":"2024-04-20T21:17:35.878458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:42:46.886405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937978+0000","last_change":"2024-04-20T21:16:54.560566+0000","last_active":"2024-04-20T21:17:35.937978+0000","last_peered":"2024-04-20T21:17:35.937978+0000","last_clean":"2024-04-20T21:17:35.937978+0000","last_became_active":"2024-04-20T21:16:54.559409+0000","last_became_peered":"2024-04-20T21:16:54.559409+0000","last_unstale":"2024-04-20T21:17:35.937978+0000","last_undegraded":"2024-04-20T21:17:35.937978+0000","last_fullsized":"2024-04-20T21:17:35.937978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:00.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938279+0000","last_change":"2024-04-20T21:16:54.559491+0000","last_active":"2024-04-20T21:17:35.938279+0000","last_peered":"2024-04-20T21:17:35.938279+0000","last_clean":"2024-04-20T21:17:35.938279+0000","last_became_active":"2024-04-20T21:16:54.558187+0000","last_became_peered":"2024-04-20T21:16:54.558187+0000","last_unstale":"2024-04-20T21:17:35.938279+0000","last_undegraded":"2024-04-20T21:17:35.938279+0000","last_fullsized":"2024-04-20T21:17:35.938279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:33.686248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186775+0000","last_change":"2024-04-20T21:16:54.564258+0000","last_active":"2024-04-20T21:17:36.186775+0000","last_peered":"2024-04-20T21:17:36.186775+0000","last_clean":"2024-04-20T21:17:36.186775+0000","last_became_active":"2024-04-20T21:16:54.563966+0000","last_became_peered":"2024-04-20T21:16:54.563966+0000","last_unstale":"2024-04-20T21:17:36.186775+0000","last_undegraded":"2024-04-20T21:17:36.186775+0000","last_fullsized":"2024-04-20T21:17:36.186775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:13:34.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186994+0000","last_change":"2024-04-20T21:16:54.573554+0000","last_active":"2024-04-20T21:17:36.186994+0000","last_peered":"2024-04-20T21:17:36.186994+0000","last_clean":"2024-04-20T21:17:36.186994+0000","last_became_active":"2024-04-20T21:16:54.571849+0000","last_became_peered":"2024-04-20T21:16:54.571849+0000","last_unstale":"2024-04-20T21:17:36.186994+0000","last_undegraded":"2024-04-20T21:17:36.186994+0000","last_fullsized":"2024-04-20T21:17:36.186994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:09.481779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186052+0000","last_change":"2024-04-20T21:16:54.571471+0000","last_active":"2024-04-20T21:17:36.186052+0000","last_peered":"2024-04-20T21:17:36.186052+0000","last_clean":"2024-04-20T21:17:36.186052+0000","last_became_active":"2024-04-20T21:16:54.571209+0000","last_became_peered":"2024-04-20T21:16:54.571209+0000","last_unstale":"2024-04-20T21:17:36.186052+0000","last_undegraded":"2024-04-20T21:17:36.186052+0000","last_fullsized":"2024-04-20T21:17:36.186052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:35.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879415+0000","last_change":"2024-04-20T21:16:54.572168+0000","last_active":"2024-04-20T21:17:35.879415+0000","last_peered":"2024-04-20T21:17:35.879415+0000","last_clean":"2024-04-20T21:17:35.879415+0000","last_became_active":"2024-04-20T21:16:54.571740+0000","last_became_peered":"2024-04-20T21:16:54.571740+0000","last_unstale":"2024-04-20T21:17:35.879415+0000","last_undegraded":"2024-04-20T21:17:35.879415+0000","last_fullsized":"2024-04-20T21:17:35.879415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:11:35.728824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186187+0000","last_change":"2024-04-20T21:16:54.569081+0000","last_active":"2024-04-20T21:17:36.186187+0000","last_peered":"2024-04-20T21:17:36.186187+0000","last_clean":"2024-04-20T21:17:36.186187+0000","last_became_active":"2024-04-20T21:16:54.568847+0000","last_became_peered":"2024-04-20T21:16:54.568847+0000","last_unstale":"2024-04-20T21:17:36.186187+0000","last_undegraded":"2024-04-20T21:17:36.186187+0000","last_fullsized":"2024-04-20T21:17:36.186187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:35.737237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878840+0000","last_change":"2024-04-20T21:16:54.571525+0000","last_active":"2024-04-20T21:17:35.878840+0000","last_peered":"2024-04-20T21:17:35.878840+0000","last_clean":"2024-04-20T21:17:35.878840+0000","last_became_active":"2024-04-20T21:16:54.570776+0000","last_became_peered":"2024-04-20T21:16:54.570776+0000","last_unstale":"2024-04-20T21:17:35.878840+0000","last_undegraded":"2024-04-20T21:17:35.878840+0000","last_fullsized":"2024-04-20T21:17:35.878840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:03.693476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883961+0000","last_change":"2024-04-20T21:16:54.570895+0000","last_active":"2024-04-20T21:17:35.883961+0000","last_peered":"2024-04-20T21:17:35.883961+0000","last_clean":"2024-04-20T21:17:35.883961+0000","last_became_active":"2024-04-20T21:16:54.570577+0000","last_became_peered":"2024-04-20T21:16:54.570577+0000","last_unstale":"2024-04-20T21:17:35.883961+0000","last_undegraded":"2024-04-20T21:17:35.883961+0000","last_fullsized":"2024-04-20T21:17:35.883961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:26.787487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878454+0000","last_change":"2024-04-20T21:16:54.581648+0000","last_active":"2024-04-20T21:17:35.878454+0000","last_peered":"2024-04-20T21:17:35.878454+0000","last_clean":"2024-04-20T21:17:35.878454+0000","last_became_active":"2024-04-20T21:16:54.581382+0000","last_became_peered":"2024-04-20T21:16:54.581382+0000","last_unstale":"2024-04-20T21:17:35.878454+0000","last_undegraded":"2024-04-20T21:17:35.878454+0000","last_fullsized":"2024-04-20T21:17:35.878454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:21.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186705+0000","last_change":"2024-04-20T21:16:54.563949+0000","last_active":"2024-04-20T21:17:36.186705+0000","last_peered":"2024-04-20T21:17:36.186705+0000","last_clean":"2024-04-20T21:17:36.186705+0000","last_became_active":"2024-04-20T21:16:54.563606+0000","last_became_peered":"2024-04-20T21:16:54.563606+0000","last_unstale":"2024-04-20T21:17:36.186705+0000","last_undegraded":"2024-04-20T21:17:36.186705+0000","last_fullsized":"2024-04-20T21:17:36.186705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:26:05.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4898,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":705311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":118203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":114451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":114487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":118461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":114361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":114521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":705367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:43.975 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:44.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.210+0000 7f7b85200640 1 Processor -- start 2024-04-20T21:17:44.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.210+0000 7f7b85200640 1 -- start start 2024-04-20T21:17:44.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b85200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b800c9eb0 0x7f7b800c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b85200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7b80073a10 0x7f7b80158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b85200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b801536d0 con 0x7f7b8014fa50 2024-04-20T21:17:44.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b85200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7b80159500 con 0x7f7b80073a10 2024-04-20T21:17:44.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b85200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7b800c76c0 con 0x7f7b800c9eb0 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b7de00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7b8014fa50 0x7f7b800c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57552/0 (socket says 172.21.15.102:57552) 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b7de00640 1 -- 172.21.15.102:0/624484111 learned_addr learned my addr 172.21.15.102:0/624484111 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b7d400640 1 --2- 172.21.15.102:0/624484111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b800c9eb0 0x7f7b800c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b7e800640 1 --2- 172.21.15.102:0/624484111 >> v2:172.21.15.193:3300/0 conn(0x7f7b80073a10 0x7f7b80158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.212+0000 7f7b7ca00640 1 -- 172.21.15.102:0/624484111 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19252696 0 0) 0x7f7b70001110 con 0x7f7b8014fa50 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.213+0000 7f7b7ca00640 1 -- 172.21.15.102:0/624484111 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b64003520 con 0x7f7b8014fa50 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.213+0000 7f7b7d400640 1 -- 172.21.15.102:0/624484111 >> v2:172.21.15.193:3300/0 conn(0x7f7b80073a10 msgr2=0x7f7b80158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.213+0000 7f7b7d400640 1 --2- 172.21.15.102:0/624484111 >> v2:172.21.15.193:3300/0 conn(0x7f7b80073a10 0x7f7b80158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.213+0000 7f7b7d400640 1 -- 172.21.15.102:0/624484111 >> v1:172.21.15.102:6789/0 conn(0x7f7b8014fa50 legacy=0x7f7b800c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.213+0000 7f7b7d400640 1 -- 172.21.15.102:0/624484111 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b800c7960 con 0x7f7b800c9eb0 2024-04-20T21:17:44.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.214+0000 7f7b7d400640 1 --2- 172.21.15.102:0/624484111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b800c9eb0 0x7f7b800c4830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0097a0 tx=0x7f7b6c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=285842e353be2b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:44.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.214+0000 7f7b7ca00640 1 -- 172.21.15.102:0/624484111 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c034070 con 0x7f7b800c9eb0 2024-04-20T21:17:44.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.214+0000 7f7b7ca00640 1 -- 172.21.15.102:0/624484111 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c030610 con 0x7f7b800c9eb0 2024-04-20T21:17:44.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.214+0000 7f7b7ca00640 1 -- 172.21.15.102:0/624484111 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0309c0 con 0x7f7b800c9eb0 2024-04-20T21:17:44.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.215+0000 7f7b85200640 1 -- 172.21.15.102:0/624484111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b800c9eb0 msgr2=0x7f7b800c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.215+0000 7f7b85200640 1 --2- 172.21.15.102:0/624484111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b800c9eb0 0x7f7b800c4830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0097a0 tx=0x7f7b6c02b960 comp rx=0 tx=0).stop 2024-04-20T21:17:44.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.215+0000 7f7b85200640 1 -- 172.21.15.102:0/624484111 shutdown_connections 2024-04-20T21:17:44.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.215+0000 7f7b85200640 1 --2- 172.21.15.102:0/624484111 >> v2:172.21.15.193:3300/0 conn(0x7f7b80073a10 0x7f7b80158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.215+0000 7f7b85200640 1 --2- 172.21.15.102:0/624484111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b800c9eb0 0x7f7b800c4830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.215+0000 7f7b85200640 1 -- 172.21.15.102:0/624484111 >> 172.21.15.102:0/624484111 conn(0x7f7b800c9390 msgr2=0x7f7b800c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:44.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.215+0000 7f7b85200640 1 -- 172.21.15.102:0/624484111 shutdown_connections 2024-04-20T21:17:44.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.215+0000 7f7b85200640 1 -- 172.21.15.102:0/624484111 wait complete. 2024-04-20T21:17:44.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.217+0000 7f7b85200640 1 Processor -- start 2024-04-20T21:17:44.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.217+0000 7f7b85200640 1 -- start start 2024-04-20T21:17:44.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b85200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b80073a10 0x7f7b8014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b85200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b800c9eb0 0x7f7b80156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b85200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b8014fa50 0x7f7b800cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b85200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800c7c00 con 0x7f7b800c9eb0 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b85200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800c3d90 con 0x7f7b80073a10 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b85200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7b800c4030 con 0x7f7b8014fa50 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b7d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b800c9eb0 0x7f7b80156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b7d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b800c9eb0 0x7f7b80156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48428/0 (socket says 172.21.15.102:48428) 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b7d400640 1 -- 172.21.15.102:0/920542514 learned_addr learned my addr 172.21.15.102:0/920542514 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b7de00640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b80073a10 0x7f7b8014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.218+0000 7f7b7e800640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b8014fa50 0x7f7b800cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b7d400640 1 -- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b8014fa50 msgr2=0x7f7b800cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b7d400640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b8014fa50 0x7f7b800cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b7d400640 1 -- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b80073a10 msgr2=0x7f7b8014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b7d400640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b80073a10 0x7f7b8014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b7d400640 1 -- 172.21.15.102:0/920542514 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b80159330 con 0x7f7b800c9eb0 2024-04-20T21:17:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b7d400640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b800c9eb0 0x7f7b80156e30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0098c0 tx=0x7f7b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b76000640 1 -- 172.21.15.102:0/920542514 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c034040 con 0x7f7b800c9eb0 2024-04-20T21:17:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b76000640 1 -- 172.21.15.102:0/920542514 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c030410 con 0x7f7b800c9eb0 2024-04-20T21:17:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b76000640 1 -- 172.21.15.102:0/920542514 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c006e10 con 0x7f7b800c9eb0 2024-04-20T21:17:44.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b80159050 con 0x7f7b800c9eb0 2024-04-20T21:17:44.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.219+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b800cc1e0 con 0x7f7b800c9eb0 2024-04-20T21:17:44.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.221+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b80074340 con 0x7f7b800c9eb0 2024-04-20T21:17:44.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.221+0000 7f7b76000640 1 -- 172.21.15.102:0/920542514 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b6c037070 con 0x7f7b800c9eb0 2024-04-20T21:17:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.223+0000 7f7b76000640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b6006c8f0 0x7f7b6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.223+0000 7f7b76000640 1 -- 172.21.15.102:0/920542514 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7b6c0add40 con 0x7f7b800c9eb0 2024-04-20T21:17:44.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.223+0000 7f7b7de00640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b6006c8f0 0x7f7b6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.224+0000 7f7b7de00640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b6006c8f0 0x7f7b6006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b70005880 tx=0x7f7b70009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.227+0000 7f7b76000640 1 -- 172.21.15.102:0/920542514 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b6c0305e0 con 0x7f7b800c9eb0 2024-04-20T21:17:44.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.423+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 --> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b800c7320 con 0x7f7b6006c8f0 2024-04-20T21:17:44.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.433+0000 7f7b76000640 1 -- 172.21.15.102:0/920542514 <== mgr.4112 v2:172.21.15.193:6832/2803621564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332649 (secure 0 0 0) 0x7f7b800c7320 con 0x7f7b6006c8f0 2024-04-20T21:17:44.438 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:44.440 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.440+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b6006c8f0 msgr2=0x7f7b6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.440+0000 7f7b85200640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b6006c8f0 0x7f7b6006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b70005880 tx=0x7f7b70009040 comp rx=0 tx=0).stop 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b800c9eb0 msgr2=0x7f7b80156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b800c9eb0 0x7f7b80156e30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0098c0 tx=0x7f7b6c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 shutdown_connections 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b8014fa50 0x7f7b800cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b800c9eb0 0x7f7b80156e30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b6006c8f0 0x7f7b6006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 --2- 172.21.15.102:0/920542514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b80073a10 0x7f7b8014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 >> 172.21.15.102:0/920542514 conn(0x7f7b800c9390 msgr2=0x7f7b8006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 shutdown_connections 2024-04-20T21:17:44.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.441+0000 7f7b85200640 1 -- 172.21.15.102:0/920542514 wait complete. 2024-04-20T21:17:44.459 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-04-20T21:17:42.673363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595266,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":210,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":319,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3481600,"data_stored":2105162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002466"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884977+0000","last_change":"2024-04-20T21:16:53.564922+0000","last_active":"2024-04-20T21:17:35.884977+0000","last_peered":"2024-04-20T21:17:35.884977+0000","last_clean":"2024-04-20T21:17:35.884977+0000","last_became_active":"2024-04-20T21:16:53.564589+0000","last_became_peered":"2024-04-20T21:16:53.564589+0000","last_unstale":"2024-04-20T21:17:35.884977+0000","last_undegraded":"2024-04-20T21:17:35.884977+0000","last_fullsized":"2024-04-20T21:17:35.884977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:47.284664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188817+0000","last_change":"2024-04-20T21:16:54.580132+0000","last_active":"2024-04-20T21:17:36.188817+0000","last_peered":"2024-04-20T21:17:36.188817+0000","last_clean":"2024-04-20T21:17:36.188817+0000","last_became_active":"2024-04-20T21:16:54.579769+0000","last_became_peered":"2024-04-20T21:16:54.579769+0000","last_unstale":"2024-04-20T21:17:36.188817+0000","last_undegraded":"2024-04-20T21:17:36.188817+0000","last_fullsized":"2024-04-20T21:17:36.188817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:00.705947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186600+0000","last_change":"2024-04-20T21:16:58.781707+0000","last_active":"2024-04-20T21:17:36.186600+0000","last_peered":"2024-04-20T21:17:36.186600+0000","last_clean":"2024-04-20T21:17:36.186600+0000","last_became_active":"2024-04-20T21:16:58.780685+0000","last_became_peered":"2024-04-20T21:16:58.780685+0000","last_unstale":"2024-04-20T21:17:36.186600+0000","last_undegraded":"2024-04-20T21:17:36.186600+0000","last_fullsized":"2024-04-20T21:17:36.186600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:50.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189794+0000","last_change":"2024-04-20T21:16:53.572137+0000","last_active":"2024-04-20T21:17:36.189794+0000","last_peered":"2024-04-20T21:17:36.189794+0000","last_clean":"2024-04-20T21:17:36.189794+0000","last_became_active":"2024-04-20T21:16:53.567907+0000","last_became_peered":"2024-04-20T21:16:53.567907+0000","last_unstale":"2024-04-20T21:17:36.189794+0000","last_undegraded":"2024-04-20T21:17:36.189794+0000","last_fullsized":"2024-04-20T21:17:36.189794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:23:28.367174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884881+0000","last_change":"2024-04-20T21:16:54.567096+0000","last_active":"2024-04-20T21:17:35.884881+0000","last_peered":"2024-04-20T21:17:35.884881+0000","last_clean":"2024-04-20T21:17:35.884881+0000","last_became_active":"2024-04-20T21:16:54.566803+0000","last_became_peered":"2024-04-20T21:16:54.566803+0000","last_unstale":"2024-04-20T21:17:35.884881+0000","last_undegraded":"2024-04-20T21:17:35.884881+0000","last_fullsized":"2024-04-20T21:17:35.884881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:13:44.730056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.191835+0000","last_change":"2024-04-20T21:17:36.191835+0000","last_active":"2024-04-20T21:17:36.191835+0000","last_peered":"2024-04-20T21:17:36.191835+0000","last_clean":"2024-04-20T21:17:36.191835+0000","last_became_active":"2024-04-20T21:17:36.191319+0000","last_became_peered":"2024-04-20T21:17:36.191319+0000","last_unstale":"2024-04-20T21:17:36.191835+0000","last_undegraded":"2024-04-20T21:17:36.191835+0000","last_fullsized":"2024-04-20T21:17:36.191835+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:29:36.899051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2,3],"acting":[4,6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186471+0000","last_change":"2024-04-20T21:16:53.554847+0000","last_active":"2024-04-20T21:17:36.186471+0000","last_peered":"2024-04-20T21:17:36.186471+0000","last_clean":"2024-04-20T21:17:36.186471+0000","last_became_active":"2024-04-20T21:16:53.554333+0000","last_became_peered":"2024-04-20T21:16:53.554333+0000","last_unstale":"2024-04-20T21:17:36.186471+0000","last_undegraded":"2024-04-20T21:17:36.186471+0000","last_fullsized":"2024-04-20T21:17:36.186471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:33.255945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878947+0000","last_change":"2024-04-20T21:16:54.556407+0000","last_active":"2024-04-20T21:17:35.878947+0000","last_peered":"2024-04-20T21:17:35.878947+0000","last_clean":"2024-04-20T21:17:35.878947+0000","last_became_active":"2024-04-20T21:16:54.556161+0000","last_became_peered":"2024-04-20T21:16:54.556161+0000","last_unstale":"2024-04-20T21:17:35.878947+0000","last_undegraded":"2024-04-20T21:17:35.878947+0000","last_fullsized":"2024-04-20T21:17:35.878947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:44:53.586714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877433+0000","last_change":"2024-04-20T21:16:58.612063+0000","last_active":"2024-04-20T21:17:35.877433+0000","last_peered":"2024-04-20T21:17:35.877433+0000","last_clean":"2024-04-20T21:17:35.877433+0000","last_became_active":"2024-04-20T21:16:58.611487+0000","last_became_peered":"2024-04-20T21:16:58.611487+0000","last_unstale":"2024-04-20T21:17:35.877433+0000","last_undegraded":"2024-04-20T21:17:35.877433+0000","last_fullsized":"2024-04-20T21:17:35.877433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:15.097731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186246+0000","last_change":"2024-04-20T21:16:53.551212+0000","last_active":"2024-04-20T21:17:36.186246+0000","last_peered":"2024-04-20T21:17:36.186246+0000","last_clean":"2024-04-20T21:17:36.186246+0000","last_became_active":"2024-04-20T21:16:53.550899+0000","last_became_peered":"2024-04-20T21:16:53.550899+0000","last_unstale":"2024-04-20T21:17:36.186246+0000","last_undegraded":"2024-04-20T21:17:36.186246+0000","last_fullsized":"2024-04-20T21:17:36.186246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:07:18.283243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878561+0000","last_change":"2024-04-20T21:16:54.557285+0000","last_active":"2024-04-20T21:17:35.878561+0000","last_peered":"2024-04-20T21:17:35.878561+0000","last_clean":"2024-04-20T21:17:35.878561+0000","last_became_active":"2024-04-20T21:16:54.556921+0000","last_became_peered":"2024-04-20T21:16:54.556921+0000","last_unstale":"2024-04-20T21:17:35.878561+0000","last_undegraded":"2024-04-20T21:17:35.878561+0000","last_fullsized":"2024-04-20T21:17:35.878561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:33.989679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877397+0000","last_change":"2024-04-20T21:16:58.617360+0000","last_active":"2024-04-20T21:17:35.877397+0000","last_peered":"2024-04-20T21:17:35.877397+0000","last_clean":"2024-04-20T21:17:35.877397+0000","last_became_active":"2024-04-20T21:16:58.616947+0000","last_became_peered":"2024-04-20T21:16:58.616947+0000","last_unstale":"2024-04-20T21:17:35.877397+0000","last_undegraded":"2024-04-20T21:17:35.877397+0000","last_fullsized":"2024-04-20T21:17:35.877397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.202955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937723+0000","last_change":"2024-04-20T21:16:54.558067+0000","last_active":"2024-04-20T21:17:35.937723+0000","last_peered":"2024-04-20T21:17:35.937723+0000","last_clean":"2024-04-20T21:17:35.937723+0000","last_became_active":"2024-04-20T21:16:54.557781+0000","last_became_peered":"2024-04-20T21:16:54.557781+0000","last_unstale":"2024-04-20T21:17:35.937723+0000","last_undegraded":"2024-04-20T21:17:35.937723+0000","last_fullsized":"2024-04-20T21:17:35.937723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:56.576081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186328+0000","last_change":"2024-04-20T21:16:53.551615+0000","last_active":"2024-04-20T21:17:36.186328+0000","last_peered":"2024-04-20T21:17:36.186328+0000","last_clean":"2024-04-20T21:17:36.186328+0000","last_became_active":"2024-04-20T21:16:53.551073+0000","last_became_peered":"2024-04-20T21:16:53.551073+0000","last_unstale":"2024-04-20T21:17:36.186328+0000","last_undegraded":"2024-04-20T21:17:36.186328+0000","last_fullsized":"2024-04-20T21:17:36.186328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:26:29.994289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877528+0000","last_change":"2024-04-20T21:16:58.617434+0000","last_active":"2024-04-20T21:17:35.877528+0000","last_peered":"2024-04-20T21:17:35.877528+0000","last_clean":"2024-04-20T21:17:35.877528+0000","last_became_active":"2024-04-20T21:16:58.617027+0000","last_became_peered":"2024-04-20T21:16:58.617027+0000","last_unstale":"2024-04-20T21:17:35.877528+0000","last_undegraded":"2024-04-20T21:17:35.877528+0000","last_fullsized":"2024-04-20T21:17:35.877528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:44:20.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188865+0000","last_change":"2024-04-20T21:16:54.585333+0000","last_active":"2024-04-20T21:17:36.188865+0000","last_peered":"2024-04-20T21:17:36.188865+0000","last_clean":"2024-04-20T21:17:36.188865+0000","last_became_active":"2024-04-20T21:16:54.584978+0000","last_became_peered":"2024-04-20T21:16:54.584978+0000","last_unstale":"2024-04-20T21:17:36.188865+0000","last_undegraded":"2024-04-20T21:17:36.188865+0000","last_fullsized":"2024-04-20T21:17:36.188865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:23.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879889+0000","last_change":"2024-04-20T21:16:53.567775+0000","last_active":"2024-04-20T21:17:35.879889+0000","last_peered":"2024-04-20T21:17:35.879889+0000","last_clean":"2024-04-20T21:17:35.879889+0000","last_became_active":"2024-04-20T21:16:53.567498+0000","last_became_peered":"2024-04-20T21:16:53.567498+0000","last_unstale":"2024-04-20T21:17:35.879889+0000","last_undegraded":"2024-04-20T21:17:35.879889+0000","last_fullsized":"2024-04-20T21:17:35.879889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:10.695659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878180+0000","last_change":"2024-04-20T21:16:58.619302+0000","last_active":"2024-04-20T21:17:35.878180+0000","last_peered":"2024-04-20T21:17:35.878180+0000","last_clean":"2024-04-20T21:17:35.878180+0000","last_became_active":"2024-04-20T21:16:58.618843+0000","last_became_peered":"2024-04-20T21:16:58.618843+0000","last_unstale":"2024-04-20T21:17:35.878180+0000","last_undegraded":"2024-04-20T21:17:35.878180+0000","last_fullsized":"2024-04-20T21:17:35.878180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:14.927199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188147+0000","last_change":"2024-04-20T21:16:54.575862+0000","last_active":"2024-04-20T21:17:36.188147+0000","last_peered":"2024-04-20T21:17:36.188147+0000","last_clean":"2024-04-20T21:17:36.188147+0000","last_became_active":"2024-04-20T21:16:54.575523+0000","last_became_peered":"2024-04-20T21:16:54.575523+0000","last_unstale":"2024-04-20T21:17:36.188147+0000","last_undegraded":"2024-04-20T21:17:36.188147+0000","last_fullsized":"2024-04-20T21:17:36.188147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:51.041414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880494+0000","last_change":"2024-04-20T21:16:53.551995+0000","last_active":"2024-04-20T21:17:35.880494+0000","last_peered":"2024-04-20T21:17:35.880494+0000","last_clean":"2024-04-20T21:17:35.880494+0000","last_became_active":"2024-04-20T21:16:53.551603+0000","last_became_peered":"2024-04-20T21:16:53.551603+0000","last_unstale":"2024-04-20T21:17:35.880494+0000","last_undegraded":"2024-04-20T21:17:35.880494+0000","last_fullsized":"2024-04-20T21:17:35.880494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:56:21.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878943+0000","last_change":"2024-04-20T21:16:58.776463+0000","last_active":"2024-04-20T21:17:35.878943+0000","last_peered":"2024-04-20T21:17:35.878943+0000","last_clean":"2024-04-20T21:17:35.878943+0000","last_became_active":"2024-04-20T21:16:58.775746+0000","last_became_peered":"2024-04-20T21:16:58.775746+0000","last_unstale":"2024-04-20T21:17:35.878943+0000","last_undegraded":"2024-04-20T21:17:35.878943+0000","last_fullsized":"2024-04-20T21:17:35.878943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:28.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878811+0000","last_change":"2024-04-20T21:16:54.566449+0000","last_active":"2024-04-20T21:17:35.878811+0000","last_peered":"2024-04-20T21:17:35.878811+0000","last_clean":"2024-04-20T21:17:35.878811+0000","last_became_active":"2024-04-20T21:16:54.566251+0000","last_became_peered":"2024-04-20T21:16:54.566251+0000","last_unstale":"2024-04-20T21:17:35.878811+0000","last_undegraded":"2024-04-20T21:17:35.878811+0000","last_fullsized":"2024-04-20T21:17:35.878811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:53.403420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185741+0000","last_change":"2024-04-20T21:16:53.564029+0000","last_active":"2024-04-20T21:17:36.185741+0000","last_peered":"2024-04-20T21:17:36.185741+0000","last_clean":"2024-04-20T21:17:36.185741+0000","last_became_active":"2024-04-20T21:16:53.563711+0000","last_became_peered":"2024-04-20T21:16:53.563711+0000","last_unstale":"2024-04-20T21:17:36.185741+0000","last_undegraded":"2024-04-20T21:17:36.185741+0000","last_fullsized":"2024-04-20T21:17:36.185741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.634644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878711+0000","last_change":"2024-04-20T21:16:58.615080+0000","last_active":"2024-04-20T21:17:35.878711+0000","last_peered":"2024-04-20T21:17:35.878711+0000","last_clean":"2024-04-20T21:17:35.878711+0000","last_became_active":"2024-04-20T21:16:58.614611+0000","last_became_peered":"2024-04-20T21:16:58.614611+0000","last_unstale":"2024-04-20T21:17:35.878711+0000","last_undegraded":"2024-04-20T21:17:35.878711+0000","last_fullsized":"2024-04-20T21:17:35.878711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:08.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884557+0000","last_change":"2024-04-20T21:16:54.578272+0000","last_active":"2024-04-20T21:17:35.884557+0000","last_peered":"2024-04-20T21:17:35.884557+0000","last_clean":"2024-04-20T21:17:35.884557+0000","last_became_active":"2024-04-20T21:16:54.577949+0000","last_became_peered":"2024-04-20T21:16:54.577949+0000","last_unstale":"2024-04-20T21:17:35.884557+0000","last_undegraded":"2024-04-20T21:17:35.884557+0000","last_fullsized":"2024-04-20T21:17:35.884557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:35:33.807091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938557+0000","last_change":"2024-04-20T21:16:53.554600+0000","last_active":"2024-04-20T21:17:35.938557+0000","last_peered":"2024-04-20T21:17:35.938557+0000","last_clean":"2024-04-20T21:17:35.938557+0000","last_became_active":"2024-04-20T21:16:53.554347+0000","last_became_peered":"2024-04-20T21:16:53.554347+0000","last_unstale":"2024-04-20T21:17:35.938557+0000","last_undegraded":"2024-04-20T21:17:35.938557+0000","last_fullsized":"2024-04-20T21:17:35.938557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:40:58.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878403+0000","last_change":"2024-04-20T21:16:58.774474+0000","last_active":"2024-04-20T21:17:35.878403+0000","last_peered":"2024-04-20T21:17:35.878403+0000","last_clean":"2024-04-20T21:17:35.878403+0000","last_became_active":"2024-04-20T21:16:58.774037+0000","last_became_peered":"2024-04-20T21:16:58.774037+0000","last_unstale":"2024-04-20T21:17:35.878403+0000","last_undegraded":"2024-04-20T21:17:35.878403+0000","last_fullsized":"2024-04-20T21:17:35.878403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:46.992667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189865+0000","last_change":"2024-04-20T21:17:36.187769+0000","last_active":"2024-04-20T21:17:36.189865+0000","last_peered":"2024-04-20T21:17:36.189865+0000","last_clean":"2024-04-20T21:17:36.189865+0000","last_became_active":"2024-04-20T21:17:36.187283+0000","last_became_peered":"2024-04-20T21:17:36.187283+0000","last_unstale":"2024-04-20T21:17:36.189865+0000","last_undegraded":"2024-04-20T21:17:36.189865+0000","last_fullsized":"2024-04-20T21:17:36.189865+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:46:58.418395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878574+0000","last_change":"2024-04-20T21:16:53.561144+0000","last_active":"2024-04-20T21:17:35.878574+0000","last_peered":"2024-04-20T21:17:35.878574+0000","last_clean":"2024-04-20T21:17:35.878574+0000","last_became_active":"2024-04-20T21:16:53.560742+0000","last_became_peered":"2024-04-20T21:16:53.560742+0000","last_unstale":"2024-04-20T21:17:35.878574+0000","last_undegraded":"2024-04-20T21:17:35.878574+0000","last_fullsized":"2024-04-20T21:17:35.878574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:39.212951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186836+0000","last_change":"2024-04-20T21:16:58.780124+0000","last_active":"2024-04-20T21:17:36.186836+0000","last_peered":"2024-04-20T21:17:36.186836+0000","last_clean":"2024-04-20T21:17:36.186836+0000","last_became_active":"2024-04-20T21:16:58.777449+0000","last_became_peered":"2024-04-20T21:16:58.777449+0000","last_unstale":"2024-04-20T21:17:36.186836+0000","last_undegraded":"2024-04-20T21:17:36.186836+0000","last_fullsized":"2024-04-20T21:17:36.186836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:07.622146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879405+0000","last_change":"2024-04-20T21:16:54.575331+0000","last_active":"2024-04-20T21:17:35.879405+0000","last_peered":"2024-04-20T21:17:35.879405+0000","last_clean":"2024-04-20T21:17:35.879405+0000","last_became_active":"2024-04-20T21:16:54.575061+0000","last_became_peered":"2024-04-20T21:16:54.575061+0000","last_unstale":"2024-04-20T21:17:35.879405+0000","last_undegraded":"2024-04-20T21:17:35.879405+0000","last_fullsized":"2024-04-20T21:17:35.879405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:47:20.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186611+0000","last_change":"2024-04-20T21:16:53.565561+0000","last_active":"2024-04-20T21:17:36.186611+0000","last_peered":"2024-04-20T21:17:36.186611+0000","last_clean":"2024-04-20T21:17:36.186611+0000","last_became_active":"2024-04-20T21:16:53.565207+0000","last_became_peered":"2024-04-20T21:16:53.565207+0000","last_unstale":"2024-04-20T21:17:36.186611+0000","last_undegraded":"2024-04-20T21:17:36.186611+0000","last_fullsized":"2024-04-20T21:17:36.186611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:07.512132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188224+0000","last_change":"2024-04-20T21:17:36.188224+0000","last_active":"2024-04-20T21:17:36.188224+0000","last_peered":"2024-04-20T21:17:36.188224+0000","last_clean":"2024-04-20T21:17:36.188224+0000","last_became_active":"2024-04-20T21:17:36.187615+0000","last_became_peered":"2024-04-20T21:17:36.187615+0000","last_unstale":"2024-04-20T21:17:36.188224+0000","last_undegraded":"2024-04-20T21:17:36.188224+0000","last_fullsized":"2024-04-20T21:17:36.188224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:45:48.216716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,2],"acting":[0,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878608+0000","last_change":"2024-04-20T21:16:54.571199+0000","last_active":"2024-04-20T21:17:35.878608+0000","last_peered":"2024-04-20T21:17:35.878608+0000","last_clean":"2024-04-20T21:17:35.878608+0000","last_became_active":"2024-04-20T21:16:54.570882+0000","last_became_peered":"2024-04-20T21:16:54.570882+0000","last_unstale":"2024-04-20T21:17:35.878608+0000","last_undegraded":"2024-04-20T21:17:35.878608+0000","last_fullsized":"2024-04-20T21:17:35.878608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:52.046176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879661+0000","last_change":"2024-04-20T21:16:53.566253+0000","last_active":"2024-04-20T21:17:35.879661+0000","last_peered":"2024-04-20T21:17:35.879661+0000","last_clean":"2024-04-20T21:17:35.879661+0000","last_became_active":"2024-04-20T21:16:53.565893+0000","last_became_peered":"2024-04-20T21:16:53.565893+0000","last_unstale":"2024-04-20T21:17:35.879661+0000","last_undegraded":"2024-04-20T21:17:35.879661+0000","last_fullsized":"2024-04-20T21:17:35.879661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:16.302801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879509+0000","last_change":"2024-04-20T21:16:58.611470+0000","last_active":"2024-04-20T21:17:35.879509+0000","last_peered":"2024-04-20T21:17:35.879509+0000","last_clean":"2024-04-20T21:17:35.879509+0000","last_became_active":"2024-04-20T21:16:58.611038+0000","last_became_peered":"2024-04-20T21:16:58.611038+0000","last_unstale":"2024-04-20T21:17:35.879509+0000","last_undegraded":"2024-04-20T21:17:35.879509+0000","last_fullsized":"2024-04-20T21:17:35.879509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:58:51.267480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877868+0000","last_change":"2024-04-20T21:16:54.577608+0000","last_active":"2024-04-20T21:17:35.877868+0000","last_peered":"2024-04-20T21:17:35.877868+0000","last_clean":"2024-04-20T21:17:35.877868+0000","last_became_active":"2024-04-20T21:16:54.577339+0000","last_became_peered":"2024-04-20T21:16:54.577339+0000","last_unstale":"2024-04-20T21:17:35.877868+0000","last_undegraded":"2024-04-20T21:17:35.877868+0000","last_fullsized":"2024-04-20T21:17:35.877868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:03.028585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879518+0000","last_change":"2024-04-20T21:16:53.547751+0000","last_active":"2024-04-20T21:17:35.879518+0000","last_peered":"2024-04-20T21:17:35.879518+0000","last_clean":"2024-04-20T21:17:35.879518+0000","last_became_active":"2024-04-20T21:16:53.547371+0000","last_became_peered":"2024-04-20T21:16:53.547371+0000","last_unstale":"2024-04-20T21:17:35.879518+0000","last_undegraded":"2024-04-20T21:17:35.879518+0000","last_fullsized":"2024-04-20T21:17:35.879518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:50.728500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878920+0000","last_change":"2024-04-20T21:16:58.622369+0000","last_active":"2024-04-20T21:17:35.878920+0000","last_peered":"2024-04-20T21:17:35.878920+0000","last_clean":"2024-04-20T21:17:35.878920+0000","last_became_active":"2024-04-20T21:16:58.621911+0000","last_became_peered":"2024-04-20T21:16:58.621911+0000","last_unstale":"2024-04-20T21:17:35.878920+0000","last_undegraded":"2024-04-20T21:17:35.878920+0000","last_fullsized":"2024-04-20T21:17:35.878920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:41.519583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878299+0000","last_change":"2024-04-20T21:16:54.579577+0000","last_active":"2024-04-20T21:17:35.878299+0000","last_peered":"2024-04-20T21:17:35.878299+0000","last_clean":"2024-04-20T21:17:35.878299+0000","last_became_active":"2024-04-20T21:16:54.579156+0000","last_became_peered":"2024-04-20T21:16:54.579156+0000","last_unstale":"2024-04-20T21:17:35.878299+0000","last_undegraded":"2024-04-20T21:17:35.878299+0000","last_fullsized":"2024-04-20T21:17:35.878299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:28:14.743667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.886735+0000","last_change":"2024-04-20T21:16:53.567995+0000","last_active":"2024-04-20T21:17:35.886735+0000","last_peered":"2024-04-20T21:17:35.886735+0000","last_clean":"2024-04-20T21:17:35.886735+0000","last_became_active":"2024-04-20T21:16:53.567601+0000","last_became_peered":"2024-04-20T21:16:53.567601+0000","last_unstale":"2024-04-20T21:17:35.886735+0000","last_undegraded":"2024-04-20T21:17:35.886735+0000","last_fullsized":"2024-04-20T21:17:35.886735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:30.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879045+0000","last_change":"2024-04-20T21:16:58.626438+0000","last_active":"2024-04-20T21:17:35.879045+0000","last_peered":"2024-04-20T21:17:35.879045+0000","last_clean":"2024-04-20T21:17:35.879045+0000","last_became_active":"2024-04-20T21:16:58.625326+0000","last_became_peered":"2024-04-20T21:16:58.625326+0000","last_unstale":"2024-04-20T21:17:35.879045+0000","last_undegraded":"2024-04-20T21:17:35.879045+0000","last_fullsized":"2024-04-20T21:17:35.879045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:06.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883279+0000","last_change":"2024-04-20T21:16:54.560925+0000","last_active":"2024-04-20T21:17:35.883279+0000","last_peered":"2024-04-20T21:17:35.883279+0000","last_clean":"2024-04-20T21:17:35.883279+0000","last_became_active":"2024-04-20T21:16:54.560615+0000","last_became_peered":"2024-04-20T21:16:54.560615+0000","last_unstale":"2024-04-20T21:17:35.883279+0000","last_undegraded":"2024-04-20T21:17:35.883279+0000","last_fullsized":"2024-04-20T21:17:35.883279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:12.815692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185774+0000","last_change":"2024-04-20T21:16:53.559359+0000","last_active":"2024-04-20T21:17:36.185774+0000","last_peered":"2024-04-20T21:17:36.185774+0000","last_clean":"2024-04-20T21:17:36.185774+0000","last_became_active":"2024-04-20T21:16:53.558955+0000","last_became_peered":"2024-04-20T21:16:53.558955+0000","last_unstale":"2024-04-20T21:17:36.185774+0000","last_undegraded":"2024-04-20T21:17:36.185774+0000","last_fullsized":"2024-04-20T21:17:36.185774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:28.664404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884324+0000","last_change":"2024-04-20T21:16:58.772175+0000","last_active":"2024-04-20T21:17:35.884324+0000","last_peered":"2024-04-20T21:17:35.884324+0000","last_clean":"2024-04-20T21:17:35.884324+0000","last_became_active":"2024-04-20T21:16:58.771732+0000","last_became_peered":"2024-04-20T21:16:58.771732+0000","last_unstale":"2024-04-20T21:17:35.884324+0000","last_undegraded":"2024-04-20T21:17:35.884324+0000","last_fullsized":"2024-04-20T21:17:35.884324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:13.155547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186877+0000","last_change":"2024-04-20T21:16:54.575375+0000","last_active":"2024-04-20T21:17:36.186877+0000","last_peered":"2024-04-20T21:17:36.186877+0000","last_clean":"2024-04-20T21:17:36.186877+0000","last_became_active":"2024-04-20T21:16:54.574194+0000","last_became_peered":"2024-04-20T21:16:54.574194+0000","last_unstale":"2024-04-20T21:17:36.186877+0000","last_undegraded":"2024-04-20T21:17:36.186877+0000","last_fullsized":"2024-04-20T21:17:36.186877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:07.948431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878007+0000","last_change":"2024-04-20T21:16:53.559374+0000","last_active":"2024-04-20T21:17:35.878007+0000","last_peered":"2024-04-20T21:17:35.878007+0000","last_clean":"2024-04-20T21:17:35.878007+0000","last_became_active":"2024-04-20T21:16:53.559133+0000","last_became_peered":"2024-04-20T21:16:53.559133+0000","last_unstale":"2024-04-20T21:17:35.878007+0000","last_undegraded":"2024-04-20T21:17:35.878007+0000","last_fullsized":"2024-04-20T21:17:35.878007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:38.576103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185908+0000","last_change":"2024-04-20T21:16:44.100529+0000","last_active":"2024-04-20T21:17:36.185908+0000","last_peered":"2024-04-20T21:17:36.185908+0000","last_clean":"2024-04-20T21:17:36.185908+0000","last_became_active":"2024-04-20T21:16:44.099590+0000","last_became_peered":"2024-04-20T21:16:44.099590+0000","last_unstale":"2024-04-20T21:17:36.185908+0000","last_undegraded":"2024-04-20T21:17:36.185908+0000","last_fullsized":"2024-04-20T21:17:36.185908+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_clean_scrub_stamp":"2024-04-20T21:16:42.782844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:08.054837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937912+0000","last_change":"2024-04-20T21:16:58.772894+0000","last_active":"2024-04-20T21:17:35.937912+0000","last_peered":"2024-04-20T21:17:35.937912+0000","last_clean":"2024-04-20T21:17:35.937912+0000","last_became_active":"2024-04-20T21:16:58.772451+0000","last_became_peered":"2024-04-20T21:16:58.772451+0000","last_unstale":"2024-04-20T21:17:35.937912+0000","last_undegraded":"2024-04-20T21:17:35.937912+0000","last_fullsized":"2024-04-20T21:17:35.937912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:29.914096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186355+0000","last_change":"2024-04-20T21:16:54.562592+0000","last_active":"2024-04-20T21:17:36.186355+0000","last_peered":"2024-04-20T21:17:36.186355+0000","last_clean":"2024-04-20T21:17:36.186355+0000","last_became_active":"2024-04-20T21:16:54.562243+0000","last_became_peered":"2024-04-20T21:16:54.562243+0000","last_unstale":"2024-04-20T21:17:36.186355+0000","last_undegraded":"2024-04-20T21:17:36.186355+0000","last_fullsized":"2024-04-20T21:17:36.186355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:52.832159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887300+0000","last_change":"2024-04-20T21:16:53.561132+0000","last_active":"2024-04-20T21:17:35.887300+0000","last_peered":"2024-04-20T21:17:35.887300+0000","last_clean":"2024-04-20T21:17:35.887300+0000","last_became_active":"2024-04-20T21:16:53.559901+0000","last_became_peered":"2024-04-20T21:16:53.559901+0000","last_unstale":"2024-04-20T21:17:35.887300+0000","last_undegraded":"2024-04-20T21:17:35.887300+0000","last_fullsized":"2024-04-20T21:17:35.887300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:20.229573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937705+0000","last_change":"2024-04-20T21:16:58.771204+0000","last_active":"2024-04-20T21:17:35.937705+0000","last_peered":"2024-04-20T21:17:35.937705+0000","last_clean":"2024-04-20T21:17:35.937705+0000","last_became_active":"2024-04-20T21:16:58.770627+0000","last_became_peered":"2024-04-20T21:16:58.770627+0000","last_unstale":"2024-04-20T21:17:35.937705+0000","last_undegraded":"2024-04-20T21:17:35.937705+0000","last_fullsized":"2024-04-20T21:17:35.937705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:39.472931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.194007+0000","last_change":"2024-04-20T21:17:36.192480+0000","last_active":"2024-04-20T21:17:36.194007+0000","last_peered":"2024-04-20T21:17:36.194007+0000","last_clean":"2024-04-20T21:17:36.194007+0000","last_became_active":"2024-04-20T21:17:36.190832+0000","last_became_peered":"2024-04-20T21:17:36.190832+0000","last_unstale":"2024-04-20T21:17:36.194007+0000","last_undegraded":"2024-04-20T21:17:36.194007+0000","last_fullsized":"2024-04-20T21:17:36.194007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:41.520453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.881260+0000","last_change":"2024-04-20T21:16:53.558554+0000","last_active":"2024-04-20T21:17:35.881260+0000","last_peered":"2024-04-20T21:17:35.881260+0000","last_clean":"2024-04-20T21:17:35.881260+0000","last_became_active":"2024-04-20T21:16:53.556934+0000","last_became_peered":"2024-04-20T21:16:53.556934+0000","last_unstale":"2024-04-20T21:17:35.881260+0000","last_undegraded":"2024-04-20T21:17:35.881260+0000","last_fullsized":"2024-04-20T21:17:35.881260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:17:52.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884183+0000","last_change":"2024-04-20T21:16:58.624410+0000","last_active":"2024-04-20T21:17:35.884183+0000","last_peered":"2024-04-20T21:17:35.884183+0000","last_clean":"2024-04-20T21:17:35.884183+0000","last_became_active":"2024-04-20T21:16:58.623677+0000","last_became_peered":"2024-04-20T21:16:58.623677+0000","last_unstale":"2024-04-20T21:17:35.884183+0000","last_undegraded":"2024-04-20T21:17:35.884183+0000","last_fullsized":"2024-04-20T21:17:35.884183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:34.465594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186797+0000","last_change":"2024-04-20T21:16:54.560217+0000","last_active":"2024-04-20T21:17:36.186797+0000","last_peered":"2024-04-20T21:17:36.186797+0000","last_clean":"2024-04-20T21:17:36.186797+0000","last_became_active":"2024-04-20T21:16:54.559947+0000","last_became_peered":"2024-04-20T21:16:54.559947+0000","last_unstale":"2024-04-20T21:17:36.186797+0000","last_undegraded":"2024-04-20T21:17:36.186797+0000","last_fullsized":"2024-04-20T21:17:36.186797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:54.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887513+0000","last_change":"2024-04-20T21:16:53.564779+0000","last_active":"2024-04-20T21:17:35.887513+0000","last_peered":"2024-04-20T21:17:35.887513+0000","last_clean":"2024-04-20T21:17:35.887513+0000","last_became_active":"2024-04-20T21:16:53.564500+0000","last_became_peered":"2024-04-20T21:16:53.564500+0000","last_unstale":"2024-04-20T21:17:35.887513+0000","last_undegraded":"2024-04-20T21:17:35.887513+0000","last_fullsized":"2024-04-20T21:17:35.887513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:00:58.673145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877151+0000","last_change":"2024-04-20T21:16:58.771524+0000","last_active":"2024-04-20T21:17:35.877151+0000","last_peered":"2024-04-20T21:17:35.877151+0000","last_clean":"2024-04-20T21:17:35.877151+0000","last_became_active":"2024-04-20T21:16:58.770672+0000","last_became_peered":"2024-04-20T21:16:58.770672+0000","last_unstale":"2024-04-20T21:17:35.877151+0000","last_undegraded":"2024-04-20T21:17:35.877151+0000","last_fullsized":"2024-04-20T21:17:35.877151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:52:27.037256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186604+0000","last_change":"2024-04-20T21:16:54.559570+0000","last_active":"2024-04-20T21:17:36.186604+0000","last_peered":"2024-04-20T21:17:36.186604+0000","last_clean":"2024-04-20T21:17:36.186604+0000","last_became_active":"2024-04-20T21:16:54.559307+0000","last_became_peered":"2024-04-20T21:16:54.559307+0000","last_unstale":"2024-04-20T21:17:36.186604+0000","last_undegraded":"2024-04-20T21:17:36.186604+0000","last_fullsized":"2024-04-20T21:17:36.186604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:04.417348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879042+0000","last_change":"2024-04-20T21:16:53.565289+0000","last_active":"2024-04-20T21:17:35.879042+0000","last_peered":"2024-04-20T21:17:35.879042+0000","last_clean":"2024-04-20T21:17:35.879042+0000","last_became_active":"2024-04-20T21:16:53.565014+0000","last_became_peered":"2024-04-20T21:16:53.565014+0000","last_unstale":"2024-04-20T21:17:35.879042+0000","last_undegraded":"2024-04-20T21:17:35.879042+0000","last_fullsized":"2024-04-20T21:17:35.879042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:22.125301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878310+0000","last_change":"2024-04-20T21:16:58.615103+0000","last_active":"2024-04-20T21:17:35.878310+0000","last_peered":"2024-04-20T21:17:35.878310+0000","last_clean":"2024-04-20T21:17:35.878310+0000","last_became_active":"2024-04-20T21:16:58.614068+0000","last_became_peered":"2024-04-20T21:16:58.614068+0000","last_unstale":"2024-04-20T21:17:35.878310+0000","last_undegraded":"2024-04-20T21:17:35.878310+0000","last_fullsized":"2024-04-20T21:17:35.878310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:09.555911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.896249+0000","last_change":"2024-04-20T21:17:35.891448+0000","last_active":"2024-04-20T21:17:35.896249+0000","last_peered":"2024-04-20T21:17:35.896249+0000","last_clean":"2024-04-20T21:17:35.896249+0000","last_became_active":"2024-04-20T21:17:35.891149+0000","last_became_peered":"2024-04-20T21:17:35.891149+0000","last_unstale":"2024-04-20T21:17:35.896249+0000","last_undegraded":"2024-04-20T21:17:35.896249+0000","last_fullsized":"2024-04-20T21:17:35.896249+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:24:05.580789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.885970+0000","last_change":"2024-04-20T21:16:53.570182+0000","last_active":"2024-04-20T21:17:35.885970+0000","last_peered":"2024-04-20T21:17:35.885970+0000","last_clean":"2024-04-20T21:17:35.885970+0000","last_became_active":"2024-04-20T21:16:53.569632+0000","last_became_peered":"2024-04-20T21:16:53.569632+0000","last_unstale":"2024-04-20T21:17:35.885970+0000","last_undegraded":"2024-04-20T21:17:35.885970+0000","last_fullsized":"2024-04-20T21:17:35.885970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:12.140497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884216+0000","last_change":"2024-04-20T21:16:58.625602+0000","last_active":"2024-04-20T21:17:35.884216+0000","last_peered":"2024-04-20T21:17:35.884216+0000","last_clean":"2024-04-20T21:17:35.884216+0000","last_became_active":"2024-04-20T21:16:58.625158+0000","last_became_peered":"2024-04-20T21:16:58.625158+0000","last_unstale":"2024-04-20T21:17:35.884216+0000","last_undegraded":"2024-04-20T21:17:35.884216+0000","last_fullsized":"2024-04-20T21:17:35.884216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:09.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938129+0000","last_change":"2024-04-20T21:16:54.578603+0000","last_active":"2024-04-20T21:17:35.938129+0000","last_peered":"2024-04-20T21:17:35.938129+0000","last_clean":"2024-04-20T21:17:35.938129+0000","last_became_active":"2024-04-20T21:16:54.578338+0000","last_became_peered":"2024-04-20T21:16:54.578338+0000","last_unstale":"2024-04-20T21:17:35.938129+0000","last_undegraded":"2024-04-20T21:17:35.938129+0000","last_fullsized":"2024-04-20T21:17:35.938129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:14:56.355343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937825+0000","last_change":"2024-04-20T21:16:53.551847+0000","last_active":"2024-04-20T21:17:35.937825+0000","last_peered":"2024-04-20T21:17:35.937825+0000","last_clean":"2024-04-20T21:17:35.937825+0000","last_became_active":"2024-04-20T21:16:53.551427+0000","last_became_peered":"2024-04-20T21:16:53.551427+0000","last_unstale":"2024-04-20T21:17:35.937825+0000","last_undegraded":"2024-04-20T21:17:35.937825+0000","last_fullsized":"2024-04-20T21:17:35.937825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:55:45.930305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189251+0000","last_change":"2024-04-20T21:16:58.777159+0000","last_active":"2024-04-20T21:17:36.189251+0000","last_peered":"2024-04-20T21:17:36.189251+0000","last_clean":"2024-04-20T21:17:36.189251+0000","last_became_active":"2024-04-20T21:16:58.776233+0000","last_became_peered":"2024-04-20T21:16:58.776233+0000","last_unstale":"2024-04-20T21:17:36.189251+0000","last_undegraded":"2024-04-20T21:17:36.189251+0000","last_fullsized":"2024-04-20T21:17:36.189251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:05.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186236+0000","last_change":"2024-04-20T21:16:54.570687+0000","last_active":"2024-04-20T21:17:36.186236+0000","last_peered":"2024-04-20T21:17:36.186236+0000","last_clean":"2024-04-20T21:17:36.186236+0000","last_became_active":"2024-04-20T21:16:54.570360+0000","last_became_peered":"2024-04-20T21:16:54.570360+0000","last_unstale":"2024-04-20T21:17:36.186236+0000","last_undegraded":"2024-04-20T21:17:36.186236+0000","last_fullsized":"2024-04-20T21:17:36.186236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:05.606229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879415+0000","last_change":"2024-04-20T21:16:53.567599+0000","last_active":"2024-04-20T21:17:35.879415+0000","last_peered":"2024-04-20T21:17:35.879415+0000","last_clean":"2024-04-20T21:17:35.879415+0000","last_became_active":"2024-04-20T21:16:53.567378+0000","last_became_peered":"2024-04-20T21:16:53.567378+0000","last_unstale":"2024-04-20T21:17:35.879415+0000","last_undegraded":"2024-04-20T21:17:35.879415+0000","last_fullsized":"2024-04-20T21:17:35.879415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:36.611887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188405+0000","last_change":"2024-04-20T21:16:58.774226+0000","last_active":"2024-04-20T21:17:36.188405+0000","last_peered":"2024-04-20T21:17:36.188405+0000","last_clean":"2024-04-20T21:17:36.188405+0000","last_became_active":"2024-04-20T21:16:58.771584+0000","last_became_peered":"2024-04-20T21:16:58.771584+0000","last_unstale":"2024-04-20T21:17:36.188405+0000","last_undegraded":"2024-04-20T21:17:36.188405+0000","last_fullsized":"2024-04-20T21:17:36.188405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:31.432077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883628+0000","last_change":"2024-04-20T21:16:54.560001+0000","last_active":"2024-04-20T21:17:35.883628+0000","last_peered":"2024-04-20T21:17:35.883628+0000","last_clean":"2024-04-20T21:17:35.883628+0000","last_became_active":"2024-04-20T21:16:54.559643+0000","last_became_peered":"2024-04-20T21:16:54.559643+0000","last_unstale":"2024-04-20T21:17:35.883628+0000","last_undegraded":"2024-04-20T21:17:35.883628+0000","last_fullsized":"2024-04-20T21:17:35.883628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:52:37.325946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884673+0000","last_change":"2024-04-20T21:16:53.570498+0000","last_active":"2024-04-20T21:17:35.884673+0000","last_peered":"2024-04-20T21:17:35.884673+0000","last_clean":"2024-04-20T21:17:35.884673+0000","last_became_active":"2024-04-20T21:16:53.570095+0000","last_became_peered":"2024-04-20T21:16:53.570095+0000","last_unstale":"2024-04-20T21:17:35.884673+0000","last_undegraded":"2024-04-20T21:17:35.884673+0000","last_fullsized":"2024-04-20T21:17:35.884673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:58:28.677814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878493+0000","last_change":"2024-04-20T21:16:58.773349+0000","last_active":"2024-04-20T21:17:35.878493+0000","last_peered":"2024-04-20T21:17:35.878493+0000","last_clean":"2024-04-20T21:17:35.878493+0000","last_became_active":"2024-04-20T21:16:58.772781+0000","last_became_peered":"2024-04-20T21:16:58.772781+0000","last_unstale":"2024-04-20T21:17:35.878493+0000","last_undegraded":"2024-04-20T21:17:35.878493+0000","last_fullsized":"2024-04-20T21:17:35.878493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:53:02.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186216+0000","last_change":"2024-04-20T21:16:54.572749+0000","last_active":"2024-04-20T21:17:36.186216+0000","last_peered":"2024-04-20T21:17:36.186216+0000","last_clean":"2024-04-20T21:17:36.186216+0000","last_became_active":"2024-04-20T21:16:54.572456+0000","last_became_peered":"2024-04-20T21:16:54.572456+0000","last_unstale":"2024-04-20T21:17:36.186216+0000","last_undegraded":"2024-04-20T21:17:36.186216+0000","last_fullsized":"2024-04-20T21:17:36.186216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.787612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'12","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.998099+0000","last_change":"2024-04-20T21:16:53.568688+0000","last_active":"2024-04-20T21:17:36.998099+0000","last_peered":"2024-04-20T21:17:36.998099+0000","last_clean":"2024-04-20T21:17:36.998099+0000","last_became_active":"2024-04-20T21:16:53.568495+0000","last_became_peered":"2024-04-20T21:16:53.568495+0000","last_unstale":"2024-04-20T21:17:36.998099+0000","last_undegraded":"2024-04-20T21:17:36.998099+0000","last_fullsized":"2024-04-20T21:17:36.998099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:00.086004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4100,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879709+0000","last_change":"2024-04-20T21:16:58.609575+0000","last_active":"2024-04-20T21:17:35.879709+0000","last_peered":"2024-04-20T21:17:35.879709+0000","last_clean":"2024-04-20T21:17:35.879709+0000","last_became_active":"2024-04-20T21:16:58.609127+0000","last_became_peered":"2024-04-20T21:16:58.609127+0000","last_unstale":"2024-04-20T21:17:35.879709+0000","last_undegraded":"2024-04-20T21:17:35.879709+0000","last_fullsized":"2024-04-20T21:17:35.879709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:02.645274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878544+0000","last_change":"2024-04-20T21:16:54.563213+0000","last_active":"2024-04-20T21:17:35.878544+0000","last_peered":"2024-04-20T21:17:35.878544+0000","last_clean":"2024-04-20T21:17:35.878544+0000","last_became_active":"2024-04-20T21:16:54.562917+0000","last_became_peered":"2024-04-20T21:16:54.562917+0000","last_unstale":"2024-04-20T21:17:35.878544+0000","last_undegraded":"2024-04-20T21:17:35.878544+0000","last_fullsized":"2024-04-20T21:17:35.878544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.155771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880903+0000","last_change":"2024-04-20T21:16:53.556988+0000","last_active":"2024-04-20T21:17:35.880903+0000","last_peered":"2024-04-20T21:17:35.880903+0000","last_clean":"2024-04-20T21:17:35.880903+0000","last_became_active":"2024-04-20T21:16:53.556672+0000","last_became_peered":"2024-04-20T21:16:53.556672+0000","last_unstale":"2024-04-20T21:17:35.880903+0000","last_undegraded":"2024-04-20T21:17:35.880903+0000","last_fullsized":"2024-04-20T21:17:35.880903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:25.219611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.939270+0000","last_change":"2024-04-20T21:17:35.939270+0000","last_active":"2024-04-20T21:17:35.939270+0000","last_peered":"2024-04-20T21:17:35.939270+0000","last_clean":"2024-04-20T21:17:35.939270+0000","last_became_active":"2024-04-20T21:17:35.938833+0000","last_became_peered":"2024-04-20T21:17:35.938833+0000","last_unstale":"2024-04-20T21:17:35.939270+0000","last_undegraded":"2024-04-20T21:17:35.939270+0000","last_fullsized":"2024-04-20T21:17:35.939270+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.790995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884930+0000","last_change":"2024-04-20T21:16:54.562128+0000","last_active":"2024-04-20T21:17:35.884930+0000","last_peered":"2024-04-20T21:17:35.884930+0000","last_clean":"2024-04-20T21:17:35.884930+0000","last_became_active":"2024-04-20T21:16:54.561495+0000","last_became_peered":"2024-04-20T21:16:54.561495+0000","last_unstale":"2024-04-20T21:17:35.884930+0000","last_undegraded":"2024-04-20T21:17:35.884930+0000","last_fullsized":"2024-04-20T21:17:35.884930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:19.582218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878813+0000","last_change":"2024-04-20T21:16:53.554881+0000","last_active":"2024-04-20T21:17:35.878813+0000","last_peered":"2024-04-20T21:17:35.878813+0000","last_clean":"2024-04-20T21:17:35.878813+0000","last_became_active":"2024-04-20T21:16:53.554590+0000","last_became_peered":"2024-04-20T21:16:53.554590+0000","last_unstale":"2024-04-20T21:17:35.878813+0000","last_undegraded":"2024-04-20T21:17:35.878813+0000","last_fullsized":"2024-04-20T21:17:35.878813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:36.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.187143+0000","last_change":"2024-04-20T21:16:58.610704+0000","last_active":"2024-04-20T21:17:36.187143+0000","last_peered":"2024-04-20T21:17:36.187143+0000","last_clean":"2024-04-20T21:17:36.187143+0000","last_became_active":"2024-04-20T21:16:58.610309+0000","last_became_peered":"2024-04-20T21:16:58.610309+0000","last_unstale":"2024-04-20T21:17:36.187143+0000","last_undegraded":"2024-04-20T21:17:36.187143+0000","last_fullsized":"2024-04-20T21:17:36.187143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:04.215050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878430+0000","last_change":"2024-04-20T21:16:54.562351+0000","last_active":"2024-04-20T21:17:35.878430+0000","last_peered":"2024-04-20T21:17:35.878430+0000","last_clean":"2024-04-20T21:17:35.878430+0000","last_became_active":"2024-04-20T21:16:54.561947+0000","last_became_peered":"2024-04-20T21:16:54.561947+0000","last_unstale":"2024-04-20T21:17:35.878430+0000","last_undegraded":"2024-04-20T21:17:35.878430+0000","last_fullsized":"2024-04-20T21:17:35.878430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:52.802328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938343+0000","last_change":"2024-04-20T21:16:53.554081+0000","last_active":"2024-04-20T21:17:35.938343+0000","last_peered":"2024-04-20T21:17:35.938343+0000","last_clean":"2024-04-20T21:17:35.938343+0000","last_became_active":"2024-04-20T21:16:53.553800+0000","last_became_peered":"2024-04-20T21:16:53.553800+0000","last_unstale":"2024-04-20T21:17:35.938343+0000","last_undegraded":"2024-04-20T21:17:35.938343+0000","last_fullsized":"2024-04-20T21:17:35.938343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:52.865485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186668+0000","last_change":"2024-04-20T21:16:58.779184+0000","last_active":"2024-04-20T21:17:36.186668+0000","last_peered":"2024-04-20T21:17:36.186668+0000","last_clean":"2024-04-20T21:17:36.186668+0000","last_became_active":"2024-04-20T21:16:58.778627+0000","last_became_peered":"2024-04-20T21:16:58.778627+0000","last_unstale":"2024-04-20T21:17:36.186668+0000","last_undegraded":"2024-04-20T21:17:36.186668+0000","last_fullsized":"2024-04-20T21:17:36.186668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:50:07.247736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884103+0000","last_change":"2024-04-20T21:16:58.774438+0000","last_active":"2024-04-20T21:17:35.884103+0000","last_peered":"2024-04-20T21:17:35.884103+0000","last_clean":"2024-04-20T21:17:35.884103+0000","last_became_active":"2024-04-20T21:16:58.774030+0000","last_became_peered":"2024-04-20T21:16:58.774030+0000","last_unstale":"2024-04-20T21:17:35.884103+0000","last_undegraded":"2024-04-20T21:17:35.884103+0000","last_fullsized":"2024-04-20T21:17:35.884103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:09:22.583239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880247+0000","last_change":"2024-04-20T21:16:54.573779+0000","last_active":"2024-04-20T21:17:35.880247+0000","last_peered":"2024-04-20T21:17:35.880247+0000","last_clean":"2024-04-20T21:17:35.880247+0000","last_became_active":"2024-04-20T21:16:54.573444+0000","last_became_peered":"2024-04-20T21:16:54.573444+0000","last_unstale":"2024-04-20T21:17:35.880247+0000","last_undegraded":"2024-04-20T21:17:35.880247+0000","last_fullsized":"2024-04-20T21:17:35.880247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:27.950525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877644+0000","last_change":"2024-04-20T21:16:53.543678+0000","last_active":"2024-04-20T21:17:35.877644+0000","last_peered":"2024-04-20T21:17:35.877644+0000","last_clean":"2024-04-20T21:17:35.877644+0000","last_became_active":"2024-04-20T21:16:53.543381+0000","last_became_peered":"2024-04-20T21:16:53.543381+0000","last_unstale":"2024-04-20T21:17:35.877644+0000","last_undegraded":"2024-04-20T21:17:35.877644+0000","last_fullsized":"2024-04-20T21:17:35.877644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:02.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938285+0000","last_change":"2024-04-20T21:16:58.626882+0000","last_active":"2024-04-20T21:17:35.938285+0000","last_peered":"2024-04-20T21:17:35.938285+0000","last_clean":"2024-04-20T21:17:35.938285+0000","last_became_active":"2024-04-20T21:16:58.626489+0000","last_became_peered":"2024-04-20T21:16:58.626489+0000","last_unstale":"2024-04-20T21:17:35.938285+0000","last_undegraded":"2024-04-20T21:17:35.938285+0000","last_fullsized":"2024-04-20T21:17:35.938285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:33:22.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.187276+0000","last_change":"2024-04-20T21:16:54.580893+0000","last_active":"2024-04-20T21:17:36.187276+0000","last_peered":"2024-04-20T21:17:36.187276+0000","last_clean":"2024-04-20T21:17:36.187276+0000","last_became_active":"2024-04-20T21:16:54.580528+0000","last_became_peered":"2024-04-20T21:16:54.580528+0000","last_unstale":"2024-04-20T21:17:36.187276+0000","last_undegraded":"2024-04-20T21:17:36.187276+0000","last_fullsized":"2024-04-20T21:17:36.187276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:14.699123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186916+0000","last_change":"2024-04-20T21:16:53.563930+0000","last_active":"2024-04-20T21:17:36.186916+0000","last_peered":"2024-04-20T21:17:36.186916+0000","last_clean":"2024-04-20T21:17:36.186916+0000","last_became_active":"2024-04-20T21:16:53.559991+0000","last_became_peered":"2024-04-20T21:16:53.559991+0000","last_unstale":"2024-04-20T21:17:36.186916+0000","last_undegraded":"2024-04-20T21:17:36.186916+0000","last_fullsized":"2024-04-20T21:17:36.186916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:22.529712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937884+0000","last_change":"2024-04-20T21:16:53.552509+0000","last_active":"2024-04-20T21:17:35.937884+0000","last_peered":"2024-04-20T21:17:35.937884+0000","last_clean":"2024-04-20T21:17:35.937884+0000","last_became_active":"2024-04-20T21:16:53.552173+0000","last_became_peered":"2024-04-20T21:16:53.552173+0000","last_unstale":"2024-04-20T21:17:35.937884+0000","last_undegraded":"2024-04-20T21:17:35.937884+0000","last_fullsized":"2024-04-20T21:17:35.937884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:14.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884531+0000","last_change":"2024-04-20T21:16:54.562849+0000","last_active":"2024-04-20T21:17:35.884531+0000","last_peered":"2024-04-20T21:17:35.884531+0000","last_clean":"2024-04-20T21:17:35.884531+0000","last_became_active":"2024-04-20T21:16:54.561283+0000","last_became_peered":"2024-04-20T21:16:54.561283+0000","last_unstale":"2024-04-20T21:17:35.884531+0000","last_undegraded":"2024-04-20T21:17:35.884531+0000","last_fullsized":"2024-04-20T21:17:35.884531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:52.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887173+0000","last_change":"2024-04-20T21:16:58.614752+0000","last_active":"2024-04-20T21:17:35.887173+0000","last_peered":"2024-04-20T21:17:35.887173+0000","last_clean":"2024-04-20T21:17:35.887173+0000","last_became_active":"2024-04-20T21:16:58.614201+0000","last_became_peered":"2024-04-20T21:16:58.614201+0000","last_unstale":"2024-04-20T21:17:35.887173+0000","last_undegraded":"2024-04-20T21:17:35.887173+0000","last_fullsized":"2024-04-20T21:17:35.887173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:46.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188988+0000","last_change":"2024-04-20T21:16:53.564303+0000","last_active":"2024-04-20T21:17:36.188988+0000","last_peered":"2024-04-20T21:17:36.188988+0000","last_clean":"2024-04-20T21:17:36.188988+0000","last_became_active":"2024-04-20T21:16:53.560203+0000","last_became_peered":"2024-04-20T21:16:53.560203+0000","last_unstale":"2024-04-20T21:17:36.188988+0000","last_undegraded":"2024-04-20T21:17:36.188988+0000","last_fullsized":"2024-04-20T21:17:36.188988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:42.691233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878072+0000","last_change":"2024-04-20T21:16:54.554499+0000","last_active":"2024-04-20T21:17:35.878072+0000","last_peered":"2024-04-20T21:17:35.878072+0000","last_clean":"2024-04-20T21:17:35.878072+0000","last_became_active":"2024-04-20T21:16:54.554230+0000","last_became_peered":"2024-04-20T21:16:54.554230+0000","last_unstale":"2024-04-20T21:17:35.878072+0000","last_undegraded":"2024-04-20T21:17:35.878072+0000","last_fullsized":"2024-04-20T21:17:35.878072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:24:31.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188251+0000","last_change":"2024-04-20T21:16:58.610424+0000","last_active":"2024-04-20T21:17:36.188251+0000","last_peered":"2024-04-20T21:17:36.188251+0000","last_clean":"2024-04-20T21:17:36.188251+0000","last_became_active":"2024-04-20T21:16:58.610025+0000","last_became_peered":"2024-04-20T21:16:58.610025+0000","last_unstale":"2024-04-20T21:17:36.188251+0000","last_undegraded":"2024-04-20T21:17:36.188251+0000","last_fullsized":"2024-04-20T21:17:36.188251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:19.290865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937939+0000","last_change":"2024-04-20T21:16:54.578801+0000","last_active":"2024-04-20T21:17:35.937939+0000","last_peered":"2024-04-20T21:17:35.937939+0000","last_clean":"2024-04-20T21:17:35.937939+0000","last_became_active":"2024-04-20T21:16:54.578543+0000","last_became_peered":"2024-04-20T21:16:54.578543+0000","last_unstale":"2024-04-20T21:17:35.937939+0000","last_undegraded":"2024-04-20T21:17:35.937939+0000","last_fullsized":"2024-04-20T21:17:35.937939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:15.042318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.886717+0000","last_change":"2024-04-20T21:16:54.565658+0000","last_active":"2024-04-20T21:17:35.886717+0000","last_peered":"2024-04-20T21:17:35.886717+0000","last_clean":"2024-04-20T21:17:35.886717+0000","last_became_active":"2024-04-20T21:16:54.565371+0000","last_became_peered":"2024-04-20T21:16:54.565371+0000","last_unstale":"2024-04-20T21:17:35.886717+0000","last_undegraded":"2024-04-20T21:17:35.886717+0000","last_fullsized":"2024-04-20T21:17:35.886717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:21.781013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879317+0000","last_change":"2024-04-20T21:16:54.568472+0000","last_active":"2024-04-20T21:17:35.879317+0000","last_peered":"2024-04-20T21:17:35.879317+0000","last_clean":"2024-04-20T21:17:35.879317+0000","last_became_active":"2024-04-20T21:16:54.568223+0000","last_became_peered":"2024-04-20T21:16:54.568223+0000","last_unstale":"2024-04-20T21:17:35.879317+0000","last_undegraded":"2024-04-20T21:17:35.879317+0000","last_fullsized":"2024-04-20T21:17:35.879317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:42:53.921935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884279+0000","last_change":"2024-04-20T21:16:54.573747+0000","last_active":"2024-04-20T21:17:35.884279+0000","last_peered":"2024-04-20T21:17:35.884279+0000","last_clean":"2024-04-20T21:17:35.884279+0000","last_became_active":"2024-04-20T21:16:54.573476+0000","last_became_peered":"2024-04-20T21:16:54.573476+0000","last_unstale":"2024-04-20T21:17:35.884279+0000","last_undegraded":"2024-04-20T21:17:35.884279+0000","last_fullsized":"2024-04-20T21:17:35.884279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:02.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188768+0000","last_change":"2024-04-20T21:16:54.577630+0000","last_active":"2024-04-20T21:17:36.188768+0000","last_peered":"2024-04-20T21:17:36.188768+0000","last_clean":"2024-04-20T21:17:36.188768+0000","last_became_active":"2024-04-20T21:16:54.577318+0000","last_became_peered":"2024-04-20T21:16:54.577318+0000","last_unstale":"2024-04-20T21:17:36.188768+0000","last_undegraded":"2024-04-20T21:17:36.188768+0000","last_fullsized":"2024-04-20T21:17:36.188768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:45.584050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879003+0000","last_change":"2024-04-20T21:16:54.579850+0000","last_active":"2024-04-20T21:17:35.879003+0000","last_peered":"2024-04-20T21:17:35.879003+0000","last_clean":"2024-04-20T21:17:35.879003+0000","last_became_active":"2024-04-20T21:16:54.579546+0000","last_became_peered":"2024-04-20T21:16:54.579546+0000","last_unstale":"2024-04-20T21:17:35.879003+0000","last_undegraded":"2024-04-20T21:17:35.879003+0000","last_fullsized":"2024-04-20T21:17:35.879003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:49:53.577077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938400+0000","last_change":"2024-04-20T21:16:54.559174+0000","last_active":"2024-04-20T21:17:35.938400+0000","last_peered":"2024-04-20T21:17:35.938400+0000","last_clean":"2024-04-20T21:17:35.938400+0000","last_became_active":"2024-04-20T21:16:54.558904+0000","last_became_peered":"2024-04-20T21:16:54.558904+0000","last_unstale":"2024-04-20T21:17:35.938400+0000","last_undegraded":"2024-04-20T21:17:35.938400+0000","last_fullsized":"2024-04-20T21:17:35.938400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:09.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189372+0000","last_change":"2024-04-20T21:16:54.574943+0000","last_active":"2024-04-20T21:17:36.189372+0000","last_peered":"2024-04-20T21:17:36.189372+0000","last_clean":"2024-04-20T21:17:36.189372+0000","last_became_active":"2024-04-20T21:16:54.574471+0000","last_became_peered":"2024-04-20T21:16:54.574471+0000","last_unstale":"2024-04-20T21:17:36.189372+0000","last_undegraded":"2024-04-20T21:17:36.189372+0000","last_fullsized":"2024-04-20T21:17:36.189372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:51:44.741339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.193375+0000","last_change":"2024-04-20T21:17:36.191229+0000","last_active":"2024-04-20T21:17:36.193375+0000","last_peered":"2024-04-20T21:17:36.193375+0000","last_clean":"2024-04-20T21:17:36.193375+0000","last_became_active":"2024-04-20T21:17:36.190776+0000","last_became_peered":"2024-04-20T21:17:36.190776+0000","last_unstale":"2024-04-20T21:17:36.193375+0000","last_undegraded":"2024-04-20T21:17:36.193375+0000","last_fullsized":"2024-04-20T21:17:36.193375+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:49:11.938154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186859+0000","last_change":"2024-04-20T21:16:54.571179+0000","last_active":"2024-04-20T21:17:36.186859+0000","last_peered":"2024-04-20T21:17:36.186859+0000","last_clean":"2024-04-20T21:17:36.186859+0000","last_became_active":"2024-04-20T21:16:54.570855+0000","last_became_peered":"2024-04-20T21:16:54.570855+0000","last_unstale":"2024-04-20T21:17:36.186859+0000","last_undegraded":"2024-04-20T21:17:36.186859+0000","last_fullsized":"2024-04-20T21:17:36.186859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:59.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879202+0000","last_change":"2024-04-20T21:16:54.559092+0000","last_active":"2024-04-20T21:17:35.879202+0000","last_peered":"2024-04-20T21:17:35.879202+0000","last_clean":"2024-04-20T21:17:35.879202+0000","last_became_active":"2024-04-20T21:16:54.558847+0000","last_became_peered":"2024-04-20T21:16:54.558847+0000","last_unstale":"2024-04-20T21:17:35.879202+0000","last_undegraded":"2024-04-20T21:17:35.879202+0000","last_fullsized":"2024-04-20T21:17:35.879202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:02.752070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884080+0000","last_change":"2024-04-20T21:16:54.571815+0000","last_active":"2024-04-20T21:17:35.884080+0000","last_peered":"2024-04-20T21:17:35.884080+0000","last_clean":"2024-04-20T21:17:35.884080+0000","last_became_active":"2024-04-20T21:16:54.570704+0000","last_became_peered":"2024-04-20T21:16:54.570704+0000","last_unstale":"2024-04-20T21:17:35.884080+0000","last_undegraded":"2024-04-20T21:17:35.884080+0000","last_fullsized":"2024-04-20T21:17:35.884080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:56.511539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938000+0000","last_change":"2024-04-20T21:16:54.555591+0000","last_active":"2024-04-20T21:17:35.938000+0000","last_peered":"2024-04-20T21:17:35.938000+0000","last_clean":"2024-04-20T21:17:35.938000+0000","last_became_active":"2024-04-20T21:16:54.555389+0000","last_became_peered":"2024-04-20T21:16:54.555389+0000","last_unstale":"2024-04-20T21:17:35.938000+0000","last_undegraded":"2024-04-20T21:17:35.938000+0000","last_fullsized":"2024-04-20T21:17:35.938000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:16:25.976707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186528+0000","last_change":"2024-04-20T21:16:54.564088+0000","last_active":"2024-04-20T21:17:36.186528+0000","last_peered":"2024-04-20T21:17:36.186528+0000","last_clean":"2024-04-20T21:17:36.186528+0000","last_became_active":"2024-04-20T21:16:54.563796+0000","last_became_peered":"2024-04-20T21:16:54.563796+0000","last_unstale":"2024-04-20T21:17:36.186528+0000","last_undegraded":"2024-04-20T21:17:36.186528+0000","last_fullsized":"2024-04-20T21:17:36.186528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:07.986736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887460+0000","last_change":"2024-04-20T21:16:54.563578+0000","last_active":"2024-04-20T21:17:35.887460+0000","last_peered":"2024-04-20T21:17:35.887460+0000","last_clean":"2024-04-20T21:17:35.887460+0000","last_became_active":"2024-04-20T21:16:54.563317+0000","last_became_peered":"2024-04-20T21:16:54.563317+0000","last_unstale":"2024-04-20T21:17:35.887460+0000","last_undegraded":"2024-04-20T21:17:35.887460+0000","last_fullsized":"2024-04-20T21:17:35.887460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:06.093675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879169+0000","last_change":"2024-04-20T21:16:54.565168+0000","last_active":"2024-04-20T21:17:35.879169+0000","last_peered":"2024-04-20T21:17:35.879169+0000","last_clean":"2024-04-20T21:17:35.879169+0000","last_became_active":"2024-04-20T21:16:54.564712+0000","last_became_peered":"2024-04-20T21:16:54.564712+0000","last_unstale":"2024-04-20T21:17:35.879169+0000","last_undegraded":"2024-04-20T21:17:35.879169+0000","last_fullsized":"2024-04-20T21:17:35.879169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:07:30.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877246+0000","last_change":"2024-04-20T21:16:54.572211+0000","last_active":"2024-04-20T21:17:35.877246+0000","last_peered":"2024-04-20T21:17:35.877246+0000","last_clean":"2024-04-20T21:17:35.877246+0000","last_became_active":"2024-04-20T21:16:54.571894+0000","last_became_peered":"2024-04-20T21:16:54.571894+0000","last_unstale":"2024-04-20T21:17:35.877246+0000","last_undegraded":"2024-04-20T21:17:35.877246+0000","last_fullsized":"2024-04-20T21:17:35.877246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:59.376514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880303+0000","last_change":"2024-04-20T21:16:54.555195+0000","last_active":"2024-04-20T21:17:35.880303+0000","last_peered":"2024-04-20T21:17:35.880303+0000","last_clean":"2024-04-20T21:17:35.880303+0000","last_became_active":"2024-04-20T21:16:54.554284+0000","last_became_peered":"2024-04-20T21:16:54.554284+0000","last_unstale":"2024-04-20T21:17:35.880303+0000","last_undegraded":"2024-04-20T21:17:35.880303+0000","last_fullsized":"2024-04-20T21:17:35.880303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:42:21.500952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883524+0000","last_change":"2024-04-20T21:16:54.565527+0000","last_active":"2024-04-20T21:17:35.883524+0000","last_peered":"2024-04-20T21:17:35.883524+0000","last_clean":"2024-04-20T21:17:35.883524+0000","last_became_active":"2024-04-20T21:16:54.565211+0000","last_became_peered":"2024-04-20T21:16:54.565211+0000","last_unstale":"2024-04-20T21:17:35.883524+0000","last_undegraded":"2024-04-20T21:17:35.883524+0000","last_fullsized":"2024-04-20T21:17:35.883524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:17:40.462670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188717+0000","last_change":"2024-04-20T21:16:54.576464+0000","last_active":"2024-04-20T21:17:36.188717+0000","last_peered":"2024-04-20T21:17:36.188717+0000","last_clean":"2024-04-20T21:17:36.188717+0000","last_became_active":"2024-04-20T21:16:54.575947+0000","last_became_peered":"2024-04-20T21:16:54.575947+0000","last_unstale":"2024-04-20T21:17:36.188717+0000","last_undegraded":"2024-04-20T21:17:36.188717+0000","last_fullsized":"2024-04-20T21:17:36.188717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:50:29.581938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878458+0000","last_change":"2024-04-20T21:16:54.558162+0000","last_active":"2024-04-20T21:17:35.878458+0000","last_peered":"2024-04-20T21:17:35.878458+0000","last_clean":"2024-04-20T21:17:35.878458+0000","last_became_active":"2024-04-20T21:16:54.557875+0000","last_became_peered":"2024-04-20T21:16:54.557875+0000","last_unstale":"2024-04-20T21:17:35.878458+0000","last_undegraded":"2024-04-20T21:17:35.878458+0000","last_fullsized":"2024-04-20T21:17:35.878458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:42:46.886405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937978+0000","last_change":"2024-04-20T21:16:54.560566+0000","last_active":"2024-04-20T21:17:35.937978+0000","last_peered":"2024-04-20T21:17:35.937978+0000","last_clean":"2024-04-20T21:17:35.937978+0000","last_became_active":"2024-04-20T21:16:54.559409+0000","last_became_peered":"2024-04-20T21:16:54.559409+0000","last_unstale":"2024-04-20T21:17:35.937978+0000","last_undegraded":"2024-04-20T21:17:35.937978+0000","last_fullsized":"2024-04-20T21:17:35.937978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:00.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938279+0000","last_change":"2024-04-20T21:16:54.559491+0000","last_active":"2024-04-20T21:17:35.938279+0000","last_peered":"2024-04-20T21:17:35.938279+0000","last_clean":"2024-04-20T21:17:35.938279+0000","last_became_active":"2024-04-20T21:16:54.558187+0000","last_became_peered":"2024-04-20T21:16:54.558187+0000","last_unstale":"2024-04-20T21:17:35.938279+0000","last_undegraded":"2024-04-20T21:17:35.938279+0000","last_fullsized":"2024-04-20T21:17:35.938279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:33.686248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186775+0000","last_change":"2024-04-20T21:16:54.564258+0000","last_active":"2024-04-20T21:17:36.186775+0000","last_peered":"2024-04-20T21:17:36.186775+0000","last_clean":"2024-04-20T21:17:36.186775+0000","last_became_active":"2024-04-20T21:16:54.563966+0000","last_became_peered":"2024-04-20T21:16:54.563966+0000","last_unstale":"2024-04-20T21:17:36.186775+0000","last_undegraded":"2024-04-20T21:17:36.186775+0000","last_fullsized":"2024-04-20T21:17:36.186775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:13:34.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186994+0000","last_change":"2024-04-20T21:16:54.573554+0000","last_active":"2024-04-20T21:17:36.186994+0000","last_peered":"2024-04-20T21:17:36.186994+0000","last_clean":"2024-04-20T21:17:36.186994+0000","last_became_active":"2024-04-20T21:16:54.571849+0000","last_became_peered":"2024-04-20T21:16:54.571849+0000","last_unstale":"2024-04-20T21:17:36.186994+0000","last_undegraded":"2024-04-20T21:17:36.186994+0000","last_fullsized":"2024-04-20T21:17:36.186994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:09.481779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186052+0000","last_change":"2024-04-20T21:16:54.571471+0000","last_active":"2024-04-20T21:17:36.186052+0000","last_peered":"2024-04-20T21:17:36.186052+0000","last_clean":"2024-04-20T21:17:36.186052+0000","last_became_active":"2024-04-20T21:16:54.571209+0000","last_became_peered":"2024-04-20T21:16:54.571209+0000","last_unstale":"2024-04-20T21:17:36.186052+0000","last_undegraded":"2024-04-20T21:17:36.186052+0000","last_fullsized":"2024-04-20T21:17:36.186052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:35.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879415+0000","last_change":"2024-04-20T21:16:54.572168+0000","last_active":"2024-04-20T21:17:35.879415+0000","last_peered":"2024-04-20T21:17:35.879415+0000","last_clean":"2024-04-20T21:17:35.879415+0000","last_became_active":"2024-04-20T21:16:54.571740+0000","last_became_peered":"2024-04-20T21:16:54.571740+0000","last_unstale":"2024-04-20T21:17:35.879415+0000","last_undegraded":"2024-04-20T21:17:35.879415+0000","last_fullsized":"2024-04-20T21:17:35.879415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:11:35.728824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186187+0000","last_change":"2024-04-20T21:16:54.569081+0000","last_active":"2024-04-20T21:17:36.186187+0000","last_peered":"2024-04-20T21:17:36.186187+0000","last_clean":"2024-04-20T21:17:36.186187+0000","last_became_active":"2024-04-20T21:16:54.568847+0000","last_became_peered":"2024-04-20T21:16:54.568847+0000","last_unstale":"2024-04-20T21:17:36.186187+0000","last_undegraded":"2024-04-20T21:17:36.186187+0000","last_fullsized":"2024-04-20T21:17:36.186187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:35.737237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878840+0000","last_change":"2024-04-20T21:16:54.571525+0000","last_active":"2024-04-20T21:17:35.878840+0000","last_peered":"2024-04-20T21:17:35.878840+0000","last_clean":"2024-04-20T21:17:35.878840+0000","last_became_active":"2024-04-20T21:16:54.570776+0000","last_became_peered":"2024-04-20T21:16:54.570776+0000","last_unstale":"2024-04-20T21:17:35.878840+0000","last_undegraded":"2024-04-20T21:17:35.878840+0000","last_fullsized":"2024-04-20T21:17:35.878840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:03.693476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883961+0000","last_change":"2024-04-20T21:16:54.570895+0000","last_active":"2024-04-20T21:17:35.883961+0000","last_peered":"2024-04-20T21:17:35.883961+0000","last_clean":"2024-04-20T21:17:35.883961+0000","last_became_active":"2024-04-20T21:16:54.570577+0000","last_became_peered":"2024-04-20T21:16:54.570577+0000","last_unstale":"2024-04-20T21:17:35.883961+0000","last_undegraded":"2024-04-20T21:17:35.883961+0000","last_fullsized":"2024-04-20T21:17:35.883961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:26.787487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878454+0000","last_change":"2024-04-20T21:16:54.581648+0000","last_active":"2024-04-20T21:17:35.878454+0000","last_peered":"2024-04-20T21:17:35.878454+0000","last_clean":"2024-04-20T21:17:35.878454+0000","last_became_active":"2024-04-20T21:16:54.581382+0000","last_became_peered":"2024-04-20T21:16:54.581382+0000","last_unstale":"2024-04-20T21:17:35.878454+0000","last_undegraded":"2024-04-20T21:17:35.878454+0000","last_fullsized":"2024-04-20T21:17:35.878454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:21.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186705+0000","last_change":"2024-04-20T21:16:54.563949+0000","last_active":"2024-04-20T21:17:36.186705+0000","last_peered":"2024-04-20T21:17:36.186705+0000","last_clean":"2024-04-20T21:17:36.186705+0000","last_became_active":"2024-04-20T21:16:54.563606+0000","last_became_peered":"2024-04-20T21:16:54.563606+0000","last_unstale":"2024-04-20T21:17:36.186705+0000","last_undegraded":"2024-04-20T21:17:36.186705+0000","last_fullsized":"2024-04-20T21:17:36.186705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:26:05.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4898,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":705311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":118203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":114451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":114487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":118461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":114361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":114521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":705367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:44.464 INFO:tasks.mds_thrash.ceph_manager:clean! 2024-04-20T21:17:44.464 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:44.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.708+0000 7fdb27800640 1 Processor -- start 2024-04-20T21:17:44.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.709+0000 7fdb27800640 1 -- start start 2024-04-20T21:17:44.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.710+0000 7fdb27800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20073da0 0x7fdb2015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.710+0000 7fdb27800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdb2006dd30 0x7fdb200ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.710+0000 7fdb27800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb2015e930 con 0x7fdb20155710 2024-04-20T21:17:44.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.710+0000 7fdb27800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fdb2015c250 con 0x7fdb2006dd30 2024-04-20T21:17:44.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.710+0000 7fdb27800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2015ebd0 con 0x7fdb20073da0 2024-04-20T21:17:44.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb24e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdb20155710 0x7fdb20071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57562/0 (socket says 172.21.15.102:57562) 2024-04-20T21:17:44.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb24e00640 1 -- 172.21.15.102:0/1607817404 learned_addr learned my addr 172.21.15.102:0/1607817404 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:44.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb1fe00640 1 --2- 172.21.15.102:0/1607817404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20073da0 0x7fdb2015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb25800640 1 --2- 172.21.15.102:0/1607817404 >> v2:172.21.15.102:3300/0 conn(0x7fdb2006dd30 0x7fdb200ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb1f400640 1 -- 172.21.15.102:0/1607817404 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510448573 0 0) 0x7fdb2015e930 con 0x7fdb20155710 2024-04-20T21:17:44.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb1fe00640 1 -- 172.21.15.102:0/1607817404 >> v2:172.21.15.102:3300/0 conn(0x7fdb2006dd30 msgr2=0x7fdb200ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:44.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb1fe00640 1 --2- 172.21.15.102:0/1607817404 >> v2:172.21.15.102:3300/0 conn(0x7fdb2006dd30 0x7fdb200ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb1fe00640 1 -- 172.21.15.102:0/1607817404 >> v1:172.21.15.102:6789/0 conn(0x7fdb20155710 legacy=0x7fdb20071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.711+0000 7fdb1fe00640 1 -- 172.21.15.102:0/1607817404 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2015eeb0 con 0x7fdb20073da0 2024-04-20T21:17:44.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.713+0000 7fdb1fe00640 1 --2- 172.21.15.102:0/1607817404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20073da0 0x7fdb2015bd10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0097a0 tx=0x7fdb0c02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=906ad81feb7d38ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:44.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.713+0000 7fdb1f400640 1 -- 172.21.15.102:0/1607817404 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c038070 con 0x7fdb20073da0 2024-04-20T21:17:44.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.713+0000 7fdb1f400640 1 -- 172.21.15.102:0/1607817404 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb0c031700 con 0x7fdb20073da0 2024-04-20T21:17:44.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.713+0000 7fdb1f400640 1 -- 172.21.15.102:0/1607817404 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c031ad0 con 0x7fdb20073da0 2024-04-20T21:17:44.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.714+0000 7fdb27800640 1 -- 172.21.15.102:0/1607817404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20073da0 msgr2=0x7fdb2015bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.714+0000 7fdb27800640 1 --2- 172.21.15.102:0/1607817404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20073da0 0x7fdb2015bd10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0097a0 tx=0x7fdb0c02b7a0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.714+0000 7fdb27800640 1 -- 172.21.15.102:0/1607817404 shutdown_connections 2024-04-20T21:17:44.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.714+0000 7fdb27800640 1 --2- 172.21.15.102:0/1607817404 >> v2:172.21.15.102:3300/0 conn(0x7fdb2006dd30 0x7fdb200ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.714+0000 7fdb27800640 1 --2- 172.21.15.102:0/1607817404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20073da0 0x7fdb2015bd10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.714+0000 7fdb27800640 1 -- 172.21.15.102:0/1607817404 >> 172.21.15.102:0/1607817404 conn(0x7fdb2006f9f0 msgr2=0x7fdb2006fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:44.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.714+0000 7fdb27800640 1 -- 172.21.15.102:0/1607817404 shutdown_connections 2024-04-20T21:17:44.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.714+0000 7fdb27800640 1 -- 172.21.15.102:0/1607817404 wait complete. 2024-04-20T21:17:44.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.715+0000 7fdb27800640 1 Processor -- start 2024-04-20T21:17:44.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.716+0000 7fdb27800640 1 -- start start 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb27800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb2006dd30 0x7fdb2015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb27800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20155710 0x7fdb2014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb27800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb2015af20 0x7fdb2014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb27800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb200725f0 con 0x7fdb2015af20 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb27800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2015f470 con 0x7fdb2006dd30 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb27800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdb20072770 con 0x7fdb20155710 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb25800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb2015af20 0x7fdb2014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb24e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb2006dd30 0x7fdb2015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb25800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb2015af20 0x7fdb2014ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48452/0 (socket says 172.21.15.102:48452) 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb25800640 1 -- 172.21.15.102:0/2676026784 learned_addr learned my addr 172.21.15.102:0/2676026784 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:44.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb1fe00640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20155710 0x7fdb2014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb25800640 1 -- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20155710 msgr2=0x7fdb2014e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb25800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20155710 0x7fdb2014e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb25800640 1 -- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb2006dd30 msgr2=0x7fdb2015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb25800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb2006dd30 0x7fdb2015d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.717+0000 7fdb25800640 1 -- 172.21.15.102:0/2676026784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb201f5170 con 0x7fdb2015af20 2024-04-20T21:17:44.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.718+0000 7fdb25800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb2015af20 0x7fdb2014ee30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800efc0 tx=0x7fdb1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:44.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.718+0000 7fdb1cc00640 1 -- 172.21.15.102:0/2676026784 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18018020 con 0x7fdb2015af20 2024-04-20T21:17:44.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.718+0000 7fdb1cc00640 1 -- 172.21.15.102:0/2676026784 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb18014480 con 0x7fdb2015af20 2024-04-20T21:17:44.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.718+0000 7fdb1cc00640 1 -- 172.21.15.102:0/2676026784 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18014850 con 0x7fdb2015af20 2024-04-20T21:17:44.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.718+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb201f45f0 con 0x7fdb2015af20 2024-04-20T21:17:44.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.718+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb201f4bb0 con 0x7fdb2015af20 2024-04-20T21:17:44.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.719+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb20068c30 con 0x7fdb2015af20 2024-04-20T21:17:44.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.720+0000 7fdb1cc00640 1 -- 172.21.15.102:0/2676026784 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb18005d10 con 0x7fdb2015af20 2024-04-20T21:17:44.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.722+0000 7fdb1cc00640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdb0006c920 0x7fdb0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:44.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.722+0000 7fdb1cc00640 1 -- 172.21.15.102:0/2676026784 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdb1808c980 con 0x7fdb2015af20 2024-04-20T21:17:44.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.722+0000 7fdb24e00640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdb0006c920 0x7fdb0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:44.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.723+0000 7fdb24e00640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdb0006c920 0x7fdb0006ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb14007fd0 tx=0x7fdb14007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:44.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.728+0000 7fdb1cc00640 1 -- 172.21.15.102:0/2676026784 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdb18022070 con 0x7fdb2015af20 2024-04-20T21:17:44.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.929+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 --> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb200c9150 con 0x7fdb0006c920 2024-04-20T21:17:44.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.938+0000 7fdb1cc00640 1 -- 172.21.15.102:0/2676026784 <== mgr.4112 v2:172.21.15.193:6832/2803621564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7fdb200c9150 con 0x7fdb0006c920 2024-04-20T21:17:44.942 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:44.944 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-20T21:17:44.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdb0006c920 msgr2=0x7fdb0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdb0006c920 0x7fdb0006ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb14007fd0 tx=0x7fdb14007de0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb2015af20 msgr2=0x7fdb2014ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb2015af20 0x7fdb2014ee30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800efc0 tx=0x7fdb1800c490 comp rx=0 tx=0).stop 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 shutdown_connections 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb2015af20 0x7fdb2014ee30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb20155710 0x7fdb2014e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdb0006c920 0x7fdb0006ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 --2- 172.21.15.102:0/2676026784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb2006dd30 0x7fdb2015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.943+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 >> 172.21.15.102:0/2676026784 conn(0x7fdb2006f9f0 msgr2=0x7fdb2006fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.944+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 shutdown_connections 2024-04-20T21:17:44.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:44.944+0000 7fdb27800640 1 -- 172.21.15.102:0/2676026784 wait complete. 2024-04-20T21:17:44.963 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-04-20T21:17:44.673986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595266,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":210,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3481600,"data_stored":2105420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.802601"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884977+0000","last_change":"2024-04-20T21:16:53.564922+0000","last_active":"2024-04-20T21:17:35.884977+0000","last_peered":"2024-04-20T21:17:35.884977+0000","last_clean":"2024-04-20T21:17:35.884977+0000","last_became_active":"2024-04-20T21:16:53.564589+0000","last_became_peered":"2024-04-20T21:16:53.564589+0000","last_unstale":"2024-04-20T21:17:35.884977+0000","last_undegraded":"2024-04-20T21:17:35.884977+0000","last_fullsized":"2024-04-20T21:17:35.884977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:47.284664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188817+0000","last_change":"2024-04-20T21:16:54.580132+0000","last_active":"2024-04-20T21:17:36.188817+0000","last_peered":"2024-04-20T21:17:36.188817+0000","last_clean":"2024-04-20T21:17:36.188817+0000","last_became_active":"2024-04-20T21:16:54.579769+0000","last_became_peered":"2024-04-20T21:16:54.579769+0000","last_unstale":"2024-04-20T21:17:36.188817+0000","last_undegraded":"2024-04-20T21:17:36.188817+0000","last_fullsized":"2024-04-20T21:17:36.188817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:00.705947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186600+0000","last_change":"2024-04-20T21:16:58.781707+0000","last_active":"2024-04-20T21:17:36.186600+0000","last_peered":"2024-04-20T21:17:36.186600+0000","last_clean":"2024-04-20T21:17:36.186600+0000","last_became_active":"2024-04-20T21:16:58.780685+0000","last_became_peered":"2024-04-20T21:16:58.780685+0000","last_unstale":"2024-04-20T21:17:36.186600+0000","last_undegraded":"2024-04-20T21:17:36.186600+0000","last_fullsized":"2024-04-20T21:17:36.186600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:50.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189794+0000","last_change":"2024-04-20T21:16:53.572137+0000","last_active":"2024-04-20T21:17:36.189794+0000","last_peered":"2024-04-20T21:17:36.189794+0000","last_clean":"2024-04-20T21:17:36.189794+0000","last_became_active":"2024-04-20T21:16:53.567907+0000","last_became_peered":"2024-04-20T21:16:53.567907+0000","last_unstale":"2024-04-20T21:17:36.189794+0000","last_undegraded":"2024-04-20T21:17:36.189794+0000","last_fullsized":"2024-04-20T21:17:36.189794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:23:28.367174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884881+0000","last_change":"2024-04-20T21:16:54.567096+0000","last_active":"2024-04-20T21:17:35.884881+0000","last_peered":"2024-04-20T21:17:35.884881+0000","last_clean":"2024-04-20T21:17:35.884881+0000","last_became_active":"2024-04-20T21:16:54.566803+0000","last_became_peered":"2024-04-20T21:16:54.566803+0000","last_unstale":"2024-04-20T21:17:35.884881+0000","last_undegraded":"2024-04-20T21:17:35.884881+0000","last_fullsized":"2024-04-20T21:17:35.884881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:13:44.730056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.191835+0000","last_change":"2024-04-20T21:17:36.191835+0000","last_active":"2024-04-20T21:17:36.191835+0000","last_peered":"2024-04-20T21:17:36.191835+0000","last_clean":"2024-04-20T21:17:36.191835+0000","last_became_active":"2024-04-20T21:17:36.191319+0000","last_became_peered":"2024-04-20T21:17:36.191319+0000","last_unstale":"2024-04-20T21:17:36.191835+0000","last_undegraded":"2024-04-20T21:17:36.191835+0000","last_fullsized":"2024-04-20T21:17:36.191835+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:29:36.899051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2,3],"acting":[4,6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186471+0000","last_change":"2024-04-20T21:16:53.554847+0000","last_active":"2024-04-20T21:17:36.186471+0000","last_peered":"2024-04-20T21:17:36.186471+0000","last_clean":"2024-04-20T21:17:36.186471+0000","last_became_active":"2024-04-20T21:16:53.554333+0000","last_became_peered":"2024-04-20T21:16:53.554333+0000","last_unstale":"2024-04-20T21:17:36.186471+0000","last_undegraded":"2024-04-20T21:17:36.186471+0000","last_fullsized":"2024-04-20T21:17:36.186471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:33.255945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878947+0000","last_change":"2024-04-20T21:16:54.556407+0000","last_active":"2024-04-20T21:17:35.878947+0000","last_peered":"2024-04-20T21:17:35.878947+0000","last_clean":"2024-04-20T21:17:35.878947+0000","last_became_active":"2024-04-20T21:16:54.556161+0000","last_became_peered":"2024-04-20T21:16:54.556161+0000","last_unstale":"2024-04-20T21:17:35.878947+0000","last_undegraded":"2024-04-20T21:17:35.878947+0000","last_fullsized":"2024-04-20T21:17:35.878947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:44:53.586714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877433+0000","last_change":"2024-04-20T21:16:58.612063+0000","last_active":"2024-04-20T21:17:35.877433+0000","last_peered":"2024-04-20T21:17:35.877433+0000","last_clean":"2024-04-20T21:17:35.877433+0000","last_became_active":"2024-04-20T21:16:58.611487+0000","last_became_peered":"2024-04-20T21:16:58.611487+0000","last_unstale":"2024-04-20T21:17:35.877433+0000","last_undegraded":"2024-04-20T21:17:35.877433+0000","last_fullsized":"2024-04-20T21:17:35.877433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:15.097731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186246+0000","last_change":"2024-04-20T21:16:53.551212+0000","last_active":"2024-04-20T21:17:36.186246+0000","last_peered":"2024-04-20T21:17:36.186246+0000","last_clean":"2024-04-20T21:17:36.186246+0000","last_became_active":"2024-04-20T21:16:53.550899+0000","last_became_peered":"2024-04-20T21:16:53.550899+0000","last_unstale":"2024-04-20T21:17:36.186246+0000","last_undegraded":"2024-04-20T21:17:36.186246+0000","last_fullsized":"2024-04-20T21:17:36.186246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:07:18.283243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878561+0000","last_change":"2024-04-20T21:16:54.557285+0000","last_active":"2024-04-20T21:17:35.878561+0000","last_peered":"2024-04-20T21:17:35.878561+0000","last_clean":"2024-04-20T21:17:35.878561+0000","last_became_active":"2024-04-20T21:16:54.556921+0000","last_became_peered":"2024-04-20T21:16:54.556921+0000","last_unstale":"2024-04-20T21:17:35.878561+0000","last_undegraded":"2024-04-20T21:17:35.878561+0000","last_fullsized":"2024-04-20T21:17:35.878561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:33.989679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877397+0000","last_change":"2024-04-20T21:16:58.617360+0000","last_active":"2024-04-20T21:17:35.877397+0000","last_peered":"2024-04-20T21:17:35.877397+0000","last_clean":"2024-04-20T21:17:35.877397+0000","last_became_active":"2024-04-20T21:16:58.616947+0000","last_became_peered":"2024-04-20T21:16:58.616947+0000","last_unstale":"2024-04-20T21:17:35.877397+0000","last_undegraded":"2024-04-20T21:17:35.877397+0000","last_fullsized":"2024-04-20T21:17:35.877397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.202955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937723+0000","last_change":"2024-04-20T21:16:54.558067+0000","last_active":"2024-04-20T21:17:35.937723+0000","last_peered":"2024-04-20T21:17:35.937723+0000","last_clean":"2024-04-20T21:17:35.937723+0000","last_became_active":"2024-04-20T21:16:54.557781+0000","last_became_peered":"2024-04-20T21:16:54.557781+0000","last_unstale":"2024-04-20T21:17:35.937723+0000","last_undegraded":"2024-04-20T21:17:35.937723+0000","last_fullsized":"2024-04-20T21:17:35.937723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:56.576081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186328+0000","last_change":"2024-04-20T21:16:53.551615+0000","last_active":"2024-04-20T21:17:36.186328+0000","last_peered":"2024-04-20T21:17:36.186328+0000","last_clean":"2024-04-20T21:17:36.186328+0000","last_became_active":"2024-04-20T21:16:53.551073+0000","last_became_peered":"2024-04-20T21:16:53.551073+0000","last_unstale":"2024-04-20T21:17:36.186328+0000","last_undegraded":"2024-04-20T21:17:36.186328+0000","last_fullsized":"2024-04-20T21:17:36.186328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:26:29.994289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877528+0000","last_change":"2024-04-20T21:16:58.617434+0000","last_active":"2024-04-20T21:17:35.877528+0000","last_peered":"2024-04-20T21:17:35.877528+0000","last_clean":"2024-04-20T21:17:35.877528+0000","last_became_active":"2024-04-20T21:16:58.617027+0000","last_became_peered":"2024-04-20T21:16:58.617027+0000","last_unstale":"2024-04-20T21:17:35.877528+0000","last_undegraded":"2024-04-20T21:17:35.877528+0000","last_fullsized":"2024-04-20T21:17:35.877528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:44:20.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188865+0000","last_change":"2024-04-20T21:16:54.585333+0000","last_active":"2024-04-20T21:17:36.188865+0000","last_peered":"2024-04-20T21:17:36.188865+0000","last_clean":"2024-04-20T21:17:36.188865+0000","last_became_active":"2024-04-20T21:16:54.584978+0000","last_became_peered":"2024-04-20T21:16:54.584978+0000","last_unstale":"2024-04-20T21:17:36.188865+0000","last_undegraded":"2024-04-20T21:17:36.188865+0000","last_fullsized":"2024-04-20T21:17:36.188865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:23.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879889+0000","last_change":"2024-04-20T21:16:53.567775+0000","last_active":"2024-04-20T21:17:35.879889+0000","last_peered":"2024-04-20T21:17:35.879889+0000","last_clean":"2024-04-20T21:17:35.879889+0000","last_became_active":"2024-04-20T21:16:53.567498+0000","last_became_peered":"2024-04-20T21:16:53.567498+0000","last_unstale":"2024-04-20T21:17:35.879889+0000","last_undegraded":"2024-04-20T21:17:35.879889+0000","last_fullsized":"2024-04-20T21:17:35.879889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:10.695659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878180+0000","last_change":"2024-04-20T21:16:58.619302+0000","last_active":"2024-04-20T21:17:35.878180+0000","last_peered":"2024-04-20T21:17:35.878180+0000","last_clean":"2024-04-20T21:17:35.878180+0000","last_became_active":"2024-04-20T21:16:58.618843+0000","last_became_peered":"2024-04-20T21:16:58.618843+0000","last_unstale":"2024-04-20T21:17:35.878180+0000","last_undegraded":"2024-04-20T21:17:35.878180+0000","last_fullsized":"2024-04-20T21:17:35.878180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:14.927199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188147+0000","last_change":"2024-04-20T21:16:54.575862+0000","last_active":"2024-04-20T21:17:36.188147+0000","last_peered":"2024-04-20T21:17:36.188147+0000","last_clean":"2024-04-20T21:17:36.188147+0000","last_became_active":"2024-04-20T21:16:54.575523+0000","last_became_peered":"2024-04-20T21:16:54.575523+0000","last_unstale":"2024-04-20T21:17:36.188147+0000","last_undegraded":"2024-04-20T21:17:36.188147+0000","last_fullsized":"2024-04-20T21:17:36.188147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:51.041414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880494+0000","last_change":"2024-04-20T21:16:53.551995+0000","last_active":"2024-04-20T21:17:35.880494+0000","last_peered":"2024-04-20T21:17:35.880494+0000","last_clean":"2024-04-20T21:17:35.880494+0000","last_became_active":"2024-04-20T21:16:53.551603+0000","last_became_peered":"2024-04-20T21:16:53.551603+0000","last_unstale":"2024-04-20T21:17:35.880494+0000","last_undegraded":"2024-04-20T21:17:35.880494+0000","last_fullsized":"2024-04-20T21:17:35.880494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:56:21.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878943+0000","last_change":"2024-04-20T21:16:58.776463+0000","last_active":"2024-04-20T21:17:35.878943+0000","last_peered":"2024-04-20T21:17:35.878943+0000","last_clean":"2024-04-20T21:17:35.878943+0000","last_became_active":"2024-04-20T21:16:58.775746+0000","last_became_peered":"2024-04-20T21:16:58.775746+0000","last_unstale":"2024-04-20T21:17:35.878943+0000","last_undegraded":"2024-04-20T21:17:35.878943+0000","last_fullsized":"2024-04-20T21:17:35.878943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:28.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878811+0000","last_change":"2024-04-20T21:16:54.566449+0000","last_active":"2024-04-20T21:17:35.878811+0000","last_peered":"2024-04-20T21:17:35.878811+0000","last_clean":"2024-04-20T21:17:35.878811+0000","last_became_active":"2024-04-20T21:16:54.566251+0000","last_became_peered":"2024-04-20T21:16:54.566251+0000","last_unstale":"2024-04-20T21:17:35.878811+0000","last_undegraded":"2024-04-20T21:17:35.878811+0000","last_fullsized":"2024-04-20T21:17:35.878811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:12:53.403420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185741+0000","last_change":"2024-04-20T21:16:53.564029+0000","last_active":"2024-04-20T21:17:36.185741+0000","last_peered":"2024-04-20T21:17:36.185741+0000","last_clean":"2024-04-20T21:17:36.185741+0000","last_became_active":"2024-04-20T21:16:53.563711+0000","last_became_peered":"2024-04-20T21:16:53.563711+0000","last_unstale":"2024-04-20T21:17:36.185741+0000","last_undegraded":"2024-04-20T21:17:36.185741+0000","last_fullsized":"2024-04-20T21:17:36.185741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.634644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878711+0000","last_change":"2024-04-20T21:16:58.615080+0000","last_active":"2024-04-20T21:17:35.878711+0000","last_peered":"2024-04-20T21:17:35.878711+0000","last_clean":"2024-04-20T21:17:35.878711+0000","last_became_active":"2024-04-20T21:16:58.614611+0000","last_became_peered":"2024-04-20T21:16:58.614611+0000","last_unstale":"2024-04-20T21:17:35.878711+0000","last_undegraded":"2024-04-20T21:17:35.878711+0000","last_fullsized":"2024-04-20T21:17:35.878711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:08.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884557+0000","last_change":"2024-04-20T21:16:54.578272+0000","last_active":"2024-04-20T21:17:35.884557+0000","last_peered":"2024-04-20T21:17:35.884557+0000","last_clean":"2024-04-20T21:17:35.884557+0000","last_became_active":"2024-04-20T21:16:54.577949+0000","last_became_peered":"2024-04-20T21:16:54.577949+0000","last_unstale":"2024-04-20T21:17:35.884557+0000","last_undegraded":"2024-04-20T21:17:35.884557+0000","last_fullsized":"2024-04-20T21:17:35.884557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:35:33.807091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938557+0000","last_change":"2024-04-20T21:16:53.554600+0000","last_active":"2024-04-20T21:17:35.938557+0000","last_peered":"2024-04-20T21:17:35.938557+0000","last_clean":"2024-04-20T21:17:35.938557+0000","last_became_active":"2024-04-20T21:16:53.554347+0000","last_became_peered":"2024-04-20T21:16:53.554347+0000","last_unstale":"2024-04-20T21:17:35.938557+0000","last_undegraded":"2024-04-20T21:17:35.938557+0000","last_fullsized":"2024-04-20T21:17:35.938557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:40:58.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878403+0000","last_change":"2024-04-20T21:16:58.774474+0000","last_active":"2024-04-20T21:17:35.878403+0000","last_peered":"2024-04-20T21:17:35.878403+0000","last_clean":"2024-04-20T21:17:35.878403+0000","last_became_active":"2024-04-20T21:16:58.774037+0000","last_became_peered":"2024-04-20T21:16:58.774037+0000","last_unstale":"2024-04-20T21:17:35.878403+0000","last_undegraded":"2024-04-20T21:17:35.878403+0000","last_fullsized":"2024-04-20T21:17:35.878403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:46.992667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189865+0000","last_change":"2024-04-20T21:17:36.187769+0000","last_active":"2024-04-20T21:17:36.189865+0000","last_peered":"2024-04-20T21:17:36.189865+0000","last_clean":"2024-04-20T21:17:36.189865+0000","last_became_active":"2024-04-20T21:17:36.187283+0000","last_became_peered":"2024-04-20T21:17:36.187283+0000","last_unstale":"2024-04-20T21:17:36.189865+0000","last_undegraded":"2024-04-20T21:17:36.189865+0000","last_fullsized":"2024-04-20T21:17:36.189865+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:46:58.418395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878574+0000","last_change":"2024-04-20T21:16:53.561144+0000","last_active":"2024-04-20T21:17:35.878574+0000","last_peered":"2024-04-20T21:17:35.878574+0000","last_clean":"2024-04-20T21:17:35.878574+0000","last_became_active":"2024-04-20T21:16:53.560742+0000","last_became_peered":"2024-04-20T21:16:53.560742+0000","last_unstale":"2024-04-20T21:17:35.878574+0000","last_undegraded":"2024-04-20T21:17:35.878574+0000","last_fullsized":"2024-04-20T21:17:35.878574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:39.212951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186836+0000","last_change":"2024-04-20T21:16:58.780124+0000","last_active":"2024-04-20T21:17:36.186836+0000","last_peered":"2024-04-20T21:17:36.186836+0000","last_clean":"2024-04-20T21:17:36.186836+0000","last_became_active":"2024-04-20T21:16:58.777449+0000","last_became_peered":"2024-04-20T21:16:58.777449+0000","last_unstale":"2024-04-20T21:17:36.186836+0000","last_undegraded":"2024-04-20T21:17:36.186836+0000","last_fullsized":"2024-04-20T21:17:36.186836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:07.622146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879405+0000","last_change":"2024-04-20T21:16:54.575331+0000","last_active":"2024-04-20T21:17:35.879405+0000","last_peered":"2024-04-20T21:17:35.879405+0000","last_clean":"2024-04-20T21:17:35.879405+0000","last_became_active":"2024-04-20T21:16:54.575061+0000","last_became_peered":"2024-04-20T21:16:54.575061+0000","last_unstale":"2024-04-20T21:17:35.879405+0000","last_undegraded":"2024-04-20T21:17:35.879405+0000","last_fullsized":"2024-04-20T21:17:35.879405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:47:20.957165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186611+0000","last_change":"2024-04-20T21:16:53.565561+0000","last_active":"2024-04-20T21:17:36.186611+0000","last_peered":"2024-04-20T21:17:36.186611+0000","last_clean":"2024-04-20T21:17:36.186611+0000","last_became_active":"2024-04-20T21:16:53.565207+0000","last_became_peered":"2024-04-20T21:16:53.565207+0000","last_unstale":"2024-04-20T21:17:36.186611+0000","last_undegraded":"2024-04-20T21:17:36.186611+0000","last_fullsized":"2024-04-20T21:17:36.186611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:07.512132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188224+0000","last_change":"2024-04-20T21:17:36.188224+0000","last_active":"2024-04-20T21:17:36.188224+0000","last_peered":"2024-04-20T21:17:36.188224+0000","last_clean":"2024-04-20T21:17:36.188224+0000","last_became_active":"2024-04-20T21:17:36.187615+0000","last_became_peered":"2024-04-20T21:17:36.187615+0000","last_unstale":"2024-04-20T21:17:36.188224+0000","last_undegraded":"2024-04-20T21:17:36.188224+0000","last_fullsized":"2024-04-20T21:17:36.188224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:45:48.216716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,2],"acting":[0,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878608+0000","last_change":"2024-04-20T21:16:54.571199+0000","last_active":"2024-04-20T21:17:35.878608+0000","last_peered":"2024-04-20T21:17:35.878608+0000","last_clean":"2024-04-20T21:17:35.878608+0000","last_became_active":"2024-04-20T21:16:54.570882+0000","last_became_peered":"2024-04-20T21:16:54.570882+0000","last_unstale":"2024-04-20T21:17:35.878608+0000","last_undegraded":"2024-04-20T21:17:35.878608+0000","last_fullsized":"2024-04-20T21:17:35.878608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:52.046176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879661+0000","last_change":"2024-04-20T21:16:53.566253+0000","last_active":"2024-04-20T21:17:35.879661+0000","last_peered":"2024-04-20T21:17:35.879661+0000","last_clean":"2024-04-20T21:17:35.879661+0000","last_became_active":"2024-04-20T21:16:53.565893+0000","last_became_peered":"2024-04-20T21:16:53.565893+0000","last_unstale":"2024-04-20T21:17:35.879661+0000","last_undegraded":"2024-04-20T21:17:35.879661+0000","last_fullsized":"2024-04-20T21:17:35.879661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:16.302801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879509+0000","last_change":"2024-04-20T21:16:58.611470+0000","last_active":"2024-04-20T21:17:35.879509+0000","last_peered":"2024-04-20T21:17:35.879509+0000","last_clean":"2024-04-20T21:17:35.879509+0000","last_became_active":"2024-04-20T21:16:58.611038+0000","last_became_peered":"2024-04-20T21:16:58.611038+0000","last_unstale":"2024-04-20T21:17:35.879509+0000","last_undegraded":"2024-04-20T21:17:35.879509+0000","last_fullsized":"2024-04-20T21:17:35.879509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:58:51.267480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877868+0000","last_change":"2024-04-20T21:16:54.577608+0000","last_active":"2024-04-20T21:17:35.877868+0000","last_peered":"2024-04-20T21:17:35.877868+0000","last_clean":"2024-04-20T21:17:35.877868+0000","last_became_active":"2024-04-20T21:16:54.577339+0000","last_became_peered":"2024-04-20T21:16:54.577339+0000","last_unstale":"2024-04-20T21:17:35.877868+0000","last_undegraded":"2024-04-20T21:17:35.877868+0000","last_fullsized":"2024-04-20T21:17:35.877868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:03.028585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879518+0000","last_change":"2024-04-20T21:16:53.547751+0000","last_active":"2024-04-20T21:17:35.879518+0000","last_peered":"2024-04-20T21:17:35.879518+0000","last_clean":"2024-04-20T21:17:35.879518+0000","last_became_active":"2024-04-20T21:16:53.547371+0000","last_became_peered":"2024-04-20T21:16:53.547371+0000","last_unstale":"2024-04-20T21:17:35.879518+0000","last_undegraded":"2024-04-20T21:17:35.879518+0000","last_fullsized":"2024-04-20T21:17:35.879518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:50.728500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878920+0000","last_change":"2024-04-20T21:16:58.622369+0000","last_active":"2024-04-20T21:17:35.878920+0000","last_peered":"2024-04-20T21:17:35.878920+0000","last_clean":"2024-04-20T21:17:35.878920+0000","last_became_active":"2024-04-20T21:16:58.621911+0000","last_became_peered":"2024-04-20T21:16:58.621911+0000","last_unstale":"2024-04-20T21:17:35.878920+0000","last_undegraded":"2024-04-20T21:17:35.878920+0000","last_fullsized":"2024-04-20T21:17:35.878920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:05:41.519583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878299+0000","last_change":"2024-04-20T21:16:54.579577+0000","last_active":"2024-04-20T21:17:35.878299+0000","last_peered":"2024-04-20T21:17:35.878299+0000","last_clean":"2024-04-20T21:17:35.878299+0000","last_became_active":"2024-04-20T21:16:54.579156+0000","last_became_peered":"2024-04-20T21:16:54.579156+0000","last_unstale":"2024-04-20T21:17:35.878299+0000","last_undegraded":"2024-04-20T21:17:35.878299+0000","last_fullsized":"2024-04-20T21:17:35.878299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:28:14.743667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.886735+0000","last_change":"2024-04-20T21:16:53.567995+0000","last_active":"2024-04-20T21:17:35.886735+0000","last_peered":"2024-04-20T21:17:35.886735+0000","last_clean":"2024-04-20T21:17:35.886735+0000","last_became_active":"2024-04-20T21:16:53.567601+0000","last_became_peered":"2024-04-20T21:16:53.567601+0000","last_unstale":"2024-04-20T21:17:35.886735+0000","last_undegraded":"2024-04-20T21:17:35.886735+0000","last_fullsized":"2024-04-20T21:17:35.886735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:30.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879045+0000","last_change":"2024-04-20T21:16:58.626438+0000","last_active":"2024-04-20T21:17:35.879045+0000","last_peered":"2024-04-20T21:17:35.879045+0000","last_clean":"2024-04-20T21:17:35.879045+0000","last_became_active":"2024-04-20T21:16:58.625326+0000","last_became_peered":"2024-04-20T21:16:58.625326+0000","last_unstale":"2024-04-20T21:17:35.879045+0000","last_undegraded":"2024-04-20T21:17:35.879045+0000","last_fullsized":"2024-04-20T21:17:35.879045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:06.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883279+0000","last_change":"2024-04-20T21:16:54.560925+0000","last_active":"2024-04-20T21:17:35.883279+0000","last_peered":"2024-04-20T21:17:35.883279+0000","last_clean":"2024-04-20T21:17:35.883279+0000","last_became_active":"2024-04-20T21:16:54.560615+0000","last_became_peered":"2024-04-20T21:16:54.560615+0000","last_unstale":"2024-04-20T21:17:35.883279+0000","last_undegraded":"2024-04-20T21:17:35.883279+0000","last_fullsized":"2024-04-20T21:17:35.883279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:12.815692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185774+0000","last_change":"2024-04-20T21:16:53.559359+0000","last_active":"2024-04-20T21:17:36.185774+0000","last_peered":"2024-04-20T21:17:36.185774+0000","last_clean":"2024-04-20T21:17:36.185774+0000","last_became_active":"2024-04-20T21:16:53.558955+0000","last_became_peered":"2024-04-20T21:16:53.558955+0000","last_unstale":"2024-04-20T21:17:36.185774+0000","last_undegraded":"2024-04-20T21:17:36.185774+0000","last_fullsized":"2024-04-20T21:17:36.185774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:28.664404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884324+0000","last_change":"2024-04-20T21:16:58.772175+0000","last_active":"2024-04-20T21:17:35.884324+0000","last_peered":"2024-04-20T21:17:35.884324+0000","last_clean":"2024-04-20T21:17:35.884324+0000","last_became_active":"2024-04-20T21:16:58.771732+0000","last_became_peered":"2024-04-20T21:16:58.771732+0000","last_unstale":"2024-04-20T21:17:35.884324+0000","last_undegraded":"2024-04-20T21:17:35.884324+0000","last_fullsized":"2024-04-20T21:17:35.884324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:13.155547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186877+0000","last_change":"2024-04-20T21:16:54.575375+0000","last_active":"2024-04-20T21:17:36.186877+0000","last_peered":"2024-04-20T21:17:36.186877+0000","last_clean":"2024-04-20T21:17:36.186877+0000","last_became_active":"2024-04-20T21:16:54.574194+0000","last_became_peered":"2024-04-20T21:16:54.574194+0000","last_unstale":"2024-04-20T21:17:36.186877+0000","last_undegraded":"2024-04-20T21:17:36.186877+0000","last_fullsized":"2024-04-20T21:17:36.186877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:07.948431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878007+0000","last_change":"2024-04-20T21:16:53.559374+0000","last_active":"2024-04-20T21:17:35.878007+0000","last_peered":"2024-04-20T21:17:35.878007+0000","last_clean":"2024-04-20T21:17:35.878007+0000","last_became_active":"2024-04-20T21:16:53.559133+0000","last_became_peered":"2024-04-20T21:16:53.559133+0000","last_unstale":"2024-04-20T21:17:35.878007+0000","last_undegraded":"2024-04-20T21:17:35.878007+0000","last_fullsized":"2024-04-20T21:17:35.878007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:38.576103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.185908+0000","last_change":"2024-04-20T21:16:44.100529+0000","last_active":"2024-04-20T21:17:36.185908+0000","last_peered":"2024-04-20T21:17:36.185908+0000","last_clean":"2024-04-20T21:17:36.185908+0000","last_became_active":"2024-04-20T21:16:44.099590+0000","last_became_peered":"2024-04-20T21:16:44.099590+0000","last_unstale":"2024-04-20T21:17:36.185908+0000","last_undegraded":"2024-04-20T21:17:36.185908+0000","last_fullsized":"2024-04-20T21:17:36.185908+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:42.782844+0000","last_clean_scrub_stamp":"2024-04-20T21:16:42.782844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:08.054837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937912+0000","last_change":"2024-04-20T21:16:58.772894+0000","last_active":"2024-04-20T21:17:35.937912+0000","last_peered":"2024-04-20T21:17:35.937912+0000","last_clean":"2024-04-20T21:17:35.937912+0000","last_became_active":"2024-04-20T21:16:58.772451+0000","last_became_peered":"2024-04-20T21:16:58.772451+0000","last_unstale":"2024-04-20T21:17:35.937912+0000","last_undegraded":"2024-04-20T21:17:35.937912+0000","last_fullsized":"2024-04-20T21:17:35.937912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:29.914096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186355+0000","last_change":"2024-04-20T21:16:54.562592+0000","last_active":"2024-04-20T21:17:36.186355+0000","last_peered":"2024-04-20T21:17:36.186355+0000","last_clean":"2024-04-20T21:17:36.186355+0000","last_became_active":"2024-04-20T21:16:54.562243+0000","last_became_peered":"2024-04-20T21:16:54.562243+0000","last_unstale":"2024-04-20T21:17:36.186355+0000","last_undegraded":"2024-04-20T21:17:36.186355+0000","last_fullsized":"2024-04-20T21:17:36.186355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:52.832159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887300+0000","last_change":"2024-04-20T21:16:53.561132+0000","last_active":"2024-04-20T21:17:35.887300+0000","last_peered":"2024-04-20T21:17:35.887300+0000","last_clean":"2024-04-20T21:17:35.887300+0000","last_became_active":"2024-04-20T21:16:53.559901+0000","last_became_peered":"2024-04-20T21:16:53.559901+0000","last_unstale":"2024-04-20T21:17:35.887300+0000","last_undegraded":"2024-04-20T21:17:35.887300+0000","last_fullsized":"2024-04-20T21:17:35.887300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:20.229573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937705+0000","last_change":"2024-04-20T21:16:58.771204+0000","last_active":"2024-04-20T21:17:35.937705+0000","last_peered":"2024-04-20T21:17:35.937705+0000","last_clean":"2024-04-20T21:17:35.937705+0000","last_became_active":"2024-04-20T21:16:58.770627+0000","last_became_peered":"2024-04-20T21:16:58.770627+0000","last_unstale":"2024-04-20T21:17:35.937705+0000","last_undegraded":"2024-04-20T21:17:35.937705+0000","last_fullsized":"2024-04-20T21:17:35.937705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:39.472931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.194007+0000","last_change":"2024-04-20T21:17:36.192480+0000","last_active":"2024-04-20T21:17:36.194007+0000","last_peered":"2024-04-20T21:17:36.194007+0000","last_clean":"2024-04-20T21:17:36.194007+0000","last_became_active":"2024-04-20T21:17:36.190832+0000","last_became_peered":"2024-04-20T21:17:36.190832+0000","last_unstale":"2024-04-20T21:17:36.194007+0000","last_undegraded":"2024-04-20T21:17:36.194007+0000","last_fullsized":"2024-04-20T21:17:36.194007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:41.520453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.881260+0000","last_change":"2024-04-20T21:16:53.558554+0000","last_active":"2024-04-20T21:17:35.881260+0000","last_peered":"2024-04-20T21:17:35.881260+0000","last_clean":"2024-04-20T21:17:35.881260+0000","last_became_active":"2024-04-20T21:16:53.556934+0000","last_became_peered":"2024-04-20T21:16:53.556934+0000","last_unstale":"2024-04-20T21:17:35.881260+0000","last_undegraded":"2024-04-20T21:17:35.881260+0000","last_fullsized":"2024-04-20T21:17:35.881260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:17:52.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884183+0000","last_change":"2024-04-20T21:16:58.624410+0000","last_active":"2024-04-20T21:17:35.884183+0000","last_peered":"2024-04-20T21:17:35.884183+0000","last_clean":"2024-04-20T21:17:35.884183+0000","last_became_active":"2024-04-20T21:16:58.623677+0000","last_became_peered":"2024-04-20T21:16:58.623677+0000","last_unstale":"2024-04-20T21:17:35.884183+0000","last_undegraded":"2024-04-20T21:17:35.884183+0000","last_fullsized":"2024-04-20T21:17:35.884183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:34.465594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186797+0000","last_change":"2024-04-20T21:16:54.560217+0000","last_active":"2024-04-20T21:17:36.186797+0000","last_peered":"2024-04-20T21:17:36.186797+0000","last_clean":"2024-04-20T21:17:36.186797+0000","last_became_active":"2024-04-20T21:16:54.559947+0000","last_became_peered":"2024-04-20T21:16:54.559947+0000","last_unstale":"2024-04-20T21:17:36.186797+0000","last_undegraded":"2024-04-20T21:17:36.186797+0000","last_fullsized":"2024-04-20T21:17:36.186797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:54.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887513+0000","last_change":"2024-04-20T21:16:53.564779+0000","last_active":"2024-04-20T21:17:35.887513+0000","last_peered":"2024-04-20T21:17:35.887513+0000","last_clean":"2024-04-20T21:17:35.887513+0000","last_became_active":"2024-04-20T21:16:53.564500+0000","last_became_peered":"2024-04-20T21:16:53.564500+0000","last_unstale":"2024-04-20T21:17:35.887513+0000","last_undegraded":"2024-04-20T21:17:35.887513+0000","last_fullsized":"2024-04-20T21:17:35.887513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:00:58.673145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877151+0000","last_change":"2024-04-20T21:16:58.771524+0000","last_active":"2024-04-20T21:17:35.877151+0000","last_peered":"2024-04-20T21:17:35.877151+0000","last_clean":"2024-04-20T21:17:35.877151+0000","last_became_active":"2024-04-20T21:16:58.770672+0000","last_became_peered":"2024-04-20T21:16:58.770672+0000","last_unstale":"2024-04-20T21:17:35.877151+0000","last_undegraded":"2024-04-20T21:17:35.877151+0000","last_fullsized":"2024-04-20T21:17:35.877151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:52:27.037256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186604+0000","last_change":"2024-04-20T21:16:54.559570+0000","last_active":"2024-04-20T21:17:36.186604+0000","last_peered":"2024-04-20T21:17:36.186604+0000","last_clean":"2024-04-20T21:17:36.186604+0000","last_became_active":"2024-04-20T21:16:54.559307+0000","last_became_peered":"2024-04-20T21:16:54.559307+0000","last_unstale":"2024-04-20T21:17:36.186604+0000","last_undegraded":"2024-04-20T21:17:36.186604+0000","last_fullsized":"2024-04-20T21:17:36.186604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:04.417348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879042+0000","last_change":"2024-04-20T21:16:53.565289+0000","last_active":"2024-04-20T21:17:35.879042+0000","last_peered":"2024-04-20T21:17:35.879042+0000","last_clean":"2024-04-20T21:17:35.879042+0000","last_became_active":"2024-04-20T21:16:53.565014+0000","last_became_peered":"2024-04-20T21:16:53.565014+0000","last_unstale":"2024-04-20T21:17:35.879042+0000","last_undegraded":"2024-04-20T21:17:35.879042+0000","last_fullsized":"2024-04-20T21:17:35.879042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:22.125301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878310+0000","last_change":"2024-04-20T21:16:58.615103+0000","last_active":"2024-04-20T21:17:35.878310+0000","last_peered":"2024-04-20T21:17:35.878310+0000","last_clean":"2024-04-20T21:17:35.878310+0000","last_became_active":"2024-04-20T21:16:58.614068+0000","last_became_peered":"2024-04-20T21:16:58.614068+0000","last_unstale":"2024-04-20T21:17:35.878310+0000","last_undegraded":"2024-04-20T21:17:35.878310+0000","last_fullsized":"2024-04-20T21:17:35.878310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:09.555911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.896249+0000","last_change":"2024-04-20T21:17:35.891448+0000","last_active":"2024-04-20T21:17:35.896249+0000","last_peered":"2024-04-20T21:17:35.896249+0000","last_clean":"2024-04-20T21:17:35.896249+0000","last_became_active":"2024-04-20T21:17:35.891149+0000","last_became_peered":"2024-04-20T21:17:35.891149+0000","last_unstale":"2024-04-20T21:17:35.896249+0000","last_undegraded":"2024-04-20T21:17:35.896249+0000","last_fullsized":"2024-04-20T21:17:35.896249+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:24:05.580789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.885970+0000","last_change":"2024-04-20T21:16:53.570182+0000","last_active":"2024-04-20T21:17:35.885970+0000","last_peered":"2024-04-20T21:17:35.885970+0000","last_clean":"2024-04-20T21:17:35.885970+0000","last_became_active":"2024-04-20T21:16:53.569632+0000","last_became_peered":"2024-04-20T21:16:53.569632+0000","last_unstale":"2024-04-20T21:17:35.885970+0000","last_undegraded":"2024-04-20T21:17:35.885970+0000","last_fullsized":"2024-04-20T21:17:35.885970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:12.140497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884216+0000","last_change":"2024-04-20T21:16:58.625602+0000","last_active":"2024-04-20T21:17:35.884216+0000","last_peered":"2024-04-20T21:17:35.884216+0000","last_clean":"2024-04-20T21:17:35.884216+0000","last_became_active":"2024-04-20T21:16:58.625158+0000","last_became_peered":"2024-04-20T21:16:58.625158+0000","last_unstale":"2024-04-20T21:17:35.884216+0000","last_undegraded":"2024-04-20T21:17:35.884216+0000","last_fullsized":"2024-04-20T21:17:35.884216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:09.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938129+0000","last_change":"2024-04-20T21:16:54.578603+0000","last_active":"2024-04-20T21:17:35.938129+0000","last_peered":"2024-04-20T21:17:35.938129+0000","last_clean":"2024-04-20T21:17:35.938129+0000","last_became_active":"2024-04-20T21:16:54.578338+0000","last_became_peered":"2024-04-20T21:16:54.578338+0000","last_unstale":"2024-04-20T21:17:35.938129+0000","last_undegraded":"2024-04-20T21:17:35.938129+0000","last_fullsized":"2024-04-20T21:17:35.938129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:14:56.355343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937825+0000","last_change":"2024-04-20T21:16:53.551847+0000","last_active":"2024-04-20T21:17:35.937825+0000","last_peered":"2024-04-20T21:17:35.937825+0000","last_clean":"2024-04-20T21:17:35.937825+0000","last_became_active":"2024-04-20T21:16:53.551427+0000","last_became_peered":"2024-04-20T21:16:53.551427+0000","last_unstale":"2024-04-20T21:17:35.937825+0000","last_undegraded":"2024-04-20T21:17:35.937825+0000","last_fullsized":"2024-04-20T21:17:35.937825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:55:45.930305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189251+0000","last_change":"2024-04-20T21:16:58.777159+0000","last_active":"2024-04-20T21:17:36.189251+0000","last_peered":"2024-04-20T21:17:36.189251+0000","last_clean":"2024-04-20T21:17:36.189251+0000","last_became_active":"2024-04-20T21:16:58.776233+0000","last_became_peered":"2024-04-20T21:16:58.776233+0000","last_unstale":"2024-04-20T21:17:36.189251+0000","last_undegraded":"2024-04-20T21:17:36.189251+0000","last_fullsized":"2024-04-20T21:17:36.189251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:05.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186236+0000","last_change":"2024-04-20T21:16:54.570687+0000","last_active":"2024-04-20T21:17:36.186236+0000","last_peered":"2024-04-20T21:17:36.186236+0000","last_clean":"2024-04-20T21:17:36.186236+0000","last_became_active":"2024-04-20T21:16:54.570360+0000","last_became_peered":"2024-04-20T21:16:54.570360+0000","last_unstale":"2024-04-20T21:17:36.186236+0000","last_undegraded":"2024-04-20T21:17:36.186236+0000","last_fullsized":"2024-04-20T21:17:36.186236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:05.606229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879415+0000","last_change":"2024-04-20T21:16:53.567599+0000","last_active":"2024-04-20T21:17:35.879415+0000","last_peered":"2024-04-20T21:17:35.879415+0000","last_clean":"2024-04-20T21:17:35.879415+0000","last_became_active":"2024-04-20T21:16:53.567378+0000","last_became_peered":"2024-04-20T21:16:53.567378+0000","last_unstale":"2024-04-20T21:17:35.879415+0000","last_undegraded":"2024-04-20T21:17:35.879415+0000","last_fullsized":"2024-04-20T21:17:35.879415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:36.611887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188405+0000","last_change":"2024-04-20T21:16:58.774226+0000","last_active":"2024-04-20T21:17:36.188405+0000","last_peered":"2024-04-20T21:17:36.188405+0000","last_clean":"2024-04-20T21:17:36.188405+0000","last_became_active":"2024-04-20T21:16:58.771584+0000","last_became_peered":"2024-04-20T21:16:58.771584+0000","last_unstale":"2024-04-20T21:17:36.188405+0000","last_undegraded":"2024-04-20T21:17:36.188405+0000","last_fullsized":"2024-04-20T21:17:36.188405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:31.432077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883628+0000","last_change":"2024-04-20T21:16:54.560001+0000","last_active":"2024-04-20T21:17:35.883628+0000","last_peered":"2024-04-20T21:17:35.883628+0000","last_clean":"2024-04-20T21:17:35.883628+0000","last_became_active":"2024-04-20T21:16:54.559643+0000","last_became_peered":"2024-04-20T21:16:54.559643+0000","last_unstale":"2024-04-20T21:17:35.883628+0000","last_undegraded":"2024-04-20T21:17:35.883628+0000","last_fullsized":"2024-04-20T21:17:35.883628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:52:37.325946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884673+0000","last_change":"2024-04-20T21:16:53.570498+0000","last_active":"2024-04-20T21:17:35.884673+0000","last_peered":"2024-04-20T21:17:35.884673+0000","last_clean":"2024-04-20T21:17:35.884673+0000","last_became_active":"2024-04-20T21:16:53.570095+0000","last_became_peered":"2024-04-20T21:16:53.570095+0000","last_unstale":"2024-04-20T21:17:35.884673+0000","last_undegraded":"2024-04-20T21:17:35.884673+0000","last_fullsized":"2024-04-20T21:17:35.884673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:58:28.677814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878493+0000","last_change":"2024-04-20T21:16:58.773349+0000","last_active":"2024-04-20T21:17:35.878493+0000","last_peered":"2024-04-20T21:17:35.878493+0000","last_clean":"2024-04-20T21:17:35.878493+0000","last_became_active":"2024-04-20T21:16:58.772781+0000","last_became_peered":"2024-04-20T21:16:58.772781+0000","last_unstale":"2024-04-20T21:17:35.878493+0000","last_undegraded":"2024-04-20T21:17:35.878493+0000","last_fullsized":"2024-04-20T21:17:35.878493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:53:02.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186216+0000","last_change":"2024-04-20T21:16:54.572749+0000","last_active":"2024-04-20T21:17:36.186216+0000","last_peered":"2024-04-20T21:17:36.186216+0000","last_clean":"2024-04-20T21:17:36.186216+0000","last_became_active":"2024-04-20T21:16:54.572456+0000","last_became_peered":"2024-04-20T21:16:54.572456+0000","last_unstale":"2024-04-20T21:17:36.186216+0000","last_undegraded":"2024-04-20T21:17:36.186216+0000","last_fullsized":"2024-04-20T21:17:36.186216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:06.787612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'12","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.998099+0000","last_change":"2024-04-20T21:16:53.568688+0000","last_active":"2024-04-20T21:17:36.998099+0000","last_peered":"2024-04-20T21:17:36.998099+0000","last_clean":"2024-04-20T21:17:36.998099+0000","last_became_active":"2024-04-20T21:16:53.568495+0000","last_became_peered":"2024-04-20T21:16:53.568495+0000","last_unstale":"2024-04-20T21:17:36.998099+0000","last_undegraded":"2024-04-20T21:17:36.998099+0000","last_fullsized":"2024-04-20T21:17:36.998099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:00.086004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4100,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879709+0000","last_change":"2024-04-20T21:16:58.609575+0000","last_active":"2024-04-20T21:17:35.879709+0000","last_peered":"2024-04-20T21:17:35.879709+0000","last_clean":"2024-04-20T21:17:35.879709+0000","last_became_active":"2024-04-20T21:16:58.609127+0000","last_became_peered":"2024-04-20T21:16:58.609127+0000","last_unstale":"2024-04-20T21:17:35.879709+0000","last_undegraded":"2024-04-20T21:17:35.879709+0000","last_fullsized":"2024-04-20T21:17:35.879709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:02.645274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878544+0000","last_change":"2024-04-20T21:16:54.563213+0000","last_active":"2024-04-20T21:17:35.878544+0000","last_peered":"2024-04-20T21:17:35.878544+0000","last_clean":"2024-04-20T21:17:35.878544+0000","last_became_active":"2024-04-20T21:16:54.562917+0000","last_became_peered":"2024-04-20T21:16:54.562917+0000","last_unstale":"2024-04-20T21:17:35.878544+0000","last_undegraded":"2024-04-20T21:17:35.878544+0000","last_fullsized":"2024-04-20T21:17:35.878544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.155771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880903+0000","last_change":"2024-04-20T21:16:53.556988+0000","last_active":"2024-04-20T21:17:35.880903+0000","last_peered":"2024-04-20T21:17:35.880903+0000","last_clean":"2024-04-20T21:17:35.880903+0000","last_became_active":"2024-04-20T21:16:53.556672+0000","last_became_peered":"2024-04-20T21:16:53.556672+0000","last_unstale":"2024-04-20T21:17:35.880903+0000","last_undegraded":"2024-04-20T21:17:35.880903+0000","last_fullsized":"2024-04-20T21:17:35.880903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:25.219611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.939270+0000","last_change":"2024-04-20T21:17:35.939270+0000","last_active":"2024-04-20T21:17:35.939270+0000","last_peered":"2024-04-20T21:17:35.939270+0000","last_clean":"2024-04-20T21:17:35.939270+0000","last_became_active":"2024-04-20T21:17:35.938833+0000","last_became_peered":"2024-04-20T21:17:35.938833+0000","last_unstale":"2024-04-20T21:17:35.939270+0000","last_undegraded":"2024-04-20T21:17:35.939270+0000","last_fullsized":"2024-04-20T21:17:35.939270+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.790995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884930+0000","last_change":"2024-04-20T21:16:54.562128+0000","last_active":"2024-04-20T21:17:35.884930+0000","last_peered":"2024-04-20T21:17:35.884930+0000","last_clean":"2024-04-20T21:17:35.884930+0000","last_became_active":"2024-04-20T21:16:54.561495+0000","last_became_peered":"2024-04-20T21:16:54.561495+0000","last_unstale":"2024-04-20T21:17:35.884930+0000","last_undegraded":"2024-04-20T21:17:35.884930+0000","last_fullsized":"2024-04-20T21:17:35.884930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:19.582218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878813+0000","last_change":"2024-04-20T21:16:53.554881+0000","last_active":"2024-04-20T21:17:35.878813+0000","last_peered":"2024-04-20T21:17:35.878813+0000","last_clean":"2024-04-20T21:17:35.878813+0000","last_became_active":"2024-04-20T21:16:53.554590+0000","last_became_peered":"2024-04-20T21:16:53.554590+0000","last_unstale":"2024-04-20T21:17:35.878813+0000","last_undegraded":"2024-04-20T21:17:35.878813+0000","last_fullsized":"2024-04-20T21:17:35.878813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:36.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.187143+0000","last_change":"2024-04-20T21:16:58.610704+0000","last_active":"2024-04-20T21:17:36.187143+0000","last_peered":"2024-04-20T21:17:36.187143+0000","last_clean":"2024-04-20T21:17:36.187143+0000","last_became_active":"2024-04-20T21:16:58.610309+0000","last_became_peered":"2024-04-20T21:16:58.610309+0000","last_unstale":"2024-04-20T21:17:36.187143+0000","last_undegraded":"2024-04-20T21:17:36.187143+0000","last_fullsized":"2024-04-20T21:17:36.187143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:04.215050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878430+0000","last_change":"2024-04-20T21:16:54.562351+0000","last_active":"2024-04-20T21:17:35.878430+0000","last_peered":"2024-04-20T21:17:35.878430+0000","last_clean":"2024-04-20T21:17:35.878430+0000","last_became_active":"2024-04-20T21:16:54.561947+0000","last_became_peered":"2024-04-20T21:16:54.561947+0000","last_unstale":"2024-04-20T21:17:35.878430+0000","last_undegraded":"2024-04-20T21:17:35.878430+0000","last_fullsized":"2024-04-20T21:17:35.878430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:52.802328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938343+0000","last_change":"2024-04-20T21:16:53.554081+0000","last_active":"2024-04-20T21:17:35.938343+0000","last_peered":"2024-04-20T21:17:35.938343+0000","last_clean":"2024-04-20T21:17:35.938343+0000","last_became_active":"2024-04-20T21:16:53.553800+0000","last_became_peered":"2024-04-20T21:16:53.553800+0000","last_unstale":"2024-04-20T21:17:35.938343+0000","last_undegraded":"2024-04-20T21:17:35.938343+0000","last_fullsized":"2024-04-20T21:17:35.938343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:36:52.865485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186668+0000","last_change":"2024-04-20T21:16:58.779184+0000","last_active":"2024-04-20T21:17:36.186668+0000","last_peered":"2024-04-20T21:17:36.186668+0000","last_clean":"2024-04-20T21:17:36.186668+0000","last_became_active":"2024-04-20T21:16:58.778627+0000","last_became_peered":"2024-04-20T21:16:58.778627+0000","last_unstale":"2024-04-20T21:17:36.186668+0000","last_undegraded":"2024-04-20T21:17:36.186668+0000","last_fullsized":"2024-04-20T21:17:36.186668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:50:07.247736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884103+0000","last_change":"2024-04-20T21:16:58.774438+0000","last_active":"2024-04-20T21:17:35.884103+0000","last_peered":"2024-04-20T21:17:35.884103+0000","last_clean":"2024-04-20T21:17:35.884103+0000","last_became_active":"2024-04-20T21:16:58.774030+0000","last_became_peered":"2024-04-20T21:16:58.774030+0000","last_unstale":"2024-04-20T21:17:35.884103+0000","last_undegraded":"2024-04-20T21:17:35.884103+0000","last_fullsized":"2024-04-20T21:17:35.884103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:09:22.583239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880247+0000","last_change":"2024-04-20T21:16:54.573779+0000","last_active":"2024-04-20T21:17:35.880247+0000","last_peered":"2024-04-20T21:17:35.880247+0000","last_clean":"2024-04-20T21:17:35.880247+0000","last_became_active":"2024-04-20T21:16:54.573444+0000","last_became_peered":"2024-04-20T21:16:54.573444+0000","last_unstale":"2024-04-20T21:17:35.880247+0000","last_undegraded":"2024-04-20T21:17:35.880247+0000","last_fullsized":"2024-04-20T21:17:35.880247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:27.950525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877644+0000","last_change":"2024-04-20T21:16:53.543678+0000","last_active":"2024-04-20T21:17:35.877644+0000","last_peered":"2024-04-20T21:17:35.877644+0000","last_clean":"2024-04-20T21:17:35.877644+0000","last_became_active":"2024-04-20T21:16:53.543381+0000","last_became_peered":"2024-04-20T21:16:53.543381+0000","last_unstale":"2024-04-20T21:17:35.877644+0000","last_undegraded":"2024-04-20T21:17:35.877644+0000","last_fullsized":"2024-04-20T21:17:35.877644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:30:02.259536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938285+0000","last_change":"2024-04-20T21:16:58.626882+0000","last_active":"2024-04-20T21:17:35.938285+0000","last_peered":"2024-04-20T21:17:35.938285+0000","last_clean":"2024-04-20T21:17:35.938285+0000","last_became_active":"2024-04-20T21:16:58.626489+0000","last_became_peered":"2024-04-20T21:16:58.626489+0000","last_unstale":"2024-04-20T21:17:35.938285+0000","last_undegraded":"2024-04-20T21:17:35.938285+0000","last_fullsized":"2024-04-20T21:17:35.938285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:33:22.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.187276+0000","last_change":"2024-04-20T21:16:54.580893+0000","last_active":"2024-04-20T21:17:36.187276+0000","last_peered":"2024-04-20T21:17:36.187276+0000","last_clean":"2024-04-20T21:17:36.187276+0000","last_became_active":"2024-04-20T21:16:54.580528+0000","last_became_peered":"2024-04-20T21:16:54.580528+0000","last_unstale":"2024-04-20T21:17:36.187276+0000","last_undegraded":"2024-04-20T21:17:36.187276+0000","last_fullsized":"2024-04-20T21:17:36.187276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:14.699123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186916+0000","last_change":"2024-04-20T21:16:53.563930+0000","last_active":"2024-04-20T21:17:36.186916+0000","last_peered":"2024-04-20T21:17:36.186916+0000","last_clean":"2024-04-20T21:17:36.186916+0000","last_became_active":"2024-04-20T21:16:53.559991+0000","last_became_peered":"2024-04-20T21:16:53.559991+0000","last_unstale":"2024-04-20T21:17:36.186916+0000","last_undegraded":"2024-04-20T21:17:36.186916+0000","last_fullsized":"2024-04-20T21:17:36.186916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:22.529712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937884+0000","last_change":"2024-04-20T21:16:53.552509+0000","last_active":"2024-04-20T21:17:35.937884+0000","last_peered":"2024-04-20T21:17:35.937884+0000","last_clean":"2024-04-20T21:17:35.937884+0000","last_became_active":"2024-04-20T21:16:53.552173+0000","last_became_peered":"2024-04-20T21:16:53.552173+0000","last_unstale":"2024-04-20T21:17:35.937884+0000","last_undegraded":"2024-04-20T21:17:35.937884+0000","last_fullsized":"2024-04-20T21:17:35.937884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:14.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884531+0000","last_change":"2024-04-20T21:16:54.562849+0000","last_active":"2024-04-20T21:17:35.884531+0000","last_peered":"2024-04-20T21:17:35.884531+0000","last_clean":"2024-04-20T21:17:35.884531+0000","last_became_active":"2024-04-20T21:16:54.561283+0000","last_became_peered":"2024-04-20T21:16:54.561283+0000","last_unstale":"2024-04-20T21:17:35.884531+0000","last_undegraded":"2024-04-20T21:17:35.884531+0000","last_fullsized":"2024-04-20T21:17:35.884531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:52.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887173+0000","last_change":"2024-04-20T21:16:58.614752+0000","last_active":"2024-04-20T21:17:35.887173+0000","last_peered":"2024-04-20T21:17:35.887173+0000","last_clean":"2024-04-20T21:17:35.887173+0000","last_became_active":"2024-04-20T21:16:58.614201+0000","last_became_peered":"2024-04-20T21:16:58.614201+0000","last_unstale":"2024-04-20T21:17:35.887173+0000","last_undegraded":"2024-04-20T21:17:35.887173+0000","last_fullsized":"2024-04-20T21:17:35.887173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:46.676469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188988+0000","last_change":"2024-04-20T21:16:53.564303+0000","last_active":"2024-04-20T21:17:36.188988+0000","last_peered":"2024-04-20T21:17:36.188988+0000","last_clean":"2024-04-20T21:17:36.188988+0000","last_became_active":"2024-04-20T21:16:53.560203+0000","last_became_peered":"2024-04-20T21:16:53.560203+0000","last_unstale":"2024-04-20T21:17:36.188988+0000","last_undegraded":"2024-04-20T21:17:36.188988+0000","last_fullsized":"2024-04-20T21:17:36.188988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:52.505002+0000","last_clean_scrub_stamp":"2024-04-20T21:16:52.505002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:42.691233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878072+0000","last_change":"2024-04-20T21:16:54.554499+0000","last_active":"2024-04-20T21:17:35.878072+0000","last_peered":"2024-04-20T21:17:35.878072+0000","last_clean":"2024-04-20T21:17:35.878072+0000","last_became_active":"2024-04-20T21:16:54.554230+0000","last_became_peered":"2024-04-20T21:16:54.554230+0000","last_unstale":"2024-04-20T21:17:35.878072+0000","last_undegraded":"2024-04-20T21:17:35.878072+0000","last_fullsized":"2024-04-20T21:17:35.878072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:24:31.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188251+0000","last_change":"2024-04-20T21:16:58.610424+0000","last_active":"2024-04-20T21:17:36.188251+0000","last_peered":"2024-04-20T21:17:36.188251+0000","last_clean":"2024-04-20T21:17:36.188251+0000","last_became_active":"2024-04-20T21:16:58.610025+0000","last_became_peered":"2024-04-20T21:16:58.610025+0000","last_unstale":"2024-04-20T21:17:36.188251+0000","last_undegraded":"2024-04-20T21:17:36.188251+0000","last_fullsized":"2024-04-20T21:17:36.188251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:57.562190+0000","last_clean_scrub_stamp":"2024-04-20T21:16:57.562190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:19.290865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937939+0000","last_change":"2024-04-20T21:16:54.578801+0000","last_active":"2024-04-20T21:17:35.937939+0000","last_peered":"2024-04-20T21:17:35.937939+0000","last_clean":"2024-04-20T21:17:35.937939+0000","last_became_active":"2024-04-20T21:16:54.578543+0000","last_became_peered":"2024-04-20T21:16:54.578543+0000","last_unstale":"2024-04-20T21:17:35.937939+0000","last_undegraded":"2024-04-20T21:17:35.937939+0000","last_fullsized":"2024-04-20T21:17:35.937939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:15.042318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.886717+0000","last_change":"2024-04-20T21:16:54.565658+0000","last_active":"2024-04-20T21:17:35.886717+0000","last_peered":"2024-04-20T21:17:35.886717+0000","last_clean":"2024-04-20T21:17:35.886717+0000","last_became_active":"2024-04-20T21:16:54.565371+0000","last_became_peered":"2024-04-20T21:16:54.565371+0000","last_unstale":"2024-04-20T21:17:35.886717+0000","last_undegraded":"2024-04-20T21:17:35.886717+0000","last_fullsized":"2024-04-20T21:17:35.886717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:21.781013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879317+0000","last_change":"2024-04-20T21:16:54.568472+0000","last_active":"2024-04-20T21:17:35.879317+0000","last_peered":"2024-04-20T21:17:35.879317+0000","last_clean":"2024-04-20T21:17:35.879317+0000","last_became_active":"2024-04-20T21:16:54.568223+0000","last_became_peered":"2024-04-20T21:16:54.568223+0000","last_unstale":"2024-04-20T21:17:35.879317+0000","last_undegraded":"2024-04-20T21:17:35.879317+0000","last_fullsized":"2024-04-20T21:17:35.879317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:42:53.921935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884279+0000","last_change":"2024-04-20T21:16:54.573747+0000","last_active":"2024-04-20T21:17:35.884279+0000","last_peered":"2024-04-20T21:17:35.884279+0000","last_clean":"2024-04-20T21:17:35.884279+0000","last_became_active":"2024-04-20T21:16:54.573476+0000","last_became_peered":"2024-04-20T21:16:54.573476+0000","last_unstale":"2024-04-20T21:17:35.884279+0000","last_undegraded":"2024-04-20T21:17:35.884279+0000","last_fullsized":"2024-04-20T21:17:35.884279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:02.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188768+0000","last_change":"2024-04-20T21:16:54.577630+0000","last_active":"2024-04-20T21:17:36.188768+0000","last_peered":"2024-04-20T21:17:36.188768+0000","last_clean":"2024-04-20T21:17:36.188768+0000","last_became_active":"2024-04-20T21:16:54.577318+0000","last_became_peered":"2024-04-20T21:16:54.577318+0000","last_unstale":"2024-04-20T21:17:36.188768+0000","last_undegraded":"2024-04-20T21:17:36.188768+0000","last_fullsized":"2024-04-20T21:17:36.188768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:45.584050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879003+0000","last_change":"2024-04-20T21:16:54.579850+0000","last_active":"2024-04-20T21:17:35.879003+0000","last_peered":"2024-04-20T21:17:35.879003+0000","last_clean":"2024-04-20T21:17:35.879003+0000","last_became_active":"2024-04-20T21:16:54.579546+0000","last_became_peered":"2024-04-20T21:16:54.579546+0000","last_unstale":"2024-04-20T21:17:35.879003+0000","last_undegraded":"2024-04-20T21:17:35.879003+0000","last_fullsized":"2024-04-20T21:17:35.879003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:49:53.577077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938400+0000","last_change":"2024-04-20T21:16:54.559174+0000","last_active":"2024-04-20T21:17:35.938400+0000","last_peered":"2024-04-20T21:17:35.938400+0000","last_clean":"2024-04-20T21:17:35.938400+0000","last_became_active":"2024-04-20T21:16:54.558904+0000","last_became_peered":"2024-04-20T21:16:54.558904+0000","last_unstale":"2024-04-20T21:17:35.938400+0000","last_undegraded":"2024-04-20T21:17:35.938400+0000","last_fullsized":"2024-04-20T21:17:35.938400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:09.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.189372+0000","last_change":"2024-04-20T21:16:54.574943+0000","last_active":"2024-04-20T21:17:36.189372+0000","last_peered":"2024-04-20T21:17:36.189372+0000","last_clean":"2024-04-20T21:17:36.189372+0000","last_became_active":"2024-04-20T21:16:54.574471+0000","last_became_peered":"2024-04-20T21:16:54.574471+0000","last_unstale":"2024-04-20T21:17:36.189372+0000","last_undegraded":"2024-04-20T21:17:36.189372+0000","last_fullsized":"2024-04-20T21:17:36.189372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:51:44.741339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.193375+0000","last_change":"2024-04-20T21:17:36.191229+0000","last_active":"2024-04-20T21:17:36.193375+0000","last_peered":"2024-04-20T21:17:36.193375+0000","last_clean":"2024-04-20T21:17:36.193375+0000","last_became_active":"2024-04-20T21:17:36.190776+0000","last_became_peered":"2024-04-20T21:17:36.190776+0000","last_unstale":"2024-04-20T21:17:36.193375+0000","last_undegraded":"2024-04-20T21:17:36.193375+0000","last_fullsized":"2024-04-20T21:17:36.193375+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:49:11.938154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186859+0000","last_change":"2024-04-20T21:16:54.571179+0000","last_active":"2024-04-20T21:17:36.186859+0000","last_peered":"2024-04-20T21:17:36.186859+0000","last_clean":"2024-04-20T21:17:36.186859+0000","last_became_active":"2024-04-20T21:16:54.570855+0000","last_became_peered":"2024-04-20T21:16:54.570855+0000","last_unstale":"2024-04-20T21:17:36.186859+0000","last_undegraded":"2024-04-20T21:17:36.186859+0000","last_fullsized":"2024-04-20T21:17:36.186859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:59.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879202+0000","last_change":"2024-04-20T21:16:54.559092+0000","last_active":"2024-04-20T21:17:35.879202+0000","last_peered":"2024-04-20T21:17:35.879202+0000","last_clean":"2024-04-20T21:17:35.879202+0000","last_became_active":"2024-04-20T21:16:54.558847+0000","last_became_peered":"2024-04-20T21:16:54.558847+0000","last_unstale":"2024-04-20T21:17:35.879202+0000","last_undegraded":"2024-04-20T21:17:35.879202+0000","last_fullsized":"2024-04-20T21:17:35.879202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:02.752070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.884080+0000","last_change":"2024-04-20T21:16:54.571815+0000","last_active":"2024-04-20T21:17:35.884080+0000","last_peered":"2024-04-20T21:17:35.884080+0000","last_clean":"2024-04-20T21:17:35.884080+0000","last_became_active":"2024-04-20T21:16:54.570704+0000","last_became_peered":"2024-04-20T21:16:54.570704+0000","last_unstale":"2024-04-20T21:17:35.884080+0000","last_undegraded":"2024-04-20T21:17:35.884080+0000","last_fullsized":"2024-04-20T21:17:35.884080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:56.511539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938000+0000","last_change":"2024-04-20T21:16:54.555591+0000","last_active":"2024-04-20T21:17:35.938000+0000","last_peered":"2024-04-20T21:17:35.938000+0000","last_clean":"2024-04-20T21:17:35.938000+0000","last_became_active":"2024-04-20T21:16:54.555389+0000","last_became_peered":"2024-04-20T21:16:54.555389+0000","last_unstale":"2024-04-20T21:17:35.938000+0000","last_undegraded":"2024-04-20T21:17:35.938000+0000","last_fullsized":"2024-04-20T21:17:35.938000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:16:25.976707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186528+0000","last_change":"2024-04-20T21:16:54.564088+0000","last_active":"2024-04-20T21:17:36.186528+0000","last_peered":"2024-04-20T21:17:36.186528+0000","last_clean":"2024-04-20T21:17:36.186528+0000","last_became_active":"2024-04-20T21:16:54.563796+0000","last_became_peered":"2024-04-20T21:16:54.563796+0000","last_unstale":"2024-04-20T21:17:36.186528+0000","last_undegraded":"2024-04-20T21:17:36.186528+0000","last_fullsized":"2024-04-20T21:17:36.186528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:07.986736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.887460+0000","last_change":"2024-04-20T21:16:54.563578+0000","last_active":"2024-04-20T21:17:35.887460+0000","last_peered":"2024-04-20T21:17:35.887460+0000","last_clean":"2024-04-20T21:17:35.887460+0000","last_became_active":"2024-04-20T21:16:54.563317+0000","last_became_peered":"2024-04-20T21:16:54.563317+0000","last_unstale":"2024-04-20T21:17:35.887460+0000","last_undegraded":"2024-04-20T21:17:35.887460+0000","last_fullsized":"2024-04-20T21:17:35.887460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:06.093675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879169+0000","last_change":"2024-04-20T21:16:54.565168+0000","last_active":"2024-04-20T21:17:35.879169+0000","last_peered":"2024-04-20T21:17:35.879169+0000","last_clean":"2024-04-20T21:17:35.879169+0000","last_became_active":"2024-04-20T21:16:54.564712+0000","last_became_peered":"2024-04-20T21:16:54.564712+0000","last_unstale":"2024-04-20T21:17:35.879169+0000","last_undegraded":"2024-04-20T21:17:35.879169+0000","last_fullsized":"2024-04-20T21:17:35.879169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:07:30.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.877246+0000","last_change":"2024-04-20T21:16:54.572211+0000","last_active":"2024-04-20T21:17:35.877246+0000","last_peered":"2024-04-20T21:17:35.877246+0000","last_clean":"2024-04-20T21:17:35.877246+0000","last_became_active":"2024-04-20T21:16:54.571894+0000","last_became_peered":"2024-04-20T21:16:54.571894+0000","last_unstale":"2024-04-20T21:17:35.877246+0000","last_undegraded":"2024-04-20T21:17:35.877246+0000","last_fullsized":"2024-04-20T21:17:35.877246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:59.376514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.880303+0000","last_change":"2024-04-20T21:16:54.555195+0000","last_active":"2024-04-20T21:17:35.880303+0000","last_peered":"2024-04-20T21:17:35.880303+0000","last_clean":"2024-04-20T21:17:35.880303+0000","last_became_active":"2024-04-20T21:16:54.554284+0000","last_became_peered":"2024-04-20T21:16:54.554284+0000","last_unstale":"2024-04-20T21:17:35.880303+0000","last_undegraded":"2024-04-20T21:17:35.880303+0000","last_fullsized":"2024-04-20T21:17:35.880303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:42:21.500952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883524+0000","last_change":"2024-04-20T21:16:54.565527+0000","last_active":"2024-04-20T21:17:35.883524+0000","last_peered":"2024-04-20T21:17:35.883524+0000","last_clean":"2024-04-20T21:17:35.883524+0000","last_became_active":"2024-04-20T21:16:54.565211+0000","last_became_peered":"2024-04-20T21:16:54.565211+0000","last_unstale":"2024-04-20T21:17:35.883524+0000","last_undegraded":"2024-04-20T21:17:35.883524+0000","last_fullsized":"2024-04-20T21:17:35.883524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:17:40.462670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.188717+0000","last_change":"2024-04-20T21:16:54.576464+0000","last_active":"2024-04-20T21:17:36.188717+0000","last_peered":"2024-04-20T21:17:36.188717+0000","last_clean":"2024-04-20T21:17:36.188717+0000","last_became_active":"2024-04-20T21:16:54.575947+0000","last_became_peered":"2024-04-20T21:16:54.575947+0000","last_unstale":"2024-04-20T21:17:36.188717+0000","last_undegraded":"2024-04-20T21:17:36.188717+0000","last_fullsized":"2024-04-20T21:17:36.188717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:50:29.581938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878458+0000","last_change":"2024-04-20T21:16:54.558162+0000","last_active":"2024-04-20T21:17:35.878458+0000","last_peered":"2024-04-20T21:17:35.878458+0000","last_clean":"2024-04-20T21:17:35.878458+0000","last_became_active":"2024-04-20T21:16:54.557875+0000","last_became_peered":"2024-04-20T21:16:54.557875+0000","last_unstale":"2024-04-20T21:17:35.878458+0000","last_undegraded":"2024-04-20T21:17:35.878458+0000","last_fullsized":"2024-04-20T21:17:35.878458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:42:46.886405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.937978+0000","last_change":"2024-04-20T21:16:54.560566+0000","last_active":"2024-04-20T21:17:35.937978+0000","last_peered":"2024-04-20T21:17:35.937978+0000","last_clean":"2024-04-20T21:17:35.937978+0000","last_became_active":"2024-04-20T21:16:54.559409+0000","last_became_peered":"2024-04-20T21:16:54.559409+0000","last_unstale":"2024-04-20T21:17:35.937978+0000","last_undegraded":"2024-04-20T21:17:35.937978+0000","last_fullsized":"2024-04-20T21:17:35.937978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:00.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.938279+0000","last_change":"2024-04-20T21:16:54.559491+0000","last_active":"2024-04-20T21:17:35.938279+0000","last_peered":"2024-04-20T21:17:35.938279+0000","last_clean":"2024-04-20T21:17:35.938279+0000","last_became_active":"2024-04-20T21:16:54.558187+0000","last_became_peered":"2024-04-20T21:16:54.558187+0000","last_unstale":"2024-04-20T21:17:35.938279+0000","last_undegraded":"2024-04-20T21:17:35.938279+0000","last_fullsized":"2024-04-20T21:17:35.938279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:33.686248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186775+0000","last_change":"2024-04-20T21:16:54.564258+0000","last_active":"2024-04-20T21:17:36.186775+0000","last_peered":"2024-04-20T21:17:36.186775+0000","last_clean":"2024-04-20T21:17:36.186775+0000","last_became_active":"2024-04-20T21:16:54.563966+0000","last_became_peered":"2024-04-20T21:16:54.563966+0000","last_unstale":"2024-04-20T21:17:36.186775+0000","last_undegraded":"2024-04-20T21:17:36.186775+0000","last_fullsized":"2024-04-20T21:17:36.186775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:13:34.741451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186994+0000","last_change":"2024-04-20T21:16:54.573554+0000","last_active":"2024-04-20T21:17:36.186994+0000","last_peered":"2024-04-20T21:17:36.186994+0000","last_clean":"2024-04-20T21:17:36.186994+0000","last_became_active":"2024-04-20T21:16:54.571849+0000","last_became_peered":"2024-04-20T21:16:54.571849+0000","last_unstale":"2024-04-20T21:17:36.186994+0000","last_undegraded":"2024-04-20T21:17:36.186994+0000","last_fullsized":"2024-04-20T21:17:36.186994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:09.481779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186052+0000","last_change":"2024-04-20T21:16:54.571471+0000","last_active":"2024-04-20T21:17:36.186052+0000","last_peered":"2024-04-20T21:17:36.186052+0000","last_clean":"2024-04-20T21:17:36.186052+0000","last_became_active":"2024-04-20T21:16:54.571209+0000","last_became_peered":"2024-04-20T21:16:54.571209+0000","last_unstale":"2024-04-20T21:17:36.186052+0000","last_undegraded":"2024-04-20T21:17:36.186052+0000","last_fullsized":"2024-04-20T21:17:36.186052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:35.458542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.879415+0000","last_change":"2024-04-20T21:16:54.572168+0000","last_active":"2024-04-20T21:17:35.879415+0000","last_peered":"2024-04-20T21:17:35.879415+0000","last_clean":"2024-04-20T21:17:35.879415+0000","last_became_active":"2024-04-20T21:16:54.571740+0000","last_became_peered":"2024-04-20T21:16:54.571740+0000","last_unstale":"2024-04-20T21:17:35.879415+0000","last_undegraded":"2024-04-20T21:17:35.879415+0000","last_fullsized":"2024-04-20T21:17:35.879415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:11:35.728824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186187+0000","last_change":"2024-04-20T21:16:54.569081+0000","last_active":"2024-04-20T21:17:36.186187+0000","last_peered":"2024-04-20T21:17:36.186187+0000","last_clean":"2024-04-20T21:17:36.186187+0000","last_became_active":"2024-04-20T21:16:54.568847+0000","last_became_peered":"2024-04-20T21:16:54.568847+0000","last_unstale":"2024-04-20T21:17:36.186187+0000","last_undegraded":"2024-04-20T21:17:36.186187+0000","last_fullsized":"2024-04-20T21:17:36.186187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:35.737237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878840+0000","last_change":"2024-04-20T21:16:54.571525+0000","last_active":"2024-04-20T21:17:35.878840+0000","last_peered":"2024-04-20T21:17:35.878840+0000","last_clean":"2024-04-20T21:17:35.878840+0000","last_became_active":"2024-04-20T21:16:54.570776+0000","last_became_peered":"2024-04-20T21:16:54.570776+0000","last_unstale":"2024-04-20T21:17:35.878840+0000","last_undegraded":"2024-04-20T21:17:35.878840+0000","last_fullsized":"2024-04-20T21:17:35.878840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:03.693476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.883961+0000","last_change":"2024-04-20T21:16:54.570895+0000","last_active":"2024-04-20T21:17:35.883961+0000","last_peered":"2024-04-20T21:17:35.883961+0000","last_clean":"2024-04-20T21:17:35.883961+0000","last_became_active":"2024-04-20T21:16:54.570577+0000","last_became_peered":"2024-04-20T21:16:54.570577+0000","last_unstale":"2024-04-20T21:17:35.883961+0000","last_undegraded":"2024-04-20T21:17:35.883961+0000","last_fullsized":"2024-04-20T21:17:35.883961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:26.787487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:35.878454+0000","last_change":"2024-04-20T21:16:54.581648+0000","last_active":"2024-04-20T21:17:35.878454+0000","last_peered":"2024-04-20T21:17:35.878454+0000","last_clean":"2024-04-20T21:17:35.878454+0000","last_became_active":"2024-04-20T21:16:54.581382+0000","last_became_peered":"2024-04-20T21:16:54.581382+0000","last_unstale":"2024-04-20T21:17:35.878454+0000","last_undegraded":"2024-04-20T21:17:35.878454+0000","last_fullsized":"2024-04-20T21:17:35.878454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:40:21.857719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T21:17:36.186705+0000","last_change":"2024-04-20T21:16:54.563949+0000","last_active":"2024-04-20T21:17:36.186705+0000","last_peered":"2024-04-20T21:17:36.186705+0000","last_clean":"2024-04-20T21:17:36.186705+0000","last_became_active":"2024-04-20T21:16:54.563606+0000","last_became_peered":"2024-04-20T21:16:54.563606+0000","last_unstale":"2024-04-20T21:17:36.186705+0000","last_undegraded":"2024-04-20T21:17:36.186705+0000","last_fullsized":"2024-04-20T21:17:36.186705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:16:53.520608+0000","last_clean_scrub_stamp":"2024-04-20T21:16:53.520608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:26:05.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4898,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":705311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":118461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":114451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":114487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":118461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":114361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":114521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":705367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:44.968 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:45.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.206+0000 7f9fc0400640 1 Processor -- start 2024-04-20T21:17:45.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.206+0000 7f9fc0400640 1 -- start start 2024-04-20T21:17:45.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.208+0000 7f9fc0400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9fb815bd10 0x7f9fb80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.208+0000 7f9fc0400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb815c150 con 0x7f9fb8075710 2024-04-20T21:17:45.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.208+0000 7f9fc0400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb80d0b30 con 0x7f9fb8155690 2024-04-20T21:17:45.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.208+0000 7f9fc0400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb80cc2f0 con 0x7f9fb815bd10 2024-04-20T21:17:45.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.208+0000 7f9fbda00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f9fb8155690 0x7f9fb80cb3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38184/0 (socket says 172.21.15.102:38184) 2024-04-20T21:17:45.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.208+0000 7f9fbda00640 1 -- 172.21.15.102:0/888594895 learned_addr learned my addr 172.21.15.102:0/888594895 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:45.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.208+0000 7f9fbe400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9fb8075710 0x7f9fb80cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57576/0 (socket says 172.21.15.102:57576) 2024-04-20T21:17:45.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fbd000640 1 --2- 172.21.15.102:0/888594895 >> v2:172.21.15.102:3300/0 conn(0x7f9fb815bd10 0x7f9fb80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fb7e00640 1 -- 172.21.15.102:0/888594895 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3299473997 0 0) 0x7f9fb80d0b30 con 0x7f9fb8155690 2024-04-20T21:17:45.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fb7e00640 1 -- 172.21.15.102:0/888594895 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f9c003520 con 0x7f9fb8155690 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fb7e00640 1 -- 172.21.15.102:0/888594895 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 614412082 0 0) 0x7f9fb0001490 con 0x7f9fb8075710 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fb7e00640 1 -- 172.21.15.102:0/888594895 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fb80d0b30 con 0x7f9fb8075710 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fbd000640 1 -- 172.21.15.102:0/888594895 >> v1:172.21.15.193:6789/0 conn(0x7f9fb8155690 legacy=0x7f9fb80cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fb7e00640 1 -- 172.21.15.102:0/888594895 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2471075712 0 0) 0x7f9f9c003520 con 0x7f9fb8155690 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fbd000640 1 -- 172.21.15.102:0/888594895 >> v1:172.21.15.102:6789/0 conn(0x7f9fb8075710 legacy=0x7f9fb80cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.209+0000 7f9fbd000640 1 -- 172.21.15.102:0/888594895 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb80d0e10 con 0x7f9fb815bd10 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.210+0000 7f9fbd000640 1 --2- 172.21.15.102:0/888594895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb815bd10 0x7f9fb80751d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009720 tx=0x7f9fa402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20729a60064af60 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fb7e00640 1 -- 172.21.15.102:0/888594895 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8001670 con 0x7f9fb815bd10 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fb7e00640 1 -- 172.21.15.102:0/888594895 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa4037070 con 0x7f9fb815bd10 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fb7e00640 1 -- 172.21.15.102:0/888594895 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40317c0 con 0x7f9fb815bd10 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fc0400640 1 -- 172.21.15.102:0/888594895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb815bd10 msgr2=0x7f9fb80751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fc0400640 1 --2- 172.21.15.102:0/888594895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb815bd10 0x7f9fb80751d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009720 tx=0x7f9fa402b860 comp rx=0 tx=0).stop 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fc0400640 1 -- 172.21.15.102:0/888594895 shutdown_connections 2024-04-20T21:17:45.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fc0400640 1 --2- 172.21.15.102:0/888594895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb815bd10 0x7f9fb80751d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fc0400640 1 -- 172.21.15.102:0/888594895 >> 172.21.15.102:0/888594895 conn(0x7f9fb80685f0 msgr2=0x7f9fb806ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:45.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fc0400640 1 -- 172.21.15.102:0/888594895 shutdown_connections 2024-04-20T21:17:45.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.211+0000 7f9fc0400640 1 -- 172.21.15.102:0/888594895 wait complete. 2024-04-20T21:17:45.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.213+0000 7f9fc0400640 1 Processor -- start 2024-04-20T21:17:45.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.213+0000 7f9fc0400640 1 -- start start 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fc0400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb8075710 0x7f9fb81f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fc0400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb8155690 0x7f9fb814b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fc0400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb815bd10 0x7f9fb814b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fc0400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80d16b0 con 0x7f9fb8075710 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fc0400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80d13d0 con 0x7f9fb8155690 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fc0400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb80cce90 con 0x7f9fb815bd10 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fbda00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb8075710 0x7f9fb81f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fbda00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb8075710 0x7f9fb81f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48470/0 (socket says 172.21.15.102:48470) 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fbda00640 1 -- 172.21.15.102:0/390265216 learned_addr learned my addr 172.21.15.102:0/390265216 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:45.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fbe400640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb815bd10 0x7f9fb814b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.214+0000 7f9fbd000640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb8155690 0x7f9fb814b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fbda00640 1 -- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb815bd10 msgr2=0x7f9fb814b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fbda00640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb815bd10 0x7f9fb814b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fbda00640 1 -- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb8155690 msgr2=0x7f9fb814b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fbda00640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb8155690 0x7f9fb814b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fbda00640 1 -- 172.21.15.102:0/390265216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb81f4f70 con 0x7f9fb8075710 2024-04-20T21:17:45.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fbda00640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb8075710 0x7f9fb81f45b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800c750 tx=0x7f9fa800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:45.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fb5600640 1 -- 172.21.15.102:0/390265216 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8018070 con 0x7f9fb8075710 2024-04-20T21:17:45.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fb5600640 1 -- 172.21.15.102:0/390265216 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa80072d0 con 0x7f9fb8075710 2024-04-20T21:17:45.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb80b4ee0 con 0x7f9fb8075710 2024-04-20T21:17:45.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fb5600640 1 -- 172.21.15.102:0/390265216 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80047e0 con 0x7f9fb8075710 2024-04-20T21:17:45.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.215+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb80b4790 con 0x7f9fb8075710 2024-04-20T21:17:45.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.217+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb8154a10 con 0x7f9fb8075710 2024-04-20T21:17:45.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.217+0000 7f9fb5600640 1 -- 172.21.15.102:0/390265216 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fa801b070 con 0x7f9fb8075710 2024-04-20T21:17:45.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.218+0000 7f9fb5600640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f9806c8f0 0x7f9f9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.218+0000 7f9fb5600640 1 -- 172.21.15.102:0/390265216 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9fa808b460 con 0x7f9fb8075710 2024-04-20T21:17:45.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.219+0000 7f9fbd000640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f9806c8f0 0x7f9f9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.219+0000 7f9fbd000640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f9806c8f0 0x7f9f9806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4036000 tx=0x7f9fa4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:45.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.223+0000 7f9fb5600640 1 -- 172.21.15.102:0/390265216 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9fb8154a10 con 0x7f9fb8075710 2024-04-20T21:17:45.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.423+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fb8159020 con 0x7f9fb8075710 2024-04-20T21:17:45.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.426+0000 7f9fb5600640 1 -- 172.21.15.102:0/390265216 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18894 (secure 0 0 0) 0x7f9fa8058da0 con 0x7f9fb8075710 2024-04-20T21:17:45.426 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:45.427 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:17:35.869503+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:45.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.429+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f9806c8f0 msgr2=0x7f9f9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.429+0000 7f9fc0400640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f9806c8f0 0x7f9f9806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4036000 tx=0x7f9fa4008030 comp rx=0 tx=0).stop 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.429+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb8075710 msgr2=0x7f9fb81f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.429+0000 7f9fc0400640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb8075710 0x7f9fb81f45b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800c750 tx=0x7f9fa800cc10 comp rx=0 tx=0).stop 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.430+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 shutdown_connections 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.430+0000 7f9fc0400640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb815bd10 0x7f9fb814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.430+0000 7f9fc0400640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f9806c8f0 0x7f9f9806edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.430+0000 7f9fc0400640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb8155690 0x7f9fb814b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.430+0000 7f9fc0400640 1 --2- 172.21.15.102:0/390265216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fb8075710 0x7f9fb81f45b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.430+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 >> 172.21.15.102:0/390265216 conn(0x7f9fb80685f0 msgr2=0x7f9fb8073300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.430+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 shutdown_connections 2024-04-20T21:17:45.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.430+0000 7f9fc0400640 1 -- 172.21.15.102:0/390265216 wait complete. 2024-04-20T21:17:45.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:16:51.795761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:16:52.983434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T21:16:56.582297+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T21:17:45.451 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.660+0000 7fd567a00640 1 Processor -- start 2024-04-20T21:17:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.661+0000 7fd567a00640 1 -- start start 2024-04-20T21:17:45.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.662+0000 7fd567a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd560153210 0x7fd56015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.662+0000 7fd567a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd5600c6ba0 0x7fd560159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.662+0000 7fd567a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd5600c9eb0 0x7fd5600c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.662+0000 7fd567a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fd5601596f0 con 0x7fd5600c6ba0 2024-04-20T21:17:45.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.662+0000 7fd567a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fd5600c44d0 con 0x7fd5600c9eb0 2024-04-20T21:17:45.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.662+0000 7fd567a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd5600c47b0 con 0x7fd560153210 2024-04-20T21:17:45.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd560153210 0x7fd56015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd560153210 0x7fd56015c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:33518/0 (socket says 172.21.15.102:33518) 2024-04-20T21:17:45.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565000640 1 -- 172.21.15.102:0/974867897 learned_addr learned my addr 172.21.15.102:0/974867897 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:45.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565a00640 1 --2- 172.21.15.102:0/974867897 >> v2:172.21.15.193:3300/0 conn(0x7fd5600c9eb0 0x7fd5600c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd55fe00640 1 --2- 172.21.15.102:0/974867897 >> v2:172.21.15.102:3300/0 conn(0x7fd5600c6ba0 0x7fd560159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565000640 1 -- 172.21.15.102:0/974867897 >> v2:172.21.15.193:3300/0 conn(0x7fd5600c9eb0 msgr2=0x7fd5600c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:45.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565000640 1 --2- 172.21.15.102:0/974867897 >> v2:172.21.15.193:3300/0 conn(0x7fd5600c9eb0 0x7fd5600c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565000640 1 -- 172.21.15.102:0/974867897 >> v2:172.21.15.102:3300/0 conn(0x7fd5600c6ba0 msgr2=0x7fd560159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:45.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565000640 1 --2- 172.21.15.102:0/974867897 >> v2:172.21.15.102:3300/0 conn(0x7fd5600c6ba0 0x7fd560159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.663+0000 7fd565000640 1 -- 172.21.15.102:0/974867897 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5600c4a90 con 0x7fd560153210 2024-04-20T21:17:45.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.665+0000 7fd565000640 1 --2- 172.21.15.102:0/974867897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd560153210 0x7fd56015c5e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd5600ca340 tx=0x7fd55002b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e759c1724df30065 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:45.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.665+0000 7fd55f400640 1 -- 172.21.15.102:0/974867897 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550034070 con 0x7fd560153210 2024-04-20T21:17:45.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.665+0000 7fd55f400640 1 -- 172.21.15.102:0/974867897 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd550030860 con 0x7fd560153210 2024-04-20T21:17:45.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.665+0000 7fd55f400640 1 -- 172.21.15.102:0/974867897 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550030c30 con 0x7fd560153210 2024-04-20T21:17:45.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.665+0000 7fd567a00640 1 -- 172.21.15.102:0/974867897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd560153210 msgr2=0x7fd56015c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.665+0000 7fd567a00640 1 --2- 172.21.15.102:0/974867897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd560153210 0x7fd56015c5e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd5600ca340 tx=0x7fd55002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.666+0000 7fd567a00640 1 -- 172.21.15.102:0/974867897 shutdown_connections 2024-04-20T21:17:45.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.666+0000 7fd567a00640 1 --2- 172.21.15.102:0/974867897 >> v2:172.21.15.193:3300/0 conn(0x7fd5600c9eb0 0x7fd5600c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.666+0000 7fd567a00640 1 --2- 172.21.15.102:0/974867897 >> v2:172.21.15.102:3300/0 conn(0x7fd5600c6ba0 0x7fd560159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.666+0000 7fd567a00640 1 --2- 172.21.15.102:0/974867897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd560153210 0x7fd56015c5e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.666+0000 7fd567a00640 1 -- 172.21.15.102:0/974867897 >> 172.21.15.102:0/974867897 conn(0x7fd5600c9390 msgr2=0x7fd5600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:45.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.666+0000 7fd567a00640 1 -- 172.21.15.102:0/974867897 shutdown_connections 2024-04-20T21:17:45.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.666+0000 7fd567a00640 1 -- 172.21.15.102:0/974867897 wait complete. 2024-04-20T21:17:45.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.667+0000 7fd567a00640 1 Processor -- start 2024-04-20T21:17:45.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.668+0000 7fd567a00640 1 -- start start 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.668+0000 7fd567a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5600c6ba0 0x7fd560151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.668+0000 7fd567a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5600c9eb0 0x7fd56014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.668+0000 7fd567a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd560153210 0x7fd56014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.668+0000 7fd567a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601535f0 con 0x7fd5600c9eb0 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.668+0000 7fd567a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd560159030 con 0x7fd560153210 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.668+0000 7fd567a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd560153770 con 0x7fd5600c6ba0 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd55fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5600c9eb0 0x7fd56014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5600c6ba0 0x7fd560151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd560153210 0x7fd56014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5600c6ba0 0x7fd560151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:33528/0 (socket says 172.21.15.102:33528) 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565000640 1 -- 172.21.15.102:0/3505236329 learned_addr learned my addr 172.21.15.102:0/3505236329 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd55fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5600c9eb0 0x7fd56014f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48492/0 (socket says 172.21.15.102:48492) 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565000640 1 -- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd560153210 msgr2=0x7fd56014fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565000640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd560153210 0x7fd56014fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565000640 1 -- 172.21.15.102:0/3505236329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5600c9eb0 msgr2=0x7fd56014f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565000640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5600c9eb0 0x7fd56014f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565000640 1 -- 172.21.15.102:0/3505236329 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd560150f30 con 0x7fd5600c6ba0 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd55fe00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5600c9eb0 0x7fd56014f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.669+0000 7fd565a00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd560153210 0x7fd56014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:45.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.670+0000 7fd565000640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5600c6ba0 0x7fd560151c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd550000c00 tx=0x7fd5500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:45.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.670+0000 7fd55cc00640 1 -- 172.21.15.102:0/3505236329 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550034040 con 0x7fd5600c6ba0 2024-04-20T21:17:45.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.670+0000 7fd55cc00640 1 -- 172.21.15.102:0/3505236329 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd550039d90 con 0x7fd5600c6ba0 2024-04-20T21:17:45.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.670+0000 7fd55cc00640 1 -- 172.21.15.102:0/3505236329 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550043610 con 0x7fd5600c6ba0 2024-04-20T21:17:45.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.670+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5600ca690 con 0x7fd5600c6ba0 2024-04-20T21:17:45.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.670+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5600cac10 con 0x7fd5600c6ba0 2024-04-20T21:17:45.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.671+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56004fd60 con 0x7fd5600c6ba0 2024-04-20T21:17:45.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.673+0000 7fd55cc00640 1 -- 172.21.15.102:0/3505236329 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd550037070 con 0x7fd5600c6ba0 2024-04-20T21:17:45.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.674+0000 7fd55cc00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd54006c8f0 0x7fd54006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:45.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.674+0000 7fd55cc00640 1 -- 172.21.15.102:0/3505236329 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd5500adf20 con 0x7fd5600c6ba0 2024-04-20T21:17:45.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.674+0000 7fd55fe00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd54006c8f0 0x7fd54006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:45.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.675+0000 7fd55fe00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd54006c8f0 0x7fd54006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5601524e0 tx=0x7fd54c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:45.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.679+0000 7fd55cc00640 1 -- 172.21.15.102:0/3505236329 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd55003e070 con 0x7fd5600c6ba0 2024-04-20T21:17:45.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.877+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5600743d0 con 0x7fd5600c6ba0 2024-04-20T21:17:45.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.878+0000 7fd55cc00640 1 -- 172.21.15.102:0/3505236329 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd55007b8f0 con 0x7fd5600c6ba0 2024-04-20T21:17:45.877 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.880+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd54006c8f0 msgr2=0x7fd54006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.880+0000 7fd567a00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd54006c8f0 0x7fd54006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5601524e0 tx=0x7fd54c008040 comp rx=0 tx=0).stop 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.880+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5600c6ba0 msgr2=0x7fd560151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.880+0000 7fd567a00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5600c6ba0 0x7fd560151c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd550000c00 tx=0x7fd5500046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.881+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 shutdown_connections 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.881+0000 7fd567a00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd54006c8f0 0x7fd54006edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.881+0000 7fd567a00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd560153210 0x7fd56014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.881+0000 7fd567a00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5600c9eb0 0x7fd56014f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.881+0000 7fd567a00640 1 --2- 172.21.15.102:0/3505236329 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5600c6ba0 0x7fd560151c30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.881+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 >> 172.21.15.102:0/3505236329 conn(0x7fd5600c9390 msgr2=0x7fd560072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.881+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 shutdown_connections 2024-04-20T21:17:45.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:45.881+0000 7fd567a00640 1 -- 172.21.15.102:0/3505236329 wait complete. 2024-04-20T21:17:45.902 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:17:46.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.100+0000 7fb7ebe00640 1 Processor -- start 2024-04-20T21:17:46.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.100+0000 7fb7ebe00640 1 -- start start 2024-04-20T21:17:46.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.101+0000 7fb7ebe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb7ec155250 0x7fb7ec0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.101+0000 7fb7ebe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb7ec0cccf0 0x7fb7ec0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.101+0000 7fb7ebe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec15e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.101+0000 7fb7ebe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb7ec0cb770 con 0x7fb7ec155250 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7ebe00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb7ec0cd3b0 con 0x7fb7ec0cccf0 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7ebe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec15b180 con 0x7fb7ec0cc590 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7eaa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb7ec155250 0x7fb7ec0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7eaa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb7ec155250 0x7fb7ec0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48494/0 (socket says 172.21.15.102:48494) 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7eaa00640 1 -- 172.21.15.102:0/1457958688 learned_addr learned my addr 172.21.15.102:0/1457958688 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7ea000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb7ec0cccf0 0x7fb7ec0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7eb400640 1 --2- 172.21.15.102:0/1457958688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7eb400640 1 -- 172.21.15.102:0/1457958688 >> v2:172.21.15.193:3300/0 conn(0x7fb7ec0cccf0 msgr2=0x7fb7ec0cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7eb400640 1 --2- 172.21.15.102:0/1457958688 >> v2:172.21.15.193:3300/0 conn(0x7fb7ec0cccf0 0x7fb7ec0cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7eb400640 1 -- 172.21.15.102:0/1457958688 >> v2:172.21.15.102:3300/0 conn(0x7fb7ec155250 msgr2=0x7fb7ec0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.102+0000 7fb7eb400640 1 --2- 172.21.15.102:0/1457958688 >> v2:172.21.15.102:3300/0 conn(0x7fb7ec155250 0x7fb7ec0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.103+0000 7fb7eb400640 1 -- 172.21.15.102:0/1457958688 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec15b420 con 0x7fb7ec0cc590 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.103+0000 7fb7ea000640 1 --2- 172.21.15.102:0/1457958688 >> v2:172.21.15.193:3300/0 conn(0x7fb7ec0cccf0 0x7fb7ec0cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.104+0000 7fb7eb400640 1 --2- 172.21.15.102:0/1457958688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec15e660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec08ebf0 tx=0x7fb7e402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c493e3d5005934b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.104+0000 7fb7e9600640 1 -- 172.21.15.102:0/1457958688 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4039070 con 0x7fb7ec0cc590 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.104+0000 7fb7e9600640 1 -- 172.21.15.102:0/1457958688 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e402cbd0 con 0x7fb7ec0cc590 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.104+0000 7fb7e9600640 1 -- 172.21.15.102:0/1457958688 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4034440 con 0x7fb7ec0cc590 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 -- 172.21.15.102:0/1457958688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 msgr2=0x7fb7ec15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/1457958688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec15e660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec08ebf0 tx=0x7fb7e402be20 comp rx=0 tx=0).stop 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 -- 172.21.15.102:0/1457958688 shutdown_connections 2024-04-20T21:17:46.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/1457958688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec15e660 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/1457958688 >> v2:172.21.15.193:3300/0 conn(0x7fb7ec0cccf0 0x7fb7ec0cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/1457958688 >> v2:172.21.15.102:3300/0 conn(0x7fb7ec155250 0x7fb7ec0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 -- 172.21.15.102:0/1457958688 >> 172.21.15.102:0/1457958688 conn(0x7fb7ec06cb40 msgr2=0x7fb7ec06cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:46.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 -- 172.21.15.102:0/1457958688 shutdown_connections 2024-04-20T21:17:46.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.105+0000 7fb7ebe00640 1 -- 172.21.15.102:0/1457958688 wait complete. 2024-04-20T21:17:46.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.107+0000 7fb7ebe00640 1 Processor -- start 2024-04-20T21:17:46.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.107+0000 7fb7ebe00640 1 -- start start 2024-04-20T21:17:46.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7ebe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7ec0cccf0 0x7fb7ec1f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7ebe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7ec155250 0x7fb7ec0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7ebe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec15bcc0 con 0x7fb7ec0cccf0 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7ebe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec15b9e0 con 0x7fb7ec155250 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7ebe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec155630 con 0x7fb7ec0cc590 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7ea000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7ec0cccf0 0x7fb7ec1f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7eaa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7eb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7ec155250 0x7fb7ec0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7eaa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:33550/0 (socket says 172.21.15.102:33550) 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.108+0000 7fb7eaa00640 1 -- 172.21.15.102:0/3087399889 learned_addr learned my addr 172.21.15.102:0/3087399889 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7eaa00640 1 -- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7ec155250 msgr2=0x7fb7ec0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7eaa00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7ec155250 0x7fb7ec0c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7eaa00640 1 -- 172.21.15.102:0/3087399889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7ec0cccf0 msgr2=0x7fb7ec1f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7eaa00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7ec0cccf0 0x7fb7ec1f0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7eaa00640 1 -- 172.21.15.102:0/3087399889 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec1e76f0 con 0x7fb7ec0cc590 2024-04-20T21:17:46.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7eb400640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7ec155250 0x7fb7ec0c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:46.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7eaa00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec153c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00e8b0 tx=0x7fb7dc00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7e2a00640 1 -- 172.21.15.102:0/3087399889 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc017070 con 0x7fb7ec0cc590 2024-04-20T21:17:46.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7e2a00640 1 -- 172.21.15.102:0/3087399889 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00c940 con 0x7fb7ec0cc590 2024-04-20T21:17:46.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec1e6ff0 con 0x7fb7ec0cc590 2024-04-20T21:17:46.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.109+0000 7fb7e2a00640 1 -- 172.21.15.102:0/3087399889 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00cd10 con 0x7fb7ec0cc590 2024-04-20T21:17:46.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.110+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec0c9710 con 0x7fb7ec0cc590 2024-04-20T21:17:46.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.111+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ec04fcf0 con 0x7fb7ec0cc590 2024-04-20T21:17:46.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.112+0000 7fb7e2a00640 1 -- 172.21.15.102:0/3087399889 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7dc01a070 con 0x7fb7ec0cc590 2024-04-20T21:17:46.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.112+0000 7fb7e2a00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb7cc06c8f0 0x7fb7cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.113+0000 7fb7e2a00640 1 -- 172.21.15.102:0/3087399889 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb7dc08bdb0 con 0x7fb7ec0cc590 2024-04-20T21:17:46.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.113+0000 7fb7ea000640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb7cc06c8f0 0x7fb7cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.114+0000 7fb7ea000640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb7cc06c8f0 0x7fb7cc06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009560 tx=0x7fb7d8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.117+0000 7fb7e2a00640 1 -- 172.21.15.102:0/3087399889 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb7dc090070 con 0x7fb7ec0cc590 2024-04-20T21:17:46.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.317+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7ec06d2a0 con 0x7fb7ec0cc590 2024-04-20T21:17:46.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.318+0000 7fb7e2a00640 1 -- 172.21.15.102:0/3087399889 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb7dc059780 con 0x7fb7ec0cc590 2024-04-20T21:17:46.318 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb7cc06c8f0 msgr2=0x7fb7cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb7cc06c8f0 0x7fb7cc06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009560 tx=0x7fb7d8009190 comp rx=0 tx=0).stop 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 msgr2=0x7fb7ec153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec153c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00e8b0 tx=0x7fb7dc00ed70 comp rx=0 tx=0).stop 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 shutdown_connections 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb7cc06c8f0 0x7fb7cc06edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7ec155250 0x7fb7ec0c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7ec0cccf0 0x7fb7ec1f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 --2- 172.21.15.102:0/3087399889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7ec0cc590 0x7fb7ec153c90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 >> 172.21.15.102:0/3087399889 conn(0x7fb7ec06cb40 msgr2=0x7fb7ec0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:46.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 shutdown_connections 2024-04-20T21:17:46.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.321+0000 7fb7ebe00640 1 -- 172.21.15.102:0/3087399889 wait complete. 2024-04-20T21:17:46.340 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:17:46.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.545+0000 7f9e85800640 1 Processor -- start 2024-04-20T21:17:46.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.545+0000 7f9e85800640 1 -- start start 2024-04-20T21:17:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e85800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e80155330 0x7f9e800c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e85800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 0x7f9e8015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e85800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e800c7fe0 con 0x7f9e80071510 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e85800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9e80070f00 con 0x7f9e80155330 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e85800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8006cb40 con 0x7f9e800710d0 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e7ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e80155330 0x7f9e800c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e7ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9e80155330 0x7f9e800c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48512/0 (socket says 172.21.15.102:48512) 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e7ea00640 1 -- 172.21.15.102:0/2989907280 learned_addr learned my addr 172.21.15.102:0/2989907280 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.547+0000 7f9e7e000640 1 --2- 172.21.15.102:0/2989907280 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 0x7f9e8015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.548+0000 7f9e7ea00640 1 -- 172.21.15.102:0/2989907280 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 msgr2=0x7f9e8015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.548+0000 7f9e7ea00640 1 --2- 172.21.15.102:0/2989907280 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 0x7f9e8015e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.548+0000 7f9e7ea00640 1 -- 172.21.15.102:0/2989907280 >> v1:172.21.15.193:6789/0 conn(0x7f9e80071510 legacy=0x7f9e8015ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.548+0000 7f9e7d600640 1 -- 172.21.15.102:0/2989907280 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3477289996 0 0) 0x7f9e800c7fe0 con 0x7f9e80071510 2024-04-20T21:17:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.548+0000 7f9e7ea00640 1 -- 172.21.15.102:0/2989907280 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8006ce20 con 0x7f9e80155330 2024-04-20T21:17:46.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.549+0000 7f9e7ea00640 1 --2- 172.21.15.102:0/2989907280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80155330 0x7f9e800c8bb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009870 tx=0x7f9e6802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b69cc87903ed370 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.549+0000 7f9e7d600640 1 -- 172.21.15.102:0/2989907280 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68034070 con 0x7f9e80155330 2024-04-20T21:17:46.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.549+0000 7f9e7d600640 1 -- 172.21.15.102:0/2989907280 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e68030840 con 0x7f9e80155330 2024-04-20T21:17:46.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.549+0000 7f9e7d600640 1 -- 172.21.15.102:0/2989907280 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68030c10 con 0x7f9e80155330 2024-04-20T21:17:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.550+0000 7f9e85800640 1 -- 172.21.15.102:0/2989907280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80155330 msgr2=0x7f9e800c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.550+0000 7f9e85800640 1 --2- 172.21.15.102:0/2989907280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80155330 0x7f9e800c8bb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009870 tx=0x7f9e6802b980 comp rx=0 tx=0).stop 2024-04-20T21:17:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.550+0000 7f9e85800640 1 -- 172.21.15.102:0/2989907280 shutdown_connections 2024-04-20T21:17:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.550+0000 7f9e85800640 1 --2- 172.21.15.102:0/2989907280 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 0x7f9e8015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.550+0000 7f9e85800640 1 --2- 172.21.15.102:0/2989907280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80155330 0x7f9e800c8bb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.550+0000 7f9e85800640 1 -- 172.21.15.102:0/2989907280 >> 172.21.15.102:0/2989907280 conn(0x7f9e800c9390 msgr2=0x7f9e800c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.550+0000 7f9e85800640 1 -- 172.21.15.102:0/2989907280 shutdown_connections 2024-04-20T21:17:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.550+0000 7f9e85800640 1 -- 172.21.15.102:0/2989907280 wait complete. 2024-04-20T21:17:46.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.552+0000 7f9e85800640 1 Processor -- start 2024-04-20T21:17:46.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.552+0000 7f9e85800640 1 -- start start 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e85800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 0x7f9e801583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e85800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80071510 0x7f9e800c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e85800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80155330 0x7f9e801516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e85800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9e800c7ca0 con 0x7f9e80071510 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e85800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8006d3e0 con 0x7f9e80155330 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e85800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8006d900 con 0x7f9e800710d0 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80071510 0x7f9e800c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80071510 0x7f9e800c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48522/0 (socket says 172.21.15.102:48522) 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7e000640 1 -- 172.21.15.102:0/700115935 learned_addr learned my addr 172.21.15.102:0/700115935 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7ea00640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 0x7f9e801583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7f400640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80155330 0x7f9e801516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7e000640 1 -- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 msgr2=0x7f9e801583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7e000640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 0x7f9e801583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7e000640 1 -- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80155330 msgr2=0x7f9e801516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.553+0000 7f9e7e000640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80155330 0x7f9e801516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.554+0000 7f9e7e000640 1 -- 172.21.15.102:0/700115935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e80155b50 con 0x7f9e80071510 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.554+0000 7f9e7e000640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80071510 0x7f9e800c48d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e930 tx=0x7f9e7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.554+0000 7f9e76a00640 1 -- 172.21.15.102:0/700115935 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70017070 con 0x7f9e80071510 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.554+0000 7f9e76a00640 1 -- 172.21.15.102:0/700115935 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7000c9a0 con 0x7f9e80071510 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.554+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e801f2000 con 0x7f9e80071510 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.554+0000 7f9e76a00640 1 -- 172.21.15.102:0/700115935 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000cd90 con 0x7f9e80071510 2024-04-20T21:17:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.554+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e801560d0 con 0x7f9e80071510 2024-04-20T21:17:46.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.555+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8004fd60 con 0x7f9e80071510 2024-04-20T21:17:46.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.556+0000 7f9e76a00640 1 -- 172.21.15.102:0/700115935 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e7001a070 con 0x7f9e80071510 2024-04-20T21:17:46.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.557+0000 7f9e76a00640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9e5c06c8f0 0x7f9e5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.557+0000 7f9e76a00640 1 -- 172.21.15.102:0/700115935 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9e7008c320 con 0x7f9e80071510 2024-04-20T21:17:46.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.558+0000 7f9e7ea00640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9e5c06c8f0 0x7f9e5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.558+0000 7f9e7ea00640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9e5c06c8f0 0x7f9e5c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e68036000 tx=0x7f9e6803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.562+0000 7f9e76a00640 1 -- 172.21.15.102:0/700115935 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e70059d20 con 0x7f9e80071510 2024-04-20T21:17:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.766+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e800c8780 con 0x7f9e80071510 2024-04-20T21:17:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.767+0000 7f9e76a00640 1 -- 172.21.15.102:0/700115935 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f9e70060030 con 0x7f9e80071510 2024-04-20T21:17:46.767 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-20T21:17:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.769+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9e5c06c8f0 msgr2=0x7f9e5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.769+0000 7f9e85800640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9e5c06c8f0 0x7f9e5c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e68036000 tx=0x7f9e6803b040 comp rx=0 tx=0).stop 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.769+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80071510 msgr2=0x7f9e800c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.769+0000 7f9e85800640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80071510 0x7f9e800c48d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e930 tx=0x7f9e7000edf0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.770+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 shutdown_connections 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.770+0000 7f9e85800640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9e5c06c8f0 0x7f9e5c06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.770+0000 7f9e85800640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80155330 0x7f9e801516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.770+0000 7f9e85800640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e80071510 0x7f9e800c48d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.770+0000 7f9e85800640 1 --2- 172.21.15.102:0/700115935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e800710d0 0x7f9e801583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.770+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 >> 172.21.15.102:0/700115935 conn(0x7f9e800c9390 msgr2=0x7f9e800c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.770+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 shutdown_connections 2024-04-20T21:17:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.770+0000 7f9e85800640 1 -- 172.21.15.102:0/700115935 wait complete. 2024-04-20T21:17:46.790 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T21:17:46.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.992+0000 7fba13800640 1 Processor -- start 2024-04-20T21:17:46.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.993+0000 7fba13800640 1 -- start start 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba13800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1534f0 0x7fba0c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba13800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fba0c158f70 0x7fba0c0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba13800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba0c0c89f0 con 0x7fba0c153110 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba13800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fba0c0c3920 con 0x7fba0c158f70 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba13800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fba0c0c91b0 con 0x7fba0c1534f0 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba10e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fba0c153110 0x7fba0c06f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57592/0 (socket says 172.21.15.102:57592) 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba10e00640 1 -- 172.21.15.102:0/885450570 learned_addr learned my addr 172.21.15.102:0/885450570 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba11800640 1 --2- 172.21.15.102:0/885450570 >> v2:172.21.15.193:3300/0 conn(0x7fba0c158f70 0x7fba0c0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.994+0000 7fba03e00640 1 --2- 172.21.15.102:0/885450570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1534f0 0x7fba0c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.995+0000 7fba0be00640 1 -- 172.21.15.102:0/885450570 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703218783 0 0) 0x7fba0c0c89f0 con 0x7fba0c153110 2024-04-20T21:17:46.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.995+0000 7fba0be00640 1 -- 172.21.15.102:0/885450570 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9f0003520 con 0x7fba0c153110 2024-04-20T21:17:46.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.995+0000 7fba03e00640 1 -- 172.21.15.102:0/885450570 >> v2:172.21.15.193:3300/0 conn(0x7fba0c158f70 msgr2=0x7fba0c0c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:46.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.995+0000 7fba03e00640 1 --2- 172.21.15.102:0/885450570 >> v2:172.21.15.193:3300/0 conn(0x7fba0c158f70 0x7fba0c0c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.995+0000 7fba03e00640 1 -- 172.21.15.102:0/885450570 >> v1:172.21.15.102:6789/0 conn(0x7fba0c153110 legacy=0x7fba0c06f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.995+0000 7fba0be00640 1 -- 172.21.15.102:0/885450570 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3108901080 0 0) 0x7fb9f0003520 con 0x7fba0c153110 2024-04-20T21:17:46.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.995+0000 7fba03e00640 1 -- 172.21.15.102:0/885450570 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c0c8cd0 con 0x7fba0c1534f0 2024-04-20T21:17:46.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.995+0000 7fba11800640 1 --2- 172.21.15.102:0/885450570 >> v2:172.21.15.193:3300/0 conn(0x7fba0c158f70 0x7fba0c0c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:46.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.996+0000 7fba03e00640 1 --2- 172.21.15.102:0/885450570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1534f0 0x7fba0c0c83d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40097a0 tx=0x7fb9f402b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5348fdf06d41ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.996+0000 7fba0be00640 1 -- 172.21.15.102:0/885450570 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4037070 con 0x7fba0c1534f0 2024-04-20T21:17:46.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.996+0000 7fba0be00640 1 -- 172.21.15.102:0/885450570 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f4030510 con 0x7fba0c1534f0 2024-04-20T21:17:46.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.996+0000 7fba0be00640 1 -- 172.21.15.102:0/885450570 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc001670 con 0x7fba0c1534f0 2024-04-20T21:17:46.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.997+0000 7fba13800640 1 -- 172.21.15.102:0/885450570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1534f0 msgr2=0x7fba0c0c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.997+0000 7fba13800640 1 --2- 172.21.15.102:0/885450570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1534f0 0x7fba0c0c83d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40097a0 tx=0x7fb9f402b710 comp rx=0 tx=0).stop 2024-04-20T21:17:46.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.997+0000 7fba13800640 1 -- 172.21.15.102:0/885450570 shutdown_connections 2024-04-20T21:17:46.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.997+0000 7fba13800640 1 --2- 172.21.15.102:0/885450570 >> v2:172.21.15.193:3300/0 conn(0x7fba0c158f70 0x7fba0c0c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.997+0000 7fba13800640 1 --2- 172.21.15.102:0/885450570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1534f0 0x7fba0c0c83d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.997+0000 7fba13800640 1 -- 172.21.15.102:0/885450570 >> 172.21.15.102:0/885450570 conn(0x7fba0c074ea0 msgr2=0x7fba0c075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:46.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.997+0000 7fba13800640 1 -- 172.21.15.102:0/885450570 shutdown_connections 2024-04-20T21:17:46.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.997+0000 7fba13800640 1 -- 172.21.15.102:0/885450570 wait complete. 2024-04-20T21:17:46.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.999+0000 7fba13800640 1 Processor -- start 2024-04-20T21:17:46.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:46.999+0000 7fba13800640 1 -- start start 2024-04-20T21:17:47.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba13800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba0c153110 0x7fba0c14a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba13800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0c158f70 0x7fba0c1f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba13800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1f50b0 0x7fba0c149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba13800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c0c99c0 con 0x7fba0c153110 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba13800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c0c96e0 con 0x7fba0c158f70 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba13800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fba0c0c9ee0 con 0x7fba0c1f50b0 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba10e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba0c153110 0x7fba0c14a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba10e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba0c153110 0x7fba0c14a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48528/0 (socket says 172.21.15.102:48528) 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba10e00640 1 -- 172.21.15.102:0/3027429328 learned_addr learned my addr 172.21.15.102:0/3027429328 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba03e00640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0c158f70 0x7fba0c1f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.000+0000 7fba11800640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1f50b0 0x7fba0c149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.001+0000 7fba03e00640 1 -- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1f50b0 msgr2=0x7fba0c149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.001+0000 7fba03e00640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1f50b0 0x7fba0c149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.001+0000 7fba03e00640 1 -- 172.21.15.102:0/3027429328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba0c153110 msgr2=0x7fba0c14a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.001+0000 7fba03e00640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba0c153110 0x7fba0c14a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.001+0000 7fba03e00640 1 -- 172.21.15.102:0/3027429328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c1f1110 con 0x7fba0c158f70 2024-04-20T21:17:47.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.001+0000 7fba03e00640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0c158f70 0x7fba0c1f4b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0c9330 tx=0x7fb9f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.001+0000 7fba09600640 1 -- 172.21.15.102:0/3027429328 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4041020 con 0x7fba0c158f70 2024-04-20T21:17:47.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.002+0000 7fba09600640 1 -- 172.21.15.102:0/3027429328 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f403c870 con 0x7fba0c158f70 2024-04-20T21:17:47.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.002+0000 7fba09600640 1 -- 172.21.15.102:0/3027429328 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4037040 con 0x7fba0c158f70 2024-04-20T21:17:47.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.002+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0c1f13d0 con 0x7fba0c158f70 2024-04-20T21:17:47.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.002+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0c1f1990 con 0x7fba0c158f70 2024-04-20T21:17:47.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.003+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba0c06e770 con 0x7fba0c158f70 2024-04-20T21:17:47.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.004+0000 7fba09600640 1 -- 172.21.15.102:0/3027429328 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9f4044020 con 0x7fba0c158f70 2024-04-20T21:17:47.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.005+0000 7fba09600640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb9e806c920 0x7fb9e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.005+0000 7fba09600640 1 -- 172.21.15.102:0/3027429328 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb9f40ad4f0 con 0x7fba0c158f70 2024-04-20T21:17:47.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.005+0000 7fba10e00640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb9e806c920 0x7fb9e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.006+0000 7fba10e00640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb9e806c920 0x7fb9e806ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc001eb0 tx=0x7fb9fc001730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.010+0000 7fba09600640 1 -- 172.21.15.102:0/3027429328 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fba0c06e770 con 0x7fba0c158f70 2024-04-20T21:17:47.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.210+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fba0c152930 con 0x7fba0c158f70 2024-04-20T21:17:47.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.212+0000 7fba09600640 1 -- 172.21.15.102:0/3027429328 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb9f404b070 con 0x7fba0c158f70 2024-04-20T21:17:47.212 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:17:47.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb9e806c920 msgr2=0x7fb9e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb9e806c920 0x7fb9e806ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc001eb0 tx=0x7fb9fc001730 comp rx=0 tx=0).stop 2024-04-20T21:17:47.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0c158f70 msgr2=0x7fba0c1f4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0c158f70 0x7fba0c1f4b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0c9330 tx=0x7fb9f40046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 shutdown_connections 2024-04-20T21:17:47.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba0c1f50b0 0x7fba0c149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb9e806c920 0x7fb9e806ede0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0c158f70 0x7fba0c1f4b70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 --2- 172.21.15.102:0/3027429328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba0c153110 0x7fba0c14a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 >> 172.21.15.102:0/3027429328 conn(0x7fba0c074ea0 msgr2=0x7fba0c0c77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.214+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 shutdown_connections 2024-04-20T21:17:47.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.215+0000 7fba13800640 1 -- 172.21.15.102:0/3027429328 wait complete. 2024-04-20T21:17:47.238 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:47.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.430+0000 7f2016a00640 1 Processor -- start 2024-04-20T21:17:47.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.430+0000 7f2016a00640 1 -- start start 2024-04-20T21:17:47.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.432+0000 7f2016a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.432+0000 7f2016a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f201015f770 con 0x7f20100cbc30 2024-04-20T21:17:47.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.432+0000 7f2016a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f201015bf60 con 0x7f2010155460 2024-04-20T21:17:47.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.432+0000 7f2016a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f201015c240 con 0x7f20100ccc60 2024-04-20T21:17:47.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.432+0000 7f2016000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f20100cbc30 0x7f201015b240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57608/0 (socket says 172.21.15.102:57608) 2024-04-20T21:17:47.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.432+0000 7f2016000640 1 -- 172.21.15.102:0/490138298 learned_addr learned my addr 172.21.15.102:0/490138298 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:47.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.432+0000 7f2014c00640 1 --2- 172.21.15.102:0/490138298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.433+0000 7f200fe00640 1 -- 172.21.15.102:0/490138298 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688575961 0 0) 0x7f201015f770 con 0x7f20100cbc30 2024-04-20T21:17:47.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.433+0000 7f200fe00640 1 -- 172.21.15.102:0/490138298 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ff8003520 con 0x7f20100cbc30 2024-04-20T21:17:47.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.433+0000 7f200fe00640 1 -- 172.21.15.102:0/490138298 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2193782389 0 0) 0x7f201015bf60 con 0x7f2010155460 2024-04-20T21:17:47.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.433+0000 7f200fe00640 1 -- 172.21.15.102:0/490138298 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f201015f770 con 0x7f2010155460 2024-04-20T21:17:47.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.433+0000 7f2014c00640 1 -- 172.21.15.102:0/490138298 >> v1:172.21.15.193:6789/0 conn(0x7f2010155460 legacy=0x7f20100c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.433+0000 7f2014c00640 1 -- 172.21.15.102:0/490138298 >> v1:172.21.15.102:6789/0 conn(0x7f20100cbc30 legacy=0x7f201015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.433+0000 7f2014c00640 1 -- 172.21.15.102:0/490138298 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201015e9b0 con 0x7f20100ccc60 2024-04-20T21:17:47.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.434+0000 7f2014c00640 1 --2- 172.21.15.102:0/490138298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100cb6f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0097d0 tx=0x7f1ffc02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5bd0e1926df956c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.434+0000 7f200fe00640 1 -- 172.21.15.102:0/490138298 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc034070 con 0x7f20100ccc60 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.434+0000 7f200fe00640 1 -- 172.21.15.102:0/490138298 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ffc030880 con 0x7f20100ccc60 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.434+0000 7f200fe00640 1 -- 172.21.15.102:0/490138298 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc030c50 con 0x7f20100ccc60 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.435+0000 7f2016a00640 1 -- 172.21.15.102:0/490138298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 msgr2=0x7f20100cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.435+0000 7f2016a00640 1 --2- 172.21.15.102:0/490138298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100cb6f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0097d0 tx=0x7f1ffc02b9c0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.435+0000 7f2016a00640 1 -- 172.21.15.102:0/490138298 shutdown_connections 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.435+0000 7f2016a00640 1 --2- 172.21.15.102:0/490138298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100cb6f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.435+0000 7f2016a00640 1 -- 172.21.15.102:0/490138298 >> 172.21.15.102:0/490138298 conn(0x7f2010072d30 msgr2=0x7f2010073140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.435+0000 7f2016a00640 1 -- 172.21.15.102:0/490138298 shutdown_connections 2024-04-20T21:17:47.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.435+0000 7f2016a00640 1 -- 172.21.15.102:0/490138298 wait complete. 2024-04-20T21:17:47.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.437+0000 7f2016a00640 1 Processor -- start 2024-04-20T21:17:47.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.437+0000 7f2016a00640 1 -- start start 2024-04-20T21:17:47.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20100cbc30 0x7f2010151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2010155460 0x7f20101f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f201015f250 con 0x7f2010155460 2024-04-20T21:17:47.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f201015ef70 con 0x7f20100cbc30 2024-04-20T21:17:47.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f20100c5f20 con 0x7f20100ccc60 2024-04-20T21:17:47.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2010155460 0x7f20101f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2014c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2015600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20100cbc30 0x7f2010151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2010155460 0x7f20101f1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48534/0 (socket says 172.21.15.102:48534) 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.438+0000 7f2016000640 1 -- 172.21.15.102:0/2296099330 learned_addr learned my addr 172.21.15.102:0/2296099330 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f2016000640 1 -- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 msgr2=0x7f20100ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f2016000640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f2016000640 1 -- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20100cbc30 msgr2=0x7f2010151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f2016000640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20100cbc30 0x7f2010151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f2016000640 1 -- 172.21.15.102:0/2296099330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101f5750 con 0x7f2010155460 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f2014c00640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f2016000640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2010155460 0x7f20101f1680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f200800cb40 tx=0x7f200800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f200d600640 1 -- 172.21.15.102:0/2296099330 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008017070 con 0x7f2010155460 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f200d600640 1 -- 172.21.15.102:0/2296099330 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200800aca0 con 0x7f2010155460 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f200d600640 1 -- 172.21.15.102:0/2296099330 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008012600 con 0x7f2010155460 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.439+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20101f48f0 con 0x7f2010155460 2024-04-20T21:17:47.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.440+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20101f4eb0 con 0x7f2010155460 2024-04-20T21:17:47.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.441+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201004fd60 con 0x7f2010155460 2024-04-20T21:17:47.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.442+0000 7f200d600640 1 -- 172.21.15.102:0/2296099330 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f200801a070 con 0x7f2010155460 2024-04-20T21:17:47.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.443+0000 7f200d600640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff006c8f0 0x7f1ff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.443+0000 7f200d600640 1 -- 172.21.15.102:0/2296099330 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f200808bd80 con 0x7f2010155460 2024-04-20T21:17:47.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.443+0000 7f2015600640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff006c8f0 0x7f1ff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.444+0000 7f2015600640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff006c8f0 0x7f1ff006edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2004007fd0 tx=0x7f2004009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.447+0000 7f200d600640 1 -- 172.21.15.102:0/2296099330 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2008059630 con 0x7f2010155460 2024-04-20T21:17:47.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.687+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20100cc250 con 0x7f2010155460 2024-04-20T21:17:47.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.689+0000 7f200d600640 1 -- 172.21.15.102:0/2296099330 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f200805d930 con 0x7f2010155460 2024-04-20T21:17:47.689 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:47.689 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:17:47.689 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:17:47.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff006c8f0 msgr2=0x7f1ff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff006c8f0 0x7f1ff006edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2004007fd0 tx=0x7f2004009040 comp rx=0 tx=0).stop 2024-04-20T21:17:47.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2010155460 msgr2=0x7f20101f1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2010155460 0x7f20101f1680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f200800cb40 tx=0x7f200800a060 comp rx=0 tx=0).stop 2024-04-20T21:17:47.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 shutdown_connections 2024-04-20T21:17:47.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2010155460 0x7f20101f1680 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20100ccc60 0x7f20100ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff006c8f0 0x7f1ff006edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 --2- 172.21.15.102:0/2296099330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20100cbc30 0x7f2010151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.691+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 >> 172.21.15.102:0/2296099330 conn(0x7f2010072d30 msgr2=0x7f20100c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.692+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 shutdown_connections 2024-04-20T21:17:47.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.692+0000 7f2016a00640 1 -- 172.21.15.102:0/2296099330 wait complete. 2024-04-20T21:17:47.710 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:47.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.910+0000 7ff3c5e00640 1 Processor -- start 2024-04-20T21:17:47.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.910+0000 7ff3c5e00640 1 -- start start 2024-04-20T21:17:47.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c5e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff3c01630a0 0x7ff3c01634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c5e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c006c2d0 con 0x7ff3c0159850 2024-04-20T21:17:47.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c5e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c00c7c00 con 0x7ff3c00c8600 2024-04-20T21:17:47.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c5e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c006e6d0 con 0x7ff3c01630a0 2024-04-20T21:17:47.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c5400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff3c01630a0 0x7ff3c01634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c4a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff3c0159850 0x7ff3c006ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60972/0 (socket says 172.21.15.102:60972) 2024-04-20T21:17:47.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c4a00640 1 -- 172.21.15.102:0/1231225036 learned_addr learned my addr 172.21.15.102:0/1231225036 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:17:47.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c5400640 1 -- 172.21.15.102:0/1231225036 >> v1:172.21.15.193:6789/0 conn(0x7ff3c00c8600 legacy=0x7ff3c0162990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.912+0000 7ff3c5400640 1 -- 172.21.15.102:0/1231225036 >> v1:172.21.15.102:6789/0 conn(0x7ff3c0159850 legacy=0x7ff3c006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.913+0000 7ff3c5400640 1 -- 172.21.15.102:0/1231225036 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c00c3810 con 0x7ff3c01630a0 2024-04-20T21:17:47.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.914+0000 7ff3c5400640 1 --2- 172.21.15.102:0/1231225036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c01630a0 0x7ff3c01634d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009d90 tx=0x7ff3b802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c6860bd5fdff3b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.914+0000 7ff3bf400640 1 -- 172.21.15.102:0/1231225036 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8039070 con 0x7ff3c01630a0 2024-04-20T21:17:47.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.914+0000 7ff3bf400640 1 -- 172.21.15.102:0/1231225036 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b802cd00 con 0x7ff3c01630a0 2024-04-20T21:17:47.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.914+0000 7ff3bf400640 1 -- 172.21.15.102:0/1231225036 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8034660 con 0x7ff3c01630a0 2024-04-20T21:17:47.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.914+0000 7ff3c5e00640 1 -- 172.21.15.102:0/1231225036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c01630a0 msgr2=0x7ff3c01634d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.914+0000 7ff3c5e00640 1 --2- 172.21.15.102:0/1231225036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c01630a0 0x7ff3c01634d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009d90 tx=0x7ff3b802bf50 comp rx=0 tx=0).stop 2024-04-20T21:17:47.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.915+0000 7ff3c5e00640 1 -- 172.21.15.102:0/1231225036 shutdown_connections 2024-04-20T21:17:47.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.915+0000 7ff3c5e00640 1 --2- 172.21.15.102:0/1231225036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c01630a0 0x7ff3c01634d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.915+0000 7ff3c5e00640 1 -- 172.21.15.102:0/1231225036 >> 172.21.15.102:0/1231225036 conn(0x7ff3c00cc820 msgr2=0x7ff3c00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.915+0000 7ff3c5e00640 1 -- 172.21.15.102:0/1231225036 shutdown_connections 2024-04-20T21:17:47.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.915+0000 7ff3c5e00640 1 -- 172.21.15.102:0/1231225036 wait complete. 2024-04-20T21:17:47.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.916+0000 7ff3c5e00640 1 Processor -- start 2024-04-20T21:17:47.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.916+0000 7ff3c5e00640 1 -- start start 2024-04-20T21:17:47.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.917+0000 7ff3c5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c00c8600 0x7ff3c014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.917+0000 7ff3c5e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3c0159850 0x7ff3c0162250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.917+0000 7ff3c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c01630a0 0x7ff3c0146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.917+0000 7ff3c5e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00c8160 con 0x7ff3c00c8600 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.917+0000 7ff3c5e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c006e970 con 0x7ff3c01630a0 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.917+0000 7ff3c5e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c00c3460 con 0x7ff3c0159850 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3c5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c01630a0 0x7ff3c0146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3c0159850 0x7ff3c0162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3c5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c01630a0 0x7ff3c0146210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:49192/0 (socket says 172.21.15.102:49192) 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3c0159850 0x7ff3c0162250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:48390/0 (socket says 172.21.15.102:48390) 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3c5400640 1 -- 172.21.15.102:0/3194896149 learned_addr learned my addr 172.21.15.102:0/3194896149 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3c4a00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c00c8600 0x7ff3c014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3bfe00640 1 -- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c01630a0 msgr2=0x7ff3c0146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3bfe00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c01630a0 0x7ff3c0146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3bfe00640 1 -- 172.21.15.102:0/3194896149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c00c8600 msgr2=0x7ff3c014c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3bfe00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c00c8600 0x7ff3c014c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3bfe00640 1 -- 172.21.15.102:0/3194896149 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c014c590 con 0x7ff3c0159850 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3c5400640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c01630a0 0x7ff3c0146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.918+0000 7ff3bfe00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3c0159850 0x7ff3c0162250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00c3c0 tx=0x7ff3ac00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.919+0000 7ff3bcc00640 1 -- 172.21.15.102:0/3194896149 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0052d0 con 0x7ff3c0159850 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.919+0000 7ff3bcc00640 1 -- 172.21.15.102:0/3194896149 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ac005490 con 0x7ff3c0159850 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.919+0000 7ff3bcc00640 1 -- 172.21.15.102:0/3194896149 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac007260 con 0x7ff3c0159850 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.919+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c0162b00 con 0x7ff3c0159850 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.919+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c0160600 con 0x7ff3c0159850 2024-04-20T21:17:47.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.920+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c004fd60 con 0x7ff3c0159850 2024-04-20T21:17:47.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.921+0000 7ff3bcc00640 1 -- 172.21.15.102:0/3194896149 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3ac007400 con 0x7ff3c0159850 2024-04-20T21:17:47.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.922+0000 7ff3bcc00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3a006c8f0 0x7ff3a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.922+0000 7ff3bcc00640 1 -- 172.21.15.102:0/3194896149 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff3ac08b320 con 0x7ff3c0159850 2024-04-20T21:17:47.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.922+0000 7ff3c4a00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3a006c8f0 0x7ff3a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.923+0000 7ff3c4a00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3a006c8f0 0x7ff3a006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4005880 tx=0x7ff3b4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:47.927+0000 7ff3bcc00640 1 -- 172.21.15.102:0/3194896149 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff3c00c7c00 con 0x7ff3c0159850 2024-04-20T21:17:48.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.128+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3c014a6a0 con 0x7ff3c0159850 2024-04-20T21:17:48.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.130+0000 7ff3bcc00640 1 -- 172.21.15.102:0/3194896149 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18894 (secure 0 0 0) 0x7ff3ac08b0e0 con 0x7ff3c0159850 2024-04-20T21:17:48.130 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:17:48.130 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:17:35.869503+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:48.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3a006c8f0 msgr2=0x7ff3a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3a006c8f0 0x7ff3a006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4005880 tx=0x7ff3b4009040 comp rx=0 tx=0).stop 2024-04-20T21:17:48.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3c0159850 msgr2=0x7ff3c0162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3c0159850 0x7ff3c0162250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00c3c0 tx=0x7ff3ac00c880 comp rx=0 tx=0).stop 2024-04-20T21:17:48.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 shutdown_connections 2024-04-20T21:17:48.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff3a006c8f0 0x7ff3a006edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c01630a0 0x7ff3c0146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3c0159850 0x7ff3c0162250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 --2- 172.21.15.102:0/3194896149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c00c8600 0x7ff3c014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 >> 172.21.15.102:0/3194896149 conn(0x7ff3c00cc820 msgr2=0x7ff3c006ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:48.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 shutdown_connections 2024-04-20T21:17:48.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:17:48.133+0000 7ff3c5e00640 1 -- 172.21.15.102:0/3194896149 wait complete. 2024-04-20T21:17:48.154 DEBUG:tasks.mds_thrash:Yielding 2024-04-20T21:17:48.154 INFO:teuthology.run_tasks:Running task workunit... 2024-04-20T21:17:48.167 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:17:48.167 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-20T21:17:48.167 INFO:tasks.workunit:timeout=6h 2024-04-20T21:17:48.167 INFO:tasks.workunit:cleanup=True 2024-04-20T21:17:48.167 DEBUG:teuthology.orchestra.run.smithi102:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:17:48.168 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2024-04-20T21:17:48.168 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 96 secs before thrashing 2024-04-20T21:17:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-20T21:17:48.222 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-20T21:17:48.222 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-04-20T21:17:48.222 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:17:48.222 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-20T21:17:48.222 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-20T21:17:48.222 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-20 21:16:59.562364133 +0000 2024-04-20T21:17:48.222 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-20 21:17:11.700143151 +0000 2024-04-20T21:17:48.223 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-20 21:16:59.562364133 +0000 2024-04-20T21:17:48.223 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-20T21:17:48.223 DEBUG:teuthology.orchestra.run.smithi102:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-20T21:17:48.313 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:17:48.374 INFO:tasks.workunit.client.0.smithi102.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-20T21:17:51.130 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:51.138 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:21.268 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:21.276 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:51.381 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:51.386 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:16.550 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 62% (7636/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-20T21:19:17.348 INFO:tasks.workunit.client.0.smithi102.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-20T21:19:17.348 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-20T21:19:17.348 INFO:tasks.workunit.client.0.smithi102.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr: git switch -c 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr:Or undo this operation with: 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr: git switch - 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-20T21:19:17.349 INFO:tasks.workunit.client.0.smithi102.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T21:19:17.350 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-20T21:19:17.350 INFO:tasks.workunit.client.0.smithi102.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-20T21:19:17.362 DEBUG:teuthology.orchestra.run.smithi102:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-20T21:19:17.431 INFO:tasks.workunit.client.0.smithi102.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T21:19:17.438 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T21:19:17.438 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T21:19:18.271 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T21:19:18.380 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T21:19:18.457 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T21:19:18.461 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T21:19:18.462 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T21:19:18.542 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T21:19:18.550 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:19:18.550 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-20T21:19:18.597 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2024-04-20T21:19:18.598 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2024-04-20T21:19:18.598 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2024-04-20T21:19:18.715 INFO:tasks.workunit.client.0.smithi102.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-04-20T21:19:21.193 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir foo 2024-04-20T21:19:21.205 INFO:tasks.workunit.client.0.smithi102.stderr:+ cp linux-2.6.33.tar.bz2 foo 2024-04-20T21:19:21.320 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir foo/.snap/barsnap 2024-04-20T21:19:21.360 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm foo/linux-2.6.33.tar.bz2 2024-04-20T21:19:21.376 INFO:tasks.workunit.client.0.smithi102.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2024-04-20T21:19:21.538 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:21.543 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:21.964 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'passed: files are identical' 2024-04-20T21:19:21.964 INFO:tasks.workunit.client.0.smithi102.stdout:passed: files are identical 2024-04-20T21:19:21.964 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir foo/.snap/barsnap 2024-04-20T21:19:21.992 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-04-20T21:19:21.992 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo OK 2024-04-20T21:19:21.993 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T21:19:21.994 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T21:19:22.087 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2024-04-20T21:19:22.088 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2024-04-20T21:19:22.149 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 1 2024-04-20T21:19:22.151 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 2 2024-04-20T21:19:22.153 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 3 2024-04-20T21:19:22.156 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -e file4 ']' 2024-04-20T21:19:22.156 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/snap1 2024-04-20T21:19:22.866 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 4 2024-04-20T21:19:22.869 INFO:tasks.workunit.client.0.smithi102.stderr:++ ls 2024-04-20T21:19:22.879 INFO:tasks.workunit.client.0.smithi102.stderr:+ now='file1 2024-04-20T21:19:22.879 INFO:tasks.workunit.client.0.smithi102.stderr:file2 2024-04-20T21:19:22.879 INFO:tasks.workunit.client.0.smithi102.stderr:file3 2024-04-20T21:19:22.880 INFO:tasks.workunit.client.0.smithi102.stderr:file4' 2024-04-20T21:19:22.881 INFO:tasks.workunit.client.0.smithi102.stderr:++ ls .snap/snap1 2024-04-20T21:19:22.889 INFO:tasks.workunit.client.0.smithi102.stderr:+ then='file1 2024-04-20T21:19:22.889 INFO:tasks.workunit.client.0.smithi102.stderr:file2 2024-04-20T21:19:22.889 INFO:tasks.workunit.client.0.smithi102.stderr:file3' 2024-04-20T21:19:22.889 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir .snap/snap1 2024-04-20T21:19:22.900 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 'file1 2024-04-20T21:19:22.900 INFO:tasks.workunit.client.0.smithi102.stderr:file2 2024-04-20T21:19:22.901 INFO:tasks.workunit.client.0.smithi102.stderr:file3 2024-04-20T21:19:22.901 INFO:tasks.workunit.client.0.smithi102.stderr:file4' = 'file1 2024-04-20T21:19:22.901 INFO:tasks.workunit.client.0.smithi102.stderr:file2 2024-04-20T21:19:22.901 INFO:tasks.workunit.client.0.smithi102.stderr:file3' ']' 2024-04-20T21:19:22.901 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 1 2024-04-20T21:19:22.919 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 2 2024-04-20T21:19:22.963 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 3 2024-04-20T21:19:22.964 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/snap1 2024-04-20T21:19:22.989 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 4 2024-04-20T21:19:23.004 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir .snap/snap1 2024-04-20T21:19:23.020 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm file1 file2 file3 file4 2024-04-20T21:19:23.032 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo OK 2024-04-20T21:19:23.032 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-04-20T21:19:23.034 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T21:19:23.034 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T21:19:23.112 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2024-04-20T21:19:23.112 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2024-04-20T21:19:23.180 INFO:tasks.workunit.client.0.smithi102.stdout:Create dir 100 to 199 ... 2024-04-20T21:19:23.200 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 100 2024-04-20T21:19:23.234 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 101 2024-04-20T21:19:23.273 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 102 2024-04-20T21:19:23.314 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 103 2024-04-20T21:19:23.353 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 104 2024-04-20T21:19:23.383 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 105 2024-04-20T21:19:23.415 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 106 2024-04-20T21:19:23.451 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 107 2024-04-20T21:19:23.482 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 108 2024-04-20T21:19:23.519 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 109 2024-04-20T21:19:23.555 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 110 2024-04-20T21:19:23.590 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 111 2024-04-20T21:19:23.625 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 112 2024-04-20T21:19:23.665 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 113 2024-04-20T21:19:23.703 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 114 2024-04-20T21:19:23.741 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 115 2024-04-20T21:19:23.779 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 116 2024-04-20T21:19:23.818 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 117 2024-04-20T21:19:23.857 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 118 2024-04-20T21:19:23.895 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 119 2024-04-20T21:19:23.934 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 120 2024-04-20T21:19:23.976 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 121 2024-04-20T21:19:24.170 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:24.273 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 122 2024-04-20T21:19:24.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.383+0000 7f6a91c00640 1 Processor -- start 2024-04-20T21:19:24.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.383+0000 7f6a91c00640 1 -- start start 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.384+0000 7f6a91c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6a8c153180 0x7f6a8c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.384+0000 7f6a91c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6a8c159380 0x7f6a8c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.384+0000 7f6a91c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a8c153560 con 0x7f6a8c0c75a0 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.384+0000 7f6a91c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6a8c0c7a70 con 0x7f6a8c153180 2024-04-20T21:19:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.384+0000 7f6a91c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6a8c0c7d30 con 0x7f6a8c159380 2024-04-20T21:19:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a91200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6a8c0c75a0 0x7f6a8c070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34978/0 (socket says 172.21.15.102:34978) 2024-04-20T21:19:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a8be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6a8c153180 0x7f6a8c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a91200640 1 -- 172.21.15.102:0/3967798280 learned_addr learned my addr 172.21.15.102:0/3967798280 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:24.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a83e00640 1 --2- 172.21.15.102:0/3967798280 >> v2:172.21.15.193:3300/0 conn(0x7f6a8c159380 0x7f6a8c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a8b400640 1 -- 172.21.15.102:0/3967798280 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 438987036 0 0) 0x7f6a8c153560 con 0x7f6a8c0c75a0 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a83e00640 1 -- 172.21.15.102:0/3967798280 >> v2:172.21.15.102:3300/0 conn(0x7f6a8c153180 msgr2=0x7f6a8c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a83e00640 1 --2- 172.21.15.102:0/3967798280 >> v2:172.21.15.102:3300/0 conn(0x7f6a8c153180 0x7f6a8c158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a83e00640 1 -- 172.21.15.102:0/3967798280 >> v1:172.21.15.102:6789/0 conn(0x7f6a8c0c75a0 legacy=0x7f6a8c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.385+0000 7f6a83e00640 1 -- 172.21.15.102:0/3967798280 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c15c4d0 con 0x7f6a8c159380 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.386+0000 7f6a83e00640 1 --2- 172.21.15.102:0/3967798280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 0x7f6a8c0c5f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c06f8d0 tx=0x7f6a7402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c338eda01010e8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a8b400640 1 -- 172.21.15.102:0/3967798280 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74035020 con 0x7f6a8c159380 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a8b400640 1 -- 172.21.15.102:0/3967798280 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a74031830 con 0x7f6a8c159380 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a8b400640 1 -- 172.21.15.102:0/3967798280 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74031c00 con 0x7f6a8c159380 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a91c00640 1 -- 172.21.15.102:0/3967798280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 msgr2=0x7f6a8c0c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a91c00640 1 --2- 172.21.15.102:0/3967798280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 0x7f6a8c0c5f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c06f8d0 tx=0x7f6a7402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a91c00640 1 -- 172.21.15.102:0/3967798280 shutdown_connections 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a91c00640 1 --2- 172.21.15.102:0/3967798280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 0x7f6a8c0c5f30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a91c00640 1 --2- 172.21.15.102:0/3967798280 >> v2:172.21.15.102:3300/0 conn(0x7f6a8c153180 0x7f6a8c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a91c00640 1 -- 172.21.15.102:0/3967798280 >> 172.21.15.102:0/3967798280 conn(0x7f6a8c0cc820 msgr2=0x7f6a8c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a91c00640 1 -- 172.21.15.102:0/3967798280 shutdown_connections 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.387+0000 7f6a91c00640 1 -- 172.21.15.102:0/3967798280 wait complete. 2024-04-20T21:19:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.389+0000 7f6a91c00640 1 Processor -- start 2024-04-20T21:19:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.389+0000 7f6a91c00640 1 -- start start 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a91c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a8c0c75a0 0x7f6a8c14d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a91c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a8c153180 0x7f6a8c151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a91c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 0x7f6a8c1f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a91c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c15d290 con 0x7f6a8c153180 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a91c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c15ca90 con 0x7f6a8c159380 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a91c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8c0c6be0 con 0x7f6a8c0c75a0 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a83e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a8c153180 0x7f6a8c151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a83e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a8c153180 0x7f6a8c151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33072/0 (socket says 172.21.15.102:33072) 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.390+0000 7f6a83e00640 1 -- 172.21.15.102:0/447203682 learned_addr learned my addr 172.21.15.102:0/447203682 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a8be00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a8c0c75a0 0x7f6a8c14d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a91200640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 0x7f6a8c1f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a83e00640 1 -- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a8c0c75a0 msgr2=0x7f6a8c14d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a83e00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a8c0c75a0 0x7f6a8c14d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a83e00640 1 -- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 msgr2=0x7f6a8c1f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a83e00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 0x7f6a8c1f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a83e00640 1 -- 172.21.15.102:0/447203682 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c14bb90 con 0x7f6a8c153180 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a91200640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 0x7f6a8c1f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a83e00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a8c153180 0x7f6a8c151e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a74000c00 tx=0x7f6a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.391+0000 7f6a88c00640 1 -- 172.21.15.102:0/447203682 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74040070 con 0x7f6a8c153180 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.392+0000 7f6a88c00640 1 -- 172.21.15.102:0/447203682 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a7403a910 con 0x7f6a8c153180 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.392+0000 7f6a88c00640 1 -- 172.21.15.102:0/447203682 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74035040 con 0x7f6a8c153180 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.392+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8c14e7c0 con 0x7f6a8c153180 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.392+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8c14ea60 con 0x7f6a8c153180 2024-04-20T21:19:24.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.393+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8c06a0c0 con 0x7f6a8c153180 2024-04-20T21:19:24.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.394+0000 7f6a88c00640 1 -- 172.21.15.102:0/447203682 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a7404b060 con 0x7f6a8c153180 2024-04-20T21:19:24.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.395+0000 7f6a88c00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6a6806c920 0x7f6a6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.395+0000 7f6a88c00640 1 -- 172.21.15.102:0/447203682 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6a740ad860 con 0x7f6a8c153180 2024-04-20T21:19:24.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.395+0000 7f6a8be00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6a6806c920 0x7f6a6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.396+0000 7f6a8be00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6a6806c920 0x7f6a6806ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009aa0 tx=0x7f6a7c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.400+0000 7f6a88c00640 1 -- 172.21.15.102:0/447203682 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6a8c06a0c0 con 0x7f6a8c153180 2024-04-20T21:19:24.573 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 123 2024-04-20T21:19:24.598 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 124 2024-04-20T21:19:24.633 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 125 2024-04-20T21:19:24.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.649+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a8c04fd60 con 0x7f6a8c153180 2024-04-20T21:19:24.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.650+0000 7f6a88c00640 1 -- 172.21.15.102:0/447203682 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f6a8c14bb90 con 0x7f6a8c153180 2024-04-20T21:19:24.650 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:24.651 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:19:24.651 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6a6806c920 msgr2=0x7f6a6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6a6806c920 0x7f6a6806ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009aa0 tx=0x7f6a7c009190 comp rx=0 tx=0).stop 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a8c153180 msgr2=0x7f6a8c151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a8c153180 0x7f6a8c151e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a74000c00 tx=0x7f6a740046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 shutdown_connections 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6a6806c920 0x7f6a6806ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c159380 0x7f6a8c1f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a8c153180 0x7f6a8c151e50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 --2- 172.21.15.102:0/447203682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a8c0c75a0 0x7f6a8c14d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 >> 172.21.15.102:0/447203682 conn(0x7f6a8c0cc820 msgr2=0x7f6a8c150fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:24.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 shutdown_connections 2024-04-20T21:19:24.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.653+0000 7f6a91c00640 1 -- 172.21.15.102:0/447203682 wait complete. 2024-04-20T21:19:24.658 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 126 2024-04-20T21:19:24.673 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:19:24.673 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:19:24.673 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:24.695 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:19:24.695 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:24.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.938+0000 7f5570200640 1 Processor -- start 2024-04-20T21:19:24.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.938+0000 7f5570200640 1 -- start start 2024-04-20T21:19:24.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f5570200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5568152170 0x7f55680c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f5570200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5568158b80 0x7f55681628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f5570200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5568071f70 con 0x7f5568158770 2024-04-20T21:19:24.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f5570200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f55680c6dc0 con 0x7f5568152170 2024-04-20T21:19:24.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f5570200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f5568072210 con 0x7f5568158b80 2024-04-20T21:19:24.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f556d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5568152170 0x7f55680c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f556d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5568152170 0x7f55680c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33084/0 (socket says 172.21.15.102:33084) 2024-04-20T21:19:24.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f556d800640 1 -- 172.21.15.102:0/3876368359 learned_addr learned my addr 172.21.15.102:0/3876368359 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:24.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.940+0000 7f556e200640 1 --2- 172.21.15.102:0/3876368359 >> v2:172.21.15.193:3300/0 conn(0x7f5568158b80 0x7f55681628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.941+0000 7f556d800640 1 -- 172.21.15.102:0/3876368359 >> v2:172.21.15.193:3300/0 conn(0x7f5568158b80 msgr2=0x7f55681628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:24.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.941+0000 7f556d800640 1 --2- 172.21.15.102:0/3876368359 >> v2:172.21.15.193:3300/0 conn(0x7f5568158b80 0x7f55681628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.941+0000 7f5567e00640 1 -- 172.21.15.102:0/3876368359 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 517862363 0 0) 0x7f5568071f70 con 0x7f5568158770 2024-04-20T21:19:24.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.941+0000 7f556d800640 1 -- 172.21.15.102:0/3876368359 >> v1:172.21.15.193:6789/0 conn(0x7f5568158770 legacy=0x7f5568070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.941+0000 7f556d800640 1 -- 172.21.15.102:0/3876368359 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55680724b0 con 0x7f5568152170 2024-04-20T21:19:24.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.942+0000 7f556d800640 1 --2- 172.21.15.102:0/3876368359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568152170 0x7f55680c8db0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f556806c190 tx=0x7f555c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc07a55403d33877 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5567e00640 1 -- 172.21.15.102:0/3876368359 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c02f850 con 0x7f5568152170 2024-04-20T21:19:24.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5567e00640 1 -- 172.21.15.102:0/3876368359 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555c02f9f0 con 0x7f5568152170 2024-04-20T21:19:24.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5567e00640 1 -- 172.21.15.102:0/3876368359 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c02fdc0 con 0x7f5568152170 2024-04-20T21:19:24.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5570200640 1 -- 172.21.15.102:0/3876368359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568152170 msgr2=0x7f55680c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5570200640 1 --2- 172.21.15.102:0/3876368359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568152170 0x7f55680c8db0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f556806c190 tx=0x7f555c02ba30 comp rx=0 tx=0).stop 2024-04-20T21:19:24.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5570200640 1 -- 172.21.15.102:0/3876368359 shutdown_connections 2024-04-20T21:19:24.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5570200640 1 --2- 172.21.15.102:0/3876368359 >> v2:172.21.15.193:3300/0 conn(0x7f5568158b80 0x7f55681628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5570200640 1 --2- 172.21.15.102:0/3876368359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568152170 0x7f55680c8db0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.943+0000 7f5570200640 1 -- 172.21.15.102:0/3876368359 >> 172.21.15.102:0/3876368359 conn(0x7f55680c9390 msgr2=0x7f55680c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:24.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.944+0000 7f5570200640 1 -- 172.21.15.102:0/3876368359 shutdown_connections 2024-04-20T21:19:24.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.944+0000 7f5570200640 1 -- 172.21.15.102:0/3876368359 wait complete. 2024-04-20T21:19:24.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.945+0000 7f5570200640 1 Processor -- start 2024-04-20T21:19:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.945+0000 7f5570200640 1 -- start start 2024-04-20T21:19:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.946+0000 7f5570200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5568152170 0x7f55680c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.946+0000 7f5570200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5568158770 0x7f55680c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.946+0000 7f5570200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568158b80 0x7f5568150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.946+0000 7f5570200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55680c9f20 con 0x7f5568158b80 2024-04-20T21:19:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.946+0000 7f5570200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5568158250 con 0x7f5568158770 2024-04-20T21:19:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.946+0000 7f5570200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5568157f20 con 0x7f5568152170 2024-04-20T21:19:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556d800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5568152170 0x7f55680c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556d800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5568152170 0x7f55680c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:57782/0 (socket says 172.21.15.102:57782) 2024-04-20T21:19:24.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5568158770 0x7f55680c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556d800640 1 -- 172.21.15.102:0/3918868284 learned_addr learned my addr 172.21.15.102:0/3918868284 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:24.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556e200640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568158b80 0x7f5568150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556d800640 1 -- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5568158770 msgr2=0x7f55680c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556d800640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5568158770 0x7f55680c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556d800640 1 -- 172.21.15.102:0/3918868284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568158b80 msgr2=0x7f5568150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556d800640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568158b80 0x7f5568150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556d800640 1 -- 172.21.15.102:0/3918868284 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556815aa00 con 0x7f5568152170 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.947+0000 7f556ce00640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5568158770 0x7f55680c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.948+0000 7f556d800640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5568152170 0x7f55680c8350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f555c02ba70 tx=0x7f555c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.948+0000 7f5565600640 1 -- 172.21.15.102:0/3918868284 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c037d10 con 0x7f5568152170 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.948+0000 7f5565600640 1 -- 172.21.15.102:0/3918868284 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555c046070 con 0x7f5568152170 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.948+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55681f1aa0 con 0x7f5568152170 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.948+0000 7f5565600640 1 -- 172.21.15.102:0/3918868284 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c041640 con 0x7f5568152170 2024-04-20T21:19:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.948+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f556815aca0 con 0x7f5568152170 2024-04-20T21:19:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.949+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5568156cb0 con 0x7f5568152170 2024-04-20T21:19:24.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.950+0000 7f5565600640 1 -- 172.21.15.102:0/3918868284 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f555c040070 con 0x7f5568152170 2024-04-20T21:19:24.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.951+0000 7f5565600640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.951+0000 7f5565600640 1 -- 172.21.15.102:0/3918868284 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f555c0ae590 con 0x7f5568152170 2024-04-20T21:19:24.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.951+0000 7f556ce00640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.952+0000 7f556ce00640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f554806c8f0 0x7f554806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5568164fc0 tx=0x7f5554009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:24.956+0000 7f5565600640 1 -- 172.21.15.102:0/3918868284 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5568156cb0 con 0x7f5568152170 2024-04-20T21:19:25.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.208+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f556815ba20 con 0x7f5568152170 2024-04-20T21:19:25.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.210+0000 7f5565600640 1 -- 172.21.15.102:0/3918868284 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f555c033030 con 0x7f5568152170 2024-04-20T21:19:25.209 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:25.209 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:19:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:19:25.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f554806c8f0 msgr2=0x7f554806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f554806c8f0 0x7f554806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5568164fc0 tx=0x7f5554009040 comp rx=0 tx=0).stop 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5568152170 msgr2=0x7f55680c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5568152170 0x7f55680c8350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f555c02ba70 tx=0x7f555c004730 comp rx=0 tx=0).stop 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 shutdown_connections 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5568158b80 0x7f5568150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f554806c8f0 0x7f554806edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5568158770 0x7f55680c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 --2- 172.21.15.102:0/3918868284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5568152170 0x7f55680c8350 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 >> 172.21.15.102:0/3918868284 conn(0x7f55680c9390 msgr2=0x7f5568070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 shutdown_connections 2024-04-20T21:19:25.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:25.212+0000 7f5570200640 1 -- 172.21.15.102:0/3918868284 wait complete. 2024-04-20T21:19:25.231 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:19:27.233 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:27.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.441+0000 7f661c000640 1 Processor -- start 2024-04-20T21:19:27.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.441+0000 7f661c000640 1 -- start start 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f661c000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6614162e40 0x7f661406c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f661c000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6614163220 0x7f6614159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f661c000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f661406d2e0 con 0x7f6614163660 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f661c000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f66140c7290 con 0x7f6614162e40 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f661c000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f66140c7570 con 0x7f6614163220 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f6619600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6614162e40 0x7f661406c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f6619600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6614162e40 0x7f661406c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33098/0 (socket says 172.21.15.102:33098) 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f6619600640 1 -- 172.21.15.102:0/4115824418 learned_addr learned my addr 172.21.15.102:0/4115824418 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:27.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.443+0000 7f6618c00640 1 --2- 172.21.15.102:0/4115824418 >> v2:172.21.15.193:3300/0 conn(0x7f6614163220 0x7f6614159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.444+0000 7f6619600640 1 -- 172.21.15.102:0/4115824418 >> v2:172.21.15.193:3300/0 conn(0x7f6614163220 msgr2=0x7f6614159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.444+0000 7f6619600640 1 --2- 172.21.15.102:0/4115824418 >> v2:172.21.15.193:3300/0 conn(0x7f6614163220 0x7f6614159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.444+0000 7f6619600640 1 -- 172.21.15.102:0/4115824418 >> v1:172.21.15.193:6789/0 conn(0x7f6614163660 legacy=0x7f66140c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.444+0000 7f6619600640 1 -- 172.21.15.102:0/4115824418 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661406d5c0 con 0x7f6614162e40 2024-04-20T21:19:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.444+0000 7f6618c00640 1 --2- 172.21.15.102:0/4115824418 >> v2:172.21.15.193:3300/0 conn(0x7f6614163220 0x7f6614159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:27.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.445+0000 7f6619600640 1 --2- 172.21.15.102:0/4115824418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614162e40 0x7f661406c9e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6614070780 tx=0x7f660802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bfe4b639849d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:27.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.446+0000 7f6613e00640 1 -- 172.21.15.102:0/4115824418 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660802f850 con 0x7f6614162e40 2024-04-20T21:19:27.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.446+0000 7f6613e00640 1 -- 172.21.15.102:0/4115824418 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660802f9f0 con 0x7f6614162e40 2024-04-20T21:19:27.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.446+0000 7f6613e00640 1 -- 172.21.15.102:0/4115824418 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660802fdc0 con 0x7f6614162e40 2024-04-20T21:19:27.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.447+0000 7f661c000640 1 -- 172.21.15.102:0/4115824418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614162e40 msgr2=0x7f661406c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:27.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.447+0000 7f661c000640 1 --2- 172.21.15.102:0/4115824418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614162e40 0x7f661406c9e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6614070780 tx=0x7f660802ba30 comp rx=0 tx=0).stop 2024-04-20T21:19:27.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.447+0000 7f661c000640 1 -- 172.21.15.102:0/4115824418 shutdown_connections 2024-04-20T21:19:27.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.447+0000 7f661c000640 1 --2- 172.21.15.102:0/4115824418 >> v2:172.21.15.193:3300/0 conn(0x7f6614163220 0x7f6614159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.447+0000 7f661c000640 1 --2- 172.21.15.102:0/4115824418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614162e40 0x7f661406c9e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.447+0000 7f661c000640 1 -- 172.21.15.102:0/4115824418 >> 172.21.15.102:0/4115824418 conn(0x7f66140c9390 msgr2=0x7f66140c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:27.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.447+0000 7f661c000640 1 -- 172.21.15.102:0/4115824418 shutdown_connections 2024-04-20T21:19:27.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.447+0000 7f661c000640 1 -- 172.21.15.102:0/4115824418 wait complete. 2024-04-20T21:19:27.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.449+0000 7f661c000640 1 Processor -- start 2024-04-20T21:19:27.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.449+0000 7f661c000640 1 -- start start 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.450+0000 7f661c000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6614162e40 0x7f66140cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.450+0000 7f661c000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614163220 0x7f66140c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.450+0000 7f661c000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6614163660 0x7f66141f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.450+0000 7f661c000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f66140c9080 con 0x7f6614163220 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.450+0000 7f661c000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f661406db80 con 0x7f6614162e40 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.450+0000 7f661c000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f661406cf20 con 0x7f6614163660 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6614162e40 0x7f66140cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6618c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614163220 0x7f66140c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6614162e40 0x7f66140cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37348/0 (socket says 172.21.15.102:37348) 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 -- 172.21.15.102:0/2945037523 learned_addr learned my addr 172.21.15.102:0/2945037523 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:27.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f661a000640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6614163660 0x7f66141f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:27.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 -- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6614163660 msgr2=0x7f66141f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:27.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6614163660 0x7f66141f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 -- 172.21.15.102:0/2945037523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614163220 msgr2=0x7f66140c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:27.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614163220 0x7f66140c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 -- 172.21.15.102:0/2945037523 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6614161d80 con 0x7f6614162e40 2024-04-20T21:19:27.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.451+0000 7f6619600640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6614162e40 0x7f66140cb950 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6608009960 tx=0x7f6608004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:27.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.452+0000 7f6611600640 1 -- 172.21.15.102:0/2945037523 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608047070 con 0x7f6614162e40 2024-04-20T21:19:27.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.452+0000 7f6611600640 1 -- 172.21.15.102:0/2945037523 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6608037d30 con 0x7f6614162e40 2024-04-20T21:19:27.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.452+0000 7f6611600640 1 -- 172.21.15.102:0/2945037523 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66080426c0 con 0x7f6614162e40 2024-04-20T21:19:27.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.452+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6614162060 con 0x7f6614162e40 2024-04-20T21:19:27.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.452+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6614162620 con 0x7f6614162e40 2024-04-20T21:19:27.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.453+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6614074340 con 0x7f6614162e40 2024-04-20T21:19:27.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.454+0000 7f6611600640 1 -- 172.21.15.102:0/2945037523 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f660804a070 con 0x7f6614162e40 2024-04-20T21:19:27.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.455+0000 7f6611600640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f65f406c8f0 0x7f65f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:27.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.455+0000 7f6611600640 1 -- 172.21.15.102:0/2945037523 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f66080ae2a0 con 0x7f6614162e40 2024-04-20T21:19:27.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.455+0000 7f6618c00640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f65f406c8f0 0x7f65f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:27.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.456+0000 7f6618c00640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f65f406c8f0 0x7f65f406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f66141536a0 tx=0x7f6600009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:27.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.460+0000 7f6611600640 1 -- 172.21.15.102:0/2945037523 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6608033030 con 0x7f6614162e40 2024-04-20T21:19:27.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.700+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f661414d4b0 con 0x7f6614162e40 2024-04-20T21:19:27.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.702+0000 7f6611600640 1 -- 172.21.15.102:0/2945037523 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f660807bd00 con 0x7f6614162e40 2024-04-20T21:19:27.701 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:27.702 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:19:27.702 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f65f406c8f0 msgr2=0x7f65f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f65f406c8f0 0x7f65f406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f66141536a0 tx=0x7f6600009190 comp rx=0 tx=0).stop 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6614162e40 msgr2=0x7f66140cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6614162e40 0x7f66140cb950 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6608009960 tx=0x7f6608004730 comp rx=0 tx=0).stop 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 shutdown_connections 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6614163660 0x7f66141f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6614163220 0x7f66140c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f65f406c8f0 0x7f65f406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 --2- 172.21.15.102:0/2945037523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6614162e40 0x7f66140cb950 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:27.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 >> 172.21.15.102:0/2945037523 conn(0x7f66140c9390 msgr2=0x7f66141589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:27.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 shutdown_connections 2024-04-20T21:19:27.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:27.705+0000 7f661c000640 1 -- 172.21.15.102:0/2945037523 wait complete. 2024-04-20T21:19:27.726 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:19:29.726 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:29.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.937+0000 7fa0b3200640 1 Processor -- start 2024-04-20T21:19:29.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.937+0000 7fa0b3200640 1 -- start start 2024-04-20T21:19:29.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.938+0000 7fa0b3200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa0b41516d0 0x7fa0b40cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:29.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.938+0000 7fa0b3200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa0b4157ff0 0x7fa0b40c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:29.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b3200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b4158420 con 0x7fa0b4151ab0 2024-04-20T21:19:29.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b3200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b40c4dd0 con 0x7fa0b4157ff0 2024-04-20T21:19:29.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b3200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b406ef60 con 0x7fa0b41516d0 2024-04-20T21:19:29.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b2800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa0b4157ff0 0x7fa0b40c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:29.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b2800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa0b4157ff0 0x7fa0b40c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50192/0 (socket says 172.21.15.102:50192) 2024-04-20T21:19:29.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b2800640 1 -- 172.21.15.102:0/3546780434 learned_addr learned my addr 172.21.15.102:0/3546780434 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:29.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b1e00640 1 --2- 172.21.15.102:0/3546780434 >> v2:172.21.15.193:3300/0 conn(0x7fa0b41516d0 0x7fa0b40cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:29.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b2800640 1 -- 172.21.15.102:0/3546780434 >> v2:172.21.15.193:3300/0 conn(0x7fa0b41516d0 msgr2=0x7fa0b40cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:29.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b2800640 1 --2- 172.21.15.102:0/3546780434 >> v2:172.21.15.193:3300/0 conn(0x7fa0b41516d0 0x7fa0b40cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:29.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b2800640 1 -- 172.21.15.102:0/3546780434 >> v1:172.21.15.193:6789/0 conn(0x7fa0b4151ab0 legacy=0x7fa0b40c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:29.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b0a00640 1 -- 172.21.15.102:0/3546780434 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663345444 0 0) 0x7fa0b4158420 con 0x7fa0b4151ab0 2024-04-20T21:19:29.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.939+0000 7fa0b2800640 1 -- 172.21.15.102:0/3546780434 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b406f240 con 0x7fa0b4157ff0 2024-04-20T21:19:29.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.940+0000 7fa0b2800640 1 --2- 172.21.15.102:0/3546780434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4157ff0 0x7fa0b40c5820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40ca3e0 tx=0x7fa0ac02e210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71efce75a5f98b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:29.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.941+0000 7fa0b0a00640 1 -- 172.21.15.102:0/3546780434 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00b0d0 con 0x7fa0b4157ff0 2024-04-20T21:19:29.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.941+0000 7fa0b0a00640 1 -- 172.21.15.102:0/3546780434 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac00b270 con 0x7fa0b4157ff0 2024-04-20T21:19:29.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.941+0000 7fa0b0a00640 1 -- 172.21.15.102:0/3546780434 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac009600 con 0x7fa0b4157ff0 2024-04-20T21:19:29.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.941+0000 7fa0b3200640 1 -- 172.21.15.102:0/3546780434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4157ff0 msgr2=0x7fa0b40c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:29.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.941+0000 7fa0b3200640 1 --2- 172.21.15.102:0/3546780434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4157ff0 0x7fa0b40c5820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40ca3e0 tx=0x7fa0ac02e210 comp rx=0 tx=0).stop 2024-04-20T21:19:29.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.941+0000 7fa0b3200640 1 -- 172.21.15.102:0/3546780434 shutdown_connections 2024-04-20T21:19:29.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.941+0000 7fa0b3200640 1 --2- 172.21.15.102:0/3546780434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4157ff0 0x7fa0b40c5820 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:29.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.941+0000 7fa0b3200640 1 --2- 172.21.15.102:0/3546780434 >> v2:172.21.15.193:3300/0 conn(0x7fa0b41516d0 0x7fa0b40cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:29.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.942+0000 7fa0b3200640 1 -- 172.21.15.102:0/3546780434 >> 172.21.15.102:0/3546780434 conn(0x7fa0b40cc820 msgr2=0x7fa0b40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:29.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.942+0000 7fa0b3200640 1 -- 172.21.15.102:0/3546780434 shutdown_connections 2024-04-20T21:19:29.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.942+0000 7fa0b3200640 1 -- 172.21.15.102:0/3546780434 wait complete. 2024-04-20T21:19:29.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.943+0000 7fa0b3200640 1 Processor -- start 2024-04-20T21:19:29.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.943+0000 7fa0b3200640 1 -- start start 2024-04-20T21:19:29.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.944+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0b41516d0 0x7fa0b4153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:29.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.944+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4151ab0 0x7fa0b40cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:29.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.944+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0b4157ff0 0x7fa0b4156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:29.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.944+0000 7fa0b3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40c5300 con 0x7fa0b4151ab0 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.944+0000 7fa0b3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40c5020 con 0x7fa0b4157ff0 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.944+0000 7fa0b3200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b40c4a40 con 0x7fa0b41516d0 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0b41516d0 0x7fa0b4153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4151ab0 0x7fa0b40cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0b41516d0 0x7fa0b4153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:43086/0 (socket says 172.21.15.102:43086) 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 -- 172.21.15.102:0/688899486 learned_addr learned my addr 172.21.15.102:0/688899486 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b2800640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0b4157ff0 0x7fa0b4156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 -- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0b4157ff0 msgr2=0x7fa0b4156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0b4157ff0 0x7fa0b4156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 -- 172.21.15.102:0/688899486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4151ab0 msgr2=0x7fa0b40cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4151ab0 0x7fa0b40cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 -- 172.21.15.102:0/688899486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b41f43b0 con 0x7fa0b41516d0 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1400640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4151ab0 0x7fa0b40cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:29.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.945+0000 7fa0b1e00640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0b41516d0 0x7fa0b4153050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400e9b0 tx=0x7fa0a400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:29.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.946+0000 7fa0aa000640 1 -- 172.21.15.102:0/688899486 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4017070 con 0x7fa0b41516d0 2024-04-20T21:19:29.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.946+0000 7fa0aa000640 1 -- 172.21.15.102:0/688899486 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a400c9a0 con 0x7fa0b41516d0 2024-04-20T21:19:29.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.946+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b41f4690 con 0x7fa0b41516d0 2024-04-20T21:19:29.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.946+0000 7fa0aa000640 1 -- 172.21.15.102:0/688899486 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400cd90 con 0x7fa0b41516d0 2024-04-20T21:19:29.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.946+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41f4c50 con 0x7fa0b41516d0 2024-04-20T21:19:29.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.947+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b404fd60 con 0x7fa0b41516d0 2024-04-20T21:19:29.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.948+0000 7fa0aa000640 1 -- 172.21.15.102:0/688899486 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0a401a070 con 0x7fa0b41516d0 2024-04-20T21:19:29.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.949+0000 7fa0aa000640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa09406c920 0x7fa09406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:29.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.949+0000 7fa0aa000640 1 -- 172.21.15.102:0/688899486 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa0a408c240 con 0x7fa0b41516d0 2024-04-20T21:19:29.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.949+0000 7fa0b1400640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa09406c920 0x7fa09406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:29.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.950+0000 7fa0b1400640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa09406c920 0x7fa09406ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4157fc0 tx=0x7fa0a0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:29.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:29.954+0000 7fa0aa000640 1 -- 172.21.15.102:0/688899486 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa0a4059c10 con 0x7fa0b41516d0 2024-04-20T21:19:30.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.192+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0b406dcb0 con 0x7fa0b41516d0 2024-04-20T21:19:30.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.193+0000 7fa0aa000640 1 -- 172.21.15.102:0/688899486 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7fa0a405df10 con 0x7fa0b41516d0 2024-04-20T21:19:30.191 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:30.192 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:19:30.192 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:19:30.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa09406c920 msgr2=0x7fa09406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:30.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa09406c920 0x7fa09406ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4157fc0 tx=0x7fa0a0009040 comp rx=0 tx=0).stop 2024-04-20T21:19:30.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0b41516d0 msgr2=0x7fa0b4153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0b41516d0 0x7fa0b4153050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400e9b0 tx=0x7fa0a400ee70 comp rx=0 tx=0).stop 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 shutdown_connections 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa09406c920 0x7fa09406ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0b4157ff0 0x7fa0b4156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0b4151ab0 0x7fa0b40cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 --2- 172.21.15.102:0/688899486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0b41516d0 0x7fa0b4153050 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 >> 172.21.15.102:0/688899486 conn(0x7fa0b40cc820 msgr2=0x7fa0b40c6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 shutdown_connections 2024-04-20T21:19:30.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:30.196+0000 7fa0b3200640 1 -- 172.21.15.102:0/688899486 wait complete. 2024-04-20T21:19:30.214 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:19:32.216 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:32.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.421+0000 7f8bd7c00640 1 Processor -- start 2024-04-20T21:19:32.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.421+0000 7f8bd7c00640 1 -- start start 2024-04-20T21:19:32.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd7c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0162e40 0x7f8bd006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd7c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8bd0163220 0x7f8bd0159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd7c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8bd0163660 0x7f8bd0159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd7c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd00c7290 con 0x7f8bd0163220 2024-04-20T21:19:32.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd7c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd00c7570 con 0x7f8bd0163660 2024-04-20T21:19:32.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd7c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd006d2e0 con 0x7f8bd0162e40 2024-04-20T21:19:32.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bcfe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8bd0163220 0x7f8bd0159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bcfe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8bd0163220 0x7f8bd0159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50206/0 (socket says 172.21.15.102:50206) 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bcfe00640 1 -- 172.21.15.102:0/3602528950 learned_addr learned my addr 172.21.15.102:0/3602528950 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd5200640 1 --2- 172.21.15.102:0/3602528950 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0162e40 0x7f8bd006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd5c00640 1 --2- 172.21.15.102:0/3602528950 >> v2:172.21.15.193:3300/0 conn(0x7f8bd0163660 0x7f8bd0159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd5200640 1 -- 172.21.15.102:0/3602528950 >> v2:172.21.15.193:3300/0 conn(0x7f8bd0163660 msgr2=0x7f8bd0159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.423+0000 7f8bd5200640 1 --2- 172.21.15.102:0/3602528950 >> v2:172.21.15.193:3300/0 conn(0x7f8bd0163660 0x7f8bd0159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.424+0000 7f8bd5200640 1 -- 172.21.15.102:0/3602528950 >> v2:172.21.15.102:3300/0 conn(0x7f8bd0163220 msgr2=0x7f8bd0159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.424+0000 7f8bd5200640 1 --2- 172.21.15.102:0/3602528950 >> v2:172.21.15.102:3300/0 conn(0x7f8bd0163220 0x7f8bd0159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.424+0000 7f8bd5200640 1 -- 172.21.15.102:0/3602528950 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd006d5c0 con 0x7f8bd0162e40 2024-04-20T21:19:32.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.424+0000 7f8bcfe00640 1 --2- 172.21.15.102:0/3602528950 >> v2:172.21.15.102:3300/0 conn(0x7f8bd0163220 0x7f8bd0159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.425+0000 7f8bd5200640 1 --2- 172.21.15.102:0/3602528950 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0162e40 0x7f8bd006c9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bd006f970 tx=0x7f8bc402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa17fc2e70fd6c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.425+0000 7f8bcf400640 1 -- 172.21.15.102:0/3602528950 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40315a0 con 0x7f8bd0162e40 2024-04-20T21:19:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.425+0000 7f8bcf400640 1 -- 172.21.15.102:0/3602528950 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc4037070 con 0x7f8bd0162e40 2024-04-20T21:19:32.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.425+0000 7f8bcf400640 1 -- 172.21.15.102:0/3602528950 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4031960 con 0x7f8bd0162e40 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 -- 172.21.15.102:0/3602528950 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0162e40 msgr2=0x7f8bd006c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/3602528950 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0162e40 0x7f8bd006c9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bd006f970 tx=0x7f8bc402b840 comp rx=0 tx=0).stop 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 -- 172.21.15.102:0/3602528950 shutdown_connections 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/3602528950 >> v2:172.21.15.193:3300/0 conn(0x7f8bd0163660 0x7f8bd0159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/3602528950 >> v2:172.21.15.102:3300/0 conn(0x7f8bd0163220 0x7f8bd0159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/3602528950 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0162e40 0x7f8bd006c9e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 -- 172.21.15.102:0/3602528950 >> 172.21.15.102:0/3602528950 conn(0x7f8bd00c9390 msgr2=0x7f8bd00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 -- 172.21.15.102:0/3602528950 shutdown_connections 2024-04-20T21:19:32.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.426+0000 7f8bd7c00640 1 -- 172.21.15.102:0/3602528950 wait complete. 2024-04-20T21:19:32.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.428+0000 7f8bd7c00640 1 Processor -- start 2024-04-20T21:19:32.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.428+0000 7f8bd7c00640 1 -- start start 2024-04-20T21:19:32.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.429+0000 7f8bd7c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bd0162e40 0x7f8bd015ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.429+0000 7f8bd7c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0163220 0x7f8bd015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.429+0000 7f8bd7c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd0163660 0x7f8bd01f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.429+0000 7f8bd7c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00c9080 con 0x7f8bd0163660 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.429+0000 7f8bd7c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd006db80 con 0x7f8bd0162e40 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.429+0000 7f8bd7c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd006cf20 con 0x7f8bd0163220 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bd0162e40 0x7f8bd015ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd0163660 0x7f8bd01f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bd0162e40 0x7f8bd015ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:38288/0 (socket says 172.21.15.102:38288) 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 -- 172.21.15.102:0/914036742 learned_addr learned my addr 172.21.15.102:0/914036742 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bcfe00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0163220 0x7f8bd015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 -- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0163220 msgr2=0x7f8bd015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0163220 0x7f8bd015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 -- 172.21.15.102:0/914036742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd0163660 msgr2=0x7f8bd01f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd0163660 0x7f8bd01f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 -- 172.21.15.102:0/914036742 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd01f5fd0 con 0x7f8bd0162e40 2024-04-20T21:19:32.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5c00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd0163660 0x7f8bd01f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:32.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.430+0000 7f8bd5200640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bd0162e40 0x7f8bd015ff80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4000c00 tx=0x7f8bc40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:32.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.431+0000 7f8bccc00640 1 -- 172.21.15.102:0/914036742 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4032630 con 0x7f8bd0162e40 2024-04-20T21:19:32.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.431+0000 7f8bccc00640 1 -- 172.21.15.102:0/914036742 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc40327f0 con 0x7f8bd0162e40 2024-04-20T21:19:32.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.431+0000 7f8bccc00640 1 -- 172.21.15.102:0/914036742 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4032be0 con 0x7f8bd0162e40 2024-04-20T21:19:32.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.431+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd015c620 con 0x7f8bd0162e40 2024-04-20T21:19:32.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.431+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd015ced0 con 0x7f8bd0162e40 2024-04-20T21:19:32.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.432+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd0074340 con 0x7f8bd0162e40 2024-04-20T21:19:32.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.433+0000 7f8bccc00640 1 -- 172.21.15.102:0/914036742 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bc404d3f0 con 0x7f8bd0162e40 2024-04-20T21:19:32.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.434+0000 7f8bccc00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bb006c8f0 0x7f8bb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.434+0000 7f8bccc00640 1 -- 172.21.15.102:0/914036742 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8bc402f030 con 0x7f8bd0162e40 2024-04-20T21:19:32.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.434+0000 7f8bcfe00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bb006c8f0 0x7f8bb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.435+0000 7f8bcfe00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bb006c8f0 0x7f8bb006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0072ea0 tx=0x7f8bbc009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:32.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.438+0000 7f8bccc00640 1 -- 172.21.15.102:0/914036742 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8bc4040070 con 0x7f8bd0162e40 2024-04-20T21:19:32.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.680+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd006f150 con 0x7f8bd0162e40 2024-04-20T21:19:32.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.681+0000 7f8bccc00640 1 -- 172.21.15.102:0/914036742 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f8bc407b9d0 con 0x7f8bd0162e40 2024-04-20T21:19:32.681 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:32.682 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:19:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:19:32.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bb006c8f0 msgr2=0x7f8bb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bb006c8f0 0x7f8bb006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0072ea0 tx=0x7f8bbc009240 comp rx=0 tx=0).stop 2024-04-20T21:19:32.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bd0162e40 msgr2=0x7f8bd015ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bd0162e40 0x7f8bd015ff80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4000c00 tx=0x7f8bc40046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 shutdown_connections 2024-04-20T21:19:32.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bd0163660 0x7f8bd01f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bd0163220 0x7f8bd015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bb006c8f0 0x7f8bb006edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 --2- 172.21.15.102:0/914036742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bd0162e40 0x7f8bd015ff80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 >> 172.21.15.102:0/914036742 conn(0x7f8bd00c9390 msgr2=0x7f8bd0157a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:32.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 shutdown_connections 2024-04-20T21:19:32.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:32.684+0000 7f8bd7c00640 1 -- 172.21.15.102:0/914036742 wait complete. 2024-04-20T21:19:32.703 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:19:34.704 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:34.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.919+0000 7fc93c200640 1 Processor -- start 2024-04-20T21:19:34.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.919+0000 7fc93c200640 1 -- start start 2024-04-20T21:19:34.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.920+0000 7fc93c200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc93415ea20 0x7fc93406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:34.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.920+0000 7fc93c200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ee00 0x7fc9340710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:34.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.920+0000 7fc93c200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9340c8f20 con 0x7fc93415f2b0 2024-04-20T21:19:34.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.920+0000 7fc93c200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc93406cb80 con 0x7fc93415ea20 2024-04-20T21:19:34.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.920+0000 7fc93c200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc93406ce60 con 0x7fc93415ee00 2024-04-20T21:19:34.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc939800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc93415ea20 0x7fc93406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:34.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc939800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc93415ea20 0x7fc93406d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50226/0 (socket says 172.21.15.102:50226) 2024-04-20T21:19:34.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc939800640 1 -- 172.21.15.102:0/2926645194 learned_addr learned my addr 172.21.15.102:0/2926645194 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:34.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc938e00640 1 --2- 172.21.15.102:0/2926645194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ee00 0x7fc9340710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:34.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc939800640 1 -- 172.21.15.102:0/2926645194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ee00 msgr2=0x7fc9340710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:34.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc939800640 1 --2- 172.21.15.102:0/2926645194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ee00 0x7fc9340710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:34.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc939800640 1 -- 172.21.15.102:0/2926645194 >> v1:172.21.15.193:6789/0 conn(0x7fc93415f2b0 legacy=0x7fc9340c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:34.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc933e00640 1 -- 172.21.15.102:0/2926645194 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537558252 0 0) 0x7fc9340c8f20 con 0x7fc93415f2b0 2024-04-20T21:19:34.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.921+0000 7fc939800640 1 -- 172.21.15.102:0/2926645194 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93406d140 con 0x7fc93415ea20 2024-04-20T21:19:34.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.923+0000 7fc939800640 1 --2- 172.21.15.102:0/2926645194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415ea20 0x7fc93406d940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc928009870 tx=0x7fc92802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c95ee2a1c37072c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:34.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.923+0000 7fc933e00640 1 -- 172.21.15.102:0/2926645194 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92802f840 con 0x7fc93415ea20 2024-04-20T21:19:34.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.923+0000 7fc933e00640 1 -- 172.21.15.102:0/2926645194 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92802f9e0 con 0x7fc93415ea20 2024-04-20T21:19:34.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.923+0000 7fc933e00640 1 -- 172.21.15.102:0/2926645194 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92802fdb0 con 0x7fc93415ea20 2024-04-20T21:19:34.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.924+0000 7fc93c200640 1 -- 172.21.15.102:0/2926645194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415ea20 msgr2=0x7fc93406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:34.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.924+0000 7fc93c200640 1 --2- 172.21.15.102:0/2926645194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415ea20 0x7fc93406d940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc928009870 tx=0x7fc92802ba20 comp rx=0 tx=0).stop 2024-04-20T21:19:34.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.924+0000 7fc93c200640 1 -- 172.21.15.102:0/2926645194 shutdown_connections 2024-04-20T21:19:34.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.924+0000 7fc93c200640 1 --2- 172.21.15.102:0/2926645194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ee00 0x7fc9340710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:34.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.924+0000 7fc93c200640 1 --2- 172.21.15.102:0/2926645194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415ea20 0x7fc93406d940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:34.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.924+0000 7fc93c200640 1 -- 172.21.15.102:0/2926645194 >> 172.21.15.102:0/2926645194 conn(0x7fc9340c9390 msgr2=0x7fc9340c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:34.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.924+0000 7fc93c200640 1 -- 172.21.15.102:0/2926645194 shutdown_connections 2024-04-20T21:19:34.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.924+0000 7fc93c200640 1 -- 172.21.15.102:0/2926645194 wait complete. 2024-04-20T21:19:34.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.926+0000 7fc93c200640 1 Processor -- start 2024-04-20T21:19:34.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.926+0000 7fc93c200640 1 -- start start 2024-04-20T21:19:34.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc93c200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ea20 0x7fc93415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:34.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc93c200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93415ee00 0x7fc9340cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:34.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc93c200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415f2b0 0x7fc93415c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:34.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc93c200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340c8c80 con 0x7fc93415f2b0 2024-04-20T21:19:34.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc93c200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340c8b00 con 0x7fc93415ee00 2024-04-20T21:19:34.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc93c200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc934154e80 con 0x7fc93415ea20 2024-04-20T21:19:34.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc939800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ea20 0x7fc93415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:34.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc939800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ea20 0x7fc93415bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:43118/0 (socket says 172.21.15.102:43118) 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc938e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93415ee00 0x7fc9340cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc939800640 1 -- 172.21.15.102:0/3737682917 learned_addr learned my addr 172.21.15.102:0/3737682917 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc93a200640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415f2b0 0x7fc93415c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc939800640 1 -- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93415ee00 msgr2=0x7fc9340cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc939800640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93415ee00 0x7fc9340cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc939800640 1 -- 172.21.15.102:0/3737682917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415f2b0 msgr2=0x7fc93415c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc939800640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415f2b0 0x7fc93415c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.927+0000 7fc939800640 1 -- 172.21.15.102:0/3737682917 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9341f1070 con 0x7fc93415ea20 2024-04-20T21:19:34.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.928+0000 7fc939800640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ea20 0x7fc93415bbf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc928009870 tx=0x7fc928004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:34.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.928+0000 7fc931600640 1 -- 172.21.15.102:0/3737682917 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928037bc0 con 0x7fc93415ea20 2024-04-20T21:19:34.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.928+0000 7fc931600640 1 -- 172.21.15.102:0/3737682917 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc928046070 con 0x7fc93415ea20 2024-04-20T21:19:34.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.928+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93415ae30 con 0x7fc93415ea20 2024-04-20T21:19:34.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.928+0000 7fc931600640 1 -- 172.21.15.102:0/3737682917 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928041400 con 0x7fc93415ea20 2024-04-20T21:19:34.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.928+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93415b3f0 con 0x7fc93415ea20 2024-04-20T21:19:34.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.929+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc934074340 con 0x7fc93415ea20 2024-04-20T21:19:34.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.930+0000 7fc931600640 1 -- 172.21.15.102:0/3737682917 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc928040070 con 0x7fc93415ea20 2024-04-20T21:19:34.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.931+0000 7fc931600640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc91406c8f0 0x7fc91406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:34.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.931+0000 7fc931600640 1 -- 172.21.15.102:0/3737682917 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc9280aeff0 con 0x7fc93415ea20 2024-04-20T21:19:34.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.932+0000 7fc938e00640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc91406c8f0 0x7fc91406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:34.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.932+0000 7fc938e00640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc91406c8f0 0x7fc91406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc93414af60 tx=0x7fc920009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:34.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:34.935+0000 7fc931600640 1 -- 172.21.15.102:0/3737682917 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc928033030 con 0x7fc93415ea20 2024-04-20T21:19:35.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.181+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc93414bbc0 con 0x7fc93415ea20 2024-04-20T21:19:35.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.182+0000 7fc931600640 1 -- 172.21.15.102:0/3737682917 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7fc92807c960 con 0x7fc93415ea20 2024-04-20T21:19:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:35.182 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:19:35.183 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:19:35.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.184+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc91406c8f0 msgr2=0x7fc91406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:35.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc91406c8f0 0x7fc91406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc93414af60 tx=0x7fc920009190 comp rx=0 tx=0).stop 2024-04-20T21:19:35.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ea20 msgr2=0x7fc93415bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:35.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ea20 0x7fc93415bbf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc928009870 tx=0x7fc928004730 comp rx=0 tx=0).stop 2024-04-20T21:19:35.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 shutdown_connections 2024-04-20T21:19:35.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc93415f2b0 0x7fc93415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:35.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc91406c8f0 0x7fc91406edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:35.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93415ee00 0x7fc9340cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:35.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 --2- 172.21.15.102:0/3737682917 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc93415ea20 0x7fc93415bbf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:35.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 >> 172.21.15.102:0/3737682917 conn(0x7fc9340c9390 msgr2=0x7fc93414fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:35.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 shutdown_connections 2024-04-20T21:19:35.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:35.185+0000 7fc93c200640 1 -- 172.21.15.102:0/3737682917 wait complete. 2024-04-20T21:19:35.204 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:19:37.206 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:37.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.417+0000 7f2a56a00640 1 Processor -- start 2024-04-20T21:19:37.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.417+0000 7f2a56a00640 1 -- start start 2024-04-20T21:19:37.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a56a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a50070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:37.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a56a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a50071330 0x7f2a501628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:37.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a56a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a50158e80 con 0x7f2a50152170 2024-04-20T21:19:37.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a56a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f2a500c6dc0 con 0x7f2a50071330 2024-04-20T21:19:37.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a56a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2a50071f70 con 0x7f2a500c8db0 2024-04-20T21:19:37.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a4fe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f2a50152170 0x7f2a50158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:50628/0 (socket says 172.21.15.102:50628) 2024-04-20T21:19:37.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a4fe00640 1 -- 172.21.15.102:0/2418337125 learned_addr learned my addr 172.21.15.102:0/2418337125 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:37.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a4f400640 1 --2- 172.21.15.102:0/2418337125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a50070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:37.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.419+0000 7f2a54a00640 1 --2- 172.21.15.102:0/2418337125 >> v2:172.21.15.102:3300/0 conn(0x7f2a50071330 0x7f2a501628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:37.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.420+0000 7f2a4ea00640 1 -- 172.21.15.102:0/2418337125 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326015762 0 0) 0x7f2a50158e80 con 0x7f2a50152170 2024-04-20T21:19:37.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.420+0000 7f2a4ea00640 1 -- 172.21.15.102:0/2418337125 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a34005730 con 0x7f2a50152170 2024-04-20T21:19:37.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.420+0000 7f2a4f400640 1 -- 172.21.15.102:0/2418337125 >> v2:172.21.15.102:3300/0 conn(0x7f2a50071330 msgr2=0x7f2a501628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:37.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.420+0000 7f2a4f400640 1 --2- 172.21.15.102:0/2418337125 >> v2:172.21.15.102:3300/0 conn(0x7f2a50071330 0x7f2a501628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.420+0000 7f2a4f400640 1 -- 172.21.15.102:0/2418337125 >> v1:172.21.15.193:6789/0 conn(0x7f2a50152170 legacy=0x7f2a50158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:37.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.420+0000 7f2a4f400640 1 -- 172.21.15.102:0/2418337125 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a50072210 con 0x7f2a500c8db0 2024-04-20T21:19:37.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.420+0000 7f2a54a00640 1 --2- 172.21.15.102:0/2418337125 >> v2:172.21.15.102:3300/0 conn(0x7f2a50071330 0x7f2a501628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:37.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.421+0000 7f2a4f400640 1 --2- 172.21.15.102:0/2418337125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a50070df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0097a0 tx=0x7f2a3c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302693d728ce75d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.421+0000 7f2a4ea00640 1 -- 172.21.15.102:0/2418337125 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c035020 con 0x7f2a500c8db0 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.421+0000 7f2a4ea00640 1 -- 172.21.15.102:0/2418337125 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3c031810 con 0x7f2a500c8db0 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.421+0000 7f2a4ea00640 1 -- 172.21.15.102:0/2418337125 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c031be0 con 0x7f2a500c8db0 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.422+0000 7f2a56a00640 1 -- 172.21.15.102:0/2418337125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 msgr2=0x7f2a50070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.422+0000 7f2a56a00640 1 --2- 172.21.15.102:0/2418337125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a50070df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0097a0 tx=0x7f2a3c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.422+0000 7f2a56a00640 1 -- 172.21.15.102:0/2418337125 shutdown_connections 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.422+0000 7f2a56a00640 1 --2- 172.21.15.102:0/2418337125 >> v2:172.21.15.102:3300/0 conn(0x7f2a50071330 0x7f2a501628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.422+0000 7f2a56a00640 1 --2- 172.21.15.102:0/2418337125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a50070df0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.422+0000 7f2a56a00640 1 -- 172.21.15.102:0/2418337125 >> 172.21.15.102:0/2418337125 conn(0x7f2a500c9390 msgr2=0x7f2a500c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.422+0000 7f2a56a00640 1 -- 172.21.15.102:0/2418337125 shutdown_connections 2024-04-20T21:19:37.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.422+0000 7f2a56a00640 1 -- 172.21.15.102:0/2418337125 wait complete. 2024-04-20T21:19:37.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.424+0000 7f2a56a00640 1 Processor -- start 2024-04-20T21:19:37.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.424+0000 7f2a56a00640 1 -- start start 2024-04-20T21:19:37.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a56a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a50071330 0x7f2a500c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a56a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a500c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a56a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a50152170 0x7f2a500c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a56a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500c9190 con 0x7f2a50152170 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a56a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500724b0 con 0x7f2a50071330 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a56a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2a50158250 con 0x7f2a500c8db0 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a54a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a50152170 0x7f2a500c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a54a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a50152170 0x7f2a500c2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50246/0 (socket says 172.21.15.102:50246) 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a54a00640 1 -- 172.21.15.102:0/485393998 learned_addr learned my addr 172.21.15.102:0/485393998 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a4f400640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a500c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a4fe00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a50071330 0x7f2a500c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a54a00640 1 -- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 msgr2=0x7f2a500c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:37.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a54a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a500c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a54a00640 1 -- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a50071330 msgr2=0x7f2a500c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:37.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a54a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a50071330 0x7f2a500c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.425+0000 7f2a54a00640 1 -- 172.21.15.102:0/485393998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a501568d0 con 0x7f2a50152170 2024-04-20T21:19:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.426+0000 7f2a54a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a50152170 0x7f2a500c2ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009f00 tx=0x7f2a4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.426+0000 7f2a47e00640 1 -- 172.21.15.102:0/485393998 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48018030 con 0x7f2a50152170 2024-04-20T21:19:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.426+0000 7f2a47e00640 1 -- 172.21.15.102:0/485393998 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4800ce50 con 0x7f2a50152170 2024-04-20T21:19:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.426+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a50156bb0 con 0x7f2a50152170 2024-04-20T21:19:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.426+0000 7f2a47e00640 1 -- 172.21.15.102:0/485393998 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48003bb0 con 0x7f2a50152170 2024-04-20T21:19:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.426+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a50157170 con 0x7f2a50152170 2024-04-20T21:19:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.427+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a500c9f20 con 0x7f2a50152170 2024-04-20T21:19:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.428+0000 7f2a47e00640 1 -- 172.21.15.102:0/485393998 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a4801b070 con 0x7f2a50152170 2024-04-20T21:19:37.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.429+0000 7f2a47e00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:37.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.429+0000 7f2a47e00640 1 -- 172.21.15.102:0/485393998 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2a4808c920 con 0x7f2a50152170 2024-04-20T21:19:37.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.429+0000 7f2a4fe00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:37.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.430+0000 7f2a4fe00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a3006c8f0 0x7f2a3006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a40005880 tx=0x7f2a40009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:37.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.433+0000 7f2a47e00640 1 -- 172.21.15.102:0/485393998 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a500c9f20 con 0x7f2a50152170 2024-04-20T21:19:37.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.678+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a500c7c60 con 0x7f2a50152170 2024-04-20T21:19:37.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.679+0000 7f2a47e00640 1 -- 172.21.15.102:0/485393998 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f2a500c7c60 con 0x7f2a50152170 2024-04-20T21:19:37.679 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:37.679 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:19:37.679 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:19:37.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a3006c8f0 msgr2=0x7f2a3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:37.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a3006c8f0 0x7f2a3006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a40005880 tx=0x7f2a40009040 comp rx=0 tx=0).stop 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a50152170 msgr2=0x7f2a500c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a50152170 0x7f2a500c2ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009f00 tx=0x7f2a4800c490 comp rx=0 tx=0).stop 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 shutdown_connections 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a50152170 0x7f2a500c2ab0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a500c8db0 0x7f2a500c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 --2- 172.21.15.102:0/485393998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a50071330 0x7f2a500c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 >> 172.21.15.102:0/485393998 conn(0x7f2a500c9390 msgr2=0x7f2a500c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 shutdown_connections 2024-04-20T21:19:37.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:37.682+0000 7f2a56a00640 1 -- 172.21.15.102:0/485393998 wait complete. 2024-04-20T21:19:37.700 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:19:39.701 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:39.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.912+0000 7f1987000640 1 Processor -- start 2024-04-20T21:19:39.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.912+0000 7f1987000640 1 -- start start 2024-04-20T21:19:39.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.914+0000 7f1987000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800c8db0 0x7f1980070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:39.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.914+0000 7f1987000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1980158e80 con 0x7f1980152170 2024-04-20T21:19:39.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.914+0000 7f1987000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1980071f70 con 0x7f1980071330 2024-04-20T21:19:39.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.914+0000 7f1987000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f19800c6dc0 con 0x7f19800c8db0 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.914+0000 7f197fe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1980152170 0x7f1980158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45650/0 (socket says 172.21.15.102:45650) 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.914+0000 7f197fe00640 1 -- 172.21.15.102:0/3974357767 learned_addr learned my addr 172.21.15.102:0/3974357767 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.914+0000 7f1977400640 1 --2- 172.21.15.102:0/3974357767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800c8db0 0x7f1980070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.915+0000 7f197f400640 1 -- 172.21.15.102:0/3974357767 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894279573 0 0) 0x7f1980158e80 con 0x7f1980152170 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.915+0000 7f197f400640 1 -- 172.21.15.102:0/3974357767 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1964003520 con 0x7f1980152170 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.915+0000 7f197f400640 1 -- 172.21.15.102:0/3974357767 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2977818994 0 0) 0x7f1980071f70 con 0x7f1980071330 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.915+0000 7f197f400640 1 -- 172.21.15.102:0/3974357767 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1980158e80 con 0x7f1980071330 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.915+0000 7f1977400640 1 -- 172.21.15.102:0/3974357767 >> v1:172.21.15.193:6789/0 conn(0x7f1980071330 legacy=0x7f19801628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.915+0000 7f1977400640 1 -- 172.21.15.102:0/3974357767 >> v1:172.21.15.102:6789/0 conn(0x7f1980152170 legacy=0x7f1980158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:39.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.915+0000 7f1977400640 1 -- 172.21.15.102:0/3974357767 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1980072210 con 0x7f19800c8db0 2024-04-20T21:19:39.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.916+0000 7f1977400640 1 --2- 172.21.15.102:0/3974357767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800c8db0 0x7f1980070df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19800723e0 tx=0x7f196802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89ae365ff25a79f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:39.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.916+0000 7f197f400640 1 -- 172.21.15.102:0/3974357767 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968035020 con 0x7f19800c8db0 2024-04-20T21:19:39.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.916+0000 7f197f400640 1 -- 172.21.15.102:0/3974357767 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1968031890 con 0x7f19800c8db0 2024-04-20T21:19:39.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.916+0000 7f197f400640 1 -- 172.21.15.102:0/3974357767 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968031c60 con 0x7f19800c8db0 2024-04-20T21:19:39.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.917+0000 7f1987000640 1 -- 172.21.15.102:0/3974357767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800c8db0 msgr2=0x7f1980070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:39.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.917+0000 7f1987000640 1 --2- 172.21.15.102:0/3974357767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800c8db0 0x7f1980070df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19800723e0 tx=0x7f196802b930 comp rx=0 tx=0).stop 2024-04-20T21:19:39.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.917+0000 7f1987000640 1 -- 172.21.15.102:0/3974357767 shutdown_connections 2024-04-20T21:19:39.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.917+0000 7f1987000640 1 --2- 172.21.15.102:0/3974357767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800c8db0 0x7f1980070df0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:39.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.917+0000 7f1987000640 1 -- 172.21.15.102:0/3974357767 >> 172.21.15.102:0/3974357767 conn(0x7f19800c9390 msgr2=0x7f19800c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:39.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.918+0000 7f1987000640 1 -- 172.21.15.102:0/3974357767 shutdown_connections 2024-04-20T21:19:39.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.918+0000 7f1987000640 1 -- 172.21.15.102:0/3974357767 wait complete. 2024-04-20T21:19:39.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.920+0000 7f1987000640 1 Processor -- start 2024-04-20T21:19:39.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.920+0000 7f1987000640 1 -- start start 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1987000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980071330 0x7f19801590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1987000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19800c8db0 0x7f19800c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1987000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1980152170 0x7f198015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1987000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19800c9190 con 0x7f19800c8db0 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1987000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19800724b0 con 0x7f1980071330 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1987000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1980158250 con 0x7f1980152170 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1977400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19800c8db0 0x7f19800c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f197fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980071330 0x7f19801590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:39.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1985000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1980152170 0x7f198015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:39.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f1977400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19800c8db0 0x7f19800c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39572/0 (socket says 172.21.15.102:39572) 2024-04-20T21:19:39.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.921+0000 7f197fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980071330 0x7f19801590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:44722/0 (socket says 172.21.15.102:44722) 2024-04-20T21:19:39.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f1977400640 1 -- 172.21.15.102:0/3738144662 learned_addr learned my addr 172.21.15.102:0/3738144662 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f1985000640 1 -- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980071330 msgr2=0x7f19801590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f1985000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980071330 0x7f19801590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f1985000640 1 -- 172.21.15.102:0/3738144662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19800c8db0 msgr2=0x7f19800c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f1985000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19800c8db0 0x7f19800c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f1985000640 1 -- 172.21.15.102:0/3738144662 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1980159620 con 0x7f1980152170 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f1977400640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19800c8db0 0x7f19800c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f197fe00640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980071330 0x7f19801590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.922+0000 7f1985000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1980152170 0x7f198015ab60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f197800c6f0 tx=0x7f197800cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.923+0000 7f197cc00640 1 -- 172.21.15.102:0/3738144662 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978003970 con 0x7f1980152170 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.923+0000 7f197cc00640 1 -- 172.21.15.102:0/3738144662 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1978003b30 con 0x7f1980152170 2024-04-20T21:19:39.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.923+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801f4490 con 0x7f1980152170 2024-04-20T21:19:39.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.923+0000 7f197cc00640 1 -- 172.21.15.102:0/3738144662 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197800a850 con 0x7f1980152170 2024-04-20T21:19:39.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.923+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801f4a50 con 0x7f1980152170 2024-04-20T21:19:39.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.924+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1980074340 con 0x7f1980152170 2024-04-20T21:19:39.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.925+0000 7f197cc00640 1 -- 172.21.15.102:0/3738144662 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f197800a9f0 con 0x7f1980152170 2024-04-20T21:19:39.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.926+0000 7f197cc00640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:39.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.926+0000 7f197cc00640 1 -- 172.21.15.102:0/3738144662 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f197808bd30 con 0x7f1980152170 2024-04-20T21:19:39.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.926+0000 7f197fe00640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:39.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.927+0000 7f197fe00640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f195c06c920 0x7f195c06ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1970000f60 tx=0x7f1970007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:39.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:39.931+0000 7f197cc00640 1 -- 172.21.15.102:0/3738144662 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1978059670 con 0x7f1980152170 2024-04-20T21:19:40.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.170+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19800c3460 con 0x7f1980152170 2024-04-20T21:19:40.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.171+0000 7f197cc00640 1 -- 172.21.15.102:0/3738144662 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6856 (secure 0 0 0) 0x7f197805d970 con 0x7f1980152170 2024-04-20T21:19:40.171 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:40.171 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:17:20.270633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.102:6835/2873333699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2873333699},{"type":"v1","addr":"172.21.15.102:6835","nonce":2873333699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-04-20T21:19:40.171 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-20T21:19:40.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f195c06c920 msgr2=0x7f195c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:40.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f195c06c920 0x7f195c06ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1970000f60 tx=0x7f1970007d50 comp rx=0 tx=0).stop 2024-04-20T21:19:40.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1980152170 msgr2=0x7f198015ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:40.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1980152170 0x7f198015ab60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f197800c6f0 tx=0x7f197800cbb0 comp rx=0 tx=0).stop 2024-04-20T21:19:40.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 shutdown_connections 2024-04-20T21:19:40.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1980152170 0x7f198015ab60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:40.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19800c8db0 0x7f19800c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:40.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:40.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 --2- 172.21.15.102:0/3738144662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980071330 0x7f19801590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:40.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 >> 172.21.15.102:0/3738144662 conn(0x7f19800c9390 msgr2=0x7f1980151400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:40.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 shutdown_connections 2024-04-20T21:19:40.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:40.174+0000 7f1987000640 1 -- 172.21.15.102:0/3738144662 wait complete. 2024-04-20T21:19:40.193 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:19:42.194 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.400+0000 7f9b14800640 1 Processor -- start 2024-04-20T21:19:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.400+0000 7f9b14800640 1 -- start start 2024-04-20T21:19:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.401+0000 7f9b14800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9b0c1554f0 0x7f9b0c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.401+0000 7f9b14800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9b0c15bfd0 0x7f9b0c0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.401+0000 7f9b14800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c073a20 con 0x7f9b0c06c550 2024-04-20T21:19:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.401+0000 7f9b14800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9b0c15b510 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.401+0000 7f9b14800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9b0c073cc0 con 0x7f9b0c1554f0 2024-04-20T21:19:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b11e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9b0c1554f0 0x7f9b0c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b12800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9b0c15bfd0 0x7f9b0c0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b11400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f9b0c06c550 0x7f9b0c0c98f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:60676/0 (socket says 172.21.15.102:60676) 2024-04-20T21:19:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b11e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9b0c1554f0 0x7f9b0c15c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:44738/0 (socket says 172.21.15.102:44738) 2024-04-20T21:19:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b11400640 1 -- 172.21.15.102:0/3744036003 learned_addr learned my addr 172.21.15.102:0/3744036003 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b11e00640 1 -- 172.21.15.102:0/3744036003 >> v2:172.21.15.102:3300/0 conn(0x7f9b0c15bfd0 msgr2=0x7f9b0c0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b11e00640 1 --2- 172.21.15.102:0/3744036003 >> v2:172.21.15.102:3300/0 conn(0x7f9b0c15bfd0 0x7f9b0c0ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b11e00640 1 -- 172.21.15.102:0/3744036003 >> v1:172.21.15.193:6789/0 conn(0x7f9b0c06c550 legacy=0x7f9b0c0c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b11e00640 1 -- 172.21.15.102:0/3744036003 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c15b7d0 con 0x7f9b0c1554f0 2024-04-20T21:19:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b10a00640 1 -- 172.21.15.102:0/3744036003 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942157103 0 0) 0x7f9b0c073a20 con 0x7f9b0c06c550 2024-04-20T21:19:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.402+0000 7f9b12800640 1 --2- 172.21.15.102:0/3744036003 >> v2:172.21.15.102:3300/0 conn(0x7f9b0c15bfd0 0x7f9b0c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:42.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.404+0000 7f9b11e00640 1 --2- 172.21.15.102:0/3744036003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1554f0 0x7f9b0c15c810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0ceb60 tx=0x7f9b0002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2399648dc8f3e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:42.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.404+0000 7f9b10a00640 1 -- 172.21.15.102:0/3744036003 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00035020 con 0x7f9b0c1554f0 2024-04-20T21:19:42.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.404+0000 7f9b10a00640 1 -- 172.21.15.102:0/3744036003 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b00031830 con 0x7f9b0c1554f0 2024-04-20T21:19:42.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.404+0000 7f9b10a00640 1 -- 172.21.15.102:0/3744036003 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00031c00 con 0x7f9b0c1554f0 2024-04-20T21:19:42.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.404+0000 7f9b14800640 1 -- 172.21.15.102:0/3744036003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1554f0 msgr2=0x7f9b0c15c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.404+0000 7f9b14800640 1 --2- 172.21.15.102:0/3744036003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1554f0 0x7f9b0c15c810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0ceb60 tx=0x7f9b0002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.405+0000 7f9b14800640 1 -- 172.21.15.102:0/3744036003 shutdown_connections 2024-04-20T21:19:42.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.405+0000 7f9b14800640 1 --2- 172.21.15.102:0/3744036003 >> v2:172.21.15.102:3300/0 conn(0x7f9b0c15bfd0 0x7f9b0c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.405+0000 7f9b14800640 1 --2- 172.21.15.102:0/3744036003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1554f0 0x7f9b0c15c810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.405+0000 7f9b14800640 1 -- 172.21.15.102:0/3744036003 >> 172.21.15.102:0/3744036003 conn(0x7f9b0c06ebf0 msgr2=0x7f9b0c06efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:42.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.405+0000 7f9b14800640 1 -- 172.21.15.102:0/3744036003 shutdown_connections 2024-04-20T21:19:42.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.405+0000 7f9b14800640 1 -- 172.21.15.102:0/3744036003 wait complete. 2024-04-20T21:19:42.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.406+0000 7f9b14800640 1 Processor -- start 2024-04-20T21:19:42.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.406+0000 7f9b14800640 1 -- start start 2024-04-20T21:19:42.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.407+0000 7f9b14800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0c06c550 0x7f9b0c157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.407+0000 7f9b14800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b0c1554f0 0x7f9b0c1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.407+0000 7f9b14800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c15bfd0 0x7f9b0c0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.407+0000 7f9b14800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c0ca000 con 0x7f9b0c06c550 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.407+0000 7f9b14800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c0736e0 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.407+0000 7f9b14800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0c1558d0 con 0x7f9b0c1554f0 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0c06c550 0x7f9b0c157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b11400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b0c1554f0 0x7f9b0c1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b12800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c15bfd0 0x7f9b0c0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0c06c550 0x7f9b0c157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39590/0 (socket says 172.21.15.102:39590) 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b11e00640 1 -- 172.21.15.102:0/1671053407 learned_addr learned my addr 172.21.15.102:0/1671053407 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b12800640 1 -- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b0c1554f0 msgr2=0x7f9b0c1f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b12800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b0c1554f0 0x7f9b0c1f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b12800640 1 -- 172.21.15.102:0/1671053407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0c06c550 msgr2=0x7f9b0c157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b12800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0c06c550 0x7f9b0c157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b12800640 1 -- 172.21.15.102:0/1671053407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c1f4f10 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.408+0000 7f9b12800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c15bfd0 0x7f9b0c0c7270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800efc0 tx=0x7f9b0800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:42.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.409+0000 7f9b06000640 1 -- 172.21.15.102:0/1671053407 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08018030 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.409+0000 7f9b06000640 1 -- 172.21.15.102:0/1671053407 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0800cd30 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.409+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c156fd0 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.409+0000 7f9b06000640 1 -- 172.21.15.102:0/1671053407 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08003bb0 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.409+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c158840 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.410+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b0c158b00 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.411+0000 7f9b06000640 1 -- 172.21.15.102:0/1671053407 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b0801b070 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.412+0000 7f9b06000640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9af006c920 0x7f9af006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.412+0000 7f9b06000640 1 -- 172.21.15.102:0/1671053407 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f9b0808c720 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.412+0000 7f9b11e00640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9af006c920 0x7f9af006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.413+0000 7f9b11e00640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9af006c920 0x7f9af006ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009560 tx=0x7f9b00008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:42.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.416+0000 7f9b06000640 1 -- 172.21.15.102:0/1671053407 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b08010040 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.664+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b0c074340 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.666+0000 7f9b06000640 1 -- 172.21.15.102:0/1671053407 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6026 (secure 0 0 0) 0x7f9b0805a090 con 0x7f9b0c15bfd0 2024-04-20T21:19:42.665 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:42.666 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:40.462088+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:19:42.666 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 10 2024-04-20T21:19:42.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.668+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9af006c920 msgr2=0x7f9af006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.668+0000 7f9b14800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9af006c920 0x7f9af006ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009560 tx=0x7f9b00008030 comp rx=0 tx=0).stop 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.668+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c15bfd0 msgr2=0x7f9b0c0c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.668+0000 7f9b14800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c15bfd0 0x7f9b0c0c7270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800efc0 tx=0x7f9b0800c490 comp rx=0 tx=0).stop 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.669+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 shutdown_connections 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.669+0000 7f9b14800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9af006c920 0x7f9af006ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.669+0000 7f9b14800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c15bfd0 0x7f9b0c0c7270 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.669+0000 7f9b14800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b0c1554f0 0x7f9b0c1f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.669+0000 7f9b14800640 1 --2- 172.21.15.102:0/1671053407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0c06c550 0x7f9b0c157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.669+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 >> 172.21.15.102:0/1671053407 conn(0x7f9b0c06ebf0 msgr2=0x7f9b0c0cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.669+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 shutdown_connections 2024-04-20T21:19:42.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.669+0000 7f9b14800640 1 -- 172.21.15.102:0/1671053407 wait complete. 2024-04-20T21:19:42.687 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:19:42.689 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:19:42.690 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.888+0000 7f0c9b000640 1 Processor -- start 2024-04-20T21:19:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.888+0000 7f0c9b000640 1 -- start start 2024-04-20T21:19:42.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.890+0000 7f0c9b000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0c940c8420 0x7f0c9415af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.890+0000 7f0c9b000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9406eec0 con 0x7f0c940cd310 2024-04-20T21:19:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.890+0000 7f0c9b000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c940c73a0 con 0x7f0c940c8aa0 2024-04-20T21:19:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.890+0000 7f0c9b000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f0c940c7f90 con 0x7f0c940c8420 2024-04-20T21:19:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.890+0000 7f0c99200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0c940c8420 0x7f0c9415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.890+0000 7f0c99c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0c940cd310 0x7f0c94158470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45662/0 (socket says 172.21.15.102:45662) 2024-04-20T21:19:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.890+0000 7f0c99c00640 1 -- 172.21.15.102:0/3077684567 learned_addr learned my addr 172.21.15.102:0/3077684567 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.891+0000 7f0c99200640 1 -- 172.21.15.102:0/3077684567 >> v1:172.21.15.193:6789/0 conn(0x7f0c940c8aa0 legacy=0x7f0c94151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.891+0000 7f0c93e00640 1 -- 172.21.15.102:0/3077684567 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080830095 0 0) 0x7f0c940c73a0 con 0x7f0c940c8aa0 2024-04-20T21:19:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.891+0000 7f0c99200640 1 -- 172.21.15.102:0/3077684567 >> v1:172.21.15.102:6789/0 conn(0x7f0c940cd310 legacy=0x7f0c94158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.891+0000 7f0c99200640 1 -- 172.21.15.102:0/3077684567 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c940c7660 con 0x7f0c940c8420 2024-04-20T21:19:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.892+0000 7f0c99200640 1 --2- 172.21.15.102:0/3077684567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8420 0x7f0c9415af20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c800097a0 tx=0x7f0c8002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b670cd2df59ca1a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.892+0000 7f0c93e00640 1 -- 172.21.15.102:0/3077684567 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80038070 con 0x7f0c940c8420 2024-04-20T21:19:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.892+0000 7f0c93e00640 1 -- 172.21.15.102:0/3077684567 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c80031700 con 0x7f0c940c8420 2024-04-20T21:19:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.892+0000 7f0c93e00640 1 -- 172.21.15.102:0/3077684567 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80031ad0 con 0x7f0c940c8420 2024-04-20T21:19:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.892+0000 7f0c9b000640 1 -- 172.21.15.102:0/3077684567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8420 msgr2=0x7f0c9415af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.892+0000 7f0c9b000640 1 --2- 172.21.15.102:0/3077684567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8420 0x7f0c9415af20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c800097a0 tx=0x7f0c8002b820 comp rx=0 tx=0).stop 2024-04-20T21:19:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.893+0000 7f0c9b000640 1 -- 172.21.15.102:0/3077684567 shutdown_connections 2024-04-20T21:19:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.893+0000 7f0c9b000640 1 --2- 172.21.15.102:0/3077684567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8420 0x7f0c9415af20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.893+0000 7f0c9b000640 1 -- 172.21.15.102:0/3077684567 >> 172.21.15.102:0/3077684567 conn(0x7f0c940cc820 msgr2=0x7f0c940ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.893+0000 7f0c9b000640 1 -- 172.21.15.102:0/3077684567 shutdown_connections 2024-04-20T21:19:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.893+0000 7f0c9b000640 1 -- 172.21.15.102:0/3077684567 wait complete. 2024-04-20T21:19:42.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.894+0000 7f0c9b000640 1 Processor -- start 2024-04-20T21:19:42.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.895+0000 7f0c9b000640 1 -- start start 2024-04-20T21:19:42.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.895+0000 7f0c9b000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c940c8420 0x7f0c9414f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.895+0000 7f0c9b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8aa0 0x7f0c941599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.895+0000 7f0c9b000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c940cd310 0x7f0c941f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.895+0000 7f0c9b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9406c120 con 0x7f0c940c8aa0 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.895+0000 7f0c9b000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0c940c7c20 con 0x7f0c940cd310 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.895+0000 7f0c9b000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9406f600 con 0x7f0c940c8420 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.896+0000 7f0c99c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c940c8420 0x7f0c9414f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.896+0000 7f0c99200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8aa0 0x7f0c941599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.896+0000 7f0c99c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c940c8420 0x7f0c9414f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56610/0 (socket says 172.21.15.102:56610) 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.896+0000 7f0c99c00640 1 -- 172.21.15.102:0/2806037666 learned_addr learned my addr 172.21.15.102:0/2806037666 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.896+0000 7f0c9a600640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c940cd310 0x7f0c941f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.896+0000 7f0c99200640 1 -- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c940c8420 msgr2=0x7f0c9414f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.896+0000 7f0c99200640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c940c8420 0x7f0c9414f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c99200640 1 -- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c940cd310 msgr2=0x7f0c941f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c99200640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c940cd310 0x7f0c941f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c99200640 1 -- 172.21.15.102:0/2806037666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c94072cf0 con 0x7f0c940c8aa0 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c99200640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8aa0 0x7f0c941599d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c80033000 tx=0x7f0c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c91600640 1 -- 172.21.15.102:0/2806037666 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80043070 con 0x7f0c940c8aa0 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c91600640 1 -- 172.21.15.102:0/2806037666 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c800339b0 con 0x7f0c940c8aa0 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c94150320 con 0x7f0c940c8aa0 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c91600640 1 -- 172.21.15.102:0/2806037666 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80038040 con 0x7f0c940c8aa0 2024-04-20T21:19:42.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.897+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c940c5440 con 0x7f0c940c8aa0 2024-04-20T21:19:42.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.899+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c70005290 con 0x7f0c940c8aa0 2024-04-20T21:19:42.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.899+0000 7f0c91600640 1 -- 172.21.15.102:0/2806037666 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c80046070 con 0x7f0c940c8aa0 2024-04-20T21:19:42.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.900+0000 7f0c91600640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c7406c8f0 0x7f0c7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:42.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.900+0000 7f0c91600640 1 -- 172.21.15.102:0/2806037666 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f0c800ade50 con 0x7f0c940c8aa0 2024-04-20T21:19:42.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.901+0000 7f0c99c00640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c7406c8f0 0x7f0c7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:42.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.901+0000 7f0c99c00640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c7406c8f0 0x7f0c7406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000f60 tx=0x7f0c88007ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:42.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:42.905+0000 7f0c91600640 1 -- 172.21.15.102:0/2806037666 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c80030070 con 0x7f0c940c8aa0 2024-04-20T21:19:43.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.148+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c70005850 con 0x7f0c940c8aa0 2024-04-20T21:19:43.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.149+0000 7f0c91600640 1 -- 172.21.15.102:0/2806037666 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6026 (secure 0 0 0) 0x7f0c8007b860 con 0x7f0c940c8aa0 2024-04-20T21:19:43.149 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:43.149 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:40.462088+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:19:43.149 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 10 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c7406c8f0 msgr2=0x7f0c7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c7406c8f0 0x7f0c7406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000f60 tx=0x7f0c88007ec0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8aa0 msgr2=0x7f0c941599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8aa0 0x7f0c941599d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c80033000 tx=0x7f0c800046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 shutdown_connections 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c7406c8f0 0x7f0c7406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c940cd310 0x7f0c941f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c940c8aa0 0x7f0c941599d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 --2- 172.21.15.102:0/2806037666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c940c8420 0x7f0c9414f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 >> 172.21.15.102:0/2806037666 conn(0x7f0c940cc820 msgr2=0x7f0c9406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 shutdown_connections 2024-04-20T21:19:43.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:43.152+0000 7f0c9b000640 1 -- 172.21.15.102:0/2806037666 wait complete. 2024-04-20T21:19:43.170 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 10, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 10, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4297": { "addr": "172.21.15.102:6833/2795763369", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 2795763369, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 2795763369, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4297, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:19:40.462088+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4297 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:19:45.171 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:45.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.381+0000 7fc198000640 1 Processor -- start 2024-04-20T21:19:45.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.381+0000 7fc198000640 1 -- start start 2024-04-20T21:19:45.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.382+0000 7fc198000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc19014fa50 0x7fc1900c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.382+0000 7fc198000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1901536d0 con 0x7fc190073a10 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.382+0000 7fc198000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1900c76c0 con 0x7fc1900c9eb0 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.382+0000 7fc198000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc1900c7960 con 0x7fc19014fa50 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.383+0000 7fc196000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc190073a10 0x7fc190163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45670/0 (socket says 172.21.15.102:45670) 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.383+0000 7fc196000640 1 -- 172.21.15.102:0/3819668231 learned_addr learned my addr 172.21.15.102:0/3819668231 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.383+0000 7fc195600640 1 --2- 172.21.15.102:0/3819668231 >> v2:172.21.15.193:3300/0 conn(0x7fc19014fa50 0x7fc1900c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.383+0000 7fc194c00640 1 --1- 172.21.15.102:0/3819668231 >> v1:172.21.15.193:6789/0 conn(0x7fc1900c9eb0 0x7fc1900c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:60696/0 (socket says 172.21.15.102:60696) 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.383+0000 7fc195600640 1 -- 172.21.15.102:0/3819668231 >> v1:172.21.15.193:6789/0 conn(0x7fc1900c9eb0 legacy=0x7fc1900c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.383+0000 7fc195600640 1 -- 172.21.15.102:0/3819668231 >> v1:172.21.15.102:6789/0 conn(0x7fc190073a10 legacy=0x7fc190163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.383+0000 7fc195600640 1 -- 172.21.15.102:0/3819668231 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1900c3d90 con 0x7fc19014fa50 2024-04-20T21:19:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.383+0000 7fc18fe00640 1 -- 172.21.15.102:0/3819668231 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805622524 0 0) 0x7fc188001040 con 0x7fc190073a10 2024-04-20T21:19:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.384+0000 7fc195600640 1 --2- 172.21.15.102:0/3819668231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc19014fa50 0x7fc1900c4830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc1900c7890 tx=0x7fc18402bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83d9230e39eb30d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.384+0000 7fc18fe00640 1 -- 172.21.15.102:0/3819668231 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184039070 con 0x7fc19014fa50 2024-04-20T21:19:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc18fe00640 1 -- 172.21.15.102:0/3819668231 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18402cd80 con 0x7fc19014fa50 2024-04-20T21:19:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc18fe00640 1 -- 172.21.15.102:0/3819668231 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184034660 con 0x7fc19014fa50 2024-04-20T21:19:45.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc198000640 1 -- 172.21.15.102:0/3819668231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc19014fa50 msgr2=0x7fc1900c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc198000640 1 --2- 172.21.15.102:0/3819668231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc19014fa50 0x7fc1900c4830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc1900c7890 tx=0x7fc18402bfd0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc198000640 1 -- 172.21.15.102:0/3819668231 shutdown_connections 2024-04-20T21:19:45.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc198000640 1 --2- 172.21.15.102:0/3819668231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc19014fa50 0x7fc1900c4830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc198000640 1 -- 172.21.15.102:0/3819668231 >> 172.21.15.102:0/3819668231 conn(0x7fc1900c9390 msgr2=0x7fc1900c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:45.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc198000640 1 -- 172.21.15.102:0/3819668231 shutdown_connections 2024-04-20T21:19:45.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.385+0000 7fc198000640 1 -- 172.21.15.102:0/3819668231 wait complete. 2024-04-20T21:19:45.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.387+0000 7fc198000640 1 Processor -- start 2024-04-20T21:19:45.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.387+0000 7fc198000640 1 -- start start 2024-04-20T21:19:45.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc198000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc190073a10 0x7fc1901542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc198000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1900c9eb0 0x7fc1901f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc198000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc19014fa50 0x7fc190156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc198000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900c4030 con 0x7fc1900c9eb0 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc198000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900c7c00 con 0x7fc190073a10 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc198000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc1900c4310 con 0x7fc19014fa50 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc194c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1900c9eb0 0x7fc1901f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc194c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1900c9eb0 0x7fc1901f1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39608/0 (socket says 172.21.15.102:39608) 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.388+0000 7fc194c00640 1 -- 172.21.15.102:0/3570397108 learned_addr learned my addr 172.21.15.102:0/3570397108 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.389+0000 7fc196000640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc19014fa50 0x7fc190156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.389+0000 7fc195600640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc190073a10 0x7fc1901542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.389+0000 7fc194c00640 1 -- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc19014fa50 msgr2=0x7fc190156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.389+0000 7fc194c00640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc19014fa50 0x7fc190156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.389+0000 7fc194c00640 1 -- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc190073a10 msgr2=0x7fc1901542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.389+0000 7fc194c00640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc190073a10 0x7fc1901542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.389+0000 7fc194c00640 1 -- 172.21.15.102:0/3570397108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1901567f0 con 0x7fc1900c9eb0 2024-04-20T21:19:45.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.389+0000 7fc194c00640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1900c9eb0 0x7fc1901f1840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c500 tx=0x7fc17c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.390+0000 7fc18d600640 1 -- 172.21.15.102:0/3570397108 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c018070 con 0x7fc1900c9eb0 2024-04-20T21:19:45.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.390+0000 7fc18d600640 1 -- 172.21.15.102:0/3570397108 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc17c007500 con 0x7fc1900c9eb0 2024-04-20T21:19:45.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.390+0000 7fc18d600640 1 -- 172.21.15.102:0/3570397108 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0049e0 con 0x7fc1900c9eb0 2024-04-20T21:19:45.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.390+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901f1080 con 0x7fc1900c9eb0 2024-04-20T21:19:45.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.390+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc190155f50 con 0x7fc1900c9eb0 2024-04-20T21:19:45.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.391+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc190074340 con 0x7fc1900c9eb0 2024-04-20T21:19:45.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.392+0000 7fc18d600640 1 -- 172.21.15.102:0/3570397108 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc17c004b80 con 0x7fc1900c9eb0 2024-04-20T21:19:45.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.393+0000 7fc18d600640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc17006c8f0 0x7fc17006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.393+0000 7fc18d600640 1 -- 172.21.15.102:0/3570397108 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7fc17c08b180 con 0x7fc1900c9eb0 2024-04-20T21:19:45.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.394+0000 7fc195600640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc17006c8f0 0x7fc17006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.394+0000 7fc195600640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc17006c8f0 0x7fc17006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc184036000 tx=0x7fc184030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.398+0000 7fc18d600640 1 -- 172.21.15.102:0/3570397108 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc17c058b00 con 0x7fc1900c9eb0 2024-04-20T21:19:45.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.655+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc190153180 con 0x7fc1900c9eb0 2024-04-20T21:19:45.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.656+0000 7fc18d600640 1 -- 172.21.15.102:0/3570397108 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6026 (secure 0 0 0) 0x7fc17c05ce00 con 0x7fc1900c9eb0 2024-04-20T21:19:45.656 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:45.657 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:40.462088+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:19:45.657 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 10 2024-04-20T21:19:45.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.659+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc17006c8f0 msgr2=0x7fc17006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.659+0000 7fc198000640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc17006c8f0 0x7fc17006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc184036000 tx=0x7fc184030000 comp rx=0 tx=0).stop 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.659+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1900c9eb0 msgr2=0x7fc1901f1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.659+0000 7fc198000640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1900c9eb0 0x7fc1901f1840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c500 tx=0x7fc17c00c9c0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.660+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 shutdown_connections 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.660+0000 7fc198000640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc19014fa50 0x7fc190156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.660+0000 7fc198000640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1900c9eb0 0x7fc1901f1840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.660+0000 7fc198000640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc17006c8f0 0x7fc17006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.660+0000 7fc198000640 1 --2- 172.21.15.102:0/3570397108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc190073a10 0x7fc1901542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.660+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 >> 172.21.15.102:0/3570397108 conn(0x7fc1900c9390 msgr2=0x7fc19006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.660+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 shutdown_connections 2024-04-20T21:19:45.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:45.660+0000 7fc198000640 1 -- 172.21.15.102:0/3570397108 wait complete. 2024-04-20T21:19:45.678 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:19:47.679 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:47.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.887+0000 7f7704400640 1 Processor -- start 2024-04-20T21:19:47.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.887+0000 7f7704400640 1 -- start start 2024-04-20T21:19:47.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.889+0000 7f7704400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76fc14fa50 0x7f76fc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.889+0000 7f7704400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76fc1536d0 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.889+0000 7f7704400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76fc0c76c0 con 0x7f76fc073a10 2024-04-20T21:19:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.889+0000 7f7704400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f76fc0c7960 con 0x7f76fc14fa50 2024-04-20T21:19:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.889+0000 7f7701a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76fc14fa50 0x7f76fc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.889+0000 7f7701a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76fc14fa50 0x7f76fc0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45332/0 (socket says 172.21.15.102:45332) 2024-04-20T21:19:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.889+0000 7f7701a00640 1 -- 172.21.15.102:0/3320336277 learned_addr learned my addr 172.21.15.102:0/3320336277 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.890+0000 7f7701a00640 1 -- 172.21.15.102:0/3320336277 >> v1:172.21.15.193:6789/0 conn(0x7f76fc073a10 legacy=0x7f76fc163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.890+0000 7f7701a00640 1 -- 172.21.15.102:0/3320336277 >> v1:172.21.15.102:6789/0 conn(0x7f76fc0c9eb0 legacy=0x7f76fc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.890+0000 7f7701a00640 1 -- 172.21.15.102:0/3320336277 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc0c3d90 con 0x7f76fc14fa50 2024-04-20T21:19:47.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.891+0000 7f7701a00640 1 --2- 172.21.15.102:0/3320336277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc14fa50 0x7f76fc0c4830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0c7890 tx=0x7f76ec02baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9b5ae7fa71fac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.891+0000 7f76fbe00640 1 -- 172.21.15.102:0/3320336277 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec02d860 con 0x7f76fc14fa50 2024-04-20T21:19:47.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.891+0000 7f76fbe00640 1 -- 172.21.15.102:0/3320336277 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ec02da00 con 0x7f76fc14fa50 2024-04-20T21:19:47.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.891+0000 7f76fbe00640 1 -- 172.21.15.102:0/3320336277 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec02ddd0 con 0x7f76fc14fa50 2024-04-20T21:19:47.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.892+0000 7f7704400640 1 -- 172.21.15.102:0/3320336277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc14fa50 msgr2=0x7f76fc0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.892+0000 7f7704400640 1 --2- 172.21.15.102:0/3320336277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc14fa50 0x7f76fc0c4830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0c7890 tx=0x7f76ec02baa0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.892+0000 7f7704400640 1 -- 172.21.15.102:0/3320336277 shutdown_connections 2024-04-20T21:19:47.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.892+0000 7f7704400640 1 --2- 172.21.15.102:0/3320336277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc14fa50 0x7f76fc0c4830 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.892+0000 7f7704400640 1 -- 172.21.15.102:0/3320336277 >> 172.21.15.102:0/3320336277 conn(0x7f76fc0c9390 msgr2=0x7f76fc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:47.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.892+0000 7f7704400640 1 -- 172.21.15.102:0/3320336277 shutdown_connections 2024-04-20T21:19:47.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.892+0000 7f7704400640 1 -- 172.21.15.102:0/3320336277 wait complete. 2024-04-20T21:19:47.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.894+0000 7f7704400640 1 Processor -- start 2024-04-20T21:19:47.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.894+0000 7f7704400640 1 -- start start 2024-04-20T21:19:47.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7704400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76fc073a10 0x7f76fc158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7704400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc0c9eb0 0x7f76fc072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7704400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76fc14fa50 0x7f76fc072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7704400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc0c4030 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7704400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc0c7c00 con 0x7f76fc073a10 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7704400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f76fc0c4310 con 0x7f76fc14fa50 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7701000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc0c9eb0 0x7f76fc072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7701000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc0c9eb0 0x7f76fc072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45348/0 (socket says 172.21.15.102:45348) 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7701000640 1 -- 172.21.15.102:0/4009849006 learned_addr learned my addr 172.21.15.102:0/4009849006 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7701a00640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76fc073a10 0x7f76fc158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.895+0000 7f7702400640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76fc14fa50 0x7f76fc072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.896+0000 7f7701000640 1 -- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76fc14fa50 msgr2=0x7f76fc072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.896+0000 7f7701000640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76fc14fa50 0x7f76fc072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.896+0000 7f7701000640 1 -- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76fc073a10 msgr2=0x7f76fc158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.896+0000 7f7701000640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76fc073a10 0x7f76fc158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.896+0000 7f7701000640 1 -- 172.21.15.102:0/4009849006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc070df0 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.896+0000 7f7701000640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc0c9eb0 0x7f76fc072130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76e800c450 tx=0x7f76e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.896+0000 7f76f9600640 1 -- 172.21.15.102:0/4009849006 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e80052d0 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.897+0000 7f76f9600640 1 -- 172.21.15.102:0/4009849006 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e8005470 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.897+0000 7f76f9600640 1 -- 172.21.15.102:0/4009849006 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8012870 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.897+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc0710b0 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.897+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc157790 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.898+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76fc074340 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.899+0000 7f76f9600640 1 -- 172.21.15.102:0/4009849006 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76e8012a10 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.900+0000 7f76f9600640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76dc06c8f0 0x7f76dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.900+0000 7f76f9600640 1 -- 172.21.15.102:0/4009849006 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f76e808bac0 con 0x7f76fc0c9eb0 2024-04-20T21:19:47.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.901+0000 7f7701a00640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76dc06c8f0 0x7f76dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.901+0000 7f7701a00640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76dc06c8f0 0x7f76dc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f76ec036000 tx=0x7f76ec008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:47.906+0000 7f76f9600640 1 -- 172.21.15.102:0/4009849006 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f76fc1536d0 con 0x7f76fc0c9eb0 2024-04-20T21:19:48.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.203+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76fc0c58f0 con 0x7f76fc0c9eb0 2024-04-20T21:19:48.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.204+0000 7f76f9600640 1 -- 172.21.15.102:0/4009849006 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6030 (secure 0 0 0) 0x7f76e8059430 con 0x7f76fc0c9eb0 2024-04-20T21:19:48.204 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:48.204 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:45.488560+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:reconnect","state_seq":46,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:19:48.204 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76dc06c8f0 msgr2=0x7f76dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76dc06c8f0 0x7f76dc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f76ec036000 tx=0x7f76ec008030 comp rx=0 tx=0).stop 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc0c9eb0 msgr2=0x7f76fc072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc0c9eb0 0x7f76fc072130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76e800c450 tx=0x7f76e800c910 comp rx=0 tx=0).stop 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 shutdown_connections 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76fc14fa50 0x7f76fc072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76fc0c9eb0 0x7f76fc072130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76dc06c8f0 0x7f76dc06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 --2- 172.21.15.102:0/4009849006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76fc073a10 0x7f76fc158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 >> 172.21.15.102:0/4009849006 conn(0x7f76fc0c9390 msgr2=0x7f76fc14b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:48.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 shutdown_connections 2024-04-20T21:19:48.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:48.207+0000 7f7704400640 1 -- 172.21.15.102:0/4009849006 wait complete. 2024-04-20T21:19:48.224 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:19:50.225 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:50.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.394+0000 7fb4afe00640 1 Processor -- start 2024-04-20T21:19:50.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.394+0000 7fb4afe00640 1 -- start start 2024-04-20T21:19:50.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.395+0000 7fb4afe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb4b0152290 0x7fb4b00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.395+0000 7fb4afe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb4b00c3700 0x7fb4b006e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.395+0000 7fb4afe00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b015c220 con 0x7fb4b006eb70 2024-04-20T21:19:50.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.395+0000 7fb4afe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b00c9180 con 0x7fb4b0152290 2024-04-20T21:19:50.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.395+0000 7fb4afe00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b00c4840 con 0x7fb4b00c3700 2024-04-20T21:19:50.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4aea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb4b0152290 0x7fb4b00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4af400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb4b006eb70 0x7fb4b00c8330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49804/0 (socket says 172.21.15.102:49804) 2024-04-20T21:19:50.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4af400640 1 -- 172.21.15.102:0/4010702538 learned_addr learned my addr 172.21.15.102:0/4010702538 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:50.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4ae000640 1 --2- 172.21.15.102:0/4010702538 >> v2:172.21.15.193:3300/0 conn(0x7fb4b00c3700 0x7fb4b006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4aea00640 1 -- 172.21.15.102:0/4010702538 >> v2:172.21.15.193:3300/0 conn(0x7fb4b00c3700 msgr2=0x7fb4b006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4aea00640 1 --2- 172.21.15.102:0/4010702538 >> v2:172.21.15.193:3300/0 conn(0x7fb4b00c3700 0x7fb4b006e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4aea00640 1 -- 172.21.15.102:0/4010702538 >> v1:172.21.15.102:6789/0 conn(0x7fb4b006eb70 legacy=0x7fb4b00c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4aea00640 1 -- 172.21.15.102:0/4010702538 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b00c4560 con 0x7fb4b0152290 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.396+0000 7fb4ad600640 1 -- 172.21.15.102:0/4010702538 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2970677533 0 0) 0x7fb4b015c220 con 0x7fb4b006eb70 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.397+0000 7fb4aea00640 1 --2- 172.21.15.102:0/4010702538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b0152290 0x7fb4b00cd310 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0069910 tx=0x7fb4a002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e03b4ac529f5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4ad600640 1 -- 172.21.15.102:0/4010702538 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0039070 con 0x7fb4b0152290 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4ad600640 1 -- 172.21.15.102:0/4010702538 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a002cd00 con 0x7fb4b0152290 2024-04-20T21:19:50.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4ad600640 1 -- 172.21.15.102:0/4010702538 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0034660 con 0x7fb4b0152290 2024-04-20T21:19:50.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4afe00640 1 -- 172.21.15.102:0/4010702538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b0152290 msgr2=0x7fb4b00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4afe00640 1 --2- 172.21.15.102:0/4010702538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b0152290 0x7fb4b00cd310 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0069910 tx=0x7fb4a002bf50 comp rx=0 tx=0).stop 2024-04-20T21:19:50.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4afe00640 1 -- 172.21.15.102:0/4010702538 shutdown_connections 2024-04-20T21:19:50.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4afe00640 1 --2- 172.21.15.102:0/4010702538 >> v2:172.21.15.193:3300/0 conn(0x7fb4b00c3700 0x7fb4b006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4afe00640 1 --2- 172.21.15.102:0/4010702538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b0152290 0x7fb4b00cd310 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4afe00640 1 -- 172.21.15.102:0/4010702538 >> 172.21.15.102:0/4010702538 conn(0x7fb4b00cc820 msgr2=0x7fb4b00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:50.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4afe00640 1 -- 172.21.15.102:0/4010702538 shutdown_connections 2024-04-20T21:19:50.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.398+0000 7fb4afe00640 1 -- 172.21.15.102:0/4010702538 wait complete. 2024-04-20T21:19:50.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.400+0000 7fb4afe00640 1 Processor -- start 2024-04-20T21:19:50.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.400+0000 7fb4afe00640 1 -- start start 2024-04-20T21:19:50.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b006eb70 0x7fb4b00caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4b00c3700 0x7fb4b00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4b0152290 0x7fb4b01ed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4afe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b015ba20 con 0x7fb4b006eb70 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4afe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b015b740 con 0x7fb4b00c3700 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4afe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b015bd00 con 0x7fb4b0152290 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4aea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b006eb70 0x7fb4b00caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4aea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b006eb70 0x7fb4b00caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45362/0 (socket says 172.21.15.102:45362) 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4aea00640 1 -- 172.21.15.102:0/782204471 learned_addr learned my addr 172.21.15.102:0/782204471 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4ae000640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4b00c3700 0x7fb4b00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.401+0000 7fb4af400640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4b0152290 0x7fb4b01ed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4aea00640 1 -- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4b0152290 msgr2=0x7fb4b01ed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4aea00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4b0152290 0x7fb4b01ed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4aea00640 1 -- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4b00c3700 msgr2=0x7fb4b00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4aea00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4b00c3700 0x7fb4b00caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4aea00640 1 -- 172.21.15.102:0/782204471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b00cb680 con 0x7fb4b006eb70 2024-04-20T21:19:50.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4aea00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b006eb70 0x7fb4b00caa20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0037040 tx=0x7fb4a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:50.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4a6a00640 1 -- 172.21.15.102:0/782204471 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0039050 con 0x7fb4b006eb70 2024-04-20T21:19:50.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4a6a00640 1 -- 172.21.15.102:0/782204471 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a0045070 con 0x7fb4b006eb70 2024-04-20T21:19:50.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4a6a00640 1 -- 172.21.15.102:0/782204471 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a003e850 con 0x7fb4b006eb70 2024-04-20T21:19:50.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.402+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b00cbc60 con 0x7fb4b006eb70 2024-04-20T21:19:50.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.403+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b0147040 con 0x7fb4b006eb70 2024-04-20T21:19:50.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.404+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b004fd60 con 0x7fb4b006eb70 2024-04-20T21:19:50.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.405+0000 7fb4a6a00640 1 -- 172.21.15.102:0/782204471 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4a003e9f0 con 0x7fb4b006eb70 2024-04-20T21:19:50.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.405+0000 7fb4a6a00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.406+0000 7fb4a6a00640 1 -- 172.21.15.102:0/782204471 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7fb4a00aeb40 con 0x7fb4b006eb70 2024-04-20T21:19:50.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.406+0000 7fb4ae000640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.406+0000 7fb4ae000640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb49006c8f0 0x7fb49006edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00ca740 tx=0x7fb49c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:50.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.410+0000 7fb4a6a00640 1 -- 172.21.15.102:0/782204471 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb4a007c670 con 0x7fb4b006eb70 2024-04-20T21:19:50.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.645+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4b00c7da0 con 0x7fb4b006eb70 2024-04-20T21:19:50.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.646+0000 7fb4a6a00640 1 -- 172.21.15.102:0/782204471 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6030 (secure 0 0 0) 0x7fb4a0080970 con 0x7fb4b006eb70 2024-04-20T21:19:50.646 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:50.646 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:45.488560+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:reconnect","state_seq":46,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:19:50.646 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-20T21:19:50.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb49006c8f0 msgr2=0x7fb49006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb49006c8f0 0x7fb49006edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00ca740 tx=0x7fb49c009190 comp rx=0 tx=0).stop 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b006eb70 msgr2=0x7fb4b00caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b006eb70 0x7fb4b00caa20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0037040 tx=0x7fb4a0004a40 comp rx=0 tx=0).stop 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 shutdown_connections 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4b0152290 0x7fb4b01ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4b00c3700 0x7fb4b00caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 --2- 172.21.15.102:0/782204471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4b006eb70 0x7fb4b00caa20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 >> 172.21.15.102:0/782204471 conn(0x7fb4b00cc820 msgr2=0x7fb4b013cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 shutdown_connections 2024-04-20T21:19:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:50.649+0000 7fb4afe00640 1 -- 172.21.15.102:0/782204471 wait complete. 2024-04-20T21:19:50.665 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:19:51.647 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:51.655 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:52.665 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:52.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.878+0000 7f279a200640 1 Processor -- start 2024-04-20T21:19:52.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.878+0000 7f279a200640 1 -- start start 2024-04-20T21:19:52.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f279a200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2794153180 0x7f2794158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f279a200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f27940c75a0 0x7f27941637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f279a200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2794153560 con 0x7f2794159380 2024-04-20T21:19:52.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f279a200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f27940c7a70 con 0x7f27940c75a0 2024-04-20T21:19:52.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f279a200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f27940c7d30 con 0x7f2794153180 2024-04-20T21:19:52.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f2799800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f27940c75a0 0x7f27941637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f2793e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2794159380 0x7f27940c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49820/0 (socket says 172.21.15.102:49820) 2024-04-20T21:19:52.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f2793e00640 1 -- 172.21.15.102:0/2512646862 learned_addr learned my addr 172.21.15.102:0/2512646862 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:52.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.880+0000 7f2798e00640 1 --2- 172.21.15.102:0/2512646862 >> v2:172.21.15.193:3300/0 conn(0x7f2794153180 0x7f2794158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.881+0000 7f2798e00640 1 -- 172.21.15.102:0/2512646862 >> v2:172.21.15.102:3300/0 conn(0x7f27940c75a0 msgr2=0x7f27941637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:52.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.881+0000 7f2798e00640 1 --2- 172.21.15.102:0/2512646862 >> v2:172.21.15.102:3300/0 conn(0x7f27940c75a0 0x7f27941637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.881+0000 7f2793400640 1 -- 172.21.15.102:0/2512646862 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784422750 0 0) 0x7f2794153560 con 0x7f2794159380 2024-04-20T21:19:52.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.881+0000 7f2798e00640 1 -- 172.21.15.102:0/2512646862 >> v1:172.21.15.102:6789/0 conn(0x7f2794159380 legacy=0x7f27940c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.881+0000 7f2798e00640 1 -- 172.21.15.102:0/2512646862 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279415c4d0 con 0x7f2794153180 2024-04-20T21:19:52.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.881+0000 7f2799800640 1 --2- 172.21.15.102:0/2512646862 >> v2:172.21.15.102:3300/0 conn(0x7f27940c75a0 0x7f27941637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:52.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.882+0000 7f2798e00640 1 --2- 172.21.15.102:0/2512646862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794153180 0x7f2794158e40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27940cd860 tx=0x7f278802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca01496d0d1f6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:52.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.882+0000 7f2793400640 1 -- 172.21.15.102:0/2512646862 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788035020 con 0x7f2794153180 2024-04-20T21:19:52.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.882+0000 7f2793400640 1 -- 172.21.15.102:0/2512646862 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2788031830 con 0x7f2794153180 2024-04-20T21:19:52.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.882+0000 7f2793400640 1 -- 172.21.15.102:0/2512646862 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788031c00 con 0x7f2794153180 2024-04-20T21:19:52.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.883+0000 7f279a200640 1 -- 172.21.15.102:0/2512646862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794153180 msgr2=0x7f2794158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.883+0000 7f279a200640 1 --2- 172.21.15.102:0/2512646862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794153180 0x7f2794158e40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27940cd860 tx=0x7f278802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.883+0000 7f279a200640 1 -- 172.21.15.102:0/2512646862 shutdown_connections 2024-04-20T21:19:52.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.883+0000 7f279a200640 1 --2- 172.21.15.102:0/2512646862 >> v2:172.21.15.102:3300/0 conn(0x7f27940c75a0 0x7f27941637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.883+0000 7f279a200640 1 --2- 172.21.15.102:0/2512646862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794153180 0x7f2794158e40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.883+0000 7f279a200640 1 -- 172.21.15.102:0/2512646862 >> 172.21.15.102:0/2512646862 conn(0x7f27940cc820 msgr2=0x7f27940ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:52.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.883+0000 7f279a200640 1 -- 172.21.15.102:0/2512646862 shutdown_connections 2024-04-20T21:19:52.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.883+0000 7f279a200640 1 -- 172.21.15.102:0/2512646862 wait complete. 2024-04-20T21:19:52.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.885+0000 7f279a200640 1 Processor -- start 2024-04-20T21:19:52.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.885+0000 7f279a200640 1 -- start start 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f279a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27940c75a0 0x7f2794156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f279a200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2794153180 0x7f27941403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f279a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794159380 0x7f2794140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f279a200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f279415d290 con 0x7f27940c75a0 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f279a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f279415ca90 con 0x7f2794159380 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f279a200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f27940c6be0 con 0x7f2794153180 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f2798e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27940c75a0 0x7f2794156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f2798e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27940c75a0 0x7f2794156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45366/0 (socket says 172.21.15.102:45366) 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f2798e00640 1 -- 172.21.15.102:0/2601104460 learned_addr learned my addr 172.21.15.102:0/2601104460 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f2793e00640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2794153180 0x7f27941403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.886+0000 7f2799800640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794159380 0x7f2794140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2798e00640 1 -- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2794153180 msgr2=0x7f27941403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2798e00640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2794153180 0x7f27941403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2798e00640 1 -- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794159380 msgr2=0x7f2794140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2798e00640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794159380 0x7f2794140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2798e00640 1 -- 172.21.15.102:0/2601104460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27940cc0e0 con 0x7f27940c75a0 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2793e00640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2794153180 0x7f27941403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2799800640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794159380 0x7f2794140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2798e00640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27940c75a0 0x7f2794156f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2788000c00 tx=0x7f27880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2790c00640 1 -- 172.21.15.102:0/2601104460 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788035040 con 0x7f27940c75a0 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2790c00640 1 -- 172.21.15.102:0/2601104460 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2788004230 con 0x7f27940c75a0 2024-04-20T21:19:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27940cc3c0 con 0x7f27940c75a0 2024-04-20T21:19:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f2790c00640 1 -- 172.21.15.102:0/2601104460 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278803a640 con 0x7f27940c75a0 2024-04-20T21:19:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.887+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279415ba80 con 0x7f27940c75a0 2024-04-20T21:19:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.889+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279415b590 con 0x7f27940c75a0 2024-04-20T21:19:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.889+0000 7f2790c00640 1 -- 172.21.15.102:0/2601104460 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2788038020 con 0x7f27940c75a0 2024-04-20T21:19:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.890+0000 7f2790c00640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f277406c8f0 0x7f277406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.890+0000 7f2790c00640 1 -- 172.21.15.102:0/2601104460 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f27880adc70 con 0x7f27940c75a0 2024-04-20T21:19:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.891+0000 7f2793e00640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f277406c8f0 0x7f277406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.891+0000 7f2793e00640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f277406c8f0 0x7f277406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f27940c8260 tx=0x7f2780007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:52.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:52.895+0000 7f2790c00640 1 -- 172.21.15.102:0/2601104460 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f278807bba0 con 0x7f27940c75a0 2024-04-20T21:19:53.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.133+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27940c3690 con 0x7f27940c75a0 2024-04-20T21:19:53.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.134+0000 7f2790c00640 1 -- 172.21.15.102:0/2601104460 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6027 (secure 0 0 0) 0x7f278807b4a0 con 0x7f27940c75a0 2024-04-20T21:19:53.134 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:53.134 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:51.506369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:rejoin","state_seq":48,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:19:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 12 2024-04-20T21:19:53.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.136+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f277406c8f0 msgr2=0x7f277406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.136+0000 7f279a200640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f277406c8f0 0x7f277406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f27940c8260 tx=0x7f2780007de0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.136+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27940c75a0 msgr2=0x7f2794156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.136+0000 7f279a200640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27940c75a0 0x7f2794156f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2788000c00 tx=0x7f27880046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.137+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 shutdown_connections 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.137+0000 7f279a200640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f277406c8f0 0x7f277406edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.137+0000 7f279a200640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2794159380 0x7f2794140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.137+0000 7f279a200640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2794153180 0x7f27941403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.137+0000 7f279a200640 1 --2- 172.21.15.102:0/2601104460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27940c75a0 0x7f2794156f20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.137+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 >> 172.21.15.102:0/2601104460 conn(0x7f27940cc820 msgr2=0x7f2794152050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:53.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.137+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 shutdown_connections 2024-04-20T21:19:53.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:53.137+0000 7f279a200640 1 -- 172.21.15.102:0/2601104460 wait complete. 2024-04-20T21:19:53.156 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:19:55.158 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:55.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.369+0000 7f89a5200640 1 Processor -- start 2024-04-20T21:19:55.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.369+0000 7f89a5200640 1 -- start start 2024-04-20T21:19:55.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.370+0000 7f89a5200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0153180 0x7f89a0158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:55.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.370+0000 7f89a5200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f89a0159380 0x7f89a00c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:55.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.370+0000 7f89a5200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a0153560 con 0x7f89a00c75a0 2024-04-20T21:19:55.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.370+0000 7f89a5200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f89a00c7a70 con 0x7f89a0159380 2024-04-20T21:19:55.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.370+0000 7f89a5200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f89a00c7d30 con 0x7f89a0153180 2024-04-20T21:19:55.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.371+0000 7f899f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0153180 0x7f89a0158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:55.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.371+0000 7f8997e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f89a0159380 0x7f89a00c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:55.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.371+0000 7f899fe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f89a00c75a0 0x7f89a0070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:36162/0 (socket says 172.21.15.102:36162) 2024-04-20T21:19:55.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.371+0000 7f899fe00640 1 -- 172.21.15.102:0/2315130163 learned_addr learned my addr 172.21.15.102:0/2315130163 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:19:55.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.371+0000 7f8997e00640 1 -- 172.21.15.102:0/2315130163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0153180 msgr2=0x7f89a0158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:55.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.371+0000 7f8997e00640 1 --2- 172.21.15.102:0/2315130163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0153180 0x7f89a0158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.371+0000 7f8997e00640 1 -- 172.21.15.102:0/2315130163 >> v1:172.21.15.193:6789/0 conn(0x7f89a00c75a0 legacy=0x7f89a0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:55.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.372+0000 7f8997e00640 1 -- 172.21.15.102:0/2315130163 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a015c4d0 con 0x7f89a0159380 2024-04-20T21:19:55.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.371+0000 7f899ea00640 1 -- 172.21.15.102:0/2315130163 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014124078 0 0) 0x7f89a0153560 con 0x7f89a00c75a0 2024-04-20T21:19:55.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.372+0000 7f899f400640 1 --2- 172.21.15.102:0/2315130163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0153180 0x7f89a0158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:55.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.373+0000 7f8997e00640 1 --2- 172.21.15.102:0/2315130163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0159380 0x7f89a00c5f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f89880097a0 tx=0x7f898802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cab125c8bd8de5db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:55.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.373+0000 7f899ea00640 1 -- 172.21.15.102:0/2315130163 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988035020 con 0x7f89a0159380 2024-04-20T21:19:55.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.373+0000 7f899ea00640 1 -- 172.21.15.102:0/2315130163 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8988031810 con 0x7f89a0159380 2024-04-20T21:19:55.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.373+0000 7f899ea00640 1 -- 172.21.15.102:0/2315130163 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988031be0 con 0x7f89a0159380 2024-04-20T21:19:55.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.373+0000 7f89a5200640 1 -- 172.21.15.102:0/2315130163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0159380 msgr2=0x7f89a00c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:55.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.373+0000 7f89a5200640 1 --2- 172.21.15.102:0/2315130163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0159380 0x7f89a00c5f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f89880097a0 tx=0x7f898802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.374+0000 7f89a5200640 1 -- 172.21.15.102:0/2315130163 shutdown_connections 2024-04-20T21:19:55.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.374+0000 7f89a5200640 1 --2- 172.21.15.102:0/2315130163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0159380 0x7f89a00c5f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.374+0000 7f89a5200640 1 --2- 172.21.15.102:0/2315130163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0153180 0x7f89a0158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.374+0000 7f89a5200640 1 -- 172.21.15.102:0/2315130163 >> 172.21.15.102:0/2315130163 conn(0x7f89a00cc820 msgr2=0x7f89a00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:55.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.374+0000 7f89a5200640 1 -- 172.21.15.102:0/2315130163 shutdown_connections 2024-04-20T21:19:55.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.374+0000 7f89a5200640 1 -- 172.21.15.102:0/2315130163 wait complete. 2024-04-20T21:19:55.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.375+0000 7f89a5200640 1 Processor -- start 2024-04-20T21:19:55.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.375+0000 7f89a5200640 1 -- start start 2024-04-20T21:19:55.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.376+0000 7f89a5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89a00c75a0 0x7f89a014c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:55.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.376+0000 7f89a5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0153180 0x7f89a014f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:55.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.376+0000 7f89a5200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0159380 0x7f89a014fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:55.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.376+0000 7f89a5200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f89a015d290 con 0x7f89a00c75a0 2024-04-20T21:19:55.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.376+0000 7f89a5200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89a015ca90 con 0x7f89a0153180 2024-04-20T21:19:55.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.376+0000 7f89a5200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f89a00c6be0 con 0x7f89a0159380 2024-04-20T21:19:55.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f899f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89a00c75a0 0x7f89a014c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:55.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f8997e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0153180 0x7f89a014f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:55.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f899fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0159380 0x7f89a014fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:55.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f8997e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0153180 0x7f89a014f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:45768/0 (socket says 172.21.15.102:45768) 2024-04-20T21:19:55.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f8997e00640 1 -- 172.21.15.102:0/3012865794 learned_addr learned my addr 172.21.15.102:0/3012865794 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:55.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f8997e00640 1 -- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0159380 msgr2=0x7f89a014fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:55.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f8997e00640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0159380 0x7f89a014fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f8997e00640 1 -- 172.21.15.102:0/3012865794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89a00c75a0 msgr2=0x7f89a014c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f8997e00640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89a00c75a0 0x7f89a014c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f8997e00640 1 -- 172.21.15.102:0/3012865794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a014a420 con 0x7f89a0153180 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.377+0000 7f899f400640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89a00c75a0 0x7f89a014c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.378+0000 7f8997e00640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0153180 0x7f89a014f750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f898802bec0 tx=0x7f89880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.378+0000 7f8997400640 1 -- 172.21.15.102:0/3012865794 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988041070 con 0x7f89a0153180 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.378+0000 7f8997400640 1 -- 172.21.15.102:0/3012865794 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f898803ad00 con 0x7f89a0153180 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.378+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a014a700 con 0x7f89a0153180 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.378+0000 7f8997400640 1 -- 172.21.15.102:0/3012865794 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988035040 con 0x7f89a0153180 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.378+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a014b880 con 0x7f89a0153180 2024-04-20T21:19:55.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.379+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a00ccf30 con 0x7f89a0153180 2024-04-20T21:19:55.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.380+0000 7f8997400640 1 -- 172.21.15.102:0/3012865794 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8988038020 con 0x7f89a0153180 2024-04-20T21:19:55.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.381+0000 7f8997400640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f897c06c8f0 0x7f897c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:55.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.381+0000 7f8997400640 1 -- 172.21.15.102:0/3012865794 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f89880adb70 con 0x7f89a0153180 2024-04-20T21:19:55.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.381+0000 7f899f400640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f897c06c8f0 0x7f897c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:55.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.382+0000 7f899f400640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f897c06c8f0 0x7f897c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8990009560 tx=0x7f8990009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:55.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.386+0000 7f8997400640 1 -- 172.21.15.102:0/3012865794 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f89a00ccf30 con 0x7f89a0153180 2024-04-20T21:19:55.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.626+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89a004fd60 con 0x7f89a0153180 2024-04-20T21:19:55.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.627+0000 7f8997400640 1 -- 172.21.15.102:0/3012865794 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6027 (secure 0 0 0) 0x7f898807b450 con 0x7f89a0153180 2024-04-20T21:19:55.627 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:55.627 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:51.506369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:rejoin","state_seq":48,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:19:55.627 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 12 2024-04-20T21:19:55.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.629+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f897c06c8f0 msgr2=0x7f897c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.629+0000 7f89a5200640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f897c06c8f0 0x7f897c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8990009560 tx=0x7f8990009190 comp rx=0 tx=0).stop 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0153180 msgr2=0x7f89a014f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0153180 0x7f89a014f750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f898802bec0 tx=0x7f89880046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 shutdown_connections 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a0159380 0x7f89a014fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f897c06c8f0 0x7f897c06edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a0153180 0x7f89a014f750 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 --2- 172.21.15.102:0/3012865794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89a00c75a0 0x7f89a014c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 >> 172.21.15.102:0/3012865794 conn(0x7f89a00cc820 msgr2=0x7f89a01493a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 shutdown_connections 2024-04-20T21:19:55.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:55.630+0000 7f89a5200640 1 -- 172.21.15.102:0/3012865794 wait complete. 2024-04-20T21:19:55.649 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 12, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 12, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4297": { "addr": "172.21.15.102:6833/2795763369", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 2795763369, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 2795763369, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4297, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 48 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:19:51.506369+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4297 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:19:57.650 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:57.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.856+0000 7f8c10e00640 1 Processor -- start 2024-04-20T21:19:57.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.856+0000 7f8c10e00640 1 -- start start 2024-04-20T21:19:57.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c10e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8c0c152290 0x7f8c0c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c10e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c0c0c8330 0x7f8c0c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c10e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0c15c220 con 0x7f8c0c0c3700 2024-04-20T21:19:57.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c10e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c0c9180 con 0x7f8c0c152290 2024-04-20T21:19:57.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c10e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c0c4840 con 0x7f8c0c0c8330 2024-04-20T21:19:57.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c0b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8c0c152290 0x7f8c0c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c0b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8c0c152290 0x7f8c0c0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43910/0 (socket says 172.21.15.102:43910) 2024-04-20T21:19:57.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c0b400640 1 -- 172.21.15.102:0/2909639034 learned_addr learned my addr 172.21.15.102:0/2909639034 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:57.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c0be00640 1 --2- 172.21.15.102:0/2909639034 >> v2:172.21.15.193:3300/0 conn(0x7f8c0c0c8330 0x7f8c0c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c0b400640 1 -- 172.21.15.102:0/2909639034 >> v2:172.21.15.193:3300/0 conn(0x7f8c0c0c8330 msgr2=0x7f8c0c0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:57.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c0b400640 1 --2- 172.21.15.102:0/2909639034 >> v2:172.21.15.193:3300/0 conn(0x7f8c0c0c8330 0x7f8c0c0c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.858+0000 7f8c0b400640 1 -- 172.21.15.102:0/2909639034 >> v1:172.21.15.193:6789/0 conn(0x7f8c0c0c3700 legacy=0x7f8c0c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:57.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.859+0000 7f8c0b400640 1 -- 172.21.15.102:0/2909639034 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c0c4560 con 0x7f8c0c152290 2024-04-20T21:19:57.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.859+0000 7f8c0be00640 1 --2- 172.21.15.102:0/2909639034 >> v2:172.21.15.193:3300/0 conn(0x7f8c0c0c8330 0x7f8c0c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:19:57.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.860+0000 7f8c0b400640 1 --2- 172.21.15.102:0/2909639034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c152290 0x7f8c0c0cd310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c06e1b0 tx=0x7f8bfc02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bddbd888641997 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:57.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.860+0000 7f8c0a000640 1 -- 172.21.15.102:0/2909639034 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc02f850 con 0x7f8c0c152290 2024-04-20T21:19:57.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.860+0000 7f8c0a000640 1 -- 172.21.15.102:0/2909639034 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc02f9f0 con 0x7f8c0c152290 2024-04-20T21:19:57.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.860+0000 7f8c0a000640 1 -- 172.21.15.102:0/2909639034 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc02fdc0 con 0x7f8c0c152290 2024-04-20T21:19:57.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.860+0000 7f8c10e00640 1 -- 172.21.15.102:0/2909639034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c152290 msgr2=0x7f8c0c0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.860+0000 7f8c10e00640 1 --2- 172.21.15.102:0/2909639034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c152290 0x7f8c0c0cd310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c06e1b0 tx=0x7f8bfc02ba30 comp rx=0 tx=0).stop 2024-04-20T21:19:57.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.861+0000 7f8c10e00640 1 -- 172.21.15.102:0/2909639034 shutdown_connections 2024-04-20T21:19:57.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.861+0000 7f8c10e00640 1 --2- 172.21.15.102:0/2909639034 >> v2:172.21.15.193:3300/0 conn(0x7f8c0c0c8330 0x7f8c0c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.861+0000 7f8c10e00640 1 --2- 172.21.15.102:0/2909639034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c152290 0x7f8c0c0cd310 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.861+0000 7f8c10e00640 1 -- 172.21.15.102:0/2909639034 >> 172.21.15.102:0/2909639034 conn(0x7f8c0c0cc820 msgr2=0x7f8c0c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:57.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.861+0000 7f8c10e00640 1 -- 172.21.15.102:0/2909639034 shutdown_connections 2024-04-20T21:19:57.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.861+0000 7f8c10e00640 1 -- 172.21.15.102:0/2909639034 wait complete. 2024-04-20T21:19:57.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.862+0000 7f8c10e00640 1 Processor -- start 2024-04-20T21:19:57.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.863+0000 7f8c10e00640 1 -- start start 2024-04-20T21:19:57.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.863+0000 7f8c10e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c0c0c3700 0x7f8c0c157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.863+0000 7f8c10e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c0c8330 0x7f8c0c1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.863+0000 7f8c10e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c152290 0x7f8c0c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.863+0000 7f8c10e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c15ba20 con 0x7f8c0c0c8330 2024-04-20T21:19:57.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.863+0000 7f8c10e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c15b740 con 0x7f8c0c152290 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.863+0000 7f8c10e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c15bd00 con 0x7f8c0c0c3700 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c0c0c3700 0x7f8c0c157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c0c0c3700 0x7f8c0c157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:46998/0 (socket says 172.21.15.102:46998) 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 -- 172.21.15.102:0/3151599727 learned_addr learned my addr 172.21.15.102:0/3151599727 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0be00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c152290 0x7f8c0c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0aa00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c0c8330 0x7f8c0c1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 -- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c152290 msgr2=0x7f8c0c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c152290 0x7f8c0c152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 -- 172.21.15.102:0/3151599727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c0c8330 msgr2=0x7f8c0c1526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c0c8330 0x7f8c0c1526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 -- 172.21.15.102:0/3151599727 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c148040 con 0x7f8c0c0c3700 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.864+0000 7f8c0b400640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c0c0c3700 0x7f8c0c157fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:57.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.865+0000 7f8c03400640 1 -- 172.21.15.102:0/3151599727 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc047070 con 0x7f8c0c0c3700 2024-04-20T21:19:57.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.865+0000 7f8c03400640 1 -- 172.21.15.102:0/3151599727 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc037cb0 con 0x7f8c0c0c3700 2024-04-20T21:19:57.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.865+0000 7f8c03400640 1 -- 172.21.15.102:0/3151599727 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc037e60 con 0x7f8c0c0c3700 2024-04-20T21:19:57.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.865+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c1efa00 con 0x7f8c0c0c3700 2024-04-20T21:19:57.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.865+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1effc0 con 0x7f8c0c0c3700 2024-04-20T21:19:57.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.866+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c04fd60 con 0x7f8c0c0c3700 2024-04-20T21:19:57.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.867+0000 7f8c03400640 1 -- 172.21.15.102:0/3151599727 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bfc04a070 con 0x7f8c0c0c3700 2024-04-20T21:19:57.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.868+0000 7f8c03400640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bec06c920 0x7f8bec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.868+0000 7f8c03400640 1 -- 172.21.15.102:0/3151599727 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f8bfc0ae2d0 con 0x7f8c0c0c3700 2024-04-20T21:19:57.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.868+0000 7f8c0aa00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bec06c920 0x7f8bec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.869+0000 7f8c0aa00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bec06c920 0x7f8bec06ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0c7aa0 tx=0x7f8bf8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:57.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:57.873+0000 7f8c03400640 1 -- 172.21.15.102:0/3151599727 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8bfc033030 con 0x7f8c0c0c3700 2024-04-20T21:19:58.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.113+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c0c06dcb0 con 0x7f8c0c0c3700 2024-04-20T21:19:58.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.115+0000 7f8c03400640 1 -- 172.21.15.102:0/3151599727 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6033 (secure 0 0 0) 0x7f8bfc07bc40 con 0x7f8c0c0c3700 2024-04-20T21:19:58.114 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:19:58.115 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:57.056685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:clientreplay","state_seq":50,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:19:58.115 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 13 2024-04-20T21:19:58.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.117+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bec06c920 msgr2=0x7f8bec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:58.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.117+0000 7f8c10e00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bec06c920 0x7f8bec06ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0c7aa0 tx=0x7f8bf8009040 comp rx=0 tx=0).stop 2024-04-20T21:19:58.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.117+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c0c0c3700 msgr2=0x7f8c0c157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:58.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.117+0000 7f8c10e00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c0c0c3700 0x7f8c0c157fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc004730 comp rx=0 tx=0).stop 2024-04-20T21:19:58.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.118+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 shutdown_connections 2024-04-20T21:19:58.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.118+0000 7f8c10e00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bec06c920 0x7f8bec06ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:58.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.118+0000 7f8c10e00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c152290 0x7f8c0c152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:58.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.118+0000 7f8c10e00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0c0c8330 0x7f8c0c1526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:58.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.118+0000 7f8c10e00640 1 --2- 172.21.15.102:0/3151599727 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c0c0c3700 0x7f8c0c157fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:58.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.118+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 >> 172.21.15.102:0/3151599727 conn(0x7f8c0c0cc820 msgr2=0x7f8c0c0c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:58.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.118+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 shutdown_connections 2024-04-20T21:19:58.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:19:58.118+0000 7f8c10e00640 1 -- 172.21.15.102:0/3151599727 wait complete. 2024-04-20T21:19:58.136 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:20:00.138 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:00.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.352+0000 7f17ea200640 1 Processor -- start 2024-04-20T21:20:00.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.352+0000 7f17ea200640 1 -- start start 2024-04-20T21:20:00.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.353+0000 7f17ea200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f17e4155530 0x7f17e415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.353+0000 7f17ea200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f17e40cb6a0 0x7f17e40cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.353+0000 7f17ea200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17e415ef60 con 0x7f17e40cc470 2024-04-20T21:20:00.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.353+0000 7f17ea200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f17e40c5ca0 con 0x7f17e4155530 2024-04-20T21:20:00.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.353+0000 7f17ea200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f17e40cd5f0 con 0x7f17e40cb6a0 2024-04-20T21:20:00.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e8e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f17e4155530 0x7f17e415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e3e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f17e40cc470 0x7f17e40cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37530/0 (socket says 172.21.15.102:37530) 2024-04-20T21:20:00.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e3e00640 1 -- 172.21.15.102:0/2895150882 learned_addr learned my addr 172.21.15.102:0/2895150882 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:20:00.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e9800640 1 --2- 172.21.15.102:0/2895150882 >> v2:172.21.15.193:3300/0 conn(0x7f17e40cb6a0 0x7f17e40cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e9800640 1 -- 172.21.15.102:0/2895150882 >> v2:172.21.15.102:3300/0 conn(0x7f17e4155530 msgr2=0x7f17e415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:00.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e9800640 1 --2- 172.21.15.102:0/2895150882 >> v2:172.21.15.102:3300/0 conn(0x7f17e4155530 0x7f17e415e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e9800640 1 -- 172.21.15.102:0/2895150882 >> v1:172.21.15.102:6789/0 conn(0x7f17e40cc470 legacy=0x7f17e40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e9800640 1 -- 172.21.15.102:0/2895150882 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e415f240 con 0x7f17e40cb6a0 2024-04-20T21:20:00.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.354+0000 7f17e8e00640 1 --2- 172.21.15.102:0/2895150882 >> v2:172.21.15.102:3300/0 conn(0x7f17e4155530 0x7f17e415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:20:00.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.356+0000 7f17e9800640 1 --2- 172.21.15.102:0/2895150882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cb6a0 0x7f17e40cbab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17e408e290 tx=0x7f17dc02bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7821fb9b1d94cf46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.356+0000 7f17e3400640 1 -- 172.21.15.102:0/2895150882 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc039070 con 0x7f17e40cb6a0 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.356+0000 7f17e3400640 1 -- 172.21.15.102:0/2895150882 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17dc02cc70 con 0x7f17e40cb6a0 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.356+0000 7f17e3400640 1 -- 172.21.15.102:0/2895150882 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc034660 con 0x7f17e40cb6a0 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.356+0000 7f17ea200640 1 -- 172.21.15.102:0/2895150882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cb6a0 msgr2=0x7f17e40cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.356+0000 7f17ea200640 1 --2- 172.21.15.102:0/2895150882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cb6a0 0x7f17e40cbab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17e408e290 tx=0x7f17dc02bec0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.357+0000 7f17ea200640 1 -- 172.21.15.102:0/2895150882 shutdown_connections 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.357+0000 7f17ea200640 1 --2- 172.21.15.102:0/2895150882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cb6a0 0x7f17e40cbab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.357+0000 7f17ea200640 1 --2- 172.21.15.102:0/2895150882 >> v2:172.21.15.102:3300/0 conn(0x7f17e4155530 0x7f17e415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.357+0000 7f17ea200640 1 -- 172.21.15.102:0/2895150882 >> 172.21.15.102:0/2895150882 conn(0x7f17e4075080 msgr2=0x7f17e4075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.357+0000 7f17ea200640 1 -- 172.21.15.102:0/2895150882 shutdown_connections 2024-04-20T21:20:00.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.357+0000 7f17ea200640 1 -- 172.21.15.102:0/2895150882 wait complete. 2024-04-20T21:20:00.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.358+0000 7f17ea200640 1 Processor -- start 2024-04-20T21:20:00.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.359+0000 7f17ea200640 1 -- start start 2024-04-20T21:20:00.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.359+0000 7f17ea200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17e40cb6a0 0x7f17e4154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.359+0000 7f17ea200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cc470 0x7f17e41f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.359+0000 7f17ea200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17e4155530 0x7f17e41420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.359+0000 7f17ea200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f17e40cd280 con 0x7f17e40cb6a0 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.359+0000 7f17ea200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17e40ccfa0 con 0x7f17e40cc470 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.359+0000 7f17ea200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f17e40ccbe0 con 0x7f17e4155530 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17e40cb6a0 0x7f17e4154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cc470 0x7f17e41f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17e40cb6a0 0x7f17e4154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43942/0 (socket says 172.21.15.102:43942) 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e9800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17e4155530 0x7f17e41420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 -- 172.21.15.102:0/3570760205 learned_addr learned my addr 172.21.15.102:0/3570760205 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 -- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17e4155530 msgr2=0x7f17e41420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17e4155530 0x7f17e41420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 -- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cc470 msgr2=0x7f17e41f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cc470 0x7f17e41f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 -- 172.21.15.102:0/3570760205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e415c7d0 con 0x7f17e40cb6a0 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e9800640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17e4155530 0x7f17e41420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e3e00640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cc470 0x7f17e41f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.360+0000 7f17e8e00640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17e40cb6a0 0x7f17e4154340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17d800e960 tx=0x7f17d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.361+0000 7f17e0c00640 1 -- 172.21.15.102:0/3570760205 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8017070 con 0x7f17e40cb6a0 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.361+0000 7f17e0c00640 1 -- 172.21.15.102:0/3570760205 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d800ca00 con 0x7f17e40cb6a0 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.361+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e40b6b10 con 0x7f17e40cb6a0 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.361+0000 7f17e0c00640 1 -- 172.21.15.102:0/3570760205 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d800cdd0 con 0x7f17e40cb6a0 2024-04-20T21:20:00.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.361+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e41e4470 con 0x7f17e40cb6a0 2024-04-20T21:20:00.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.362+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e404fd60 con 0x7f17e40cb6a0 2024-04-20T21:20:00.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.363+0000 7f17e0c00640 1 -- 172.21.15.102:0/3570760205 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17d801a070 con 0x7f17e40cb6a0 2024-04-20T21:20:00.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.364+0000 7f17e0c00640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17c406c8f0 0x7f17c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.364+0000 7f17e0c00640 1 -- 172.21.15.102:0/3570760205 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f17d808bdc0 con 0x7f17e40cb6a0 2024-04-20T21:20:00.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.365+0000 7f17e3e00640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17c406c8f0 0x7f17c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.365+0000 7f17e3e00640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17c406c8f0 0x7f17c406edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b39e0 tx=0x7f17d0007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.373+0000 7f17e0c00640 1 -- 172.21.15.102:0/3570760205 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f17d8059730 con 0x7f17e40cb6a0 2024-04-20T21:20:00.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.612+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17e40cc1a0 con 0x7f17e40cb6a0 2024-04-20T21:20:00.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.613+0000 7f17e0c00640 1 -- 172.21.15.102:0/3570760205 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6033 (secure 0 0 0) 0x7f17d805da30 con 0x7f17e40cb6a0 2024-04-20T21:20:00.613 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:20:00.613 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:57.056685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:clientreplay","state_seq":50,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:20:00.613 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 13 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17c406c8f0 msgr2=0x7f17c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17c406c8f0 0x7f17c406edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b39e0 tx=0x7f17d0007e80 comp rx=0 tx=0).stop 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17e40cb6a0 msgr2=0x7f17e4154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17e40cb6a0 0x7f17e4154340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17d800e960 tx=0x7f17d800ee20 comp rx=0 tx=0).stop 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 shutdown_connections 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17e4155530 0x7f17e41420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17c406c8f0 0x7f17c406edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17e40cc470 0x7f17e41f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 --2- 172.21.15.102:0/3570760205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17e40cb6a0 0x7f17e4154340 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 >> 172.21.15.102:0/3570760205 conn(0x7f17e4075080 msgr2=0x7f17e4144470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:00.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 shutdown_connections 2024-04-20T21:20:00.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:00.616+0000 7f17ea200640 1 -- 172.21.15.102:0/3570760205 wait complete. 2024-04-20T21:20:00.633 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:20:02.634 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:02.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.854+0000 7f8693e00640 1 Processor -- start 2024-04-20T21:20:02.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.854+0000 7f8693e00640 1 -- start start 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.855+0000 7f8693e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.855+0000 7f8693e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f868c0c9eb0 0x7f868c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.855+0000 7f8693e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868c1536d0 con 0x7f868c073a10 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.855+0000 7f8693e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f868c159500 con 0x7f868c0c9eb0 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.855+0000 7f8693e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f868c0c76c0 con 0x7f868c14fa50 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f8690a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f868c0c9eb0 0x7f868c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f8690a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f868c0c9eb0 0x7f868c158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43958/0 (socket says 172.21.15.102:43958) 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f8690a00640 1 -- 172.21.15.102:0/3144319188 learned_addr learned my addr 172.21.15.102:0/3144319188 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:20:02.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f8691400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f868be00640 1 -- 172.21.15.102:0/3144319188 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768211421 0 0) 0x7f8684001490 con 0x7f868c073a10 2024-04-20T21:20:02.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f8691400640 1 -- 172.21.15.102:0/3144319188 >> v2:172.21.15.102:3300/0 conn(0x7f868c0c9eb0 msgr2=0x7f868c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:02.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f8691400640 1 --2- 172.21.15.102:0/3144319188 >> v2:172.21.15.102:3300/0 conn(0x7f868c0c9eb0 0x7f868c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f8691400640 1 -- 172.21.15.102:0/3144319188 >> v1:172.21.15.193:6789/0 conn(0x7f868c073a10 legacy=0x7f868c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.856+0000 7f8691400640 1 -- 172.21.15.102:0/3144319188 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c0c7960 con 0x7f868c14fa50 2024-04-20T21:20:02.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.857+0000 7f8691400640 1 --2- 172.21.15.102:0/3144319188 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c0c4830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f868c0c7b30 tx=0x7f868002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9cb6578c9ccff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:02.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.858+0000 7f868be00640 1 -- 172.21.15.102:0/3144319188 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680035020 con 0x7f868c14fa50 2024-04-20T21:20:02.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.858+0000 7f868be00640 1 -- 172.21.15.102:0/3144319188 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8680031890 con 0x7f868c14fa50 2024-04-20T21:20:02.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.858+0000 7f868be00640 1 -- 172.21.15.102:0/3144319188 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680031c60 con 0x7f868c14fa50 2024-04-20T21:20:02.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.859+0000 7f8693e00640 1 -- 172.21.15.102:0/3144319188 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 msgr2=0x7f868c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.859+0000 7f8693e00640 1 --2- 172.21.15.102:0/3144319188 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c0c4830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f868c0c7b30 tx=0x7f868002b930 comp rx=0 tx=0).stop 2024-04-20T21:20:02.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.859+0000 7f8693e00640 1 -- 172.21.15.102:0/3144319188 shutdown_connections 2024-04-20T21:20:02.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.859+0000 7f8693e00640 1 --2- 172.21.15.102:0/3144319188 >> v2:172.21.15.102:3300/0 conn(0x7f868c0c9eb0 0x7f868c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.859+0000 7f8693e00640 1 --2- 172.21.15.102:0/3144319188 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c0c4830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.859+0000 7f8693e00640 1 -- 172.21.15.102:0/3144319188 >> 172.21.15.102:0/3144319188 conn(0x7f868c0c9390 msgr2=0x7f868c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:02.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.859+0000 7f8693e00640 1 -- 172.21.15.102:0/3144319188 shutdown_connections 2024-04-20T21:20:02.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.859+0000 7f8693e00640 1 -- 172.21.15.102:0/3144319188 wait complete. 2024-04-20T21:20:02.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.862+0000 7f8693e00640 1 Processor -- start 2024-04-20T21:20:02.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.862+0000 7f8693e00640 1 -- start start 2024-04-20T21:20:02.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.863+0000 7f8693e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c073a10 0x7f868c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.863+0000 7f8693e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868c0c9eb0 0x7f868c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.863+0000 7f8693e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.863+0000 7f8693e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f868c0c7c00 con 0x7f868c0c9eb0 2024-04-20T21:20:02.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.863+0000 7f8693e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f868c0c3d90 con 0x7f868c073a10 2024-04-20T21:20:02.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.863+0000 7f8693e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f868c0c4030 con 0x7f868c14fa50 2024-04-20T21:20:02.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c073a10 0x7f868c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8690a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868c0c9eb0 0x7f868c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c073a10 0x7f868c0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:42752/0 (socket says 172.21.15.102:42752) 2024-04-20T21:20:02.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 -- 172.21.15.102:0/3958946247 learned_addr learned my addr 172.21.15.102:0/3958946247 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:20:02.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691e00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 -- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 msgr2=0x7f868c06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c06d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 -- 172.21.15.102:0/3958946247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868c0c9eb0 msgr2=0x7f868c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868c0c9eb0 0x7f868c06d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 -- 172.21.15.102:0/3958946247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c158f90 con 0x7f868c073a10 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.864+0000 7f8691400640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c073a10 0x7f868c0cb7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f868002bf40 tx=0x7f86800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.865+0000 7f8689600640 1 -- 172.21.15.102:0/3958946247 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680035040 con 0x7f868c073a10 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.865+0000 7f8689600640 1 -- 172.21.15.102:0/3958946247 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8680030cb0 con 0x7f868c073a10 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.865+0000 7f8689600640 1 -- 172.21.15.102:0/3958946247 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680030e60 con 0x7f868c073a10 2024-04-20T21:20:02.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.865+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c159230 con 0x7f868c073a10 2024-04-20T21:20:02.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.865+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c157f20 con 0x7f868c073a10 2024-04-20T21:20:02.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.867+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c074340 con 0x7f868c073a10 2024-04-20T21:20:02.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.867+0000 7f8689600640 1 -- 172.21.15.102:0/3958946247 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8680038020 con 0x7f868c073a10 2024-04-20T21:20:02.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.868+0000 7f8689600640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f866c06c8f0 0x7f866c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.868+0000 7f8689600640 1 -- 172.21.15.102:0/3958946247 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f86800ae280 con 0x7f868c073a10 2024-04-20T21:20:02.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.868+0000 7f8690a00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f866c06c8f0 0x7f866c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.869+0000 7f8690a00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f866c06c8f0 0x7f866c06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f868c071690 tx=0x7f8678009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:02.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:02.873+0000 7f8689600640 1 -- 172.21.15.102:0/3958946247 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f868002d040 con 0x7f868c073a10 2024-04-20T21:20:03.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.168+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f868c0c7320 con 0x7f868c073a10 2024-04-20T21:20:03.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.169+0000 7f8689600640 1 -- 172.21.15.102:0/3958946247 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6033 (secure 0 0 0) 0x7f868007bc90 con 0x7f868c073a10 2024-04-20T21:20:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:20:03.169 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:19:57.056685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:clientreplay","state_seq":50,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:20:03.170 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 13 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f866c06c8f0 msgr2=0x7f866c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f866c06c8f0 0x7f866c06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f868c071690 tx=0x7f8678009240 comp rx=0 tx=0).stop 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c073a10 msgr2=0x7f868c0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c073a10 0x7f868c0cb7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f868002bf40 tx=0x7f86800046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 shutdown_connections 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f868c14fa50 0x7f868c06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f868c0c9eb0 0x7f868c06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f866c06c8f0 0x7f866c06edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 --2- 172.21.15.102:0/3958946247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c073a10 0x7f868c0cb7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 >> 172.21.15.102:0/3958946247 conn(0x7f868c0c9390 msgr2=0x7f868c06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:03.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.172+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 shutdown_connections 2024-04-20T21:20:03.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:03.173+0000 7f8693e00640 1 -- 172.21.15.102:0/3958946247 wait complete. 2024-04-20T21:20:03.190 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:20:05.191 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:05.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.400+0000 7fc1bfe00640 1 Processor -- start 2024-04-20T21:20:05.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.400+0000 7fc1bfe00640 1 -- start start 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.401+0000 7fc1bfe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1c0153180 0x7fc1c0158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bfe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc1c00c75a0 0x7fc1c01637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bfe00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c0153560 con 0x7fc1c0159380 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bfe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c00c7a70 con 0x7fc1c0153180 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bfe00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c00c7d30 con 0x7fc1c00c75a0 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1c0153180 0x7fc1c0158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1c0153180 0x7fc1c0158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43976/0 (socket says 172.21.15.102:43976) 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bea00640 1 -- 172.21.15.102:0/715789193 learned_addr learned my addr 172.21.15.102:0/715789193 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bf400640 1 --2- 172.21.15.102:0/715789193 >> v2:172.21.15.193:3300/0 conn(0x7fc1c00c75a0 0x7fc1c01637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bea00640 1 -- 172.21.15.102:0/715789193 >> v2:172.21.15.193:3300/0 conn(0x7fc1c00c75a0 msgr2=0x7fc1c01637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bea00640 1 --2- 172.21.15.102:0/715789193 >> v2:172.21.15.193:3300/0 conn(0x7fc1c00c75a0 0x7fc1c01637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bea00640 1 -- 172.21.15.102:0/715789193 >> v1:172.21.15.193:6789/0 conn(0x7fc1c0159380 legacy=0x7fc1c00c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.402+0000 7fc1bea00640 1 -- 172.21.15.102:0/715789193 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c015c4d0 con 0x7fc1c0153180 2024-04-20T21:20:05.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.403+0000 7fc1bea00640 1 --2- 172.21.15.102:0/715789193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c0153180 0x7fc1c0158e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0159760 tx=0x7fc1b002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af0e4634e82b59d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:05.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.404+0000 7fc1be000640 1 -- 172.21.15.102:0/715789193 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0034070 con 0x7fc1c0153180 2024-04-20T21:20:05.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.404+0000 7fc1be000640 1 -- 172.21.15.102:0/715789193 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b0030860 con 0x7fc1c0153180 2024-04-20T21:20:05.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.404+0000 7fc1be000640 1 -- 172.21.15.102:0/715789193 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0030c30 con 0x7fc1c0153180 2024-04-20T21:20:05.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.404+0000 7fc1bfe00640 1 -- 172.21.15.102:0/715789193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c0153180 msgr2=0x7fc1c0158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.404+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/715789193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c0153180 0x7fc1c0158e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0159760 tx=0x7fc1b002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.405+0000 7fc1bfe00640 1 -- 172.21.15.102:0/715789193 shutdown_connections 2024-04-20T21:20:05.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.405+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/715789193 >> v2:172.21.15.193:3300/0 conn(0x7fc1c00c75a0 0x7fc1c01637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.405+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/715789193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c0153180 0x7fc1c0158e40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.405+0000 7fc1bfe00640 1 -- 172.21.15.102:0/715789193 >> 172.21.15.102:0/715789193 conn(0x7fc1c00cc820 msgr2=0x7fc1c00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.405+0000 7fc1bfe00640 1 -- 172.21.15.102:0/715789193 shutdown_connections 2024-04-20T21:20:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.405+0000 7fc1bfe00640 1 -- 172.21.15.102:0/715789193 wait complete. 2024-04-20T21:20:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.406+0000 7fc1bfe00640 1 Processor -- start 2024-04-20T21:20:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.406+0000 7fc1bfe00640 1 -- start start 2024-04-20T21:20:05.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.407+0000 7fc1bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c00c75a0 0x7fc1c0145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.407+0000 7fc1bfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1c0153180 0x7fc1c01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.407+0000 7fc1bfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c0159380 0x7fc1c0157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.407+0000 7fc1bfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c015d290 con 0x7fc1c00c75a0 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.407+0000 7fc1bfe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c015ca90 con 0x7fc1c0159380 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.407+0000 7fc1bfe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c00c6be0 con 0x7fc1c0153180 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c00c75a0 0x7fc1c0145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1b7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1c0153180 0x7fc1c01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c00c75a0 0x7fc1c0145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43990/0 (socket says 172.21.15.102:43990) 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 -- 172.21.15.102:0/2195718742 learned_addr learned my addr 172.21.15.102:0/2195718742 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bf400640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c0159380 0x7fc1c0157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 -- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1c0153180 msgr2=0x7fc1c01578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1c0153180 0x7fc1c01578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 -- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c0159380 msgr2=0x7fc1c0157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c0159380 0x7fc1c0157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 -- 172.21.15.102:0/2195718742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c00cb1f0 con 0x7fc1c00c75a0 2024-04-20T21:20:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1b7e00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1c0153180 0x7fc1c01578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:20:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.408+0000 7fc1bea00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c00c75a0 0x7fc1c0145bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0000c00 tx=0x7fc1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.409+0000 7fc1b7400640 1 -- 172.21.15.102:0/2195718742 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0034040 con 0x7fc1c00c75a0 2024-04-20T21:20:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.409+0000 7fc1b7400640 1 -- 172.21.15.102:0/2195718742 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b0039d90 con 0x7fc1c00c75a0 2024-04-20T21:20:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.409+0000 7fc1b7400640 1 -- 172.21.15.102:0/2195718742 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0043610 con 0x7fc1c00c75a0 2024-04-20T21:20:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.409+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c0159170 con 0x7fc1c00c75a0 2024-04-20T21:20:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.409+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c00caf10 con 0x7fc1c00c75a0 2024-04-20T21:20:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.410+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c004fd60 con 0x7fc1c00c75a0 2024-04-20T21:20:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.411+0000 7fc1b7400640 1 -- 172.21.15.102:0/2195718742 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1b0037070 con 0x7fc1c00c75a0 2024-04-20T21:20:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.412+0000 7fc1b7400640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc19c06c920 0x7fc19c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.412+0000 7fc1b7400640 1 -- 172.21.15.102:0/2195718742 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7fc1b00adff0 con 0x7fc1c00c75a0 2024-04-20T21:20:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.412+0000 7fc1b7e00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc19c06c920 0x7fc19c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.413+0000 7fc1b7e00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc19c06c920 0x7fc19c06ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0156960 tx=0x7fc1a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:05.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.416+0000 7fc1b7400640 1 -- 172.21.15.102:0/2195718742 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc1b003e070 con 0x7fc1c00c75a0 2024-04-20T21:20:05.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.663+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1c0072100 con 0x7fc1c00c75a0 2024-04-20T21:20:05.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.664+0000 7fc1b7400640 1 -- 172.21.15.102:0/2195718742 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6034 (secure 0 0 0) 0x7fc1b007b8f0 con 0x7fc1c00c75a0 2024-04-20T21:20:05.664 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:20:05.664 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:20:05.664 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 14 2024-04-20T21:20:05.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc19c06c920 msgr2=0x7fc19c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc19c06c920 0x7fc19c06ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0156960 tx=0x7fc1a8009040 comp rx=0 tx=0).stop 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c00c75a0 msgr2=0x7fc1c0145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c00c75a0 0x7fc1c0145bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0000c00 tx=0x7fc1b00046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 shutdown_connections 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc19c06c920 0x7fc19c06ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c0159380 0x7fc1c0157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1c0153180 0x7fc1c01578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 --2- 172.21.15.102:0/2195718742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c00c75a0 0x7fc1c0145bb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 >> 172.21.15.102:0/2195718742 conn(0x7fc1c00cc820 msgr2=0x7fc1c0151720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 shutdown_connections 2024-04-20T21:20:05.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:05.667+0000 7fc1bfe00640 1 -- 172.21.15.102:0/2195718742 wait complete. 2024-04-20T21:20:05.685 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=4300 2024-04-20T21:20:05.685 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.c 2024-04-20T21:20:08.622 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 127 2024-04-20T21:20:08.666 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 128 2024-04-20T21:20:08.697 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 129 2024-04-20T21:20:08.726 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 130 2024-04-20T21:20:08.772 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 131 2024-04-20T21:20:08.821 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 132 2024-04-20T21:20:08.872 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 133 2024-04-20T21:20:08.925 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 134 2024-04-20T21:20:08.960 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 135 2024-04-20T21:20:08.999 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 136 2024-04-20T21:20:09.034 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 137 2024-04-20T21:20:09.073 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 138 2024-04-20T21:20:09.114 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 139 2024-04-20T21:20:09.152 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 140 2024-04-20T21:20:09.268 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 141 2024-04-20T21:20:09.321 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 142 2024-04-20T21:20:09.374 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 143 2024-04-20T21:20:09.428 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 144 2024-04-20T21:20:09.471 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 145 2024-04-20T21:20:09.524 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 146 2024-04-20T21:20:09.550 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 147 2024-04-20T21:20:09.685 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:20:09.686 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:20:09.686 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:20:09.691 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:20:09.691 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:09.787 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-04-20T21:20:09.788+0000 7f1ce8600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:20:09.805 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 148 2024-04-20T21:20:09.822 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 149 2024-04-20T21:20:09.841 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 150 2024-04-20T21:20:09.854 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:20:09.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.890+0000 7ff463e00640 1 Processor -- start 2024-04-20T21:20:09.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.890+0000 7ff463e00640 1 -- start start 2024-04-20T21:20:09.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.891+0000 7ff463e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff464155530 0x7ff46415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.891+0000 7ff463e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4640cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.891+0000 7ff463e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff46415ef60 con 0x7ff4640cc470 2024-04-20T21:20:09.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.891+0000 7ff463e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff4640c5ca0 con 0x7ff464155530 2024-04-20T21:20:09.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.891+0000 7ff463e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff4640cd5f0 con 0x7ff4640cb6a0 2024-04-20T21:20:09.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.892+0000 7ff462a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff464155530 0x7ff46415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.892+0000 7ff463400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4640cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.892+0000 7ff462000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff4640cc470 0x7ff4640cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:54760/0 (socket says 172.21.15.102:54760) 2024-04-20T21:20:09.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.892+0000 7ff462000640 1 -- 172.21.15.102:0/3854207414 learned_addr learned my addr 172.21.15.102:0/3854207414 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:20:09.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.892+0000 7ff462a00640 1 -- 172.21.15.102:0/3854207414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 msgr2=0x7ff4640cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:09.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.892+0000 7ff462a00640 1 --2- 172.21.15.102:0/3854207414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4640cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.892+0000 7ff462a00640 1 -- 172.21.15.102:0/3854207414 >> v1:172.21.15.193:6789/0 conn(0x7ff4640cc470 legacy=0x7ff4640cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.892+0000 7ff462a00640 1 -- 172.21.15.102:0/3854207414 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46415f240 con 0x7ff464155530 2024-04-20T21:20:09.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.893+0000 7ff462a00640 1 --2- 172.21.15.102:0/3854207414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff464155530 0x7ff46415e640 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff4540097a0 tx=0x7ff45402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2240a704fc507e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.893+0000 7ff461600640 1 -- 172.21.15.102:0/3854207414 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454035020 con 0x7ff464155530 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.893+0000 7ff461600640 1 -- 172.21.15.102:0/3854207414 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff454031810 con 0x7ff464155530 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.893+0000 7ff461600640 1 -- 172.21.15.102:0/3854207414 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454031be0 con 0x7ff464155530 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.894+0000 7ff463e00640 1 -- 172.21.15.102:0/3854207414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff464155530 msgr2=0x7ff46415e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.894+0000 7ff463e00640 1 --2- 172.21.15.102:0/3854207414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff464155530 0x7ff46415e640 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff4540097a0 tx=0x7ff45402b8b0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.894+0000 7ff463e00640 1 -- 172.21.15.102:0/3854207414 shutdown_connections 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.894+0000 7ff463e00640 1 --2- 172.21.15.102:0/3854207414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4640cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.894+0000 7ff463e00640 1 --2- 172.21.15.102:0/3854207414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff464155530 0x7ff46415e640 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.894+0000 7ff463e00640 1 -- 172.21.15.102:0/3854207414 >> 172.21.15.102:0/3854207414 conn(0x7ff464075080 msgr2=0x7ff464075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.894+0000 7ff463e00640 1 -- 172.21.15.102:0/3854207414 shutdown_connections 2024-04-20T21:20:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.894+0000 7ff463e00640 1 -- 172.21.15.102:0/3854207414 wait complete. 2024-04-20T21:20:09.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.896+0000 7ff463e00640 1 Processor -- start 2024-04-20T21:20:09.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.896+0000 7ff463e00640 1 -- start start 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.896+0000 7ff463e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4641f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff463e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4640cc470 0x7ff464152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff463e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff464155530 0x7ff4641e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff463e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff4640cd280 con 0x7ff464155530 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff463e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4640ccfa0 con 0x7ff4640cc470 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff463e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff4640ccbe0 con 0x7ff4640cb6a0 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4640cc470 0x7ff464152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4641f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4641f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:37162/0 (socket says 172.21.15.102:37162) 2024-04-20T21:20:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4640cc470 0x7ff464152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:59258/0 (socket says 172.21.15.102:59258) 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462a00640 1 -- 172.21.15.102:0/753219531 learned_addr learned my addr 172.21.15.102:0/753219531 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff463400640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff464155530 0x7ff4641e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462a00640 1 -- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4640cc470 msgr2=0x7ff464152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462a00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4640cc470 0x7ff464152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462a00640 1 -- 172.21.15.102:0/753219531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff464155530 msgr2=0x7ff4641e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462a00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff464155530 0x7ff4641e53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462a00640 1 -- 172.21.15.102:0/753219531 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff464153730 con 0x7ff4640cb6a0 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.897+0000 7ff462000640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4640cc470 0x7ff464152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:20:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.898+0000 7ff462a00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4641f21a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff4540098c0 tx=0x7ff4540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.898+0000 7ff45aa00640 1 -- 172.21.15.102:0/753219531 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454035040 con 0x7ff4640cb6a0 2024-04-20T21:20:09.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.898+0000 7ff45aa00640 1 -- 172.21.15.102:0/753219531 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff454039cf0 con 0x7ff4640cb6a0 2024-04-20T21:20:09.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.898+0000 7ff45aa00640 1 -- 172.21.15.102:0/753219531 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454039ea0 con 0x7ff4640cb6a0 2024-04-20T21:20:09.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.898+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff464153cd0 con 0x7ff4640cb6a0 2024-04-20T21:20:09.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.898+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641f16c0 con 0x7ff4640cb6a0 2024-04-20T21:20:09.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.899+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46404fd60 con 0x7ff4640cb6a0 2024-04-20T21:20:09.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.900+0000 7ff45aa00640 1 -- 172.21.15.102:0/753219531 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff454038020 con 0x7ff4640cb6a0 2024-04-20T21:20:09.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.901+0000 7ff45aa00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff44406c8f0 0x7ff44406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.901+0000 7ff45aa00640 1 -- 172.21.15.102:0/753219531 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6933+0+0 (secure 0 0 0) 0x7ff4540adc60 con 0x7ff4640cb6a0 2024-04-20T21:20:09.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.901+0000 7ff462000640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff44406c8f0 0x7ff44406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.902+0000 7ff462000640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff44406c8f0 0x7ff44406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff464152e00 tx=0x7ff450007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:09.906+0000 7ff45aa00640 1 -- 172.21.15.102:0/753219531 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff45407b4d0 con 0x7ff4640cb6a0 2024-04-20T21:20:10.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.144+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4640cc9c0 con 0x7ff4640cb6a0 2024-04-20T21:20:10.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.145+0000 7ff45aa00640 1 -- 172.21.15.102:0/753219531 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6034 (secure 0 0 0) 0x7ff45407f7d0 con 0x7ff4640cb6a0 2024-04-20T21:20:10.145 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:20:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:20:10.145 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 14 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff44406c8f0 msgr2=0x7ff44406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff44406c8f0 0x7ff44406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff464152e00 tx=0x7ff450007e80 comp rx=0 tx=0).stop 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 msgr2=0x7ff4641f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4641f21a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff4540098c0 tx=0x7ff4540046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 shutdown_connections 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff464155530 0x7ff4641e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff44406c8f0 0x7ff44406edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4640cc470 0x7ff464152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 --2- 172.21.15.102:0/753219531 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4640cb6a0 0x7ff4641f21a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 >> 172.21.15.102:0/753219531 conn(0x7ff464075080 msgr2=0x7ff46414fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 shutdown_connections 2024-04-20T21:20:10.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:10.148+0000 7ff463e00640 1 -- 172.21.15.102:0/753219531 wait complete. 2024-04-20T21:20:10.164 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 151 2024-04-20T21:20:10.167 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-20T21:20:10.189 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 152 2024-04-20T21:20:10.228 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 153 2024-04-20T21:20:10.266 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 154 2024-04-20T21:20:10.310 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 155 2024-04-20T21:20:10.362 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 156 2024-04-20T21:20:10.417 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 157 2024-04-20T21:20:10.472 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 158 2024-04-20T21:20:10.526 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 159 2024-04-20T21:20:10.566 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 160 2024-04-20T21:20:10.609 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 161 2024-04-20T21:20:10.643 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 162 2024-04-20T21:20:10.671 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 163 2024-04-20T21:20:10.712 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 164 2024-04-20T21:20:10.765 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 165 2024-04-20T21:20:10.816 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 166 2024-04-20T21:20:10.870 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 167 2024-04-20T21:20:10.907 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 168 2024-04-20T21:20:10.944 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 169 2024-04-20T21:20:10.981 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 170 2024-04-20T21:20:11.017 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 171 2024-04-20T21:20:11.055 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 172 2024-04-20T21:20:11.094 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 173 2024-04-20T21:20:11.133 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 174 2024-04-20T21:20:11.173 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 175 2024-04-20T21:20:11.212 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 176 2024-04-20T21:20:11.250 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 177 2024-04-20T21:20:11.290 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 178 2024-04-20T21:20:11.329 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 179 2024-04-20T21:20:11.365 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 180 2024-04-20T21:20:11.408 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 181 2024-04-20T21:20:11.448 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 182 2024-04-20T21:20:12.168 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:12.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.377+0000 7f6b9ec00640 1 Processor -- start 2024-04-20T21:20:12.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.377+0000 7f6b9ec00640 1 -- start start 2024-04-20T21:20:12.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b9ec00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c68e0 0x7f6b980c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b9ec00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6b980c86d0 0x7f6b9806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b9ec00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b980c8b40 con 0x7f6b98153430 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b9ec00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6b980c8dd0 con 0x7f6b980c86d0 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b9ec00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6b98159310 con 0x7f6b980c68e0 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b9cc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6b980c86d0 0x7f6b9806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b97400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c68e0 0x7f6b980c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b97400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c68e0 0x7f6b980c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:37176/0 (socket says 172.21.15.102:37176) 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b97400640 1 -- 172.21.15.102:0/4092595868 learned_addr learned my addr 172.21.15.102:0/4092595868 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:20:12.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.379+0000 7f6b97e00640 1 --1- 172.21.15.102:0/4092595868 >> v1:172.21.15.102:6789/0 conn(0x7f6b98153430 0x7f6b980c95a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57050/0 (socket says 172.21.15.102:57050) 2024-04-20T21:20:12.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.380+0000 7f6b97400640 1 -- 172.21.15.102:0/4092595868 >> v2:172.21.15.193:3300/0 conn(0x7f6b980c86d0 msgr2=0x7f6b9806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:12.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.380+0000 7f6b97400640 1 --2- 172.21.15.102:0/4092595868 >> v2:172.21.15.193:3300/0 conn(0x7f6b980c86d0 0x7f6b9806ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.380+0000 7f6b97400640 1 -- 172.21.15.102:0/4092595868 >> v1:172.21.15.102:6789/0 conn(0x7f6b98153430 legacy=0x7f6b980c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:12.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.380+0000 7f6b97400640 1 -- 172.21.15.102:0/4092595868 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b981595f0 con 0x7f6b980c68e0 2024-04-20T21:20:12.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.381+0000 7f6b97400640 1 --2- 172.21.15.102:0/4092595868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c68e0 0x7f6b980c8190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009870 tx=0x7f6b8402ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1ead20a778bebf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:12.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.381+0000 7f6b96a00640 1 -- 172.21.15.102:0/4092595868 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8402f830 con 0x7f6b980c68e0 2024-04-20T21:20:12.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.381+0000 7f6b96a00640 1 -- 172.21.15.102:0/4092595868 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8402f9d0 con 0x7f6b980c68e0 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.381+0000 7f6b96a00640 1 -- 172.21.15.102:0/4092595868 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8402fda0 con 0x7f6b980c68e0 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.382+0000 7f6b9ec00640 1 -- 172.21.15.102:0/4092595868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c68e0 msgr2=0x7f6b980c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.382+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/4092595868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c68e0 0x7f6b980c8190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009870 tx=0x7f6b8402ba10 comp rx=0 tx=0).stop 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.382+0000 7f6b9ec00640 1 -- 172.21.15.102:0/4092595868 shutdown_connections 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.382+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/4092595868 >> v2:172.21.15.193:3300/0 conn(0x7f6b980c86d0 0x7f6b9806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.382+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/4092595868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c68e0 0x7f6b980c8190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.382+0000 7f6b9ec00640 1 -- 172.21.15.102:0/4092595868 >> 172.21.15.102:0/4092595868 conn(0x7f6b98072f00 msgr2=0x7f6b98073310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.382+0000 7f6b9ec00640 1 -- 172.21.15.102:0/4092595868 shutdown_connections 2024-04-20T21:20:12.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.382+0000 7f6b9ec00640 1 -- 172.21.15.102:0/4092595868 wait complete. 2024-04-20T21:20:12.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.383+0000 7f6b9ec00640 1 Processor -- start 2024-04-20T21:20:12.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.384+0000 7f6b9ec00640 1 -- start start 2024-04-20T21:20:12.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.384+0000 7f6b9ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b980c68e0 0x7f6b9814faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:12.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.384+0000 7f6b9ec00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c86d0 0x7f6b980cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.384+0000 7f6b9ec00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b98153430 0x7f6b981e41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.384+0000 7f6b9ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6b98159e90 con 0x7f6b980c68e0 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.384+0000 7f6b9ec00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b98159bb0 con 0x7f6b98153430 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b9ec00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6b98153810 con 0x7f6b980c86d0 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b980c68e0 0x7f6b9814faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b9cc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b98153430 0x7f6b981e41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b980c68e0 0x7f6b9814faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38522/0 (socket says 172.21.15.102:38522) 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97e00640 1 -- 172.21.15.102:0/2978426276 learned_addr learned my addr 172.21.15.102:0/2978426276 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:20:12.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c86d0 0x7f6b980cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:12.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97400640 1 -- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b98153430 msgr2=0x7f6b981e41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97400640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b98153430 0x7f6b981e41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97400640 1 -- 172.21.15.102:0/2978426276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b980c68e0 msgr2=0x7f6b9814faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97400640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b980c68e0 0x7f6b9814faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.385+0000 7f6b97400640 1 -- 172.21.15.102:0/2978426276 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9815a300 con 0x7f6b980c86d0 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.386+0000 7f6b97e00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b980c68e0 0x7f6b9814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.386+0000 7f6b97400640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c86d0 0x7f6b980cb1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b9815a230 tx=0x7f6b84004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.386+0000 7f6b8fe00640 1 -- 172.21.15.102:0/2978426276 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84047070 con 0x7f6b980c86d0 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.386+0000 7f6b8fe00640 1 -- 172.21.15.102:0/2978426276 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b84037be0 con 0x7f6b980c86d0 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.386+0000 7f6b8fe00640 1 -- 172.21.15.102:0/2978426276 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84042480 con 0x7f6b980c86d0 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.386+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b981e4d30 con 0x7f6b980c86d0 2024-04-20T21:20:12.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.386+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9815a570 con 0x7f6b980c86d0 2024-04-20T21:20:12.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.387+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b98068c30 con 0x7f6b980c86d0 2024-04-20T21:20:12.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.388+0000 7f6b8fe00640 1 -- 172.21.15.102:0/2978426276 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b8402d3c0 con 0x7f6b980c86d0 2024-04-20T21:20:12.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.389+0000 7f6b8fe00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6b7806c8f0 0x7f6b7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:12.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.389+0000 7f6b8fe00640 1 -- 172.21.15.102:0/2978426276 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6933+0+0 (secure 0 0 0) 0x7f6b840ae150 con 0x7f6b980c86d0 2024-04-20T21:20:12.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.390+0000 7f6b97e00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6b7806c8f0 0x7f6b7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:12.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.390+0000 7f6b97e00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6b7806c8f0 0x7f6b7806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b88004e00 tx=0x7f6b88007f10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:12.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.394+0000 7f6b8fe00640 1 -- 172.21.15.102:0/2978426276 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6b84033030 con 0x7f6b980c86d0 2024-04-20T21:20:12.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.644+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b980cf6e0 con 0x7f6b980c86d0 2024-04-20T21:20:12.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.645+0000 7f6b8fe00640 1 -- 172.21.15.102:0/2978426276 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6862 (secure 0 0 0) 0x7f6b8407ba50 con 0x7f6b980c86d0 2024-04-20T21:20:12.644 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:20:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:20:12.645 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 15 2024-04-20T21:20:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6b7806c8f0 msgr2=0x7f6b7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6b7806c8f0 0x7f6b7806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b88004e00 tx=0x7f6b88007f10 comp rx=0 tx=0).stop 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c86d0 msgr2=0x7f6b980cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c86d0 0x7f6b980cb1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b9815a230 tx=0x7f6b84004730 comp rx=0 tx=0).stop 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 shutdown_connections 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6b7806c8f0 0x7f6b7806edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b98153430 0x7f6b981e41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6b980c86d0 0x7f6b980cb1a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 --2- 172.21.15.102:0/2978426276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b980c68e0 0x7f6b9814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 >> 172.21.15.102:0/2978426276 conn(0x7f6b98072f00 msgr2=0x7f6b981524a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 shutdown_connections 2024-04-20T21:20:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:20:12.648+0000 7f6b9ec00640 1 -- 172.21.15.102:0/2978426276 wait complete. 2024-04-20T21:20:12.669 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:20:12.670 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 52 secs before thrashing 2024-04-20T21:20:12.688 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 183 2024-04-20T21:20:12.726 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 184 2024-04-20T21:20:12.765 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 185 2024-04-20T21:20:12.802 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 186 2024-04-20T21:20:12.840 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 187 2024-04-20T21:20:13.462 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 188 2024-04-20T21:20:13.501 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 189 2024-04-20T21:20:13.538 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 190 2024-04-20T21:20:13.577 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 191 2024-04-20T21:20:13.616 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 192 2024-04-20T21:20:13.809 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 193 2024-04-20T21:20:13.846 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 194 2024-04-20T21:20:13.887 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 195 2024-04-20T21:20:13.926 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 196 2024-04-20T21:20:13.963 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 197 2024-04-20T21:20:14.001 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 198 2024-04-20T21:20:14.039 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 199 2024-04-20T21:20:14.076 INFO:tasks.workunit.client.0.smithi102.stdout:Take first snapshot .snap/test1 2024-04-20T21:20:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:Create dir 200 to 299 ... 2024-04-20T21:20:14.215 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 200 2024-04-20T21:20:14.619 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 201 2024-04-20T21:20:14.656 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 202 2024-04-20T21:20:14.693 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 203 2024-04-20T21:20:14.724 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 204 2024-04-20T21:20:14.751 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 205 2024-04-20T21:20:14.785 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 206 2024-04-20T21:20:14.821 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 207 2024-04-20T21:20:14.858 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 208 2024-04-20T21:20:14.893 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 209 2024-04-20T21:20:14.928 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 210 2024-04-20T21:20:14.964 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 211 2024-04-20T21:20:15.001 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 212 2024-04-20T21:20:15.879 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 213 2024-04-20T21:20:15.911 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 214 2024-04-20T21:20:16.207 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 215 2024-04-20T21:20:16.235 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 216 2024-04-20T21:20:16.273 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 217 2024-04-20T21:20:16.299 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 218 2024-04-20T21:20:16.337 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 219 2024-04-20T21:20:16.363 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 220 2024-04-20T21:20:16.400 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 221 2024-04-20T21:20:16.437 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 222 2024-04-20T21:20:16.472 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 223 2024-04-20T21:20:16.769 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 224 2024-04-20T21:20:16.809 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 225 2024-04-20T21:20:17.047 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 226 2024-04-20T21:20:17.086 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 227 2024-04-20T21:20:17.113 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 228 2024-04-20T21:20:17.152 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 229 2024-04-20T21:20:17.182 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 230 2024-04-20T21:20:17.224 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 231 2024-04-20T21:20:17.733 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 232 2024-04-20T21:20:17.760 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 233 2024-04-20T21:20:17.785 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 234 2024-04-20T21:20:17.811 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 235 2024-04-20T21:20:17.839 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 236 2024-04-20T21:20:17.871 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 237 2024-04-20T21:20:17.906 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 238 2024-04-20T21:20:17.943 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 239 2024-04-20T21:20:17.977 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 240 2024-04-20T21:20:18.003 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 241 2024-04-20T21:20:18.040 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 242 2024-04-20T21:20:18.076 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 243 2024-04-20T21:20:18.113 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 244 2024-04-20T21:20:18.148 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 245 2024-04-20T21:20:18.183 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 246 2024-04-20T21:20:18.219 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 247 2024-04-20T21:20:18.254 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 248 2024-04-20T21:20:18.290 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 249 2024-04-20T21:20:18.328 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 250 2024-04-20T21:20:19.162 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 251 2024-04-20T21:20:19.211 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 252 2024-04-20T21:20:19.281 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 253 2024-04-20T21:20:19.624 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 254 2024-04-20T21:20:20.620 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 255 2024-04-20T21:20:20.648 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 256 2024-04-20T21:20:20.672 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 257 2024-04-20T21:20:20.697 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 258 2024-04-20T21:20:20.717 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 259 2024-04-20T21:20:20.736 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 260 2024-04-20T21:20:20.754 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 261 2024-04-20T21:20:21.053 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 262 2024-04-20T21:20:21.091 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 263 2024-04-20T21:20:21.126 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 264 2024-04-20T21:20:21.160 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 265 2024-04-20T21:20:21.196 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 266 2024-04-20T21:20:21.232 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 267 2024-04-20T21:20:21.336 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 268 2024-04-20T21:20:21.373 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 269 2024-04-20T21:20:21.408 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 270 2024-04-20T21:20:21.444 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 271 2024-04-20T21:20:21.478 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 272 2024-04-20T21:20:21.517 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 273 2024-04-20T21:20:21.797 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:21.805 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:22.075 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 274 2024-04-20T21:20:22.117 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 275 2024-04-20T21:20:22.166 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 276 2024-04-20T21:20:22.219 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 277 2024-04-20T21:20:22.269 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 278 2024-04-20T21:20:22.315 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 279 2024-04-20T21:20:22.365 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 280 2024-04-20T21:20:22.414 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 281 2024-04-20T21:20:22.465 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 282 2024-04-20T21:20:22.510 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 283 2024-04-20T21:20:22.554 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 284 2024-04-20T21:20:22.589 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 285 2024-04-20T21:20:22.626 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 286 2024-04-20T21:20:22.660 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 287 2024-04-20T21:20:22.693 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 288 2024-04-20T21:20:22.729 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 289 2024-04-20T21:20:22.759 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 290 2024-04-20T21:20:22.788 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 291 2024-04-20T21:20:22.824 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 292 2024-04-20T21:20:22.860 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 293 2024-04-20T21:20:22.895 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 294 2024-04-20T21:20:22.947 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 295 2024-04-20T21:20:22.998 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 296 2024-04-20T21:20:23.049 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 297 2024-04-20T21:20:23.098 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 298 2024-04-20T21:20:23.135 INFO:tasks.workunit.client.0.smithi102.stdout: create dir 299 2024-04-20T21:20:23.173 INFO:tasks.workunit.client.0.smithi102.stdout:Create a snapshot in every first level dir ... 2024-04-20T21:20:23.879 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/.snap/snap-subdir-test 2024-04-20T21:20:23.917 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/30 file after the snapshot 2024-04-20T21:20:23.920 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/31 file after the snapshot 2024-04-20T21:20:23.924 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/32 file after the snapshot 2024-04-20T21:20:23.927 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/33 file after the snapshot 2024-04-20T21:20:23.930 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/34 file after the snapshot 2024-04-20T21:20:23.933 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/35 file after the snapshot 2024-04-20T21:20:23.936 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/36 file after the snapshot 2024-04-20T21:20:23.940 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/37 file after the snapshot 2024-04-20T21:20:23.943 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/38 file after the snapshot 2024-04-20T21:20:23.947 INFO:tasks.workunit.client.0.smithi102.stdout: create 100/39 file after the snapshot 2024-04-20T21:20:23.950 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/.snap/snap-subdir-test 2024-04-20T21:20:23.991 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/30 file after the snapshot 2024-04-20T21:20:23.995 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/31 file after the snapshot 2024-04-20T21:20:23.999 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/32 file after the snapshot 2024-04-20T21:20:24.003 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/33 file after the snapshot 2024-04-20T21:20:24.007 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/34 file after the snapshot 2024-04-20T21:20:24.011 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/35 file after the snapshot 2024-04-20T21:20:24.015 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/36 file after the snapshot 2024-04-20T21:20:24.019 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/37 file after the snapshot 2024-04-20T21:20:24.023 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/38 file after the snapshot 2024-04-20T21:20:24.028 INFO:tasks.workunit.client.0.smithi102.stdout: create 101/39 file after the snapshot 2024-04-20T21:20:24.031 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/.snap/snap-subdir-test 2024-04-20T21:20:24.895 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/30 file after the snapshot 2024-04-20T21:20:24.897 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/31 file after the snapshot 2024-04-20T21:20:24.900 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/32 file after the snapshot 2024-04-20T21:20:24.902 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/33 file after the snapshot 2024-04-20T21:20:24.905 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/34 file after the snapshot 2024-04-20T21:20:24.906 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/35 file after the snapshot 2024-04-20T21:20:24.909 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/36 file after the snapshot 2024-04-20T21:20:24.912 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/37 file after the snapshot 2024-04-20T21:20:24.915 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/38 file after the snapshot 2024-04-20T21:20:24.918 INFO:tasks.workunit.client.0.smithi102.stdout: create 102/39 file after the snapshot 2024-04-20T21:20:24.921 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/.snap/snap-subdir-test 2024-04-20T21:20:25.493 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/30 file after the snapshot 2024-04-20T21:20:25.495 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/31 file after the snapshot 2024-04-20T21:20:25.497 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/32 file after the snapshot 2024-04-20T21:20:25.499 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/33 file after the snapshot 2024-04-20T21:20:25.501 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/34 file after the snapshot 2024-04-20T21:20:25.503 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/35 file after the snapshot 2024-04-20T21:20:25.506 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/36 file after the snapshot 2024-04-20T21:20:25.509 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/37 file after the snapshot 2024-04-20T21:20:25.512 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/38 file after the snapshot 2024-04-20T21:20:25.515 INFO:tasks.workunit.client.0.smithi102.stdout: create 103/39 file after the snapshot 2024-04-20T21:20:25.518 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/.snap/snap-subdir-test 2024-04-20T21:20:25.565 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/30 file after the snapshot 2024-04-20T21:20:25.568 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/31 file after the snapshot 2024-04-20T21:20:25.570 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/32 file after the snapshot 2024-04-20T21:20:25.573 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/33 file after the snapshot 2024-04-20T21:20:25.576 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/34 file after the snapshot 2024-04-20T21:20:25.579 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/35 file after the snapshot 2024-04-20T21:20:25.582 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/36 file after the snapshot 2024-04-20T21:20:25.584 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/37 file after the snapshot 2024-04-20T21:20:25.587 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/38 file after the snapshot 2024-04-20T21:20:25.590 INFO:tasks.workunit.client.0.smithi102.stdout: create 104/39 file after the snapshot 2024-04-20T21:20:25.593 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/.snap/snap-subdir-test 2024-04-20T21:20:25.640 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/30 file after the snapshot 2024-04-20T21:20:25.643 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/31 file after the snapshot 2024-04-20T21:20:25.645 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/32 file after the snapshot 2024-04-20T21:20:25.648 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/33 file after the snapshot 2024-04-20T21:20:25.651 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/34 file after the snapshot 2024-04-20T21:20:25.654 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/35 file after the snapshot 2024-04-20T21:20:25.657 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/36 file after the snapshot 2024-04-20T21:20:25.660 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/37 file after the snapshot 2024-04-20T21:20:25.663 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/38 file after the snapshot 2024-04-20T21:20:25.666 INFO:tasks.workunit.client.0.smithi102.stdout: create 105/39 file after the snapshot 2024-04-20T21:20:25.668 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/.snap/snap-subdir-test 2024-04-20T21:20:25.712 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/30 file after the snapshot 2024-04-20T21:20:25.714 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/31 file after the snapshot 2024-04-20T21:20:25.717 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/32 file after the snapshot 2024-04-20T21:20:25.720 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/33 file after the snapshot 2024-04-20T21:20:25.722 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/34 file after the snapshot 2024-04-20T21:20:25.725 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/35 file after the snapshot 2024-04-20T21:20:25.728 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/36 file after the snapshot 2024-04-20T21:20:25.731 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/37 file after the snapshot 2024-04-20T21:20:25.734 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/38 file after the snapshot 2024-04-20T21:20:25.736 INFO:tasks.workunit.client.0.smithi102.stdout: create 106/39 file after the snapshot 2024-04-20T21:20:25.739 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/.snap/snap-subdir-test 2024-04-20T21:20:25.782 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/30 file after the snapshot 2024-04-20T21:20:25.785 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/31 file after the snapshot 2024-04-20T21:20:25.788 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/32 file after the snapshot 2024-04-20T21:20:25.791 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/33 file after the snapshot 2024-04-20T21:20:25.794 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/34 file after the snapshot 2024-04-20T21:20:25.796 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/35 file after the snapshot 2024-04-20T21:20:25.799 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/36 file after the snapshot 2024-04-20T21:20:25.802 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/37 file after the snapshot 2024-04-20T21:20:25.805 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/38 file after the snapshot 2024-04-20T21:20:25.807 INFO:tasks.workunit.client.0.smithi102.stdout: create 107/39 file after the snapshot 2024-04-20T21:20:25.810 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/.snap/snap-subdir-test 2024-04-20T21:20:25.851 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/30 file after the snapshot 2024-04-20T21:20:25.853 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/31 file after the snapshot 2024-04-20T21:20:26.117 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/32 file after the snapshot 2024-04-20T21:20:26.119 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/33 file after the snapshot 2024-04-20T21:20:26.122 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/34 file after the snapshot 2024-04-20T21:20:26.125 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/35 file after the snapshot 2024-04-20T21:20:26.128 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/36 file after the snapshot 2024-04-20T21:20:26.131 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/37 file after the snapshot 2024-04-20T21:20:26.134 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/38 file after the snapshot 2024-04-20T21:20:26.545 INFO:tasks.workunit.client.0.smithi102.stdout: create 108/39 file after the snapshot 2024-04-20T21:20:26.547 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/.snap/snap-subdir-test 2024-04-20T21:20:26.595 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/30 file after the snapshot 2024-04-20T21:20:26.598 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/31 file after the snapshot 2024-04-20T21:20:26.601 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/32 file after the snapshot 2024-04-20T21:20:26.604 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/33 file after the snapshot 2024-04-20T21:20:26.607 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/34 file after the snapshot 2024-04-20T21:20:26.610 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/35 file after the snapshot 2024-04-20T21:20:26.612 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/36 file after the snapshot 2024-04-20T21:20:26.615 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/37 file after the snapshot 2024-04-20T21:20:26.845 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/38 file after the snapshot 2024-04-20T21:20:27.051 INFO:tasks.workunit.client.0.smithi102.stdout: create 109/39 file after the snapshot 2024-04-20T21:20:27.054 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/.snap/snap-subdir-test 2024-04-20T21:20:27.092 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/30 file after the snapshot 2024-04-20T21:20:27.095 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/31 file after the snapshot 2024-04-20T21:20:27.098 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/32 file after the snapshot 2024-04-20T21:20:27.100 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/33 file after the snapshot 2024-04-20T21:20:27.103 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/34 file after the snapshot 2024-04-20T21:20:27.105 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/35 file after the snapshot 2024-04-20T21:20:27.108 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/36 file after the snapshot 2024-04-20T21:20:27.111 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/37 file after the snapshot 2024-04-20T21:20:27.114 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/38 file after the snapshot 2024-04-20T21:20:27.116 INFO:tasks.workunit.client.0.smithi102.stdout: create 110/39 file after the snapshot 2024-04-20T21:20:27.119 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/.snap/snap-subdir-test 2024-04-20T21:20:27.155 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/30 file after the snapshot 2024-04-20T21:20:27.158 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/31 file after the snapshot 2024-04-20T21:20:27.161 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/32 file after the snapshot 2024-04-20T21:20:27.164 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/33 file after the snapshot 2024-04-20T21:20:27.166 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/34 file after the snapshot 2024-04-20T21:20:27.169 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/35 file after the snapshot 2024-04-20T21:20:27.172 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/36 file after the snapshot 2024-04-20T21:20:27.174 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/37 file after the snapshot 2024-04-20T21:20:27.177 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/38 file after the snapshot 2024-04-20T21:20:27.180 INFO:tasks.workunit.client.0.smithi102.stdout: create 111/39 file after the snapshot 2024-04-20T21:20:27.183 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/.snap/snap-subdir-test 2024-04-20T21:20:27.218 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/30 file after the snapshot 2024-04-20T21:20:27.220 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/31 file after the snapshot 2024-04-20T21:20:27.222 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/32 file after the snapshot 2024-04-20T21:20:27.224 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/33 file after the snapshot 2024-04-20T21:20:27.226 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/34 file after the snapshot 2024-04-20T21:20:27.228 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/35 file after the snapshot 2024-04-20T21:20:27.230 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/36 file after the snapshot 2024-04-20T21:20:27.233 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/37 file after the snapshot 2024-04-20T21:20:27.236 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/38 file after the snapshot 2024-04-20T21:20:27.239 INFO:tasks.workunit.client.0.smithi102.stdout: create 112/39 file after the snapshot 2024-04-20T21:20:27.242 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/.snap/snap-subdir-test 2024-04-20T21:20:27.272 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/30 file after the snapshot 2024-04-20T21:20:27.275 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/31 file after the snapshot 2024-04-20T21:20:27.278 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/32 file after the snapshot 2024-04-20T21:20:27.281 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/33 file after the snapshot 2024-04-20T21:20:27.284 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/34 file after the snapshot 2024-04-20T21:20:27.287 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/35 file after the snapshot 2024-04-20T21:20:27.290 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/36 file after the snapshot 2024-04-20T21:20:27.293 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/37 file after the snapshot 2024-04-20T21:20:27.296 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/38 file after the snapshot 2024-04-20T21:20:27.299 INFO:tasks.workunit.client.0.smithi102.stdout: create 113/39 file after the snapshot 2024-04-20T21:20:27.302 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/.snap/snap-subdir-test 2024-04-20T21:20:27.333 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/30 file after the snapshot 2024-04-20T21:20:27.335 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/31 file after the snapshot 2024-04-20T21:20:27.338 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/32 file after the snapshot 2024-04-20T21:20:27.341 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/33 file after the snapshot 2024-04-20T21:20:27.344 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/34 file after the snapshot 2024-04-20T21:20:27.347 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/35 file after the snapshot 2024-04-20T21:20:27.350 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/36 file after the snapshot 2024-04-20T21:20:27.353 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/37 file after the snapshot 2024-04-20T21:20:27.356 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/38 file after the snapshot 2024-04-20T21:20:27.359 INFO:tasks.workunit.client.0.smithi102.stdout: create 114/39 file after the snapshot 2024-04-20T21:20:27.362 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/.snap/snap-subdir-test 2024-04-20T21:20:27.394 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/30 file after the snapshot 2024-04-20T21:20:27.397 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/31 file after the snapshot 2024-04-20T21:20:27.400 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/32 file after the snapshot 2024-04-20T21:20:27.403 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/33 file after the snapshot 2024-04-20T21:20:27.406 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/34 file after the snapshot 2024-04-20T21:20:27.409 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/35 file after the snapshot 2024-04-20T21:20:27.412 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/36 file after the snapshot 2024-04-20T21:20:27.414 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/37 file after the snapshot 2024-04-20T21:20:27.416 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/38 file after the snapshot 2024-04-20T21:20:27.418 INFO:tasks.workunit.client.0.smithi102.stdout: create 115/39 file after the snapshot 2024-04-20T21:20:27.419 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/.snap/snap-subdir-test 2024-04-20T21:20:27.456 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/30 file after the snapshot 2024-04-20T21:20:27.459 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/31 file after the snapshot 2024-04-20T21:20:27.462 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/32 file after the snapshot 2024-04-20T21:20:27.465 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/33 file after the snapshot 2024-04-20T21:20:27.467 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/34 file after the snapshot 2024-04-20T21:20:27.470 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/35 file after the snapshot 2024-04-20T21:20:27.473 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/36 file after the snapshot 2024-04-20T21:20:27.476 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/37 file after the snapshot 2024-04-20T21:20:27.479 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/38 file after the snapshot 2024-04-20T21:20:27.482 INFO:tasks.workunit.client.0.smithi102.stdout: create 116/39 file after the snapshot 2024-04-20T21:20:27.484 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/.snap/snap-subdir-test 2024-04-20T21:20:27.523 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/30 file after the snapshot 2024-04-20T21:20:27.525 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/31 file after the snapshot 2024-04-20T21:20:27.528 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/32 file after the snapshot 2024-04-20T21:20:27.530 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/33 file after the snapshot 2024-04-20T21:20:27.532 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/34 file after the snapshot 2024-04-20T21:20:27.535 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/35 file after the snapshot 2024-04-20T21:20:27.537 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/36 file after the snapshot 2024-04-20T21:20:27.540 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/37 file after the snapshot 2024-04-20T21:20:27.542 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/38 file after the snapshot 2024-04-20T21:20:27.545 INFO:tasks.workunit.client.0.smithi102.stdout: create 117/39 file after the snapshot 2024-04-20T21:20:27.547 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/.snap/snap-subdir-test 2024-04-20T21:20:27.579 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/30 file after the snapshot 2024-04-20T21:20:27.582 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/31 file after the snapshot 2024-04-20T21:20:27.584 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/32 file after the snapshot 2024-04-20T21:20:27.587 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/33 file after the snapshot 2024-04-20T21:20:27.589 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/34 file after the snapshot 2024-04-20T21:20:27.592 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/35 file after the snapshot 2024-04-20T21:20:27.593 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/36 file after the snapshot 2024-04-20T21:20:27.596 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/37 file after the snapshot 2024-04-20T21:20:27.598 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/38 file after the snapshot 2024-04-20T21:20:27.601 INFO:tasks.workunit.client.0.smithi102.stdout: create 118/39 file after the snapshot 2024-04-20T21:20:27.604 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/.snap/snap-subdir-test 2024-04-20T21:20:27.641 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/30 file after the snapshot 2024-04-20T21:20:27.645 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/31 file after the snapshot 2024-04-20T21:20:27.648 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/32 file after the snapshot 2024-04-20T21:20:27.651 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/33 file after the snapshot 2024-04-20T21:20:27.654 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/34 file after the snapshot 2024-04-20T21:20:27.659 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/35 file after the snapshot 2024-04-20T21:20:27.662 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/36 file after the snapshot 2024-04-20T21:20:27.665 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/37 file after the snapshot 2024-04-20T21:20:27.667 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/38 file after the snapshot 2024-04-20T21:20:27.670 INFO:tasks.workunit.client.0.smithi102.stdout: create 119/39 file after the snapshot 2024-04-20T21:20:27.673 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/.snap/snap-subdir-test 2024-04-20T21:20:27.719 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/30 file after the snapshot 2024-04-20T21:20:27.721 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/31 file after the snapshot 2024-04-20T21:20:27.723 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/32 file after the snapshot 2024-04-20T21:20:27.724 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/33 file after the snapshot 2024-04-20T21:20:27.726 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/34 file after the snapshot 2024-04-20T21:20:27.728 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/35 file after the snapshot 2024-04-20T21:20:27.730 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/36 file after the snapshot 2024-04-20T21:20:27.732 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/37 file after the snapshot 2024-04-20T21:20:27.735 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/38 file after the snapshot 2024-04-20T21:20:27.738 INFO:tasks.workunit.client.0.smithi102.stdout: create 120/39 file after the snapshot 2024-04-20T21:20:27.741 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/.snap/snap-subdir-test 2024-04-20T21:20:27.785 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/30 file after the snapshot 2024-04-20T21:20:27.787 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/31 file after the snapshot 2024-04-20T21:20:27.789 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/32 file after the snapshot 2024-04-20T21:20:27.791 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/33 file after the snapshot 2024-04-20T21:20:27.792 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/34 file after the snapshot 2024-04-20T21:20:27.794 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/35 file after the snapshot 2024-04-20T21:20:27.796 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/36 file after the snapshot 2024-04-20T21:20:27.798 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/37 file after the snapshot 2024-04-20T21:20:27.800 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/38 file after the snapshot 2024-04-20T21:20:27.802 INFO:tasks.workunit.client.0.smithi102.stdout: create 121/39 file after the snapshot 2024-04-20T21:20:27.804 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/.snap/snap-subdir-test 2024-04-20T21:20:27.847 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/30 file after the snapshot 2024-04-20T21:20:27.849 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/31 file after the snapshot 2024-04-20T21:20:27.852 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/32 file after the snapshot 2024-04-20T21:20:27.855 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/33 file after the snapshot 2024-04-20T21:20:27.858 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/34 file after the snapshot 2024-04-20T21:20:27.860 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/35 file after the snapshot 2024-04-20T21:20:27.863 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/36 file after the snapshot 2024-04-20T21:20:27.866 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/37 file after the snapshot 2024-04-20T21:20:27.869 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/38 file after the snapshot 2024-04-20T21:20:27.871 INFO:tasks.workunit.client.0.smithi102.stdout: create 122/39 file after the snapshot 2024-04-20T21:20:27.874 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/.snap/snap-subdir-test 2024-04-20T21:20:27.919 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/30 file after the snapshot 2024-04-20T21:20:27.921 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/31 file after the snapshot 2024-04-20T21:20:27.924 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/32 file after the snapshot 2024-04-20T21:20:27.927 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/33 file after the snapshot 2024-04-20T21:20:27.930 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/34 file after the snapshot 2024-04-20T21:20:27.933 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/35 file after the snapshot 2024-04-20T21:20:27.935 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/36 file after the snapshot 2024-04-20T21:20:27.938 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/37 file after the snapshot 2024-04-20T21:20:27.941 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/38 file after the snapshot 2024-04-20T21:20:27.944 INFO:tasks.workunit.client.0.smithi102.stdout: create 123/39 file after the snapshot 2024-04-20T21:20:27.947 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/.snap/snap-subdir-test 2024-04-20T21:20:27.985 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/30 file after the snapshot 2024-04-20T21:20:27.988 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/31 file after the snapshot 2024-04-20T21:20:27.991 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/32 file after the snapshot 2024-04-20T21:20:27.994 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/33 file after the snapshot 2024-04-20T21:20:27.997 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/34 file after the snapshot 2024-04-20T21:20:27.999 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/35 file after the snapshot 2024-04-20T21:20:28.002 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/36 file after the snapshot 2024-04-20T21:20:28.005 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/37 file after the snapshot 2024-04-20T21:20:28.008 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/38 file after the snapshot 2024-04-20T21:20:28.011 INFO:tasks.workunit.client.0.smithi102.stdout: create 124/39 file after the snapshot 2024-04-20T21:20:28.014 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/.snap/snap-subdir-test 2024-04-20T21:20:28.053 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/30 file after the snapshot 2024-04-20T21:20:28.056 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/31 file after the snapshot 2024-04-20T21:20:28.058 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/32 file after the snapshot 2024-04-20T21:20:28.061 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/33 file after the snapshot 2024-04-20T21:20:28.064 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/34 file after the snapshot 2024-04-20T21:20:28.067 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/35 file after the snapshot 2024-04-20T21:20:28.069 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/36 file after the snapshot 2024-04-20T21:20:28.071 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/37 file after the snapshot 2024-04-20T21:20:28.073 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/38 file after the snapshot 2024-04-20T21:20:28.075 INFO:tasks.workunit.client.0.smithi102.stdout: create 125/39 file after the snapshot 2024-04-20T21:20:28.077 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/.snap/snap-subdir-test 2024-04-20T21:20:28.112 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/30 file after the snapshot 2024-04-20T21:20:28.115 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/31 file after the snapshot 2024-04-20T21:20:28.118 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/32 file after the snapshot 2024-04-20T21:20:28.120 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/33 file after the snapshot 2024-04-20T21:20:28.123 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/34 file after the snapshot 2024-04-20T21:20:28.126 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/35 file after the snapshot 2024-04-20T21:20:28.128 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/36 file after the snapshot 2024-04-20T21:20:28.131 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/37 file after the snapshot 2024-04-20T21:20:28.134 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/38 file after the snapshot 2024-04-20T21:20:28.137 INFO:tasks.workunit.client.0.smithi102.stdout: create 126/39 file after the snapshot 2024-04-20T21:20:28.139 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/.snap/snap-subdir-test 2024-04-20T21:20:28.178 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/30 file after the snapshot 2024-04-20T21:20:28.180 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/31 file after the snapshot 2024-04-20T21:20:28.183 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/32 file after the snapshot 2024-04-20T21:20:28.185 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/33 file after the snapshot 2024-04-20T21:20:28.188 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/34 file after the snapshot 2024-04-20T21:20:28.190 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/35 file after the snapshot 2024-04-20T21:20:28.193 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/36 file after the snapshot 2024-04-20T21:20:28.195 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/37 file after the snapshot 2024-04-20T21:20:28.198 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/38 file after the snapshot 2024-04-20T21:20:28.200 INFO:tasks.workunit.client.0.smithi102.stdout: create 127/39 file after the snapshot 2024-04-20T21:20:28.203 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/.snap/snap-subdir-test 2024-04-20T21:20:28.241 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/30 file after the snapshot 2024-04-20T21:20:28.244 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/31 file after the snapshot 2024-04-20T21:20:28.247 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/32 file after the snapshot 2024-04-20T21:20:28.249 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/33 file after the snapshot 2024-04-20T21:20:28.252 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/34 file after the snapshot 2024-04-20T21:20:28.255 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/35 file after the snapshot 2024-04-20T21:20:28.258 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/36 file after the snapshot 2024-04-20T21:20:28.261 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/37 file after the snapshot 2024-04-20T21:20:28.264 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/38 file after the snapshot 2024-04-20T21:20:28.266 INFO:tasks.workunit.client.0.smithi102.stdout: create 128/39 file after the snapshot 2024-04-20T21:20:28.269 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/.snap/snap-subdir-test 2024-04-20T21:20:28.303 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/30 file after the snapshot 2024-04-20T21:20:28.305 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/31 file after the snapshot 2024-04-20T21:20:28.307 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/32 file after the snapshot 2024-04-20T21:20:28.309 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/33 file after the snapshot 2024-04-20T21:20:28.311 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/34 file after the snapshot 2024-04-20T21:20:28.313 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/35 file after the snapshot 2024-04-20T21:20:28.315 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/36 file after the snapshot 2024-04-20T21:20:28.318 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/37 file after the snapshot 2024-04-20T21:20:28.321 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/38 file after the snapshot 2024-04-20T21:20:28.324 INFO:tasks.workunit.client.0.smithi102.stdout: create 129/39 file after the snapshot 2024-04-20T21:20:28.327 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/.snap/snap-subdir-test 2024-04-20T21:20:28.365 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/30 file after the snapshot 2024-04-20T21:20:28.368 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/31 file after the snapshot 2024-04-20T21:20:28.371 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/32 file after the snapshot 2024-04-20T21:20:28.373 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/33 file after the snapshot 2024-04-20T21:20:28.376 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/34 file after the snapshot 2024-04-20T21:20:28.378 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/35 file after the snapshot 2024-04-20T21:20:28.381 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/36 file after the snapshot 2024-04-20T21:20:28.384 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/37 file after the snapshot 2024-04-20T21:20:28.386 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/38 file after the snapshot 2024-04-20T21:20:28.389 INFO:tasks.workunit.client.0.smithi102.stdout: create 130/39 file after the snapshot 2024-04-20T21:20:28.654 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/.snap/snap-subdir-test 2024-04-20T21:20:28.686 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/30 file after the snapshot 2024-04-20T21:20:28.688 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/31 file after the snapshot 2024-04-20T21:20:28.690 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/32 file after the snapshot 2024-04-20T21:20:28.692 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/33 file after the snapshot 2024-04-20T21:20:28.694 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/34 file after the snapshot 2024-04-20T21:20:28.696 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/35 file after the snapshot 2024-04-20T21:20:28.698 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/36 file after the snapshot 2024-04-20T21:20:28.700 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/37 file after the snapshot 2024-04-20T21:20:28.702 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/38 file after the snapshot 2024-04-20T21:20:28.704 INFO:tasks.workunit.client.0.smithi102.stdout: create 131/39 file after the snapshot 2024-04-20T21:20:28.707 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/.snap/snap-subdir-test 2024-04-20T21:20:28.961 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/30 file after the snapshot 2024-04-20T21:20:28.963 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/31 file after the snapshot 2024-04-20T21:20:28.965 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/32 file after the snapshot 2024-04-20T21:20:28.966 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/33 file after the snapshot 2024-04-20T21:20:28.969 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/34 file after the snapshot 2024-04-20T21:20:28.970 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/35 file after the snapshot 2024-04-20T21:20:28.972 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/36 file after the snapshot 2024-04-20T21:20:28.974 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/37 file after the snapshot 2024-04-20T21:20:28.976 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/38 file after the snapshot 2024-04-20T21:20:28.980 INFO:tasks.workunit.client.0.smithi102.stdout: create 132/39 file after the snapshot 2024-04-20T21:20:28.982 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/.snap/snap-subdir-test 2024-04-20T21:20:29.038 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/30 file after the snapshot 2024-04-20T21:20:29.040 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/31 file after the snapshot 2024-04-20T21:20:29.041 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/32 file after the snapshot 2024-04-20T21:20:29.043 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/33 file after the snapshot 2024-04-20T21:20:29.046 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/34 file after the snapshot 2024-04-20T21:20:29.049 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/35 file after the snapshot 2024-04-20T21:20:29.051 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/36 file after the snapshot 2024-04-20T21:20:29.054 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/37 file after the snapshot 2024-04-20T21:20:29.057 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/38 file after the snapshot 2024-04-20T21:20:29.060 INFO:tasks.workunit.client.0.smithi102.stdout: create 133/39 file after the snapshot 2024-04-20T21:20:29.063 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/.snap/snap-subdir-test 2024-04-20T21:20:29.121 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/30 file after the snapshot 2024-04-20T21:20:29.124 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/31 file after the snapshot 2024-04-20T21:20:29.125 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/32 file after the snapshot 2024-04-20T21:20:29.127 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/33 file after the snapshot 2024-04-20T21:20:29.130 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/34 file after the snapshot 2024-04-20T21:20:29.133 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/35 file after the snapshot 2024-04-20T21:20:29.136 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/36 file after the snapshot 2024-04-20T21:20:29.139 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/37 file after the snapshot 2024-04-20T21:20:29.141 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/38 file after the snapshot 2024-04-20T21:20:29.144 INFO:tasks.workunit.client.0.smithi102.stdout: create 134/39 file after the snapshot 2024-04-20T21:20:29.147 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/.snap/snap-subdir-test 2024-04-20T21:20:29.206 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/30 file after the snapshot 2024-04-20T21:20:29.209 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/31 file after the snapshot 2024-04-20T21:20:29.212 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/32 file after the snapshot 2024-04-20T21:20:29.215 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/33 file after the snapshot 2024-04-20T21:20:29.218 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/34 file after the snapshot 2024-04-20T21:20:29.221 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/35 file after the snapshot 2024-04-20T21:20:29.224 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/36 file after the snapshot 2024-04-20T21:20:29.227 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/37 file after the snapshot 2024-04-20T21:20:29.230 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/38 file after the snapshot 2024-04-20T21:20:29.853 INFO:tasks.workunit.client.0.smithi102.stdout: create 135/39 file after the snapshot 2024-04-20T21:20:30.162 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/.snap/snap-subdir-test 2024-04-20T21:20:30.217 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/30 file after the snapshot 2024-04-20T21:20:30.219 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/31 file after the snapshot 2024-04-20T21:20:30.222 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/32 file after the snapshot 2024-04-20T21:20:30.225 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/33 file after the snapshot 2024-04-20T21:20:30.228 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/34 file after the snapshot 2024-04-20T21:20:30.231 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/35 file after the snapshot 2024-04-20T21:20:30.234 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/36 file after the snapshot 2024-04-20T21:20:30.237 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/37 file after the snapshot 2024-04-20T21:20:30.240 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/38 file after the snapshot 2024-04-20T21:20:30.242 INFO:tasks.workunit.client.0.smithi102.stdout: create 136/39 file after the snapshot 2024-04-20T21:20:30.245 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/.snap/snap-subdir-test 2024-04-20T21:20:30.300 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/30 file after the snapshot 2024-04-20T21:20:30.303 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/31 file after the snapshot 2024-04-20T21:20:30.306 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/32 file after the snapshot 2024-04-20T21:20:30.309 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/33 file after the snapshot 2024-04-20T21:20:30.312 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/34 file after the snapshot 2024-04-20T21:20:30.315 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/35 file after the snapshot 2024-04-20T21:20:30.318 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/36 file after the snapshot 2024-04-20T21:20:30.320 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/37 file after the snapshot 2024-04-20T21:20:30.323 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/38 file after the snapshot 2024-04-20T21:20:30.326 INFO:tasks.workunit.client.0.smithi102.stdout: create 137/39 file after the snapshot 2024-04-20T21:20:30.329 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/.snap/snap-subdir-test 2024-04-20T21:20:30.383 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/30 file after the snapshot 2024-04-20T21:20:30.386 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/31 file after the snapshot 2024-04-20T21:20:30.389 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/32 file after the snapshot 2024-04-20T21:20:30.392 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/33 file after the snapshot 2024-04-20T21:20:30.395 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/34 file after the snapshot 2024-04-20T21:20:30.398 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/35 file after the snapshot 2024-04-20T21:20:30.401 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/36 file after the snapshot 2024-04-20T21:20:30.404 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/37 file after the snapshot 2024-04-20T21:20:30.406 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/38 file after the snapshot 2024-04-20T21:20:30.409 INFO:tasks.workunit.client.0.smithi102.stdout: create 138/39 file after the snapshot 2024-04-20T21:20:30.412 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/.snap/snap-subdir-test 2024-04-20T21:20:30.469 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/30 file after the snapshot 2024-04-20T21:20:30.471 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/31 file after the snapshot 2024-04-20T21:20:30.473 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/32 file after the snapshot 2024-04-20T21:20:30.475 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/33 file after the snapshot 2024-04-20T21:20:30.477 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/34 file after the snapshot 2024-04-20T21:20:30.480 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/35 file after the snapshot 2024-04-20T21:20:30.483 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/36 file after the snapshot 2024-04-20T21:20:30.485 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/37 file after the snapshot 2024-04-20T21:20:30.488 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/38 file after the snapshot 2024-04-20T21:20:30.490 INFO:tasks.workunit.client.0.smithi102.stdout: create 139/39 file after the snapshot 2024-04-20T21:20:30.493 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/.snap/snap-subdir-test 2024-04-20T21:20:30.553 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/30 file after the snapshot 2024-04-20T21:20:30.555 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/31 file after the snapshot 2024-04-20T21:20:30.558 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/32 file after the snapshot 2024-04-20T21:20:30.561 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/33 file after the snapshot 2024-04-20T21:20:30.563 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/34 file after the snapshot 2024-04-20T21:20:30.829 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/35 file after the snapshot 2024-04-20T21:20:30.832 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/36 file after the snapshot 2024-04-20T21:20:30.834 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/37 file after the snapshot 2024-04-20T21:20:30.837 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/38 file after the snapshot 2024-04-20T21:20:30.840 INFO:tasks.workunit.client.0.smithi102.stdout: create 140/39 file after the snapshot 2024-04-20T21:20:30.843 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/.snap/snap-subdir-test 2024-04-20T21:20:30.895 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/30 file after the snapshot 2024-04-20T21:20:30.897 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/31 file after the snapshot 2024-04-20T21:20:30.899 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/32 file after the snapshot 2024-04-20T21:20:30.901 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/33 file after the snapshot 2024-04-20T21:20:30.903 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/34 file after the snapshot 2024-04-20T21:20:30.905 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/35 file after the snapshot 2024-04-20T21:20:30.908 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/36 file after the snapshot 2024-04-20T21:20:30.911 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/37 file after the snapshot 2024-04-20T21:20:30.914 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/38 file after the snapshot 2024-04-20T21:20:30.916 INFO:tasks.workunit.client.0.smithi102.stdout: create 141/39 file after the snapshot 2024-04-20T21:20:30.920 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/.snap/snap-subdir-test 2024-04-20T21:20:30.958 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/30 file after the snapshot 2024-04-20T21:20:30.960 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/31 file after the snapshot 2024-04-20T21:20:30.962 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/32 file after the snapshot 2024-04-20T21:20:30.965 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/33 file after the snapshot 2024-04-20T21:20:30.967 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/34 file after the snapshot 2024-04-20T21:20:30.970 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/35 file after the snapshot 2024-04-20T21:20:30.972 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/36 file after the snapshot 2024-04-20T21:20:30.975 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/37 file after the snapshot 2024-04-20T21:20:30.978 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/38 file after the snapshot 2024-04-20T21:20:30.981 INFO:tasks.workunit.client.0.smithi102.stdout: create 142/39 file after the snapshot 2024-04-20T21:20:30.984 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/.snap/snap-subdir-test 2024-04-20T21:20:31.025 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/30 file after the snapshot 2024-04-20T21:20:31.028 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/31 file after the snapshot 2024-04-20T21:20:31.031 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/32 file after the snapshot 2024-04-20T21:20:31.034 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/33 file after the snapshot 2024-04-20T21:20:31.037 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/34 file after the snapshot 2024-04-20T21:20:31.040 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/35 file after the snapshot 2024-04-20T21:20:31.043 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/36 file after the snapshot 2024-04-20T21:20:31.046 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/37 file after the snapshot 2024-04-20T21:20:31.049 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/38 file after the snapshot 2024-04-20T21:20:31.052 INFO:tasks.workunit.client.0.smithi102.stdout: create 143/39 file after the snapshot 2024-04-20T21:20:31.055 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/.snap/snap-subdir-test 2024-04-20T21:20:31.096 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/30 file after the snapshot 2024-04-20T21:20:31.099 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/31 file after the snapshot 2024-04-20T21:20:31.101 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/32 file after the snapshot 2024-04-20T21:20:31.104 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/33 file after the snapshot 2024-04-20T21:20:31.106 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/34 file after the snapshot 2024-04-20T21:20:31.109 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/35 file after the snapshot 2024-04-20T21:20:31.112 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/36 file after the snapshot 2024-04-20T21:20:31.115 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/37 file after the snapshot 2024-04-20T21:20:31.118 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/38 file after the snapshot 2024-04-20T21:20:31.120 INFO:tasks.workunit.client.0.smithi102.stdout: create 144/39 file after the snapshot 2024-04-20T21:20:31.123 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/.snap/snap-subdir-test 2024-04-20T21:20:31.167 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/30 file after the snapshot 2024-04-20T21:20:31.169 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/31 file after the snapshot 2024-04-20T21:20:31.172 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/32 file after the snapshot 2024-04-20T21:20:31.174 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/33 file after the snapshot 2024-04-20T21:20:31.177 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/34 file after the snapshot 2024-04-20T21:20:31.180 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/35 file after the snapshot 2024-04-20T21:20:31.182 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/36 file after the snapshot 2024-04-20T21:20:31.185 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/37 file after the snapshot 2024-04-20T21:20:31.188 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/38 file after the snapshot 2024-04-20T21:20:31.191 INFO:tasks.workunit.client.0.smithi102.stdout: create 145/39 file after the snapshot 2024-04-20T21:20:31.193 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/.snap/snap-subdir-test 2024-04-20T21:20:31.239 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/30 file after the snapshot 2024-04-20T21:20:31.242 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/31 file after the snapshot 2024-04-20T21:20:31.244 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/32 file after the snapshot 2024-04-20T21:20:31.247 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/33 file after the snapshot 2024-04-20T21:20:31.250 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/34 file after the snapshot 2024-04-20T21:20:31.252 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/35 file after the snapshot 2024-04-20T21:20:31.255 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/36 file after the snapshot 2024-04-20T21:20:31.258 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/37 file after the snapshot 2024-04-20T21:20:31.260 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/38 file after the snapshot 2024-04-20T21:20:31.263 INFO:tasks.workunit.client.0.smithi102.stdout: create 146/39 file after the snapshot 2024-04-20T21:20:31.266 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/.snap/snap-subdir-test 2024-04-20T21:20:31.308 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/30 file after the snapshot 2024-04-20T21:20:31.311 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/31 file after the snapshot 2024-04-20T21:20:31.313 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/32 file after the snapshot 2024-04-20T21:20:31.316 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/33 file after the snapshot 2024-04-20T21:20:31.319 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/34 file after the snapshot 2024-04-20T21:20:31.321 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/35 file after the snapshot 2024-04-20T21:20:31.324 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/36 file after the snapshot 2024-04-20T21:20:31.327 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/37 file after the snapshot 2024-04-20T21:20:31.330 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/38 file after the snapshot 2024-04-20T21:20:31.332 INFO:tasks.workunit.client.0.smithi102.stdout: create 147/39 file after the snapshot 2024-04-20T21:20:31.335 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/.snap/snap-subdir-test 2024-04-20T21:20:31.619 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/30 file after the snapshot 2024-04-20T21:20:31.622 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/31 file after the snapshot 2024-04-20T21:20:31.624 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/32 file after the snapshot 2024-04-20T21:20:31.627 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/33 file after the snapshot 2024-04-20T21:20:31.630 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/34 file after the snapshot 2024-04-20T21:20:31.632 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/35 file after the snapshot 2024-04-20T21:20:31.635 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/36 file after the snapshot 2024-04-20T21:20:31.638 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/37 file after the snapshot 2024-04-20T21:20:31.640 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/38 file after the snapshot 2024-04-20T21:20:31.643 INFO:tasks.workunit.client.0.smithi102.stdout: create 148/39 file after the snapshot 2024-04-20T21:20:31.645 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/.snap/snap-subdir-test 2024-04-20T21:20:31.686 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/30 file after the snapshot 2024-04-20T21:20:31.688 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/31 file after the snapshot 2024-04-20T21:20:31.689 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/32 file after the snapshot 2024-04-20T21:20:31.691 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/33 file after the snapshot 2024-04-20T21:20:31.692 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/34 file after the snapshot 2024-04-20T21:20:31.694 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/35 file after the snapshot 2024-04-20T21:20:31.696 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/36 file after the snapshot 2024-04-20T21:20:31.698 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/37 file after the snapshot 2024-04-20T21:20:31.700 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/38 file after the snapshot 2024-04-20T21:20:31.706 INFO:tasks.workunit.client.0.smithi102.stdout: create 149/39 file after the snapshot 2024-04-20T21:20:31.706 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/.snap/snap-subdir-test 2024-04-20T21:20:31.742 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/30 file after the snapshot 2024-04-20T21:20:31.744 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/31 file after the snapshot 2024-04-20T21:20:31.747 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/32 file after the snapshot 2024-04-20T21:20:31.750 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/33 file after the snapshot 2024-04-20T21:20:31.753 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/34 file after the snapshot 2024-04-20T21:20:31.755 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/35 file after the snapshot 2024-04-20T21:20:31.758 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/36 file after the snapshot 2024-04-20T21:20:31.761 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/37 file after the snapshot 2024-04-20T21:20:31.764 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/38 file after the snapshot 2024-04-20T21:20:31.767 INFO:tasks.workunit.client.0.smithi102.stdout: create 150/39 file after the snapshot 2024-04-20T21:20:31.770 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/.snap/snap-subdir-test 2024-04-20T21:20:31.806 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/30 file after the snapshot 2024-04-20T21:20:31.808 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/31 file after the snapshot 2024-04-20T21:20:31.811 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/32 file after the snapshot 2024-04-20T21:20:31.814 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/33 file after the snapshot 2024-04-20T21:20:31.817 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/34 file after the snapshot 2024-04-20T21:20:31.820 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/35 file after the snapshot 2024-04-20T21:20:31.823 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/36 file after the snapshot 2024-04-20T21:20:31.826 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/37 file after the snapshot 2024-04-20T21:20:31.829 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/38 file after the snapshot 2024-04-20T21:20:31.832 INFO:tasks.workunit.client.0.smithi102.stdout: create 151/39 file after the snapshot 2024-04-20T21:20:31.834 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/.snap/snap-subdir-test 2024-04-20T21:20:31.870 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/30 file after the snapshot 2024-04-20T21:20:31.873 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/31 file after the snapshot 2024-04-20T21:20:31.876 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/32 file after the snapshot 2024-04-20T21:20:31.879 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/33 file after the snapshot 2024-04-20T21:20:31.881 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/34 file after the snapshot 2024-04-20T21:20:31.884 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/35 file after the snapshot 2024-04-20T21:20:31.887 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/36 file after the snapshot 2024-04-20T21:20:31.890 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/37 file after the snapshot 2024-04-20T21:20:31.893 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/38 file after the snapshot 2024-04-20T21:20:31.895 INFO:tasks.workunit.client.0.smithi102.stdout: create 152/39 file after the snapshot 2024-04-20T21:20:31.898 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/.snap/snap-subdir-test 2024-04-20T21:20:31.936 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/30 file after the snapshot 2024-04-20T21:20:31.939 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/31 file after the snapshot 2024-04-20T21:20:31.941 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/32 file after the snapshot 2024-04-20T21:20:31.944 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/33 file after the snapshot 2024-04-20T21:20:31.947 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/34 file after the snapshot 2024-04-20T21:20:31.950 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/35 file after the snapshot 2024-04-20T21:20:31.953 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/36 file after the snapshot 2024-04-20T21:20:31.955 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/37 file after the snapshot 2024-04-20T21:20:31.958 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/38 file after the snapshot 2024-04-20T21:20:31.961 INFO:tasks.workunit.client.0.smithi102.stdout: create 153/39 file after the snapshot 2024-04-20T21:20:31.963 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/.snap/snap-subdir-test 2024-04-20T21:20:32.001 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/30 file after the snapshot 2024-04-20T21:20:32.004 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/31 file after the snapshot 2024-04-20T21:20:32.006 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/32 file after the snapshot 2024-04-20T21:20:32.009 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/33 file after the snapshot 2024-04-20T21:20:32.012 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/34 file after the snapshot 2024-04-20T21:20:32.015 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/35 file after the snapshot 2024-04-20T21:20:32.018 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/36 file after the snapshot 2024-04-20T21:20:32.021 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/37 file after the snapshot 2024-04-20T21:20:32.024 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/38 file after the snapshot 2024-04-20T21:20:32.026 INFO:tasks.workunit.client.0.smithi102.stdout: create 154/39 file after the snapshot 2024-04-20T21:20:32.029 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/.snap/snap-subdir-test 2024-04-20T21:20:32.067 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/30 file after the snapshot 2024-04-20T21:20:32.069 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/31 file after the snapshot 2024-04-20T21:20:32.072 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/32 file after the snapshot 2024-04-20T21:20:32.417 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/33 file after the snapshot 2024-04-20T21:20:32.419 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/34 file after the snapshot 2024-04-20T21:20:32.422 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/35 file after the snapshot 2024-04-20T21:20:32.425 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/36 file after the snapshot 2024-04-20T21:20:32.428 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/37 file after the snapshot 2024-04-20T21:20:32.430 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/38 file after the snapshot 2024-04-20T21:20:32.433 INFO:tasks.workunit.client.0.smithi102.stdout: create 155/39 file after the snapshot 2024-04-20T21:20:32.436 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/.snap/snap-subdir-test 2024-04-20T21:20:32.482 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/30 file after the snapshot 2024-04-20T21:20:32.485 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/31 file after the snapshot 2024-04-20T21:20:32.488 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/32 file after the snapshot 2024-04-20T21:20:32.491 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/33 file after the snapshot 2024-04-20T21:20:32.494 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/34 file after the snapshot 2024-04-20T21:20:32.496 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/35 file after the snapshot 2024-04-20T21:20:32.499 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/36 file after the snapshot 2024-04-20T21:20:32.502 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/37 file after the snapshot 2024-04-20T21:20:32.505 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/38 file after the snapshot 2024-04-20T21:20:32.508 INFO:tasks.workunit.client.0.smithi102.stdout: create 156/39 file after the snapshot 2024-04-20T21:20:32.510 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/.snap/snap-subdir-test 2024-04-20T21:20:32.560 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/30 file after the snapshot 2024-04-20T21:20:32.561 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/31 file after the snapshot 2024-04-20T21:20:32.563 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/32 file after the snapshot 2024-04-20T21:20:32.564 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/33 file after the snapshot 2024-04-20T21:20:32.566 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/34 file after the snapshot 2024-04-20T21:20:32.568 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/35 file after the snapshot 2024-04-20T21:20:32.569 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/36 file after the snapshot 2024-04-20T21:20:32.571 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/37 file after the snapshot 2024-04-20T21:20:32.572 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/38 file after the snapshot 2024-04-20T21:20:32.574 INFO:tasks.workunit.client.0.smithi102.stdout: create 157/39 file after the snapshot 2024-04-20T21:20:32.576 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/.snap/snap-subdir-test 2024-04-20T21:20:32.620 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/30 file after the snapshot 2024-04-20T21:20:32.623 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/31 file after the snapshot 2024-04-20T21:20:32.626 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/32 file after the snapshot 2024-04-20T21:20:32.628 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/33 file after the snapshot 2024-04-20T21:20:32.631 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/34 file after the snapshot 2024-04-20T21:20:32.634 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/35 file after the snapshot 2024-04-20T21:20:32.637 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/36 file after the snapshot 2024-04-20T21:20:32.639 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/37 file after the snapshot 2024-04-20T21:20:32.642 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/38 file after the snapshot 2024-04-20T21:20:32.645 INFO:tasks.workunit.client.0.smithi102.stdout: create 158/39 file after the snapshot 2024-04-20T21:20:32.647 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/.snap/snap-subdir-test 2024-04-20T21:20:32.688 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/30 file after the snapshot 2024-04-20T21:20:32.690 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/31 file after the snapshot 2024-04-20T21:20:32.691 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/32 file after the snapshot 2024-04-20T21:20:32.693 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/33 file after the snapshot 2024-04-20T21:20:32.696 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/34 file after the snapshot 2024-04-20T21:20:32.698 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/35 file after the snapshot 2024-04-20T21:20:32.701 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/36 file after the snapshot 2024-04-20T21:20:32.704 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/37 file after the snapshot 2024-04-20T21:20:32.707 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/38 file after the snapshot 2024-04-20T21:20:32.710 INFO:tasks.workunit.client.0.smithi102.stdout: create 159/39 file after the snapshot 2024-04-20T21:20:32.713 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/.snap/snap-subdir-test 2024-04-20T21:20:32.756 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/30 file after the snapshot 2024-04-20T21:20:32.758 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/31 file after the snapshot 2024-04-20T21:20:32.761 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/32 file after the snapshot 2024-04-20T21:20:32.764 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/33 file after the snapshot 2024-04-20T21:20:32.767 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/34 file after the snapshot 2024-04-20T21:20:32.770 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/35 file after the snapshot 2024-04-20T21:20:32.772 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/36 file after the snapshot 2024-04-20T21:20:32.775 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/37 file after the snapshot 2024-04-20T21:20:32.778 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/38 file after the snapshot 2024-04-20T21:20:32.781 INFO:tasks.workunit.client.0.smithi102.stdout: create 160/39 file after the snapshot 2024-04-20T21:20:32.784 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/.snap/snap-subdir-test 2024-04-20T21:20:32.827 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/30 file after the snapshot 2024-04-20T21:20:32.830 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/31 file after the snapshot 2024-04-20T21:20:32.833 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/32 file after the snapshot 2024-04-20T21:20:32.836 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/33 file after the snapshot 2024-04-20T21:20:32.839 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/34 file after the snapshot 2024-04-20T21:20:32.842 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/35 file after the snapshot 2024-04-20T21:20:32.845 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/36 file after the snapshot 2024-04-20T21:20:32.847 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/37 file after the snapshot 2024-04-20T21:20:32.850 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/38 file after the snapshot 2024-04-20T21:20:32.853 INFO:tasks.workunit.client.0.smithi102.stdout: create 161/39 file after the snapshot 2024-04-20T21:20:32.856 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/.snap/snap-subdir-test 2024-04-20T21:20:32.899 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/30 file after the snapshot 2024-04-20T21:20:32.902 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/31 file after the snapshot 2024-04-20T21:20:32.904 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/32 file after the snapshot 2024-04-20T21:20:32.907 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/33 file after the snapshot 2024-04-20T21:20:32.910 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/34 file after the snapshot 2024-04-20T21:20:32.913 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/35 file after the snapshot 2024-04-20T21:20:32.915 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/36 file after the snapshot 2024-04-20T21:20:32.918 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/37 file after the snapshot 2024-04-20T21:20:32.921 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/38 file after the snapshot 2024-04-20T21:20:32.923 INFO:tasks.workunit.client.0.smithi102.stdout: create 162/39 file after the snapshot 2024-04-20T21:20:32.926 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/.snap/snap-subdir-test 2024-04-20T21:20:32.971 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/30 file after the snapshot 2024-04-20T21:20:32.973 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/31 file after the snapshot 2024-04-20T21:20:32.975 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/32 file after the snapshot 2024-04-20T21:20:32.977 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/33 file after the snapshot 2024-04-20T21:20:32.978 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/34 file after the snapshot 2024-04-20T21:20:32.980 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/35 file after the snapshot 2024-04-20T21:20:32.982 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/36 file after the snapshot 2024-04-20T21:20:32.983 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/37 file after the snapshot 2024-04-20T21:20:32.985 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/38 file after the snapshot 2024-04-20T21:20:32.987 INFO:tasks.workunit.client.0.smithi102.stdout: create 163/39 file after the snapshot 2024-04-20T21:20:32.989 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/.snap/snap-subdir-test 2024-04-20T21:20:33.028 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/30 file after the snapshot 2024-04-20T21:20:33.030 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/31 file after the snapshot 2024-04-20T21:20:33.033 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/32 file after the snapshot 2024-04-20T21:20:33.035 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/33 file after the snapshot 2024-04-20T21:20:33.038 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/34 file after the snapshot 2024-04-20T21:20:33.041 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/35 file after the snapshot 2024-04-20T21:20:33.043 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/36 file after the snapshot 2024-04-20T21:20:33.046 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/37 file after the snapshot 2024-04-20T21:20:33.049 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/38 file after the snapshot 2024-04-20T21:20:33.052 INFO:tasks.workunit.client.0.smithi102.stdout: create 164/39 file after the snapshot 2024-04-20T21:20:33.055 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/.snap/snap-subdir-test 2024-04-20T21:20:33.104 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/30 file after the snapshot 2024-04-20T21:20:33.106 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/31 file after the snapshot 2024-04-20T21:20:33.109 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/32 file after the snapshot 2024-04-20T21:20:33.112 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/33 file after the snapshot 2024-04-20T21:20:33.115 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/34 file after the snapshot 2024-04-20T21:20:33.119 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/35 file after the snapshot 2024-04-20T21:20:33.122 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/36 file after the snapshot 2024-04-20T21:20:33.125 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/37 file after the snapshot 2024-04-20T21:20:33.127 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/38 file after the snapshot 2024-04-20T21:20:33.130 INFO:tasks.workunit.client.0.smithi102.stdout: create 165/39 file after the snapshot 2024-04-20T21:20:33.133 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/.snap/snap-subdir-test 2024-04-20T21:20:33.185 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/30 file after the snapshot 2024-04-20T21:20:33.187 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/31 file after the snapshot 2024-04-20T21:20:33.189 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/32 file after the snapshot 2024-04-20T21:20:33.191 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/33 file after the snapshot 2024-04-20T21:20:33.193 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/34 file after the snapshot 2024-04-20T21:20:33.195 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/35 file after the snapshot 2024-04-20T21:20:33.197 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/36 file after the snapshot 2024-04-20T21:20:33.199 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/37 file after the snapshot 2024-04-20T21:20:33.202 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/38 file after the snapshot 2024-04-20T21:20:33.205 INFO:tasks.workunit.client.0.smithi102.stdout: create 166/39 file after the snapshot 2024-04-20T21:20:33.208 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/.snap/snap-subdir-test 2024-04-20T21:20:33.259 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/30 file after the snapshot 2024-04-20T21:20:33.261 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/31 file after the snapshot 2024-04-20T21:20:33.263 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/32 file after the snapshot 2024-04-20T21:20:33.265 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/33 file after the snapshot 2024-04-20T21:20:33.266 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/34 file after the snapshot 2024-04-20T21:20:33.268 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/35 file after the snapshot 2024-04-20T21:20:33.270 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/36 file after the snapshot 2024-04-20T21:20:33.272 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/37 file after the snapshot 2024-04-20T21:20:33.274 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/38 file after the snapshot 2024-04-20T21:20:33.277 INFO:tasks.workunit.client.0.smithi102.stdout: create 167/39 file after the snapshot 2024-04-20T21:20:33.279 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/.snap/snap-subdir-test 2024-04-20T21:20:33.331 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/30 file after the snapshot 2024-04-20T21:20:33.333 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/31 file after the snapshot 2024-04-20T21:20:33.334 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/32 file after the snapshot 2024-04-20T21:20:33.336 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/33 file after the snapshot 2024-04-20T21:20:33.339 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/34 file after the snapshot 2024-04-20T21:20:33.341 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/35 file after the snapshot 2024-04-20T21:20:33.346 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/36 file after the snapshot 2024-04-20T21:20:33.347 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/37 file after the snapshot 2024-04-20T21:20:33.350 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/38 file after the snapshot 2024-04-20T21:20:33.353 INFO:tasks.workunit.client.0.smithi102.stdout: create 168/39 file after the snapshot 2024-04-20T21:20:33.355 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/.snap/snap-subdir-test 2024-04-20T21:20:33.405 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/30 file after the snapshot 2024-04-20T21:20:33.407 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/31 file after the snapshot 2024-04-20T21:20:33.410 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/32 file after the snapshot 2024-04-20T21:20:33.413 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/33 file after the snapshot 2024-04-20T21:20:33.415 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/34 file after the snapshot 2024-04-20T21:20:33.418 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/35 file after the snapshot 2024-04-20T21:20:33.421 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/36 file after the snapshot 2024-04-20T21:20:33.424 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/37 file after the snapshot 2024-04-20T21:20:33.426 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/38 file after the snapshot 2024-04-20T21:20:33.429 INFO:tasks.workunit.client.0.smithi102.stdout: create 169/39 file after the snapshot 2024-04-20T21:20:33.432 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/.snap/snap-subdir-test 2024-04-20T21:20:33.485 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/30 file after the snapshot 2024-04-20T21:20:33.487 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/31 file after the snapshot 2024-04-20T21:20:33.489 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/32 file after the snapshot 2024-04-20T21:20:33.491 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/33 file after the snapshot 2024-04-20T21:20:33.492 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/34 file after the snapshot 2024-04-20T21:20:33.494 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/35 file after the snapshot 2024-04-20T21:20:33.757 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/36 file after the snapshot 2024-04-20T21:20:33.759 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/37 file after the snapshot 2024-04-20T21:20:33.762 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/38 file after the snapshot 2024-04-20T21:20:33.765 INFO:tasks.workunit.client.0.smithi102.stdout: create 170/39 file after the snapshot 2024-04-20T21:20:33.768 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/.snap/snap-subdir-test 2024-04-20T21:20:33.814 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/30 file after the snapshot 2024-04-20T21:20:33.817 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/31 file after the snapshot 2024-04-20T21:20:33.820 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/32 file after the snapshot 2024-04-20T21:20:33.823 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/33 file after the snapshot 2024-04-20T21:20:33.826 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/34 file after the snapshot 2024-04-20T21:20:33.828 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/35 file after the snapshot 2024-04-20T21:20:33.841 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/36 file after the snapshot 2024-04-20T21:20:33.844 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/37 file after the snapshot 2024-04-20T21:20:33.847 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/38 file after the snapshot 2024-04-20T21:20:33.849 INFO:tasks.workunit.client.0.smithi102.stdout: create 171/39 file after the snapshot 2024-04-20T21:20:33.852 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/.snap/snap-subdir-test 2024-04-20T21:20:33.903 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/30 file after the snapshot 2024-04-20T21:20:33.905 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/31 file after the snapshot 2024-04-20T21:20:33.908 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/32 file after the snapshot 2024-04-20T21:20:33.911 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/33 file after the snapshot 2024-04-20T21:20:33.914 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/34 file after the snapshot 2024-04-20T21:20:33.916 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/35 file after the snapshot 2024-04-20T21:20:33.919 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/36 file after the snapshot 2024-04-20T21:20:33.922 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/37 file after the snapshot 2024-04-20T21:20:33.926 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/38 file after the snapshot 2024-04-20T21:20:33.928 INFO:tasks.workunit.client.0.smithi102.stdout: create 172/39 file after the snapshot 2024-04-20T21:20:33.931 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/.snap/snap-subdir-test 2024-04-20T21:20:34.000 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/30 file after the snapshot 2024-04-20T21:20:34.002 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/31 file after the snapshot 2024-04-20T21:20:34.005 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/32 file after the snapshot 2024-04-20T21:20:34.008 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/33 file after the snapshot 2024-04-20T21:20:34.011 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/34 file after the snapshot 2024-04-20T21:20:34.013 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/35 file after the snapshot 2024-04-20T21:20:34.016 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/36 file after the snapshot 2024-04-20T21:20:34.019 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/37 file after the snapshot 2024-04-20T21:20:34.022 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/38 file after the snapshot 2024-04-20T21:20:34.025 INFO:tasks.workunit.client.0.smithi102.stdout: create 173/39 file after the snapshot 2024-04-20T21:20:34.028 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/.snap/snap-subdir-test 2024-04-20T21:20:34.081 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/30 file after the snapshot 2024-04-20T21:20:34.083 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/31 file after the snapshot 2024-04-20T21:20:34.085 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/32 file after the snapshot 2024-04-20T21:20:34.087 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/33 file after the snapshot 2024-04-20T21:20:34.089 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/34 file after the snapshot 2024-04-20T21:20:34.091 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/35 file after the snapshot 2024-04-20T21:20:34.094 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/36 file after the snapshot 2024-04-20T21:20:34.097 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/37 file after the snapshot 2024-04-20T21:20:34.099 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/38 file after the snapshot 2024-04-20T21:20:34.102 INFO:tasks.workunit.client.0.smithi102.stdout: create 174/39 file after the snapshot 2024-04-20T21:20:34.105 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/.snap/snap-subdir-test 2024-04-20T21:20:34.158 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/30 file after the snapshot 2024-04-20T21:20:34.160 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/31 file after the snapshot 2024-04-20T21:20:34.166 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/32 file after the snapshot 2024-04-20T21:20:34.166 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/33 file after the snapshot 2024-04-20T21:20:34.168 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/34 file after the snapshot 2024-04-20T21:20:34.171 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/35 file after the snapshot 2024-04-20T21:20:34.173 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/36 file after the snapshot 2024-04-20T21:20:34.176 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/37 file after the snapshot 2024-04-20T21:20:34.178 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/38 file after the snapshot 2024-04-20T21:20:34.181 INFO:tasks.workunit.client.0.smithi102.stdout: create 175/39 file after the snapshot 2024-04-20T21:20:34.183 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/.snap/snap-subdir-test 2024-04-20T21:20:34.244 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/30 file after the snapshot 2024-04-20T21:20:34.246 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/31 file after the snapshot 2024-04-20T21:20:34.249 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/32 file after the snapshot 2024-04-20T21:20:34.252 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/33 file after the snapshot 2024-04-20T21:20:34.254 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/34 file after the snapshot 2024-04-20T21:20:34.256 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/35 file after the snapshot 2024-04-20T21:20:34.259 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/36 file after the snapshot 2024-04-20T21:20:34.263 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/37 file after the snapshot 2024-04-20T21:20:34.267 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/38 file after the snapshot 2024-04-20T21:20:34.271 INFO:tasks.workunit.client.0.smithi102.stdout: create 176/39 file after the snapshot 2024-04-20T21:20:34.275 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/.snap/snap-subdir-test 2024-04-20T21:20:34.332 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/30 file after the snapshot 2024-04-20T21:20:34.336 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/31 file after the snapshot 2024-04-20T21:20:34.338 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/32 file after the snapshot 2024-04-20T21:20:34.340 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/33 file after the snapshot 2024-04-20T21:20:34.342 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/34 file after the snapshot 2024-04-20T21:20:34.607 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/35 file after the snapshot 2024-04-20T21:20:34.719 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/36 file after the snapshot 2024-04-20T21:20:34.720 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/37 file after the snapshot 2024-04-20T21:20:34.720 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/38 file after the snapshot 2024-04-20T21:20:34.720 INFO:tasks.workunit.client.0.smithi102.stdout: create 177/39 file after the snapshot 2024-04-20T21:20:34.720 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/.snap/snap-subdir-test 2024-04-20T21:20:34.991 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/30 file after the snapshot 2024-04-20T21:20:35.341 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/31 file after the snapshot 2024-04-20T21:20:35.810 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/32 file after the snapshot 2024-04-20T21:20:35.855 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/33 file after the snapshot 2024-04-20T21:20:35.857 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/34 file after the snapshot 2024-04-20T21:20:35.860 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/35 file after the snapshot 2024-04-20T21:20:35.863 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/36 file after the snapshot 2024-04-20T21:20:35.866 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/37 file after the snapshot 2024-04-20T21:20:35.868 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/38 file after the snapshot 2024-04-20T21:20:35.871 INFO:tasks.workunit.client.0.smithi102.stdout: create 178/39 file after the snapshot 2024-04-20T21:20:35.874 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/.snap/snap-subdir-test 2024-04-20T21:20:36.881 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/30 file after the snapshot 2024-04-20T21:20:36.884 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/31 file after the snapshot 2024-04-20T21:20:36.887 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/32 file after the snapshot 2024-04-20T21:20:36.890 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/33 file after the snapshot 2024-04-20T21:20:36.893 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/34 file after the snapshot 2024-04-20T21:20:36.896 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/35 file after the snapshot 2024-04-20T21:20:36.899 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/36 file after the snapshot 2024-04-20T21:20:36.902 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/37 file after the snapshot 2024-04-20T21:20:36.905 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/38 file after the snapshot 2024-04-20T21:20:36.908 INFO:tasks.workunit.client.0.smithi102.stdout: create 179/39 file after the snapshot 2024-04-20T21:20:36.910 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/.snap/snap-subdir-test 2024-04-20T21:20:36.956 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/30 file after the snapshot 2024-04-20T21:20:36.959 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/31 file after the snapshot 2024-04-20T21:20:36.961 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/32 file after the snapshot 2024-04-20T21:20:36.964 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/33 file after the snapshot 2024-04-20T21:20:36.967 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/34 file after the snapshot 2024-04-20T21:20:36.970 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/35 file after the snapshot 2024-04-20T21:20:36.972 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/36 file after the snapshot 2024-04-20T21:20:36.975 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/37 file after the snapshot 2024-04-20T21:20:36.978 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/38 file after the snapshot 2024-04-20T21:20:36.981 INFO:tasks.workunit.client.0.smithi102.stdout: create 180/39 file after the snapshot 2024-04-20T21:20:36.984 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/.snap/snap-subdir-test 2024-04-20T21:20:37.031 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/30 file after the snapshot 2024-04-20T21:20:37.033 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/31 file after the snapshot 2024-04-20T21:20:37.036 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/32 file after the snapshot 2024-04-20T21:20:37.039 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/33 file after the snapshot 2024-04-20T21:20:37.042 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/34 file after the snapshot 2024-04-20T21:20:37.044 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/35 file after the snapshot 2024-04-20T21:20:37.047 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/36 file after the snapshot 2024-04-20T21:20:37.050 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/37 file after the snapshot 2024-04-20T21:20:37.053 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/38 file after the snapshot 2024-04-20T21:20:37.056 INFO:tasks.workunit.client.0.smithi102.stdout: create 181/39 file after the snapshot 2024-04-20T21:20:37.059 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/.snap/snap-subdir-test 2024-04-20T21:20:37.112 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/30 file after the snapshot 2024-04-20T21:20:37.114 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/31 file after the snapshot 2024-04-20T21:20:37.116 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/32 file after the snapshot 2024-04-20T21:20:37.118 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/33 file after the snapshot 2024-04-20T21:20:37.120 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/34 file after the snapshot 2024-04-20T21:20:37.122 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/35 file after the snapshot 2024-04-20T21:20:37.124 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/36 file after the snapshot 2024-04-20T21:20:37.126 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/37 file after the snapshot 2024-04-20T21:20:37.128 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/38 file after the snapshot 2024-04-20T21:20:37.130 INFO:tasks.workunit.client.0.smithi102.stdout: create 182/39 file after the snapshot 2024-04-20T21:20:37.132 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/.snap/snap-subdir-test 2024-04-20T21:20:37.182 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/30 file after the snapshot 2024-04-20T21:20:37.184 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/31 file after the snapshot 2024-04-20T21:20:37.186 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/32 file after the snapshot 2024-04-20T21:20:37.188 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/33 file after the snapshot 2024-04-20T21:20:37.190 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/34 file after the snapshot 2024-04-20T21:20:37.192 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/35 file after the snapshot 2024-04-20T21:20:37.194 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/36 file after the snapshot 2024-04-20T21:20:37.197 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/37 file after the snapshot 2024-04-20T21:20:37.200 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/38 file after the snapshot 2024-04-20T21:20:37.203 INFO:tasks.workunit.client.0.smithi102.stdout: create 183/39 file after the snapshot 2024-04-20T21:20:37.206 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/.snap/snap-subdir-test 2024-04-20T21:20:37.250 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/30 file after the snapshot 2024-04-20T21:20:37.252 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/31 file after the snapshot 2024-04-20T21:20:37.255 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/32 file after the snapshot 2024-04-20T21:20:37.258 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/33 file after the snapshot 2024-04-20T21:20:37.260 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/34 file after the snapshot 2024-04-20T21:20:37.263 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/35 file after the snapshot 2024-04-20T21:20:37.265 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/36 file after the snapshot 2024-04-20T21:20:37.268 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/37 file after the snapshot 2024-04-20T21:20:37.271 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/38 file after the snapshot 2024-04-20T21:20:37.273 INFO:tasks.workunit.client.0.smithi102.stdout: create 184/39 file after the snapshot 2024-04-20T21:20:37.276 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/.snap/snap-subdir-test 2024-04-20T21:20:37.320 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/30 file after the snapshot 2024-04-20T21:20:37.323 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/31 file after the snapshot 2024-04-20T21:20:37.326 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/32 file after the snapshot 2024-04-20T21:20:37.328 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/33 file after the snapshot 2024-04-20T21:20:37.331 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/34 file after the snapshot 2024-04-20T21:20:37.334 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/35 file after the snapshot 2024-04-20T21:20:37.336 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/36 file after the snapshot 2024-04-20T21:20:37.339 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/37 file after the snapshot 2024-04-20T21:20:37.342 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/38 file after the snapshot 2024-04-20T21:20:37.344 INFO:tasks.workunit.client.0.smithi102.stdout: create 185/39 file after the snapshot 2024-04-20T21:20:37.347 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/.snap/snap-subdir-test 2024-04-20T21:20:37.390 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/30 file after the snapshot 2024-04-20T21:20:37.393 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/31 file after the snapshot 2024-04-20T21:20:37.396 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/32 file after the snapshot 2024-04-20T21:20:37.398 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/33 file after the snapshot 2024-04-20T21:20:37.401 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/34 file after the snapshot 2024-04-20T21:20:37.404 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/35 file after the snapshot 2024-04-20T21:20:37.406 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/36 file after the snapshot 2024-04-20T21:20:37.409 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/37 file after the snapshot 2024-04-20T21:20:37.412 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/38 file after the snapshot 2024-04-20T21:20:37.415 INFO:tasks.workunit.client.0.smithi102.stdout: create 186/39 file after the snapshot 2024-04-20T21:20:37.417 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/.snap/snap-subdir-test 2024-04-20T21:20:37.464 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/30 file after the snapshot 2024-04-20T21:20:37.467 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/31 file after the snapshot 2024-04-20T21:20:37.469 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/32 file after the snapshot 2024-04-20T21:20:37.472 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/33 file after the snapshot 2024-04-20T21:20:37.475 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/34 file after the snapshot 2024-04-20T21:20:37.478 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/35 file after the snapshot 2024-04-20T21:20:37.480 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/36 file after the snapshot 2024-04-20T21:20:37.483 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/37 file after the snapshot 2024-04-20T21:20:37.485 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/38 file after the snapshot 2024-04-20T21:20:37.488 INFO:tasks.workunit.client.0.smithi102.stdout: create 187/39 file after the snapshot 2024-04-20T21:20:37.491 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/.snap/snap-subdir-test 2024-04-20T21:20:37.536 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/30 file after the snapshot 2024-04-20T21:20:37.538 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/31 file after the snapshot 2024-04-20T21:20:37.541 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/32 file after the snapshot 2024-04-20T21:20:37.543 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/33 file after the snapshot 2024-04-20T21:20:37.546 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/34 file after the snapshot 2024-04-20T21:20:37.549 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/35 file after the snapshot 2024-04-20T21:20:37.551 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/36 file after the snapshot 2024-04-20T21:20:37.554 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/37 file after the snapshot 2024-04-20T21:20:37.557 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/38 file after the snapshot 2024-04-20T21:20:37.560 INFO:tasks.workunit.client.0.smithi102.stdout: create 188/39 file after the snapshot 2024-04-20T21:20:37.562 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/.snap/snap-subdir-test 2024-04-20T21:20:37.614 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/30 file after the snapshot 2024-04-20T21:20:37.617 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/31 file after the snapshot 2024-04-20T21:20:37.619 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/32 file after the snapshot 2024-04-20T21:20:37.622 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/33 file after the snapshot 2024-04-20T21:20:37.624 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/34 file after the snapshot 2024-04-20T21:20:37.627 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/35 file after the snapshot 2024-04-20T21:20:37.630 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/36 file after the snapshot 2024-04-20T21:20:37.632 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/37 file after the snapshot 2024-04-20T21:20:37.635 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/38 file after the snapshot 2024-04-20T21:20:37.638 INFO:tasks.workunit.client.0.smithi102.stdout: create 189/39 file after the snapshot 2024-04-20T21:20:37.641 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/.snap/snap-subdir-test 2024-04-20T21:20:37.686 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/30 file after the snapshot 2024-04-20T21:20:37.688 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/31 file after the snapshot 2024-04-20T21:20:37.691 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/32 file after the snapshot 2024-04-20T21:20:37.693 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/33 file after the snapshot 2024-04-20T21:20:37.697 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/34 file after the snapshot 2024-04-20T21:20:37.699 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/35 file after the snapshot 2024-04-20T21:20:37.702 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/36 file after the snapshot 2024-04-20T21:20:37.704 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/37 file after the snapshot 2024-04-20T21:20:37.707 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/38 file after the snapshot 2024-04-20T21:20:37.709 INFO:tasks.workunit.client.0.smithi102.stdout: create 190/39 file after the snapshot 2024-04-20T21:20:37.712 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/.snap/snap-subdir-test 2024-04-20T21:20:37.758 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/30 file after the snapshot 2024-04-20T21:20:37.759 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/31 file after the snapshot 2024-04-20T21:20:37.761 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/32 file after the snapshot 2024-04-20T21:20:37.762 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/33 file after the snapshot 2024-04-20T21:20:37.764 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/34 file after the snapshot 2024-04-20T21:20:37.766 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/35 file after the snapshot 2024-04-20T21:20:37.768 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/36 file after the snapshot 2024-04-20T21:20:37.769 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/37 file after the snapshot 2024-04-20T21:20:37.771 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/38 file after the snapshot 2024-04-20T21:20:37.773 INFO:tasks.workunit.client.0.smithi102.stdout: create 191/39 file after the snapshot 2024-04-20T21:20:37.775 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/.snap/snap-subdir-test 2024-04-20T21:20:37.812 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/30 file after the snapshot 2024-04-20T21:20:37.815 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/31 file after the snapshot 2024-04-20T21:20:37.818 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/32 file after the snapshot 2024-04-20T21:20:37.821 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/33 file after the snapshot 2024-04-20T21:20:37.823 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/34 file after the snapshot 2024-04-20T21:20:37.826 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/35 file after the snapshot 2024-04-20T21:20:37.828 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/36 file after the snapshot 2024-04-20T21:20:37.831 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/37 file after the snapshot 2024-04-20T21:20:37.834 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/38 file after the snapshot 2024-04-20T21:20:37.837 INFO:tasks.workunit.client.0.smithi102.stdout: create 192/39 file after the snapshot 2024-04-20T21:20:37.842 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/.snap/snap-subdir-test 2024-04-20T21:20:37.885 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/30 file after the snapshot 2024-04-20T21:20:37.888 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/31 file after the snapshot 2024-04-20T21:20:37.892 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/32 file after the snapshot 2024-04-20T21:20:37.896 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/33 file after the snapshot 2024-04-20T21:20:37.900 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/34 file after the snapshot 2024-04-20T21:20:37.904 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/35 file after the snapshot 2024-04-20T21:20:37.908 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/36 file after the snapshot 2024-04-20T21:20:37.913 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/37 file after the snapshot 2024-04-20T21:20:37.917 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/38 file after the snapshot 2024-04-20T21:20:37.921 INFO:tasks.workunit.client.0.smithi102.stdout: create 193/39 file after the snapshot 2024-04-20T21:20:37.925 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/.snap/snap-subdir-test 2024-04-20T21:20:37.979 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/30 file after the snapshot 2024-04-20T21:20:37.982 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/31 file after the snapshot 2024-04-20T21:20:37.986 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/32 file after the snapshot 2024-04-20T21:20:37.990 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/33 file after the snapshot 2024-04-20T21:20:37.994 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/34 file after the snapshot 2024-04-20T21:20:37.998 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/35 file after the snapshot 2024-04-20T21:20:38.003 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/36 file after the snapshot 2024-04-20T21:20:38.007 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/37 file after the snapshot 2024-04-20T21:20:38.011 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/38 file after the snapshot 2024-04-20T21:20:38.015 INFO:tasks.workunit.client.0.smithi102.stdout: create 194/39 file after the snapshot 2024-04-20T21:20:38.018 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/.snap/snap-subdir-test 2024-04-20T21:20:38.062 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/30 file after the snapshot 2024-04-20T21:20:38.065 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/31 file after the snapshot 2024-04-20T21:20:38.068 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/32 file after the snapshot 2024-04-20T21:20:38.070 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/33 file after the snapshot 2024-04-20T21:20:38.073 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/34 file after the snapshot 2024-04-20T21:20:38.076 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/35 file after the snapshot 2024-04-20T21:20:38.079 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/36 file after the snapshot 2024-04-20T21:20:38.082 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/37 file after the snapshot 2024-04-20T21:20:38.085 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/38 file after the snapshot 2024-04-20T21:20:38.088 INFO:tasks.workunit.client.0.smithi102.stdout: create 195/39 file after the snapshot 2024-04-20T21:20:38.091 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/.snap/snap-subdir-test 2024-04-20T21:20:38.135 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/30 file after the snapshot 2024-04-20T21:20:38.138 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/31 file after the snapshot 2024-04-20T21:20:38.141 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/32 file after the snapshot 2024-04-20T21:20:38.144 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/33 file after the snapshot 2024-04-20T21:20:38.147 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/34 file after the snapshot 2024-04-20T21:20:38.149 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/35 file after the snapshot 2024-04-20T21:20:38.152 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/36 file after the snapshot 2024-04-20T21:20:38.155 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/37 file after the snapshot 2024-04-20T21:20:38.158 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/38 file after the snapshot 2024-04-20T21:20:38.162 INFO:tasks.workunit.client.0.smithi102.stdout: create 196/39 file after the snapshot 2024-04-20T21:20:38.166 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/.snap/snap-subdir-test 2024-04-20T21:20:38.210 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/30 file after the snapshot 2024-04-20T21:20:38.213 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/31 file after the snapshot 2024-04-20T21:20:38.216 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/32 file after the snapshot 2024-04-20T21:20:38.219 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/33 file after the snapshot 2024-04-20T21:20:38.221 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/34 file after the snapshot 2024-04-20T21:20:38.224 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/35 file after the snapshot 2024-04-20T21:20:38.227 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/36 file after the snapshot 2024-04-20T21:20:38.230 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/37 file after the snapshot 2024-04-20T21:20:38.233 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/38 file after the snapshot 2024-04-20T21:20:38.236 INFO:tasks.workunit.client.0.smithi102.stdout: create 197/39 file after the snapshot 2024-04-20T21:20:38.239 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/.snap/snap-subdir-test 2024-04-20T21:20:38.827 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/30 file after the snapshot 2024-04-20T21:20:38.830 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/31 file after the snapshot 2024-04-20T21:20:38.833 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/32 file after the snapshot 2024-04-20T21:20:38.836 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/33 file after the snapshot 2024-04-20T21:20:38.839 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/34 file after the snapshot 2024-04-20T21:20:38.841 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/35 file after the snapshot 2024-04-20T21:20:38.844 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/36 file after the snapshot 2024-04-20T21:20:38.847 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/37 file after the snapshot 2024-04-20T21:20:39.524 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/38 file after the snapshot 2024-04-20T21:20:39.527 INFO:tasks.workunit.client.0.smithi102.stdout: create 198/39 file after the snapshot 2024-04-20T21:20:39.531 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/.snap/snap-subdir-test 2024-04-20T21:20:39.593 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/30 file after the snapshot 2024-04-20T21:20:39.595 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/31 file after the snapshot 2024-04-20T21:20:39.598 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/32 file after the snapshot 2024-04-20T21:20:39.601 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/33 file after the snapshot 2024-04-20T21:20:39.603 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/34 file after the snapshot 2024-04-20T21:20:39.606 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/35 file after the snapshot 2024-04-20T21:20:39.609 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/36 file after the snapshot 2024-04-20T21:20:39.611 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/37 file after the snapshot 2024-04-20T21:20:39.614 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/38 file after the snapshot 2024-04-20T21:20:39.617 INFO:tasks.workunit.client.0.smithi102.stdout: create 199/39 file after the snapshot 2024-04-20T21:20:39.620 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/.snap/snap-subdir-test 2024-04-20T21:20:39.679 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/30 file after the snapshot 2024-04-20T21:20:39.680 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/31 file after the snapshot 2024-04-20T21:20:39.683 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/32 file after the snapshot 2024-04-20T21:20:39.686 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/33 file after the snapshot 2024-04-20T21:20:39.688 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/34 file after the snapshot 2024-04-20T21:20:39.691 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/35 file after the snapshot 2024-04-20T21:20:39.694 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/36 file after the snapshot 2024-04-20T21:20:39.696 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/37 file after the snapshot 2024-04-20T21:20:39.700 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/38 file after the snapshot 2024-04-20T21:20:39.702 INFO:tasks.workunit.client.0.smithi102.stdout: create 200/39 file after the snapshot 2024-04-20T21:20:39.705 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/.snap/snap-subdir-test 2024-04-20T21:20:39.848 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/30 file after the snapshot 2024-04-20T21:20:40.101 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/31 file after the snapshot 2024-04-20T21:20:40.103 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/32 file after the snapshot 2024-04-20T21:20:40.105 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/33 file after the snapshot 2024-04-20T21:20:40.108 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/34 file after the snapshot 2024-04-20T21:20:40.111 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/35 file after the snapshot 2024-04-20T21:20:40.114 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/36 file after the snapshot 2024-04-20T21:20:40.117 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/37 file after the snapshot 2024-04-20T21:20:40.119 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/38 file after the snapshot 2024-04-20T21:20:40.122 INFO:tasks.workunit.client.0.smithi102.stdout: create 201/39 file after the snapshot 2024-04-20T21:20:40.125 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/.snap/snap-subdir-test 2024-04-20T21:20:40.889 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/30 file after the snapshot 2024-04-20T21:20:40.890 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/31 file after the snapshot 2024-04-20T21:20:40.892 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/32 file after the snapshot 2024-04-20T21:20:40.894 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/33 file after the snapshot 2024-04-20T21:20:40.896 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/34 file after the snapshot 2024-04-20T21:20:40.898 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/35 file after the snapshot 2024-04-20T21:20:40.900 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/36 file after the snapshot 2024-04-20T21:20:40.902 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/37 file after the snapshot 2024-04-20T21:20:40.906 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/38 file after the snapshot 2024-04-20T21:20:40.907 INFO:tasks.workunit.client.0.smithi102.stdout: create 202/39 file after the snapshot 2024-04-20T21:20:40.910 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/.snap/snap-subdir-test 2024-04-20T21:20:40.968 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/30 file after the snapshot 2024-04-20T21:20:40.971 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/31 file after the snapshot 2024-04-20T21:20:40.974 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/32 file after the snapshot 2024-04-20T21:20:40.976 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/33 file after the snapshot 2024-04-20T21:20:40.979 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/34 file after the snapshot 2024-04-20T21:20:40.983 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/35 file after the snapshot 2024-04-20T21:20:40.985 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/36 file after the snapshot 2024-04-20T21:20:40.988 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/37 file after the snapshot 2024-04-20T21:20:40.991 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/38 file after the snapshot 2024-04-20T21:20:40.994 INFO:tasks.workunit.client.0.smithi102.stdout: create 203/39 file after the snapshot 2024-04-20T21:20:40.997 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/.snap/snap-subdir-test 2024-04-20T21:20:41.054 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/30 file after the snapshot 2024-04-20T21:20:41.057 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/31 file after the snapshot 2024-04-20T21:20:41.060 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/32 file after the snapshot 2024-04-20T21:20:41.063 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/33 file after the snapshot 2024-04-20T21:20:41.066 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/34 file after the snapshot 2024-04-20T21:20:41.069 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/35 file after the snapshot 2024-04-20T21:20:41.072 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/36 file after the snapshot 2024-04-20T21:20:41.075 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/37 file after the snapshot 2024-04-20T21:20:41.077 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/38 file after the snapshot 2024-04-20T21:20:41.080 INFO:tasks.workunit.client.0.smithi102.stdout: create 204/39 file after the snapshot 2024-04-20T21:20:41.083 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/.snap/snap-subdir-test 2024-04-20T21:20:41.141 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/30 file after the snapshot 2024-04-20T21:20:41.144 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/31 file after the snapshot 2024-04-20T21:20:41.147 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/32 file after the snapshot 2024-04-20T21:20:41.150 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/33 file after the snapshot 2024-04-20T21:20:41.153 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/34 file after the snapshot 2024-04-20T21:20:41.158 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/35 file after the snapshot 2024-04-20T21:20:41.158 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/36 file after the snapshot 2024-04-20T21:20:41.161 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/37 file after the snapshot 2024-04-20T21:20:41.164 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/38 file after the snapshot 2024-04-20T21:20:41.166 INFO:tasks.workunit.client.0.smithi102.stdout: create 205/39 file after the snapshot 2024-04-20T21:20:41.169 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/.snap/snap-subdir-test 2024-04-20T21:20:41.223 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/30 file after the snapshot 2024-04-20T21:20:41.226 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/31 file after the snapshot 2024-04-20T21:20:41.228 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/32 file after the snapshot 2024-04-20T21:20:41.231 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/33 file after the snapshot 2024-04-20T21:20:41.234 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/34 file after the snapshot 2024-04-20T21:20:41.237 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/35 file after the snapshot 2024-04-20T21:20:41.239 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/36 file after the snapshot 2024-04-20T21:20:41.242 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/37 file after the snapshot 2024-04-20T21:20:41.245 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/38 file after the snapshot 2024-04-20T21:20:41.247 INFO:tasks.workunit.client.0.smithi102.stdout: create 206/39 file after the snapshot 2024-04-20T21:20:41.250 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/.snap/snap-subdir-test 2024-04-20T21:20:41.306 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/30 file after the snapshot 2024-04-20T21:20:41.309 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/31 file after the snapshot 2024-04-20T21:20:41.314 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/32 file after the snapshot 2024-04-20T21:20:41.314 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/33 file after the snapshot 2024-04-20T21:20:41.317 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/34 file after the snapshot 2024-04-20T21:20:41.319 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/35 file after the snapshot 2024-04-20T21:20:41.322 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/36 file after the snapshot 2024-04-20T21:20:41.325 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/37 file after the snapshot 2024-04-20T21:20:41.328 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/38 file after the snapshot 2024-04-20T21:20:41.330 INFO:tasks.workunit.client.0.smithi102.stdout: create 207/39 file after the snapshot 2024-04-20T21:20:41.333 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/.snap/snap-subdir-test 2024-04-20T21:20:41.389 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/30 file after the snapshot 2024-04-20T21:20:41.391 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/31 file after the snapshot 2024-04-20T21:20:41.394 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/32 file after the snapshot 2024-04-20T21:20:41.608 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/33 file after the snapshot 2024-04-20T21:20:41.612 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/34 file after the snapshot 2024-04-20T21:20:41.616 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/35 file after the snapshot 2024-04-20T21:20:41.620 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/36 file after the snapshot 2024-04-20T21:20:41.625 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/37 file after the snapshot 2024-04-20T21:20:41.629 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/38 file after the snapshot 2024-04-20T21:20:41.633 INFO:tasks.workunit.client.0.smithi102.stdout: create 208/39 file after the snapshot 2024-04-20T21:20:41.637 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/.snap/snap-subdir-test 2024-04-20T21:20:41.685 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/30 file after the snapshot 2024-04-20T21:20:41.688 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/31 file after the snapshot 2024-04-20T21:20:41.690 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/32 file after the snapshot 2024-04-20T21:20:41.692 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/33 file after the snapshot 2024-04-20T21:20:41.694 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/34 file after the snapshot 2024-04-20T21:20:41.697 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/35 file after the snapshot 2024-04-20T21:20:41.700 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/36 file after the snapshot 2024-04-20T21:20:41.703 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/37 file after the snapshot 2024-04-20T21:20:41.706 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/38 file after the snapshot 2024-04-20T21:20:41.709 INFO:tasks.workunit.client.0.smithi102.stdout: create 209/39 file after the snapshot 2024-04-20T21:20:41.712 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/.snap/snap-subdir-test 2024-04-20T21:20:41.755 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/30 file after the snapshot 2024-04-20T21:20:41.757 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/31 file after the snapshot 2024-04-20T21:20:41.759 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/32 file after the snapshot 2024-04-20T21:20:41.761 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/33 file after the snapshot 2024-04-20T21:20:41.763 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/34 file after the snapshot 2024-04-20T21:20:41.766 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/35 file after the snapshot 2024-04-20T21:20:41.768 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/36 file after the snapshot 2024-04-20T21:20:41.771 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/37 file after the snapshot 2024-04-20T21:20:41.774 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/38 file after the snapshot 2024-04-20T21:20:41.777 INFO:tasks.workunit.client.0.smithi102.stdout: create 210/39 file after the snapshot 2024-04-20T21:20:41.780 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/.snap/snap-subdir-test 2024-04-20T21:20:41.826 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/30 file after the snapshot 2024-04-20T21:20:41.827 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/31 file after the snapshot 2024-04-20T21:20:41.829 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/32 file after the snapshot 2024-04-20T21:20:41.831 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/33 file after the snapshot 2024-04-20T21:20:41.833 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/34 file after the snapshot 2024-04-20T21:20:41.834 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/35 file after the snapshot 2024-04-20T21:20:41.836 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/36 file after the snapshot 2024-04-20T21:20:41.838 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/37 file after the snapshot 2024-04-20T21:20:41.840 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/38 file after the snapshot 2024-04-20T21:20:41.841 INFO:tasks.workunit.client.0.smithi102.stdout: create 211/39 file after the snapshot 2024-04-20T21:20:41.843 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/.snap/snap-subdir-test 2024-04-20T21:20:41.890 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/30 file after the snapshot 2024-04-20T21:20:41.893 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/31 file after the snapshot 2024-04-20T21:20:41.897 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/32 file after the snapshot 2024-04-20T21:20:41.900 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/33 file after the snapshot 2024-04-20T21:20:41.902 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/34 file after the snapshot 2024-04-20T21:20:41.904 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/35 file after the snapshot 2024-04-20T21:20:41.907 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/36 file after the snapshot 2024-04-20T21:20:41.910 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/37 file after the snapshot 2024-04-20T21:20:41.912 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/38 file after the snapshot 2024-04-20T21:20:41.915 INFO:tasks.workunit.client.0.smithi102.stdout: create 212/39 file after the snapshot 2024-04-20T21:20:41.918 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/.snap/snap-subdir-test 2024-04-20T21:20:42.048 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/30 file after the snapshot 2024-04-20T21:20:42.051 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/31 file after the snapshot 2024-04-20T21:20:42.054 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/32 file after the snapshot 2024-04-20T21:20:42.057 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/33 file after the snapshot 2024-04-20T21:20:42.060 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/34 file after the snapshot 2024-04-20T21:20:42.063 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/35 file after the snapshot 2024-04-20T21:20:42.066 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/36 file after the snapshot 2024-04-20T21:20:42.068 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/37 file after the snapshot 2024-04-20T21:20:42.071 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/38 file after the snapshot 2024-04-20T21:20:42.074 INFO:tasks.workunit.client.0.smithi102.stdout: create 213/39 file after the snapshot 2024-04-20T21:20:42.077 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/.snap/snap-subdir-test 2024-04-20T21:20:42.124 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/30 file after the snapshot 2024-04-20T21:20:42.130 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/31 file after the snapshot 2024-04-20T21:20:42.389 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/32 file after the snapshot 2024-04-20T21:20:42.391 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/33 file after the snapshot 2024-04-20T21:20:42.394 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/34 file after the snapshot 2024-04-20T21:20:42.397 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/35 file after the snapshot 2024-04-20T21:20:42.400 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/36 file after the snapshot 2024-04-20T21:20:42.403 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/37 file after the snapshot 2024-04-20T21:20:42.405 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/38 file after the snapshot 2024-04-20T21:20:42.408 INFO:tasks.workunit.client.0.smithi102.stdout: create 214/39 file after the snapshot 2024-04-20T21:20:42.411 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/.snap/snap-subdir-test 2024-04-20T21:20:42.466 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/30 file after the snapshot 2024-04-20T21:20:42.467 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/31 file after the snapshot 2024-04-20T21:20:42.469 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/32 file after the snapshot 2024-04-20T21:20:42.471 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/33 file after the snapshot 2024-04-20T21:20:42.472 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/34 file after the snapshot 2024-04-20T21:20:42.474 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/35 file after the snapshot 2024-04-20T21:20:42.476 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/36 file after the snapshot 2024-04-20T21:20:42.477 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/37 file after the snapshot 2024-04-20T21:20:42.479 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/38 file after the snapshot 2024-04-20T21:20:42.481 INFO:tasks.workunit.client.0.smithi102.stdout: create 215/39 file after the snapshot 2024-04-20T21:20:42.483 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/.snap/snap-subdir-test 2024-04-20T21:20:42.531 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/30 file after the snapshot 2024-04-20T21:20:42.533 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/31 file after the snapshot 2024-04-20T21:20:42.535 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/32 file after the snapshot 2024-04-20T21:20:42.538 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/33 file after the snapshot 2024-04-20T21:20:42.538 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/34 file after the snapshot 2024-04-20T21:20:42.539 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/35 file after the snapshot 2024-04-20T21:20:42.540 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/36 file after the snapshot 2024-04-20T21:20:42.541 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/37 file after the snapshot 2024-04-20T21:20:42.542 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/38 file after the snapshot 2024-04-20T21:20:42.846 INFO:tasks.workunit.client.0.smithi102.stdout: create 216/39 file after the snapshot 2024-04-20T21:20:43.001 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/.snap/snap-subdir-test 2024-04-20T21:20:43.049 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/30 file after the snapshot 2024-04-20T21:20:43.053 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/31 file after the snapshot 2024-04-20T21:20:43.057 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/32 file after the snapshot 2024-04-20T21:20:43.061 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/33 file after the snapshot 2024-04-20T21:20:43.065 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/34 file after the snapshot 2024-04-20T21:20:43.069 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/35 file after the snapshot 2024-04-20T21:20:43.073 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/36 file after the snapshot 2024-04-20T21:20:43.077 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/37 file after the snapshot 2024-04-20T21:20:43.081 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/38 file after the snapshot 2024-04-20T21:20:43.086 INFO:tasks.workunit.client.0.smithi102.stdout: create 217/39 file after the snapshot 2024-04-20T21:20:43.091 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/.snap/snap-subdir-test 2024-04-20T21:20:43.131 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/30 file after the snapshot 2024-04-20T21:20:43.134 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/31 file after the snapshot 2024-04-20T21:20:43.135 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/32 file after the snapshot 2024-04-20T21:20:43.137 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/33 file after the snapshot 2024-04-20T21:20:43.139 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/34 file after the snapshot 2024-04-20T21:20:43.140 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/35 file after the snapshot 2024-04-20T21:20:43.142 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/36 file after the snapshot 2024-04-20T21:20:43.144 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/37 file after the snapshot 2024-04-20T21:20:43.146 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/38 file after the snapshot 2024-04-20T21:20:43.150 INFO:tasks.workunit.client.0.smithi102.stdout: create 218/39 file after the snapshot 2024-04-20T21:20:43.151 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/.snap/snap-subdir-test 2024-04-20T21:20:43.189 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/30 file after the snapshot 2024-04-20T21:20:43.192 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/31 file after the snapshot 2024-04-20T21:20:43.195 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/32 file after the snapshot 2024-04-20T21:20:43.197 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/33 file after the snapshot 2024-04-20T21:20:43.200 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/34 file after the snapshot 2024-04-20T21:20:43.203 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/35 file after the snapshot 2024-04-20T21:20:43.205 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/36 file after the snapshot 2024-04-20T21:20:43.208 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/37 file after the snapshot 2024-04-20T21:20:43.211 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/38 file after the snapshot 2024-04-20T21:20:43.214 INFO:tasks.workunit.client.0.smithi102.stdout: create 219/39 file after the snapshot 2024-04-20T21:20:43.216 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/.snap/snap-subdir-test 2024-04-20T21:20:43.256 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/30 file after the snapshot 2024-04-20T21:20:43.259 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/31 file after the snapshot 2024-04-20T21:20:43.261 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/32 file after the snapshot 2024-04-20T21:20:43.264 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/33 file after the snapshot 2024-04-20T21:20:43.267 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/34 file after the snapshot 2024-04-20T21:20:43.269 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/35 file after the snapshot 2024-04-20T21:20:43.272 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/36 file after the snapshot 2024-04-20T21:20:43.275 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/37 file after the snapshot 2024-04-20T21:20:43.278 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/38 file after the snapshot 2024-04-20T21:20:43.280 INFO:tasks.workunit.client.0.smithi102.stdout: create 220/39 file after the snapshot 2024-04-20T21:20:43.283 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/.snap/snap-subdir-test 2024-04-20T21:20:43.323 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/30 file after the snapshot 2024-04-20T21:20:43.325 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/31 file after the snapshot 2024-04-20T21:20:43.328 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/32 file after the snapshot 2024-04-20T21:20:43.331 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/33 file after the snapshot 2024-04-20T21:20:43.334 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/34 file after the snapshot 2024-04-20T21:20:43.336 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/35 file after the snapshot 2024-04-20T21:20:43.338 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/36 file after the snapshot 2024-04-20T21:20:43.341 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/37 file after the snapshot 2024-04-20T21:20:43.343 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/38 file after the snapshot 2024-04-20T21:20:43.346 INFO:tasks.workunit.client.0.smithi102.stdout: create 221/39 file after the snapshot 2024-04-20T21:20:43.348 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/.snap/snap-subdir-test 2024-04-20T21:20:43.388 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/30 file after the snapshot 2024-04-20T21:20:43.390 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/31 file after the snapshot 2024-04-20T21:20:43.393 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/32 file after the snapshot 2024-04-20T21:20:43.396 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/33 file after the snapshot 2024-04-20T21:20:43.399 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/34 file after the snapshot 2024-04-20T21:20:43.402 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/35 file after the snapshot 2024-04-20T21:20:43.404 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/36 file after the snapshot 2024-04-20T21:20:43.407 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/37 file after the snapshot 2024-04-20T21:20:43.410 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/38 file after the snapshot 2024-04-20T21:20:43.413 INFO:tasks.workunit.client.0.smithi102.stdout: create 222/39 file after the snapshot 2024-04-20T21:20:43.415 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/.snap/snap-subdir-test 2024-04-20T21:20:43.454 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/30 file after the snapshot 2024-04-20T21:20:43.457 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/31 file after the snapshot 2024-04-20T21:20:43.459 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/32 file after the snapshot 2024-04-20T21:20:43.462 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/33 file after the snapshot 2024-04-20T21:20:43.465 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/34 file after the snapshot 2024-04-20T21:20:43.467 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/35 file after the snapshot 2024-04-20T21:20:43.470 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/36 file after the snapshot 2024-04-20T21:20:43.473 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/37 file after the snapshot 2024-04-20T21:20:43.475 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/38 file after the snapshot 2024-04-20T21:20:43.478 INFO:tasks.workunit.client.0.smithi102.stdout: create 223/39 file after the snapshot 2024-04-20T21:20:43.481 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/.snap/snap-subdir-test 2024-04-20T21:20:43.519 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/30 file after the snapshot 2024-04-20T21:20:43.522 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/31 file after the snapshot 2024-04-20T21:20:43.525 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/32 file after the snapshot 2024-04-20T21:20:43.527 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/33 file after the snapshot 2024-04-20T21:20:43.530 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/34 file after the snapshot 2024-04-20T21:20:43.533 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/35 file after the snapshot 2024-04-20T21:20:43.536 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/36 file after the snapshot 2024-04-20T21:20:43.539 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/37 file after the snapshot 2024-04-20T21:20:43.542 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/38 file after the snapshot 2024-04-20T21:20:43.544 INFO:tasks.workunit.client.0.smithi102.stdout: create 224/39 file after the snapshot 2024-04-20T21:20:43.547 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/.snap/snap-subdir-test 2024-04-20T21:20:43.584 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/30 file after the snapshot 2024-04-20T21:20:43.586 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/31 file after the snapshot 2024-04-20T21:20:43.588 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/32 file after the snapshot 2024-04-20T21:20:43.590 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/33 file after the snapshot 2024-04-20T21:20:43.592 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/34 file after the snapshot 2024-04-20T21:20:43.594 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/35 file after the snapshot 2024-04-20T21:20:43.596 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/36 file after the snapshot 2024-04-20T21:20:43.597 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/37 file after the snapshot 2024-04-20T21:20:43.599 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/38 file after the snapshot 2024-04-20T21:20:43.601 INFO:tasks.workunit.client.0.smithi102.stdout: create 225/39 file after the snapshot 2024-04-20T21:20:43.603 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/.snap/snap-subdir-test 2024-04-20T21:20:43.642 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/30 file after the snapshot 2024-04-20T21:20:43.644 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/31 file after the snapshot 2024-04-20T21:20:43.647 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/32 file after the snapshot 2024-04-20T21:20:43.650 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/33 file after the snapshot 2024-04-20T21:20:43.652 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/34 file after the snapshot 2024-04-20T21:20:43.655 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/35 file after the snapshot 2024-04-20T21:20:43.658 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/36 file after the snapshot 2024-04-20T21:20:43.661 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/37 file after the snapshot 2024-04-20T21:20:43.664 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/38 file after the snapshot 2024-04-20T21:20:43.667 INFO:tasks.workunit.client.0.smithi102.stdout: create 226/39 file after the snapshot 2024-04-20T21:20:43.670 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/.snap/snap-subdir-test 2024-04-20T21:20:43.719 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/30 file after the snapshot 2024-04-20T21:20:43.721 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/31 file after the snapshot 2024-04-20T21:20:43.724 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/32 file after the snapshot 2024-04-20T21:20:43.726 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/33 file after the snapshot 2024-04-20T21:20:43.729 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/34 file after the snapshot 2024-04-20T21:20:43.730 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/35 file after the snapshot 2024-04-20T21:20:43.733 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/36 file after the snapshot 2024-04-20T21:20:43.736 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/37 file after the snapshot 2024-04-20T21:20:43.738 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/38 file after the snapshot 2024-04-20T21:20:43.741 INFO:tasks.workunit.client.0.smithi102.stdout: create 227/39 file after the snapshot 2024-04-20T21:20:43.744 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/.snap/snap-subdir-test 2024-04-20T21:20:43.793 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/30 file after the snapshot 2024-04-20T21:20:43.794 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/31 file after the snapshot 2024-04-20T21:20:43.796 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/32 file after the snapshot 2024-04-20T21:20:43.797 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/33 file after the snapshot 2024-04-20T21:20:43.799 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/34 file after the snapshot 2024-04-20T21:20:43.801 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/35 file after the snapshot 2024-04-20T21:20:43.803 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/36 file after the snapshot 2024-04-20T21:20:43.805 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/37 file after the snapshot 2024-04-20T21:20:43.807 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/38 file after the snapshot 2024-04-20T21:20:43.810 INFO:tasks.workunit.client.0.smithi102.stdout: create 228/39 file after the snapshot 2024-04-20T21:20:43.813 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/.snap/snap-subdir-test 2024-04-20T21:20:43.846 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/30 file after the snapshot 2024-04-20T21:20:43.849 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/31 file after the snapshot 2024-04-20T21:20:43.852 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/32 file after the snapshot 2024-04-20T21:20:43.854 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/33 file after the snapshot 2024-04-20T21:20:43.857 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/34 file after the snapshot 2024-04-20T21:20:43.860 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/35 file after the snapshot 2024-04-20T21:20:43.863 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/36 file after the snapshot 2024-04-20T21:20:43.865 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/37 file after the snapshot 2024-04-20T21:20:43.869 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/38 file after the snapshot 2024-04-20T21:20:43.871 INFO:tasks.workunit.client.0.smithi102.stdout: create 229/39 file after the snapshot 2024-04-20T21:20:43.874 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/.snap/snap-subdir-test 2024-04-20T21:20:43.909 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/30 file after the snapshot 2024-04-20T21:20:43.911 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/31 file after the snapshot 2024-04-20T21:20:44.314 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/32 file after the snapshot 2024-04-20T21:20:44.316 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/33 file after the snapshot 2024-04-20T21:20:44.319 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/34 file after the snapshot 2024-04-20T21:20:44.322 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/35 file after the snapshot 2024-04-20T21:20:44.325 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/36 file after the snapshot 2024-04-20T21:20:44.328 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/37 file after the snapshot 2024-04-20T21:20:44.330 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/38 file after the snapshot 2024-04-20T21:20:44.333 INFO:tasks.workunit.client.0.smithi102.stdout: create 230/39 file after the snapshot 2024-04-20T21:20:44.336 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/.snap/snap-subdir-test 2024-04-20T21:20:44.376 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/30 file after the snapshot 2024-04-20T21:20:44.382 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/31 file after the snapshot 2024-04-20T21:20:44.384 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/32 file after the snapshot 2024-04-20T21:20:44.388 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/33 file after the snapshot 2024-04-20T21:20:44.440 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/34 file after the snapshot 2024-04-20T21:20:44.444 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/35 file after the snapshot 2024-04-20T21:20:44.448 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/36 file after the snapshot 2024-04-20T21:20:44.452 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/37 file after the snapshot 2024-04-20T21:20:44.456 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/38 file after the snapshot 2024-04-20T21:20:44.460 INFO:tasks.workunit.client.0.smithi102.stdout: create 231/39 file after the snapshot 2024-04-20T21:20:44.464 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/.snap/snap-subdir-test 2024-04-20T21:20:44.860 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/30 file after the snapshot 2024-04-20T21:20:44.863 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/31 file after the snapshot 2024-04-20T21:20:44.867 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/32 file after the snapshot 2024-04-20T21:20:44.871 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/33 file after the snapshot 2024-04-20T21:20:44.875 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/34 file after the snapshot 2024-04-20T21:20:44.879 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/35 file after the snapshot 2024-04-20T21:20:44.884 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/36 file after the snapshot 2024-04-20T21:20:44.888 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/37 file after the snapshot 2024-04-20T21:20:44.892 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/38 file after the snapshot 2024-04-20T21:20:44.896 INFO:tasks.workunit.client.0.smithi102.stdout: create 232/39 file after the snapshot 2024-04-20T21:20:44.900 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/.snap/snap-subdir-test 2024-04-20T21:20:44.950 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/30 file after the snapshot 2024-04-20T21:20:44.951 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/31 file after the snapshot 2024-04-20T21:20:44.953 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/32 file after the snapshot 2024-04-20T21:20:44.955 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/33 file after the snapshot 2024-04-20T21:20:44.957 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/34 file after the snapshot 2024-04-20T21:20:44.958 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/35 file after the snapshot 2024-04-20T21:20:44.960 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/36 file after the snapshot 2024-04-20T21:20:44.962 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/37 file after the snapshot 2024-04-20T21:20:44.963 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/38 file after the snapshot 2024-04-20T21:20:44.965 INFO:tasks.workunit.client.0.smithi102.stdout: create 233/39 file after the snapshot 2024-04-20T21:20:44.968 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/.snap/snap-subdir-test 2024-04-20T21:20:45.017 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/30 file after the snapshot 2024-04-20T21:20:45.020 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/31 file after the snapshot 2024-04-20T21:20:45.023 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/32 file after the snapshot 2024-04-20T21:20:45.025 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/33 file after the snapshot 2024-04-20T21:20:45.028 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/34 file after the snapshot 2024-04-20T21:20:45.031 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/35 file after the snapshot 2024-04-20T21:20:45.034 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/36 file after the snapshot 2024-04-20T21:20:45.036 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/37 file after the snapshot 2024-04-20T21:20:45.039 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/38 file after the snapshot 2024-04-20T21:20:45.042 INFO:tasks.workunit.client.0.smithi102.stdout: create 234/39 file after the snapshot 2024-04-20T21:20:45.045 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/.snap/snap-subdir-test 2024-04-20T21:20:45.096 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/30 file after the snapshot 2024-04-20T21:20:45.099 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/31 file after the snapshot 2024-04-20T21:20:45.342 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/32 file after the snapshot 2024-04-20T21:20:45.344 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/33 file after the snapshot 2024-04-20T21:20:45.347 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/34 file after the snapshot 2024-04-20T21:20:45.350 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/35 file after the snapshot 2024-04-20T21:20:45.353 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/36 file after the snapshot 2024-04-20T21:20:45.355 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/37 file after the snapshot 2024-04-20T21:20:45.358 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/38 file after the snapshot 2024-04-20T21:20:45.361 INFO:tasks.workunit.client.0.smithi102.stdout: create 235/39 file after the snapshot 2024-04-20T21:20:45.364 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/.snap/snap-subdir-test 2024-04-20T21:20:45.418 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/30 file after the snapshot 2024-04-20T21:20:45.420 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/31 file after the snapshot 2024-04-20T21:20:45.422 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/32 file after the snapshot 2024-04-20T21:20:45.425 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/33 file after the snapshot 2024-04-20T21:20:45.427 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/34 file after the snapshot 2024-04-20T21:20:45.430 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/35 file after the snapshot 2024-04-20T21:20:45.432 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/36 file after the snapshot 2024-04-20T21:20:45.435 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/37 file after the snapshot 2024-04-20T21:20:45.437 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/38 file after the snapshot 2024-04-20T21:20:45.440 INFO:tasks.workunit.client.0.smithi102.stdout: create 236/39 file after the snapshot 2024-04-20T21:20:45.443 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/.snap/snap-subdir-test 2024-04-20T21:20:45.493 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/30 file after the snapshot 2024-04-20T21:20:45.495 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/31 file after the snapshot 2024-04-20T21:20:45.496 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/32 file after the snapshot 2024-04-20T21:20:45.498 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/33 file after the snapshot 2024-04-20T21:20:45.500 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/34 file after the snapshot 2024-04-20T21:20:45.502 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/35 file after the snapshot 2024-04-20T21:20:45.503 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/36 file after the snapshot 2024-04-20T21:20:45.507 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/37 file after the snapshot 2024-04-20T21:20:45.511 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/38 file after the snapshot 2024-04-20T21:20:45.516 INFO:tasks.workunit.client.0.smithi102.stdout: create 237/39 file after the snapshot 2024-04-20T21:20:45.520 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/.snap/snap-subdir-test 2024-04-20T21:20:45.567 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/30 file after the snapshot 2024-04-20T21:20:45.570 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/31 file after the snapshot 2024-04-20T21:20:45.572 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/32 file after the snapshot 2024-04-20T21:20:45.574 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/33 file after the snapshot 2024-04-20T21:20:45.576 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/34 file after the snapshot 2024-04-20T21:20:45.579 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/35 file after the snapshot 2024-04-20T21:20:45.582 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/36 file after the snapshot 2024-04-20T21:20:45.586 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/37 file after the snapshot 2024-04-20T21:20:45.590 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/38 file after the snapshot 2024-04-20T21:20:45.594 INFO:tasks.workunit.client.0.smithi102.stdout: create 238/39 file after the snapshot 2024-04-20T21:20:45.599 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/.snap/snap-subdir-test 2024-04-20T21:20:45.650 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/30 file after the snapshot 2024-04-20T21:20:45.653 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/31 file after the snapshot 2024-04-20T21:20:45.656 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/32 file after the snapshot 2024-04-20T21:20:45.660 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/33 file after the snapshot 2024-04-20T21:20:45.663 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/34 file after the snapshot 2024-04-20T21:20:45.665 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/35 file after the snapshot 2024-04-20T21:20:45.668 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/36 file after the snapshot 2024-04-20T21:20:45.671 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/37 file after the snapshot 2024-04-20T21:20:45.674 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/38 file after the snapshot 2024-04-20T21:20:45.676 INFO:tasks.workunit.client.0.smithi102.stdout: create 239/39 file after the snapshot 2024-04-20T21:20:45.679 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/.snap/snap-subdir-test 2024-04-20T21:20:45.725 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/30 file after the snapshot 2024-04-20T21:20:45.726 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/31 file after the snapshot 2024-04-20T21:20:45.728 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/32 file after the snapshot 2024-04-20T21:20:45.730 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/33 file after the snapshot 2024-04-20T21:20:45.731 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/34 file after the snapshot 2024-04-20T21:20:45.733 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/35 file after the snapshot 2024-04-20T21:20:45.735 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/36 file after the snapshot 2024-04-20T21:20:45.736 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/37 file after the snapshot 2024-04-20T21:20:45.738 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/38 file after the snapshot 2024-04-20T21:20:45.740 INFO:tasks.workunit.client.0.smithi102.stdout: create 240/39 file after the snapshot 2024-04-20T21:20:45.741 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/.snap/snap-subdir-test 2024-04-20T21:20:45.787 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/30 file after the snapshot 2024-04-20T21:20:45.790 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/31 file after the snapshot 2024-04-20T21:20:45.793 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/32 file after the snapshot 2024-04-20T21:20:45.795 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/33 file after the snapshot 2024-04-20T21:20:45.798 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/34 file after the snapshot 2024-04-20T21:20:45.801 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/35 file after the snapshot 2024-04-20T21:20:45.803 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/36 file after the snapshot 2024-04-20T21:20:45.806 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/37 file after the snapshot 2024-04-20T21:20:45.809 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/38 file after the snapshot 2024-04-20T21:20:45.812 INFO:tasks.workunit.client.0.smithi102.stdout: create 241/39 file after the snapshot 2024-04-20T21:20:45.814 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/.snap/snap-subdir-test 2024-04-20T21:20:45.854 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/30 file after the snapshot 2024-04-20T21:20:45.856 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/31 file after the snapshot 2024-04-20T21:20:45.858 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/32 file after the snapshot 2024-04-20T21:20:45.860 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/33 file after the snapshot 2024-04-20T21:20:45.861 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/34 file after the snapshot 2024-04-20T21:20:45.863 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/35 file after the snapshot 2024-04-20T21:20:45.865 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/36 file after the snapshot 2024-04-20T21:20:45.868 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/37 file after the snapshot 2024-04-20T21:20:45.871 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/38 file after the snapshot 2024-04-20T21:20:45.873 INFO:tasks.workunit.client.0.smithi102.stdout: create 242/39 file after the snapshot 2024-04-20T21:20:45.876 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/.snap/snap-subdir-test 2024-04-20T21:20:45.918 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/30 file after the snapshot 2024-04-20T21:20:45.920 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/31 file after the snapshot 2024-04-20T21:20:45.922 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/32 file after the snapshot 2024-04-20T21:20:45.925 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/33 file after the snapshot 2024-04-20T21:20:45.928 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/34 file after the snapshot 2024-04-20T21:20:45.930 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/35 file after the snapshot 2024-04-20T21:20:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/36 file after the snapshot 2024-04-20T21:20:45.936 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/37 file after the snapshot 2024-04-20T21:20:45.938 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/38 file after the snapshot 2024-04-20T21:20:45.941 INFO:tasks.workunit.client.0.smithi102.stdout: create 243/39 file after the snapshot 2024-04-20T21:20:45.944 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/.snap/snap-subdir-test 2024-04-20T21:20:45.985 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/30 file after the snapshot 2024-04-20T21:20:45.987 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/31 file after the snapshot 2024-04-20T21:20:45.990 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/32 file after the snapshot 2024-04-20T21:20:45.992 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/33 file after the snapshot 2024-04-20T21:20:45.995 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/34 file after the snapshot 2024-04-20T21:20:45.997 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/35 file after the snapshot 2024-04-20T21:20:46.000 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/36 file after the snapshot 2024-04-20T21:20:46.003 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/37 file after the snapshot 2024-04-20T21:20:46.006 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/38 file after the snapshot 2024-04-20T21:20:46.008 INFO:tasks.workunit.client.0.smithi102.stdout: create 244/39 file after the snapshot 2024-04-20T21:20:46.011 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/.snap/snap-subdir-test 2024-04-20T21:20:46.053 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/30 file after the snapshot 2024-04-20T21:20:46.055 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/31 file after the snapshot 2024-04-20T21:20:46.058 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/32 file after the snapshot 2024-04-20T21:20:46.061 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/33 file after the snapshot 2024-04-20T21:20:46.064 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/34 file after the snapshot 2024-04-20T21:20:46.067 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/35 file after the snapshot 2024-04-20T21:20:46.069 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/36 file after the snapshot 2024-04-20T21:20:46.072 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/37 file after the snapshot 2024-04-20T21:20:46.074 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/38 file after the snapshot 2024-04-20T21:20:46.077 INFO:tasks.workunit.client.0.smithi102.stdout: create 245/39 file after the snapshot 2024-04-20T21:20:46.080 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/.snap/snap-subdir-test 2024-04-20T21:20:46.796 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/30 file after the snapshot 2024-04-20T21:20:46.798 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/31 file after the snapshot 2024-04-20T21:20:46.799 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/32 file after the snapshot 2024-04-20T21:20:46.801 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/33 file after the snapshot 2024-04-20T21:20:46.803 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/34 file after the snapshot 2024-04-20T21:20:46.805 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/35 file after the snapshot 2024-04-20T21:20:46.806 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/36 file after the snapshot 2024-04-20T21:20:46.808 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/37 file after the snapshot 2024-04-20T21:20:46.810 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/38 file after the snapshot 2024-04-20T21:20:46.811 INFO:tasks.workunit.client.0.smithi102.stdout: create 246/39 file after the snapshot 2024-04-20T21:20:46.813 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/.snap/snap-subdir-test 2024-04-20T21:20:46.866 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/30 file after the snapshot 2024-04-20T21:20:46.869 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/31 file after the snapshot 2024-04-20T21:20:46.872 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/32 file after the snapshot 2024-04-20T21:20:46.875 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/33 file after the snapshot 2024-04-20T21:20:46.877 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/34 file after the snapshot 2024-04-20T21:20:46.880 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/35 file after the snapshot 2024-04-20T21:20:46.883 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/36 file after the snapshot 2024-04-20T21:20:46.886 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/37 file after the snapshot 2024-04-20T21:20:46.888 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/38 file after the snapshot 2024-04-20T21:20:46.891 INFO:tasks.workunit.client.0.smithi102.stdout: create 247/39 file after the snapshot 2024-04-20T21:20:46.894 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/.snap/snap-subdir-test 2024-04-20T21:20:46.944 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/30 file after the snapshot 2024-04-20T21:20:46.947 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/31 file after the snapshot 2024-04-20T21:20:46.949 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/32 file after the snapshot 2024-04-20T21:20:46.952 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/33 file after the snapshot 2024-04-20T21:20:46.955 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/34 file after the snapshot 2024-04-20T21:20:46.958 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/35 file after the snapshot 2024-04-20T21:20:46.961 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/36 file after the snapshot 2024-04-20T21:20:46.963 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/37 file after the snapshot 2024-04-20T21:20:46.966 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/38 file after the snapshot 2024-04-20T21:20:46.969 INFO:tasks.workunit.client.0.smithi102.stdout: create 248/39 file after the snapshot 2024-04-20T21:20:46.972 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/.snap/snap-subdir-test 2024-04-20T21:20:47.016 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/30 file after the snapshot 2024-04-20T21:20:47.018 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/31 file after the snapshot 2024-04-20T21:20:47.019 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/32 file after the snapshot 2024-04-20T21:20:47.021 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/33 file after the snapshot 2024-04-20T21:20:47.023 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/34 file after the snapshot 2024-04-20T21:20:47.025 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/35 file after the snapshot 2024-04-20T21:20:47.027 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/36 file after the snapshot 2024-04-20T21:20:47.030 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/37 file after the snapshot 2024-04-20T21:20:47.032 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/38 file after the snapshot 2024-04-20T21:20:47.039 INFO:tasks.workunit.client.0.smithi102.stdout: create 249/39 file after the snapshot 2024-04-20T21:20:47.039 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/.snap/snap-subdir-test 2024-04-20T21:20:47.088 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/30 file after the snapshot 2024-04-20T21:20:47.091 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/31 file after the snapshot 2024-04-20T21:20:47.093 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/32 file after the snapshot 2024-04-20T21:20:47.096 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/33 file after the snapshot 2024-04-20T21:20:47.099 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/34 file after the snapshot 2024-04-20T21:20:47.101 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/35 file after the snapshot 2024-04-20T21:20:47.104 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/36 file after the snapshot 2024-04-20T21:20:47.107 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/37 file after the snapshot 2024-04-20T21:20:47.110 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/38 file after the snapshot 2024-04-20T21:20:47.112 INFO:tasks.workunit.client.0.smithi102.stdout: create 250/39 file after the snapshot 2024-04-20T21:20:47.115 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/.snap/snap-subdir-test 2024-04-20T21:20:47.166 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/30 file after the snapshot 2024-04-20T21:20:47.168 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/31 file after the snapshot 2024-04-20T21:20:47.171 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/32 file after the snapshot 2024-04-20T21:20:47.174 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/33 file after the snapshot 2024-04-20T21:20:47.176 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/34 file after the snapshot 2024-04-20T21:20:47.179 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/35 file after the snapshot 2024-04-20T21:20:47.182 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/36 file after the snapshot 2024-04-20T21:20:47.185 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/37 file after the snapshot 2024-04-20T21:20:47.188 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/38 file after the snapshot 2024-04-20T21:20:47.190 INFO:tasks.workunit.client.0.smithi102.stdout: create 251/39 file after the snapshot 2024-04-20T21:20:47.193 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/.snap/snap-subdir-test 2024-04-20T21:20:47.244 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/30 file after the snapshot 2024-04-20T21:20:47.246 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/31 file after the snapshot 2024-04-20T21:20:47.249 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/32 file after the snapshot 2024-04-20T21:20:47.251 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/33 file after the snapshot 2024-04-20T21:20:47.254 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/34 file after the snapshot 2024-04-20T21:20:47.257 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/35 file after the snapshot 2024-04-20T21:20:47.260 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/36 file after the snapshot 2024-04-20T21:20:47.262 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/37 file after the snapshot 2024-04-20T21:20:47.265 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/38 file after the snapshot 2024-04-20T21:20:47.268 INFO:tasks.workunit.client.0.smithi102.stdout: create 252/39 file after the snapshot 2024-04-20T21:20:47.271 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/.snap/snap-subdir-test 2024-04-20T21:20:47.320 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/30 file after the snapshot 2024-04-20T21:20:47.323 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/31 file after the snapshot 2024-04-20T21:20:47.325 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/32 file after the snapshot 2024-04-20T21:20:47.328 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/33 file after the snapshot 2024-04-20T21:20:47.330 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/34 file after the snapshot 2024-04-20T21:20:47.333 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/35 file after the snapshot 2024-04-20T21:20:47.336 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/36 file after the snapshot 2024-04-20T21:20:47.338 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/37 file after the snapshot 2024-04-20T21:20:47.341 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/38 file after the snapshot 2024-04-20T21:20:47.344 INFO:tasks.workunit.client.0.smithi102.stdout: create 253/39 file after the snapshot 2024-04-20T21:20:47.347 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/.snap/snap-subdir-test 2024-04-20T21:20:47.397 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/30 file after the snapshot 2024-04-20T21:20:47.400 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/31 file after the snapshot 2024-04-20T21:20:47.403 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/32 file after the snapshot 2024-04-20T21:20:47.405 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/33 file after the snapshot 2024-04-20T21:20:47.408 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/34 file after the snapshot 2024-04-20T21:20:47.411 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/35 file after the snapshot 2024-04-20T21:20:47.413 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/36 file after the snapshot 2024-04-20T21:20:47.416 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/37 file after the snapshot 2024-04-20T21:20:47.419 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/38 file after the snapshot 2024-04-20T21:20:47.421 INFO:tasks.workunit.client.0.smithi102.stdout: create 254/39 file after the snapshot 2024-04-20T21:20:47.424 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/.snap/snap-subdir-test 2024-04-20T21:20:47.473 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/30 file after the snapshot 2024-04-20T21:20:47.476 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/31 file after the snapshot 2024-04-20T21:20:47.478 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/32 file after the snapshot 2024-04-20T21:20:47.481 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/33 file after the snapshot 2024-04-20T21:20:47.484 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/34 file after the snapshot 2024-04-20T21:20:47.486 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/35 file after the snapshot 2024-04-20T21:20:47.490 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/36 file after the snapshot 2024-04-20T21:20:47.493 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/37 file after the snapshot 2024-04-20T21:20:47.495 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/38 file after the snapshot 2024-04-20T21:20:47.498 INFO:tasks.workunit.client.0.smithi102.stdout: create 255/39 file after the snapshot 2024-04-20T21:20:47.501 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/.snap/snap-subdir-test 2024-04-20T21:20:47.539 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/30 file after the snapshot 2024-04-20T21:20:47.542 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/31 file after the snapshot 2024-04-20T21:20:47.544 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/32 file after the snapshot 2024-04-20T21:20:47.547 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/33 file after the snapshot 2024-04-20T21:20:47.549 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/34 file after the snapshot 2024-04-20T21:20:47.552 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/35 file after the snapshot 2024-04-20T21:20:47.554 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/36 file after the snapshot 2024-04-20T21:20:47.557 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/37 file after the snapshot 2024-04-20T21:20:47.559 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/38 file after the snapshot 2024-04-20T21:20:47.562 INFO:tasks.workunit.client.0.smithi102.stdout: create 256/39 file after the snapshot 2024-04-20T21:20:47.565 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/.snap/snap-subdir-test 2024-04-20T21:20:47.608 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/30 file after the snapshot 2024-04-20T21:20:47.611 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/31 file after the snapshot 2024-04-20T21:20:47.612 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/32 file after the snapshot 2024-04-20T21:20:47.614 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/33 file after the snapshot 2024-04-20T21:20:47.616 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/34 file after the snapshot 2024-04-20T21:20:47.618 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/35 file after the snapshot 2024-04-20T21:20:47.620 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/36 file after the snapshot 2024-04-20T21:20:47.622 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/37 file after the snapshot 2024-04-20T21:20:47.624 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/38 file after the snapshot 2024-04-20T21:20:47.626 INFO:tasks.workunit.client.0.smithi102.stdout: create 257/39 file after the snapshot 2024-04-20T21:20:47.629 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/.snap/snap-subdir-test 2024-04-20T21:20:47.879 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/30 file after the snapshot 2024-04-20T21:20:48.488 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/31 file after the snapshot 2024-04-20T21:20:48.491 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/32 file after the snapshot 2024-04-20T21:20:48.494 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/33 file after the snapshot 2024-04-20T21:20:48.497 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/34 file after the snapshot 2024-04-20T21:20:48.499 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/35 file after the snapshot 2024-04-20T21:20:48.502 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/36 file after the snapshot 2024-04-20T21:20:48.505 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/37 file after the snapshot 2024-04-20T21:20:48.507 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/38 file after the snapshot 2024-04-20T21:20:48.510 INFO:tasks.workunit.client.0.smithi102.stdout: create 258/39 file after the snapshot 2024-04-20T21:20:48.513 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/.snap/snap-subdir-test 2024-04-20T21:20:48.554 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/30 file after the snapshot 2024-04-20T21:20:48.557 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/31 file after the snapshot 2024-04-20T21:20:48.560 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/32 file after the snapshot 2024-04-20T21:20:48.562 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/33 file after the snapshot 2024-04-20T21:20:48.565 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/34 file after the snapshot 2024-04-20T21:20:48.567 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/35 file after the snapshot 2024-04-20T21:20:48.570 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/36 file after the snapshot 2024-04-20T21:20:48.573 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/37 file after the snapshot 2024-04-20T21:20:48.576 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/38 file after the snapshot 2024-04-20T21:20:48.578 INFO:tasks.workunit.client.0.smithi102.stdout: create 259/39 file after the snapshot 2024-04-20T21:20:48.581 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/.snap/snap-subdir-test 2024-04-20T21:20:48.633 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/30 file after the snapshot 2024-04-20T21:20:48.634 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/31 file after the snapshot 2024-04-20T21:20:48.636 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/32 file after the snapshot 2024-04-20T21:20:48.638 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/33 file after the snapshot 2024-04-20T21:20:48.640 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/34 file after the snapshot 2024-04-20T21:20:48.641 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/35 file after the snapshot 2024-04-20T21:20:48.642 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/36 file after the snapshot 2024-04-20T21:20:48.644 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/37 file after the snapshot 2024-04-20T21:20:48.646 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/38 file after the snapshot 2024-04-20T21:20:48.647 INFO:tasks.workunit.client.0.smithi102.stdout: create 260/39 file after the snapshot 2024-04-20T21:20:48.649 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/.snap/snap-subdir-test 2024-04-20T21:20:48.698 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/30 file after the snapshot 2024-04-20T21:20:48.699 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/31 file after the snapshot 2024-04-20T21:20:48.701 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/32 file after the snapshot 2024-04-20T21:20:48.703 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/33 file after the snapshot 2024-04-20T21:20:48.704 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/34 file after the snapshot 2024-04-20T21:20:48.706 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/35 file after the snapshot 2024-04-20T21:20:48.708 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/36 file after the snapshot 2024-04-20T21:20:48.709 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/37 file after the snapshot 2024-04-20T21:20:48.712 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/38 file after the snapshot 2024-04-20T21:20:48.714 INFO:tasks.workunit.client.0.smithi102.stdout: create 261/39 file after the snapshot 2024-04-20T21:20:48.717 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/.snap/snap-subdir-test 2024-04-20T21:20:48.768 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/30 file after the snapshot 2024-04-20T21:20:48.770 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/31 file after the snapshot 2024-04-20T21:20:48.772 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/32 file after the snapshot 2024-04-20T21:20:48.774 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/33 file after the snapshot 2024-04-20T21:20:48.776 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/34 file after the snapshot 2024-04-20T21:20:48.778 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/35 file after the snapshot 2024-04-20T21:20:48.781 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/36 file after the snapshot 2024-04-20T21:20:48.783 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/37 file after the snapshot 2024-04-20T21:20:48.786 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/38 file after the snapshot 2024-04-20T21:20:48.788 INFO:tasks.workunit.client.0.smithi102.stdout: create 262/39 file after the snapshot 2024-04-20T21:20:48.791 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/.snap/snap-subdir-test 2024-04-20T21:20:48.842 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/30 file after the snapshot 2024-04-20T21:20:48.845 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/31 file after the snapshot 2024-04-20T21:20:48.848 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/32 file after the snapshot 2024-04-20T21:20:48.851 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/33 file after the snapshot 2024-04-20T21:20:48.853 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/34 file after the snapshot 2024-04-20T21:20:48.856 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/35 file after the snapshot 2024-04-20T21:20:48.859 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/36 file after the snapshot 2024-04-20T21:20:48.862 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/37 file after the snapshot 2024-04-20T21:20:48.865 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/38 file after the snapshot 2024-04-20T21:20:48.867 INFO:tasks.workunit.client.0.smithi102.stdout: create 263/39 file after the snapshot 2024-04-20T21:20:48.870 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/.snap/snap-subdir-test 2024-04-20T21:20:48.921 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/30 file after the snapshot 2024-04-20T21:20:48.924 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/31 file after the snapshot 2024-04-20T21:20:49.712 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/32 file after the snapshot 2024-04-20T21:20:49.714 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/33 file after the snapshot 2024-04-20T21:20:49.717 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/34 file after the snapshot 2024-04-20T21:20:49.719 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/35 file after the snapshot 2024-04-20T21:20:49.722 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/36 file after the snapshot 2024-04-20T21:20:49.725 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/37 file after the snapshot 2024-04-20T21:20:49.728 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/38 file after the snapshot 2024-04-20T21:20:49.731 INFO:tasks.workunit.client.0.smithi102.stdout: create 264/39 file after the snapshot 2024-04-20T21:20:49.734 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/.snap/snap-subdir-test 2024-04-20T21:20:49.789 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/30 file after the snapshot 2024-04-20T21:20:49.791 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/31 file after the snapshot 2024-04-20T21:20:49.794 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/32 file after the snapshot 2024-04-20T21:20:49.797 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/33 file after the snapshot 2024-04-20T21:20:49.800 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/34 file after the snapshot 2024-04-20T21:20:49.803 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/35 file after the snapshot 2024-04-20T21:20:49.805 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/36 file after the snapshot 2024-04-20T21:20:49.808 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/37 file after the snapshot 2024-04-20T21:20:49.811 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/38 file after the snapshot 2024-04-20T21:20:49.814 INFO:tasks.workunit.client.0.smithi102.stdout: create 265/39 file after the snapshot 2024-04-20T21:20:49.817 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/.snap/snap-subdir-test 2024-04-20T21:20:49.878 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/30 file after the snapshot 2024-04-20T21:20:49.881 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/31 file after the snapshot 2024-04-20T21:20:49.883 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/32 file after the snapshot 2024-04-20T21:20:49.887 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/33 file after the snapshot 2024-04-20T21:20:49.891 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/34 file after the snapshot 2024-04-20T21:20:49.895 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/35 file after the snapshot 2024-04-20T21:20:49.899 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/36 file after the snapshot 2024-04-20T21:20:49.903 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/37 file after the snapshot 2024-04-20T21:20:49.908 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/38 file after the snapshot 2024-04-20T21:20:49.912 INFO:tasks.workunit.client.0.smithi102.stdout: create 266/39 file after the snapshot 2024-04-20T21:20:49.916 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/.snap/snap-subdir-test 2024-04-20T21:20:49.991 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/30 file after the snapshot 2024-04-20T21:20:49.997 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/31 file after the snapshot 2024-04-20T21:20:50.006 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/32 file after the snapshot 2024-04-20T21:20:50.010 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/33 file after the snapshot 2024-04-20T21:20:50.012 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/34 file after the snapshot 2024-04-20T21:20:50.016 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/35 file after the snapshot 2024-04-20T21:20:50.017 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/36 file after the snapshot 2024-04-20T21:20:50.019 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/37 file after the snapshot 2024-04-20T21:20:50.021 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/38 file after the snapshot 2024-04-20T21:20:50.023 INFO:tasks.workunit.client.0.smithi102.stdout: create 267/39 file after the snapshot 2024-04-20T21:20:50.028 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/.snap/snap-subdir-test 2024-04-20T21:20:50.642 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/30 file after the snapshot 2024-04-20T21:20:51.173 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/31 file after the snapshot 2024-04-20T21:20:51.176 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/32 file after the snapshot 2024-04-20T21:20:51.179 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/33 file after the snapshot 2024-04-20T21:20:51.181 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/34 file after the snapshot 2024-04-20T21:20:51.184 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/35 file after the snapshot 2024-04-20T21:20:51.187 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/36 file after the snapshot 2024-04-20T21:20:51.189 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/37 file after the snapshot 2024-04-20T21:20:51.192 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/38 file after the snapshot 2024-04-20T21:20:51.195 INFO:tasks.workunit.client.0.smithi102.stdout: create 268/39 file after the snapshot 2024-04-20T21:20:51.197 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/.snap/snap-subdir-test 2024-04-20T21:20:51.389 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/30 file after the snapshot 2024-04-20T21:20:51.391 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/31 file after the snapshot 2024-04-20T21:20:51.392 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/32 file after the snapshot 2024-04-20T21:20:51.394 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/33 file after the snapshot 2024-04-20T21:20:51.395 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/34 file after the snapshot 2024-04-20T21:20:51.397 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/35 file after the snapshot 2024-04-20T21:20:51.399 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/36 file after the snapshot 2024-04-20T21:20:51.401 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/37 file after the snapshot 2024-04-20T21:20:51.403 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/38 file after the snapshot 2024-04-20T21:20:51.405 INFO:tasks.workunit.client.0.smithi102.stdout: create 269/39 file after the snapshot 2024-04-20T21:20:51.408 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/.snap/snap-subdir-test 2024-04-20T21:20:51.458 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/30 file after the snapshot 2024-04-20T21:20:51.461 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/31 file after the snapshot 2024-04-20T21:20:51.463 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/32 file after the snapshot 2024-04-20T21:20:51.466 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/33 file after the snapshot 2024-04-20T21:20:51.468 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/34 file after the snapshot 2024-04-20T21:20:51.471 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/35 file after the snapshot 2024-04-20T21:20:51.474 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/36 file after the snapshot 2024-04-20T21:20:51.476 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/37 file after the snapshot 2024-04-20T21:20:51.479 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/38 file after the snapshot 2024-04-20T21:20:51.482 INFO:tasks.workunit.client.0.smithi102.stdout: create 270/39 file after the snapshot 2024-04-20T21:20:51.485 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/.snap/snap-subdir-test 2024-04-20T21:20:51.815 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/30 file after the snapshot 2024-04-20T21:20:51.817 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/31 file after the snapshot 2024-04-20T21:20:51.819 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/32 file after the snapshot 2024-04-20T21:20:51.821 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/33 file after the snapshot 2024-04-20T21:20:51.823 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/34 file after the snapshot 2024-04-20T21:20:51.825 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/35 file after the snapshot 2024-04-20T21:20:51.827 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/36 file after the snapshot 2024-04-20T21:20:51.830 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/37 file after the snapshot 2024-04-20T21:20:51.832 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/38 file after the snapshot 2024-04-20T21:20:51.834 INFO:tasks.workunit.client.0.smithi102.stdout: create 271/39 file after the snapshot 2024-04-20T21:20:51.837 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/.snap/snap-subdir-test 2024-04-20T21:20:51.886 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/30 file after the snapshot 2024-04-20T21:20:51.889 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/31 file after the snapshot 2024-04-20T21:20:51.892 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/32 file after the snapshot 2024-04-20T21:20:51.894 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/33 file after the snapshot 2024-04-20T21:20:51.897 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/34 file after the snapshot 2024-04-20T21:20:51.900 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/35 file after the snapshot 2024-04-20T21:20:51.902 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/36 file after the snapshot 2024-04-20T21:20:51.905 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/37 file after the snapshot 2024-04-20T21:20:51.908 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/38 file after the snapshot 2024-04-20T21:20:51.911 INFO:tasks.workunit.client.0.smithi102.stdout: create 272/39 file after the snapshot 2024-04-20T21:20:51.913 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/.snap/snap-subdir-test 2024-04-20T21:20:51.917 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:51.923 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:51.959 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/30 file after the snapshot 2024-04-20T21:20:51.961 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/31 file after the snapshot 2024-04-20T21:20:51.962 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/32 file after the snapshot 2024-04-20T21:20:51.964 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/33 file after the snapshot 2024-04-20T21:20:51.965 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/34 file after the snapshot 2024-04-20T21:20:51.967 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/35 file after the snapshot 2024-04-20T21:20:51.968 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/36 file after the snapshot 2024-04-20T21:20:51.970 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/37 file after the snapshot 2024-04-20T21:20:51.971 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/38 file after the snapshot 2024-04-20T21:20:51.973 INFO:tasks.workunit.client.0.smithi102.stdout: create 273/39 file after the snapshot 2024-04-20T21:20:51.974 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/.snap/snap-subdir-test 2024-04-20T21:20:52.021 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/30 file after the snapshot 2024-04-20T21:20:52.023 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/31 file after the snapshot 2024-04-20T21:20:52.024 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/32 file after the snapshot 2024-04-20T21:20:52.026 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/33 file after the snapshot 2024-04-20T21:20:52.027 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/34 file after the snapshot 2024-04-20T21:20:52.028 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/35 file after the snapshot 2024-04-20T21:20:52.030 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/36 file after the snapshot 2024-04-20T21:20:52.032 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/37 file after the snapshot 2024-04-20T21:20:52.035 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/38 file after the snapshot 2024-04-20T21:20:52.037 INFO:tasks.workunit.client.0.smithi102.stdout: create 274/39 file after the snapshot 2024-04-20T21:20:52.040 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/.snap/snap-subdir-test 2024-04-20T21:20:52.101 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/30 file after the snapshot 2024-04-20T21:20:52.103 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/31 file after the snapshot 2024-04-20T21:20:52.105 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/32 file after the snapshot 2024-04-20T21:20:52.107 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/33 file after the snapshot 2024-04-20T21:20:52.109 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/34 file after the snapshot 2024-04-20T21:20:52.111 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/35 file after the snapshot 2024-04-20T21:20:52.113 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/36 file after the snapshot 2024-04-20T21:20:52.116 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/37 file after the snapshot 2024-04-20T21:20:52.118 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/38 file after the snapshot 2024-04-20T21:20:52.121 INFO:tasks.workunit.client.0.smithi102.stdout: create 275/39 file after the snapshot 2024-04-20T21:20:52.124 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/.snap/snap-subdir-test 2024-04-20T21:20:52.843 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/30 file after the snapshot 2024-04-20T21:20:52.846 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/31 file after the snapshot 2024-04-20T21:20:52.849 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/32 file after the snapshot 2024-04-20T21:20:52.851 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/33 file after the snapshot 2024-04-20T21:20:52.854 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/34 file after the snapshot 2024-04-20T21:20:52.857 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/35 file after the snapshot 2024-04-20T21:20:52.859 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/36 file after the snapshot 2024-04-20T21:20:52.862 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/37 file after the snapshot 2024-04-20T21:20:52.865 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/38 file after the snapshot 2024-04-20T21:20:52.868 INFO:tasks.workunit.client.0.smithi102.stdout: create 276/39 file after the snapshot 2024-04-20T21:20:52.871 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/.snap/snap-subdir-test 2024-04-20T21:20:52.912 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/30 file after the snapshot 2024-04-20T21:20:52.915 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/31 file after the snapshot 2024-04-20T21:20:52.917 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/32 file after the snapshot 2024-04-20T21:20:52.920 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/33 file after the snapshot 2024-04-20T21:20:52.923 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/34 file after the snapshot 2024-04-20T21:20:52.926 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/35 file after the snapshot 2024-04-20T21:20:52.928 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/36 file after the snapshot 2024-04-20T21:20:52.931 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/37 file after the snapshot 2024-04-20T21:20:52.934 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/38 file after the snapshot 2024-04-20T21:20:52.936 INFO:tasks.workunit.client.0.smithi102.stdout: create 277/39 file after the snapshot 2024-04-20T21:20:52.939 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/.snap/snap-subdir-test 2024-04-20T21:20:52.981 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/30 file after the snapshot 2024-04-20T21:20:52.984 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/31 file after the snapshot 2024-04-20T21:20:52.987 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/32 file after the snapshot 2024-04-20T21:20:52.989 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/33 file after the snapshot 2024-04-20T21:20:52.992 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/34 file after the snapshot 2024-04-20T21:20:52.995 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/35 file after the snapshot 2024-04-20T21:20:52.998 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/36 file after the snapshot 2024-04-20T21:20:53.000 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/37 file after the snapshot 2024-04-20T21:20:53.003 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/38 file after the snapshot 2024-04-20T21:20:53.006 INFO:tasks.workunit.client.0.smithi102.stdout: create 278/39 file after the snapshot 2024-04-20T21:20:53.009 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/.snap/snap-subdir-test 2024-04-20T21:20:53.318 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/30 file after the snapshot 2024-04-20T21:20:53.319 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/31 file after the snapshot 2024-04-20T21:20:53.321 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/32 file after the snapshot 2024-04-20T21:20:53.322 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/33 file after the snapshot 2024-04-20T21:20:53.324 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/34 file after the snapshot 2024-04-20T21:20:53.325 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/35 file after the snapshot 2024-04-20T21:20:53.327 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/36 file after the snapshot 2024-04-20T21:20:53.328 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/37 file after the snapshot 2024-04-20T21:20:53.330 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/38 file after the snapshot 2024-04-20T21:20:53.333 INFO:tasks.workunit.client.0.smithi102.stdout: create 279/39 file after the snapshot 2024-04-20T21:20:53.334 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/.snap/snap-subdir-test 2024-04-20T21:20:53.383 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/30 file after the snapshot 2024-04-20T21:20:53.386 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/31 file after the snapshot 2024-04-20T21:20:53.389 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/32 file after the snapshot 2024-04-20T21:20:53.391 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/33 file after the snapshot 2024-04-20T21:20:53.394 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/34 file after the snapshot 2024-04-20T21:20:53.397 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/35 file after the snapshot 2024-04-20T21:20:53.400 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/36 file after the snapshot 2024-04-20T21:20:53.402 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/37 file after the snapshot 2024-04-20T21:20:53.405 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/38 file after the snapshot 2024-04-20T21:20:53.408 INFO:tasks.workunit.client.0.smithi102.stdout: create 280/39 file after the snapshot 2024-04-20T21:20:53.411 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/.snap/snap-subdir-test 2024-04-20T21:20:53.456 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/30 file after the snapshot 2024-04-20T21:20:53.457 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/31 file after the snapshot 2024-04-20T21:20:53.459 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/32 file after the snapshot 2024-04-20T21:20:53.461 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/33 file after the snapshot 2024-04-20T21:20:53.463 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/34 file after the snapshot 2024-04-20T21:20:53.464 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/35 file after the snapshot 2024-04-20T21:20:53.466 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/36 file after the snapshot 2024-04-20T21:20:53.468 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/37 file after the snapshot 2024-04-20T21:20:53.469 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/38 file after the snapshot 2024-04-20T21:20:53.789 INFO:tasks.workunit.client.0.smithi102.stdout: create 281/39 file after the snapshot 2024-04-20T21:20:53.819 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/.snap/snap-subdir-test 2024-04-20T21:20:53.857 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/30 file after the snapshot 2024-04-20T21:20:53.860 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/31 file after the snapshot 2024-04-20T21:20:53.862 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/32 file after the snapshot 2024-04-20T21:20:53.865 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/33 file after the snapshot 2024-04-20T21:20:53.869 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/34 file after the snapshot 2024-04-20T21:20:53.871 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/35 file after the snapshot 2024-04-20T21:20:53.874 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/36 file after the snapshot 2024-04-20T21:20:53.876 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/37 file after the snapshot 2024-04-20T21:20:53.879 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/38 file after the snapshot 2024-04-20T21:20:53.882 INFO:tasks.workunit.client.0.smithi102.stdout: create 282/39 file after the snapshot 2024-04-20T21:20:53.885 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/.snap/snap-subdir-test 2024-04-20T21:20:53.934 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/30 file after the snapshot 2024-04-20T21:20:53.936 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/31 file after the snapshot 2024-04-20T21:20:53.938 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/32 file after the snapshot 2024-04-20T21:20:53.941 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/33 file after the snapshot 2024-04-20T21:20:53.943 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/34 file after the snapshot 2024-04-20T21:20:53.945 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/35 file after the snapshot 2024-04-20T21:20:53.948 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/36 file after the snapshot 2024-04-20T21:20:53.951 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/37 file after the snapshot 2024-04-20T21:20:53.953 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/38 file after the snapshot 2024-04-20T21:20:53.956 INFO:tasks.workunit.client.0.smithi102.stdout: create 283/39 file after the snapshot 2024-04-20T21:20:53.959 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/.snap/snap-subdir-test 2024-04-20T21:20:54.003 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/30 file after the snapshot 2024-04-20T21:20:54.006 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/31 file after the snapshot 2024-04-20T21:20:54.009 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/32 file after the snapshot 2024-04-20T21:20:54.012 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/33 file after the snapshot 2024-04-20T21:20:54.014 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/34 file after the snapshot 2024-04-20T21:20:54.018 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/35 file after the snapshot 2024-04-20T21:20:54.020 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/36 file after the snapshot 2024-04-20T21:20:54.023 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/37 file after the snapshot 2024-04-20T21:20:54.026 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/38 file after the snapshot 2024-04-20T21:20:54.029 INFO:tasks.workunit.client.0.smithi102.stdout: create 284/39 file after the snapshot 2024-04-20T21:20:54.032 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/.snap/snap-subdir-test 2024-04-20T21:20:54.085 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/30 file after the snapshot 2024-04-20T21:20:54.087 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/31 file after the snapshot 2024-04-20T21:20:54.090 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/32 file after the snapshot 2024-04-20T21:20:54.094 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/33 file after the snapshot 2024-04-20T21:20:54.098 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/34 file after the snapshot 2024-04-20T21:20:54.102 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/35 file after the snapshot 2024-04-20T21:20:54.105 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/36 file after the snapshot 2024-04-20T21:20:54.109 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/37 file after the snapshot 2024-04-20T21:20:54.113 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/38 file after the snapshot 2024-04-20T21:20:54.117 INFO:tasks.workunit.client.0.smithi102.stdout: create 285/39 file after the snapshot 2024-04-20T21:20:54.121 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/.snap/snap-subdir-test 2024-04-20T21:20:55.152 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/30 file after the snapshot 2024-04-20T21:20:55.157 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/31 file after the snapshot 2024-04-20T21:20:55.159 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/32 file after the snapshot 2024-04-20T21:20:55.421 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/33 file after the snapshot 2024-04-20T21:20:55.425 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/34 file after the snapshot 2024-04-20T21:20:55.429 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/35 file after the snapshot 2024-04-20T21:20:55.433 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/36 file after the snapshot 2024-04-20T21:20:55.437 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/37 file after the snapshot 2024-04-20T21:20:55.442 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/38 file after the snapshot 2024-04-20T21:20:55.446 INFO:tasks.workunit.client.0.smithi102.stdout: create 286/39 file after the snapshot 2024-04-20T21:20:55.450 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/.snap/snap-subdir-test 2024-04-20T21:20:55.508 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/30 file after the snapshot 2024-04-20T21:20:55.512 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/31 file after the snapshot 2024-04-20T21:20:55.516 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/32 file after the snapshot 2024-04-20T21:20:55.520 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/33 file after the snapshot 2024-04-20T21:20:55.524 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/34 file after the snapshot 2024-04-20T21:20:55.528 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/35 file after the snapshot 2024-04-20T21:20:55.532 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/36 file after the snapshot 2024-04-20T21:20:55.536 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/37 file after the snapshot 2024-04-20T21:20:55.540 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/38 file after the snapshot 2024-04-20T21:20:55.544 INFO:tasks.workunit.client.0.smithi102.stdout: create 287/39 file after the snapshot 2024-04-20T21:20:55.548 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/.snap/snap-subdir-test 2024-04-20T21:20:55.609 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/30 file after the snapshot 2024-04-20T21:20:55.612 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/31 file after the snapshot 2024-04-20T21:20:55.616 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/32 file after the snapshot 2024-04-20T21:20:55.621 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/33 file after the snapshot 2024-04-20T21:20:55.625 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/34 file after the snapshot 2024-04-20T21:20:55.629 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/35 file after the snapshot 2024-04-20T21:20:55.633 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/36 file after the snapshot 2024-04-20T21:20:55.637 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/37 file after the snapshot 2024-04-20T21:20:55.641 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/38 file after the snapshot 2024-04-20T21:20:55.645 INFO:tasks.workunit.client.0.smithi102.stdout: create 288/39 file after the snapshot 2024-04-20T21:20:55.649 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/.snap/snap-subdir-test 2024-04-20T21:20:55.703 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/30 file after the snapshot 2024-04-20T21:20:55.706 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/31 file after the snapshot 2024-04-20T21:20:55.709 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/32 file after the snapshot 2024-04-20T21:20:55.714 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/33 file after the snapshot 2024-04-20T21:20:55.718 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/34 file after the snapshot 2024-04-20T21:20:55.722 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/35 file after the snapshot 2024-04-20T21:20:55.726 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/36 file after the snapshot 2024-04-20T21:20:55.730 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/37 file after the snapshot 2024-04-20T21:20:55.735 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/38 file after the snapshot 2024-04-20T21:20:55.739 INFO:tasks.workunit.client.0.smithi102.stdout: create 289/39 file after the snapshot 2024-04-20T21:20:55.744 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/.snap/snap-subdir-test 2024-04-20T21:20:55.803 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/30 file after the snapshot 2024-04-20T21:20:55.805 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/31 file after the snapshot 2024-04-20T21:20:55.807 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/32 file after the snapshot 2024-04-20T21:20:55.808 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/33 file after the snapshot 2024-04-20T21:20:55.811 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/34 file after the snapshot 2024-04-20T21:20:55.813 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/35 file after the snapshot 2024-04-20T21:20:55.816 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/36 file after the snapshot 2024-04-20T21:20:55.819 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/37 file after the snapshot 2024-04-20T21:20:55.821 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/38 file after the snapshot 2024-04-20T21:20:55.824 INFO:tasks.workunit.client.0.smithi102.stdout: create 290/39 file after the snapshot 2024-04-20T21:20:55.827 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/.snap/snap-subdir-test 2024-04-20T21:20:55.886 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/30 file after the snapshot 2024-04-20T21:20:55.887 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/31 file after the snapshot 2024-04-20T21:20:55.889 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/32 file after the snapshot 2024-04-20T21:20:55.891 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/33 file after the snapshot 2024-04-20T21:20:55.893 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/34 file after the snapshot 2024-04-20T21:20:55.895 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/35 file after the snapshot 2024-04-20T21:20:55.896 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/36 file after the snapshot 2024-04-20T21:20:55.898 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/37 file after the snapshot 2024-04-20T21:20:55.900 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/38 file after the snapshot 2024-04-20T21:20:55.902 INFO:tasks.workunit.client.0.smithi102.stdout: create 291/39 file after the snapshot 2024-04-20T21:20:55.905 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/.snap/snap-subdir-test 2024-04-20T21:20:55.962 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/30 file after the snapshot 2024-04-20T21:20:55.964 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/31 file after the snapshot 2024-04-20T21:20:55.966 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/32 file after the snapshot 2024-04-20T21:20:55.968 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/33 file after the snapshot 2024-04-20T21:20:55.971 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/34 file after the snapshot 2024-04-20T21:20:55.973 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/35 file after the snapshot 2024-04-20T21:20:55.976 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/36 file after the snapshot 2024-04-20T21:20:55.979 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/37 file after the snapshot 2024-04-20T21:20:55.981 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/38 file after the snapshot 2024-04-20T21:20:55.984 INFO:tasks.workunit.client.0.smithi102.stdout: create 292/39 file after the snapshot 2024-04-20T21:20:55.987 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/.snap/snap-subdir-test 2024-04-20T21:20:56.050 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/30 file after the snapshot 2024-04-20T21:20:56.053 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/31 file after the snapshot 2024-04-20T21:20:56.056 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/32 file after the snapshot 2024-04-20T21:20:56.058 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/33 file after the snapshot 2024-04-20T21:20:56.061 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/34 file after the snapshot 2024-04-20T21:20:56.064 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/35 file after the snapshot 2024-04-20T21:20:56.067 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/36 file after the snapshot 2024-04-20T21:20:56.069 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/37 file after the snapshot 2024-04-20T21:20:56.071 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/38 file after the snapshot 2024-04-20T21:20:56.073 INFO:tasks.workunit.client.0.smithi102.stdout: create 293/39 file after the snapshot 2024-04-20T21:20:56.075 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/.snap/snap-subdir-test 2024-04-20T21:20:56.136 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/30 file after the snapshot 2024-04-20T21:20:56.138 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/31 file after the snapshot 2024-04-20T21:20:56.140 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/32 file after the snapshot 2024-04-20T21:20:56.142 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/33 file after the snapshot 2024-04-20T21:20:56.144 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/34 file after the snapshot 2024-04-20T21:20:56.146 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/35 file after the snapshot 2024-04-20T21:20:56.149 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/36 file after the snapshot 2024-04-20T21:20:56.152 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/37 file after the snapshot 2024-04-20T21:20:56.154 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/38 file after the snapshot 2024-04-20T21:20:56.157 INFO:tasks.workunit.client.0.smithi102.stdout: create 294/39 file after the snapshot 2024-04-20T21:20:56.160 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/.snap/snap-subdir-test 2024-04-20T21:20:56.224 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/30 file after the snapshot 2024-04-20T21:20:56.228 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/31 file after the snapshot 2024-04-20T21:20:56.232 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/32 file after the snapshot 2024-04-20T21:20:56.236 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/33 file after the snapshot 2024-04-20T21:20:56.240 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/34 file after the snapshot 2024-04-20T21:20:56.244 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/35 file after the snapshot 2024-04-20T21:20:56.249 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/36 file after the snapshot 2024-04-20T21:20:56.253 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/37 file after the snapshot 2024-04-20T21:20:56.257 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/38 file after the snapshot 2024-04-20T21:20:56.261 INFO:tasks.workunit.client.0.smithi102.stdout: create 295/39 file after the snapshot 2024-04-20T21:20:56.265 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/.snap/snap-subdir-test 2024-04-20T21:20:56.308 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/30 file after the snapshot 2024-04-20T21:20:56.312 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/31 file after the snapshot 2024-04-20T21:20:56.316 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/32 file after the snapshot 2024-04-20T21:20:56.320 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/33 file after the snapshot 2024-04-20T21:20:56.324 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/34 file after the snapshot 2024-04-20T21:20:56.328 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/35 file after the snapshot 2024-04-20T21:20:56.333 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/36 file after the snapshot 2024-04-20T21:20:56.337 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/37 file after the snapshot 2024-04-20T21:20:56.341 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/38 file after the snapshot 2024-04-20T21:20:56.344 INFO:tasks.workunit.client.0.smithi102.stdout: create 296/39 file after the snapshot 2024-04-20T21:20:56.347 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/.snap/snap-subdir-test 2024-04-20T21:20:56.387 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/30 file after the snapshot 2024-04-20T21:20:56.390 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/31 file after the snapshot 2024-04-20T21:20:56.393 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/32 file after the snapshot 2024-04-20T21:20:56.396 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/33 file after the snapshot 2024-04-20T21:20:56.399 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/34 file after the snapshot 2024-04-20T21:20:56.402 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/35 file after the snapshot 2024-04-20T21:20:56.405 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/36 file after the snapshot 2024-04-20T21:20:56.408 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/37 file after the snapshot 2024-04-20T21:20:56.410 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/38 file after the snapshot 2024-04-20T21:20:56.413 INFO:tasks.workunit.client.0.smithi102.stdout: create 297/39 file after the snapshot 2024-04-20T21:20:56.416 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/.snap/snap-subdir-test 2024-04-20T21:20:56.473 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/30 file after the snapshot 2024-04-20T21:20:56.476 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/31 file after the snapshot 2024-04-20T21:20:56.478 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/32 file after the snapshot 2024-04-20T21:20:56.481 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/33 file after the snapshot 2024-04-20T21:20:56.484 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/34 file after the snapshot 2024-04-20T21:20:56.487 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/35 file after the snapshot 2024-04-20T21:20:56.490 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/36 file after the snapshot 2024-04-20T21:20:56.493 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/37 file after the snapshot 2024-04-20T21:20:56.495 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/38 file after the snapshot 2024-04-20T21:20:56.498 INFO:tasks.workunit.client.0.smithi102.stdout: create 298/39 file after the snapshot 2024-04-20T21:20:56.501 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/.snap/snap-subdir-test 2024-04-20T21:20:56.540 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/30 file after the snapshot 2024-04-20T21:20:56.542 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/31 file after the snapshot 2024-04-20T21:20:56.545 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/32 file after the snapshot 2024-04-20T21:20:56.548 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/33 file after the snapshot 2024-04-20T21:20:56.551 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/34 file after the snapshot 2024-04-20T21:20:56.553 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/35 file after the snapshot 2024-04-20T21:20:56.556 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/36 file after the snapshot 2024-04-20T21:20:56.559 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/37 file after the snapshot 2024-04-20T21:20:56.561 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/38 file after the snapshot 2024-04-20T21:20:56.564 INFO:tasks.workunit.client.0.smithi102.stdout: create 299/39 file after the snapshot 2024-04-20T21:20:56.567 INFO:tasks.workunit.client.0.smithi102.stdout:Take second snapshot .snap/test2 2024-04-20T21:20:56.623 INFO:tasks.workunit.client.0.smithi102.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2024-04-20T21:20:57.370 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2024-04-20T21:20:57.371 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2024-04-20T21:20:57.372 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2024-04-20T21:20:57.373 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2024-04-20T21:20:57.374 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2024-04-20T21:20:57.375 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2024-04-20T21:20:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2024-04-20T21:20:59.125 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2024-04-20T21:20:59.125 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2024-04-20T21:20:59.125 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2024-04-20T21:20:59.125 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2024-04-20T21:20:59.125 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2024-04-20T21:20:59.126 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2024-04-20T21:20:59.127 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2024-04-20T21:20:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2024-04-20T21:20:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2024-04-20T21:20:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2024-04-20T21:20:59.131 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2024-04-20T21:20:59.131 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2024-04-20T21:20:59.131 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2024-04-20T21:20:59.593 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2024-04-20T21:20:59.594 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2024-04-20T21:20:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2024-04-20T21:20:59.596 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2024-04-20T21:20:59.597 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2024-04-20T21:20:59.598 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2024-04-20T21:20:59.599 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2024-04-20T21:21:00.039 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2024-04-20T21:21:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2024-04-20T21:21:00.041 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2024-04-20T21:21:00.042 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2024-04-20T21:21:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2024-04-20T21:21:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2024-04-20T21:21:00.045 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2024-04-20T21:21:01.913 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2024-04-20T21:21:01.914 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2024-04-20T21:21:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2024-04-20T21:21:01.916 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2024-04-20T21:21:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2024-04-20T21:21:01.918 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2024-04-20T21:21:01.919 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2024-04-20T21:21:01.919 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2024-04-20T21:21:01.919 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2024-04-20T21:21:01.919 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2024-04-20T21:21:02.410 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2024-04-20T21:21:02.410 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2024-04-20T21:21:02.410 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2024-04-20T21:21:02.410 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2024-04-20T21:21:02.410 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2024-04-20T21:21:02.410 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2024-04-20T21:21:02.410 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2024-04-20T21:21:02.410 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2024-04-20T21:21:02.411 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2024-04-20T21:21:02.412 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2024-04-20T21:21:02.413 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2024-04-20T21:21:02.414 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2024-04-20T21:21:02.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2024-04-20T21:21:02.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2024-04-20T21:21:02.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2024-04-20T21:21:02.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2024-04-20T21:21:02.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2024-04-20T21:21:02.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2024-04-20T21:21:02.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2024-04-20T21:21:02.991 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2024-04-20T21:21:02.991 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2024-04-20T21:21:02.991 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2024-04-20T21:21:02.991 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2024-04-20T21:21:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2024-04-20T21:21:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2024-04-20T21:21:02.994 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2024-04-20T21:21:02.995 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2024-04-20T21:21:02.996 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2024-04-20T21:21:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2024-04-20T21:21:02.998 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2024-04-20T21:21:04.670 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2024-04-20T21:21:04.804 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2024-04-20T21:21:04.805 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2024-04-20T21:21:04.806 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2024-04-20T21:21:04.807 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2024-04-20T21:21:04.808 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2024-04-20T21:21:04.809 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2024-04-20T21:21:04.809 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2024-04-20T21:21:04.809 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2024-04-20T21:21:04.809 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2024-04-20T21:21:04.809 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2024-04-20T21:21:04.809 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2024-04-20T21:21:04.809 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2024-04-20T21:21:04.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.882+0000 7f7c51e00640 1 Processor -- start 2024-04-20T21:21:04.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.882+0000 7f7c51e00640 1 -- start start 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.883+0000 7f7c51e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7c4c0cc470 0x7f7c4c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.883+0000 7f7c51e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7c4c0cdfc0 0x7f7c4c0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.883+0000 7f7c51e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c4c15ef60 con 0x7f7c4c155530 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.883+0000 7f7c51e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f7c4c0c5ca0 con 0x7f7c4c0cdfc0 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.883+0000 7f7c51e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7c4c0cd5f0 con 0x7f7c4c0cc470 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.883+0000 7f7c50a00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f7c4c155530 0x7f7c4c15e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:60826/0 (socket says 172.21.15.102:60826) 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.883+0000 7f7c50a00640 1 -- 172.21.15.102:0/3739620748 learned_addr learned my addr 172.21.15.102:0/3739620748 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.884+0000 7f7c4be00640 1 --2- 172.21.15.102:0/3739620748 >> v2:172.21.15.193:3300/0 conn(0x7f7c4c0cc470 0x7f7c4c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.884+0000 7f7c51400640 1 --2- 172.21.15.102:0/3739620748 >> v2:172.21.15.102:3300/0 conn(0x7f7c4c0cdfc0 0x7f7c4c0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.884+0000 7f7c4b400640 1 -- 172.21.15.102:0/3739620748 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661869669 0 0) 0x7f7c4c15ef60 con 0x7f7c4c155530 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.884+0000 7f7c4b400640 1 -- 172.21.15.102:0/3739620748 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c34003520 con 0x7f7c4c155530 2024-04-20T21:21:04.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.884+0000 7f7c4be00640 1 -- 172.21.15.102:0/3739620748 >> v2:172.21.15.102:3300/0 conn(0x7f7c4c0cdfc0 msgr2=0x7f7c4c0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:04.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.884+0000 7f7c4be00640 1 --2- 172.21.15.102:0/3739620748 >> v2:172.21.15.102:3300/0 conn(0x7f7c4c0cdfc0 0x7f7c4c0cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.884+0000 7f7c4be00640 1 -- 172.21.15.102:0/3739620748 >> v1:172.21.15.193:6789/0 conn(0x7f7c4c155530 legacy=0x7f7c4c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.884+0000 7f7c4be00640 1 -- 172.21.15.102:0/3739620748 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c15f240 con 0x7f7c4c0cc470 2024-04-20T21:21:04.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.885+0000 7f7c4be00640 1 --2- 172.21.15.102:0/3739620748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c0cda80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c06caf0 tx=0x7f7c3802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880e7519ba1c3cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.886+0000 7f7c4b400640 1 -- 172.21.15.102:0/3739620748 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38035020 con 0x7f7c4c0cc470 2024-04-20T21:21:04.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.886+0000 7f7c4b400640 1 -- 172.21.15.102:0/3739620748 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c38031830 con 0x7f7c4c0cc470 2024-04-20T21:21:04.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.886+0000 7f7c4b400640 1 -- 172.21.15.102:0/3739620748 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38031c00 con 0x7f7c4c0cc470 2024-04-20T21:21:04.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.887+0000 7f7c51e00640 1 -- 172.21.15.102:0/3739620748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 msgr2=0x7f7c4c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.887+0000 7f7c51e00640 1 --2- 172.21.15.102:0/3739620748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c0cda80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c06caf0 tx=0x7f7c3802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.887+0000 7f7c51e00640 1 -- 172.21.15.102:0/3739620748 shutdown_connections 2024-04-20T21:21:04.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.887+0000 7f7c51e00640 1 --2- 172.21.15.102:0/3739620748 >> v2:172.21.15.102:3300/0 conn(0x7f7c4c0cdfc0 0x7f7c4c0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.887+0000 7f7c51e00640 1 --2- 172.21.15.102:0/3739620748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c0cda80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.887+0000 7f7c51e00640 1 -- 172.21.15.102:0/3739620748 >> 172.21.15.102:0/3739620748 conn(0x7f7c4c075080 msgr2=0x7f7c4c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:04.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.887+0000 7f7c51e00640 1 -- 172.21.15.102:0/3739620748 shutdown_connections 2024-04-20T21:21:04.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.887+0000 7f7c51e00640 1 -- 172.21.15.102:0/3739620748 wait complete. 2024-04-20T21:21:04.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.889+0000 7f7c51e00640 1 Processor -- start 2024-04-20T21:21:04.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.889+0000 7f7c51e00640 1 -- start start 2024-04-20T21:21:04.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c51e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c1f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c51e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c4c0cdfc0 0x7f7c4c144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c4c155530 0x7f7c4c0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c51e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0cd280 con 0x7f7c4c155530 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c51e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0ccfa0 con 0x7f7c4c0cc470 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c51e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4c0ccbe0 con 0x7f7c4c0cdfc0 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c50a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c1f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c4c0cdfc0 0x7f7c4c144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c51400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c4c155530 0x7f7c4c0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c50a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c1f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:48310/0 (socket says 172.21.15.102:48310) 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c4c0cdfc0 0x7f7c4c144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:60164/0 (socket says 172.21.15.102:60164) 2024-04-20T21:21:04.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.890+0000 7f7c50a00640 1 -- 172.21.15.102:0/1960556361 learned_addr learned my addr 172.21.15.102:0/1960556361 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:04.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c50a00640 1 -- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c4c0cdfc0 msgr2=0x7f7c4c144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c50a00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c4c0cdfc0 0x7f7c4c144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c50a00640 1 -- 172.21.15.102:0/1960556361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c4c155530 msgr2=0x7f7c4c0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c50a00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c4c155530 0x7f7c4c0b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c50a00640 1 -- 172.21.15.102:0/1960556361 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c1f20e0 con 0x7f7c4c0cc470 2024-04-20T21:21:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c4be00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c4c0cdfc0 0x7f7c4c144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c50a00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c1f1ba0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c40001bf0 tx=0x7f7c4000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c48c00640 1 -- 172.21.15.102:0/1960556361 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000ad50 con 0x7f7c4c0cc470 2024-04-20T21:21:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c48c00640 1 -- 172.21.15.102:0/1960556361 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c400058b0 con 0x7f7c4c0cc470 2024-04-20T21:21:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c0b6c80 con 0x7f7c4c0cc470 2024-04-20T21:21:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c48c00640 1 -- 172.21.15.102:0/1960556361 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40005ce0 con 0x7f7c4c0cc470 2024-04-20T21:21:04.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.891+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c0b49e0 con 0x7f7c4c0cc470 2024-04-20T21:21:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.893+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4c04fd60 con 0x7f7c4c0cc470 2024-04-20T21:21:04.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.894+0000 7f7c48c00640 1 -- 172.21.15.102:0/1960556361 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c40007070 con 0x7f7c4c0cc470 2024-04-20T21:21:04.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.895+0000 7f7c48c00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c2c06c8f0 0x7f7c2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.895+0000 7f7c48c00640 1 -- 172.21.15.102:0/1960556361 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6877+0+0 (secure 0 0 0) 0x7f7c40093fe0 con 0x7f7c4c0cc470 2024-04-20T21:21:04.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.895+0000 7f7c4be00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c2c06c8f0 0x7f7c2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.896+0000 7f7c4be00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c2c06c8f0 0x7f7c2c06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7c38036000 tx=0x7f7c38008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:04.899+0000 7f7c48c00640 1 -- 172.21.15.102:0/1960556361 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c400618b0 con 0x7f7c4c0cc470 2024-04-20T21:21:05.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.152+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c4c0cc1a0 con 0x7f7c4c0cc470 2024-04-20T21:21:05.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.153+0000 7f7c48c00640 1 -- 172.21.15.102:0/1960556361 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6862 (secure 0 0 0) 0x7f7c40065bb0 con 0x7f7c4c0cc470 2024-04-20T21:21:05.153 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:05.153 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:21:05.153 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 15 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c2c06c8f0 msgr2=0x7f7c2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c2c06c8f0 0x7f7c2c06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7c38036000 tx=0x7f7c38008030 comp rx=0 tx=0).stop 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 msgr2=0x7f7c4c1f1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c1f1ba0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c40001bf0 tx=0x7f7c4000a430 comp rx=0 tx=0).stop 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 shutdown_connections 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c4c155530 0x7f7c4c0b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c4c0cdfc0 0x7f7c4c144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c2c06c8f0 0x7f7c2c06edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 --2- 172.21.15.102:0/1960556361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c4c0cc470 0x7f7c4c1f1ba0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 >> 172.21.15.102:0/1960556361 conn(0x7f7c4c075080 msgr2=0x7f7c4c0ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 shutdown_connections 2024-04-20T21:21:05.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.156+0000 7f7c51e00640 1 -- 172.21.15.102:0/1960556361 wait complete. 2024-04-20T21:21:05.174 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-04-20T21:21:05.174 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:21:05.175 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:21:05.200 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:21:05.201 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:05.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.456+0000 7f8b7a000640 1 Processor -- start 2024-04-20T21:21:05.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.456+0000 7f8b7a000640 1 -- start start 2024-04-20T21:21:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.457+0000 7f8b7a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b740c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.457+0000 7f8b7a000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b741536d0 con 0x7f8b7414fa50 2024-04-20T21:21:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.457+0000 7f8b7a000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b740c76c0 con 0x7f8b74073a10 2024-04-20T21:21:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.457+0000 7f8b7a000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8b740c7960 con 0x7f8b740c9eb0 2024-04-20T21:21:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b73400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8b7414fa50 0x7f8b740c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53426/0 (socket says 172.21.15.102:53426) 2024-04-20T21:21:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b73400640 1 -- 172.21.15.102:0/1418167810 learned_addr learned my addr 172.21.15.102:0/1418167810 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:05.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b6be00640 1 --2- 172.21.15.102:0/1418167810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b740c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b72a00640 1 -- 172.21.15.102:0/1418167810 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761966146 0 0) 0x7f8b64001110 con 0x7f8b7414fa50 2024-04-20T21:21:05.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b72a00640 1 -- 172.21.15.102:0/1418167810 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b58003520 con 0x7f8b7414fa50 2024-04-20T21:21:05.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b72a00640 1 -- 172.21.15.102:0/1418167810 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321703551 0 0) 0x7f8b740c76c0 con 0x7f8b74073a10 2024-04-20T21:21:05.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b72a00640 1 -- 172.21.15.102:0/1418167810 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b64001110 con 0x7f8b74073a10 2024-04-20T21:21:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b6be00640 1 -- 172.21.15.102:0/1418167810 >> v1:172.21.15.193:6789/0 conn(0x7f8b74073a10 legacy=0x7f8b74163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b6be00640 1 -- 172.21.15.102:0/1418167810 >> v1:172.21.15.102:6789/0 conn(0x7f8b7414fa50 legacy=0x7f8b740c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.458+0000 7f8b6be00640 1 -- 172.21.15.102:0/1418167810 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b740c3d90 con 0x7f8b740c9eb0 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.459+0000 7f8b6be00640 1 --2- 172.21.15.102:0/1418167810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b740c4830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b740c3f60 tx=0x7f8b5c02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1569a0c8bdcfacb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b72a00640 1 -- 172.21.15.102:0/1418167810 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c034070 con 0x7f8b740c9eb0 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b72a00640 1 -- 172.21.15.102:0/1418167810 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b5c030880 con 0x7f8b740c9eb0 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b72a00640 1 -- 172.21.15.102:0/1418167810 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c030c50 con 0x7f8b740c9eb0 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b7a000640 1 -- 172.21.15.102:0/1418167810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 msgr2=0x7f8b740c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b7a000640 1 --2- 172.21.15.102:0/1418167810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b740c4830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b740c3f60 tx=0x7f8b5c02b9c0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b7a000640 1 -- 172.21.15.102:0/1418167810 shutdown_connections 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b7a000640 1 --2- 172.21.15.102:0/1418167810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b740c4830 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b7a000640 1 -- 172.21.15.102:0/1418167810 >> 172.21.15.102:0/1418167810 conn(0x7f8b740c9390 msgr2=0x7f8b740c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b7a000640 1 -- 172.21.15.102:0/1418167810 shutdown_connections 2024-04-20T21:21:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.460+0000 7f8b7a000640 1 -- 172.21.15.102:0/1418167810 wait complete. 2024-04-20T21:21:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.462+0000 7f8b7a000640 1 Processor -- start 2024-04-20T21:21:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.462+0000 7f8b7a000640 1 -- start start 2024-04-20T21:21:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.463+0000 7f8b7a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b74073a10 0x7f8b741f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.463+0000 7f8b7a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b74155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.463+0000 7f8b7a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b7414fa50 0x7f8b740cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.463+0000 7f8b7a000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8b740c4030 con 0x7f8b74073a10 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.463+0000 7f8b7a000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b740c7c00 con 0x7f8b7414fa50 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.463+0000 7f8b7a000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8b740c4310 con 0x7f8b740c9eb0 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b74073a10 0x7f8b741f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b6be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b74155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b7414fa50 0x7f8b740cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b74073a10 0x7f8b741f17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45642/0 (socket says 172.21.15.102:45642) 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 -- 172.21.15.102:0/3200747124 learned_addr learned my addr 172.21.15.102:0/3200747124 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b6be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b74155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:60188/0 (socket says 172.21.15.102:60188) 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 -- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 msgr2=0x7f8b74155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b74155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 -- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b7414fa50 msgr2=0x7f8b740cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b7414fa50 0x7f8b740cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 -- 172.21.15.102:0/3200747124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b740cb320 con 0x7f8b74073a10 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.464+0000 7f8b73400640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b74073a10 0x7f8b741f17c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c760 tx=0x7f8b6400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.465+0000 7f8b6b400640 1 -- 172.21.15.102:0/3200747124 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64017070 con 0x7f8b74073a10 2024-04-20T21:21:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.465+0000 7f8b6b400640 1 -- 172.21.15.102:0/3200747124 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b64005af0 con 0x7f8b74073a10 2024-04-20T21:21:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.465+0000 7f8b6b400640 1 -- 172.21.15.102:0/3200747124 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64007070 con 0x7f8b74073a10 2024-04-20T21:21:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.465+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b740cb5c0 con 0x7f8b74073a10 2024-04-20T21:21:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.465+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b740cb880 con 0x7f8b74073a10 2024-04-20T21:21:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.466+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b74074340 con 0x7f8b74073a10 2024-04-20T21:21:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.467+0000 7f8b6b400640 1 -- 172.21.15.102:0/3200747124 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b6401a070 con 0x7f8b74073a10 2024-04-20T21:21:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.468+0000 7f8b6b400640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8b5006c8f0 0x7f8b5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.468+0000 7f8b6b400640 1 -- 172.21.15.102:0/3200747124 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6877+0+0 (secure 0 0 0) 0x7f8b6408b330 con 0x7f8b74073a10 2024-04-20T21:21:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.469+0000 7f8b6be00640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8b5006c8f0 0x7f8b5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.469+0000 7f8b6be00640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8b5006c8f0 0x7f8b5006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c036000 tx=0x7f8b5c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:05.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.473+0000 7f8b6b400640 1 -- 172.21.15.102:0/3200747124 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8b741536d0 con 0x7f8b74073a10 2024-04-20T21:21:05.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.719+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b741568e0 con 0x7f8b74073a10 2024-04-20T21:21:05.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.720+0000 7f8b6b400640 1 -- 172.21.15.102:0/3200747124 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6862 (secure 0 0 0) 0x7f8b64058d30 con 0x7f8b74073a10 2024-04-20T21:21:05.720 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:05.720 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:21:05.720 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 15 2024-04-20T21:21:05.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.722+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8b5006c8f0 msgr2=0x7f8b5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.722+0000 7f8b7a000640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8b5006c8f0 0x7f8b5006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c036000 tx=0x7f8b5c03b040 comp rx=0 tx=0).stop 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.722+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b74073a10 msgr2=0x7f8b741f17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.722+0000 7f8b7a000640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b74073a10 0x7f8b741f17c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c760 tx=0x7f8b6400cc20 comp rx=0 tx=0).stop 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.723+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 shutdown_connections 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.723+0000 7f8b7a000640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8b5006c8f0 0x7f8b5006edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.723+0000 7f8b7a000640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b7414fa50 0x7f8b740cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.723+0000 7f8b7a000640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b740c9eb0 0x7f8b74155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.723+0000 7f8b7a000640 1 --2- 172.21.15.102:0/3200747124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b74073a10 0x7f8b741f17c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.723+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 >> 172.21.15.102:0/3200747124 conn(0x7f8b740c9390 msgr2=0x7f8b740c88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.723+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 shutdown_connections 2024-04-20T21:21:05.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:05.723+0000 7f8b7a000640 1 -- 172.21.15.102:0/3200747124 wait complete. 2024-04-20T21:21:05.742 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:21:07.742 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:07.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.956+0000 7f488b200640 1 Processor -- start 2024-04-20T21:21:07.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.956+0000 7f488b200640 1 -- start start 2024-04-20T21:21:07.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.957+0000 7f488b200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f488c0c75a0 0x7f488c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488b200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f488c073c70 0x7f488c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488b200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f488c153560 con 0x7f488c153180 2024-04-20T21:21:07.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488b200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f488c0c7a70 con 0x7f488c0c75a0 2024-04-20T21:21:07.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488b200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f488c0c7d30 con 0x7f488c073c70 2024-04-20T21:21:07.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f4889e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f488c153180 0x7f488c158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51580/0 (socket says 172.21.15.102:51580) 2024-04-20T21:21:07.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f4889e00640 1 -- 172.21.15.102:0/1247101920 learned_addr learned my addr 172.21.15.102:0/1247101920 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:07.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488a800640 1 --2- 172.21.15.102:0/1247101920 >> v2:172.21.15.193:3300/0 conn(0x7f488c073c70 0x7f488c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f4889400640 1 --2- 172.21.15.102:0/1247101920 >> v2:172.21.15.102:3300/0 conn(0x7f488c0c75a0 0x7f488c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f4888a00640 1 -- 172.21.15.102:0/1247101920 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557145919 0 0) 0x7f488c153560 con 0x7f488c153180 2024-04-20T21:21:07.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f4888a00640 1 -- 172.21.15.102:0/1247101920 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4870003520 con 0x7f488c153180 2024-04-20T21:21:07.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488a800640 1 -- 172.21.15.102:0/1247101920 >> v2:172.21.15.102:3300/0 conn(0x7f488c0c75a0 msgr2=0x7f488c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:07.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488a800640 1 --2- 172.21.15.102:0/1247101920 >> v2:172.21.15.102:3300/0 conn(0x7f488c0c75a0 0x7f488c0c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488a800640 1 -- 172.21.15.102:0/1247101920 >> v1:172.21.15.193:6789/0 conn(0x7f488c153180 legacy=0x7f488c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.958+0000 7f488a800640 1 -- 172.21.15.102:0/1247101920 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c15c4d0 con 0x7f488c073c70 2024-04-20T21:21:07.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.960+0000 7f488a800640 1 --2- 172.21.15.102:0/1247101920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c073c70 0x7f488c0c6470 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f488c0c7980 tx=0x7f488402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fb4dca9f7349261 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:07.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.960+0000 7f4888a00640 1 -- 172.21.15.102:0/1247101920 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884039070 con 0x7f488c073c70 2024-04-20T21:21:07.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.960+0000 7f4888a00640 1 -- 172.21.15.102:0/1247101920 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488402cc80 con 0x7f488c073c70 2024-04-20T21:21:07.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.960+0000 7f4888a00640 1 -- 172.21.15.102:0/1247101920 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884034640 con 0x7f488c073c70 2024-04-20T21:21:07.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.961+0000 7f488b200640 1 -- 172.21.15.102:0/1247101920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c073c70 msgr2=0x7f488c0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.961+0000 7f488b200640 1 --2- 172.21.15.102:0/1247101920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c073c70 0x7f488c0c6470 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f488c0c7980 tx=0x7f488402bed0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.961+0000 7f488b200640 1 -- 172.21.15.102:0/1247101920 shutdown_connections 2024-04-20T21:21:07.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.961+0000 7f488b200640 1 --2- 172.21.15.102:0/1247101920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c073c70 0x7f488c0c6470 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.961+0000 7f488b200640 1 --2- 172.21.15.102:0/1247101920 >> v2:172.21.15.102:3300/0 conn(0x7f488c0c75a0 0x7f488c0c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.961+0000 7f488b200640 1 -- 172.21.15.102:0/1247101920 >> 172.21.15.102:0/1247101920 conn(0x7f488c0cc820 msgr2=0x7f488c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:07.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.961+0000 7f488b200640 1 -- 172.21.15.102:0/1247101920 shutdown_connections 2024-04-20T21:21:07.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.961+0000 7f488b200640 1 -- 172.21.15.102:0/1247101920 wait complete. 2024-04-20T21:21:07.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.963+0000 7f488b200640 1 Processor -- start 2024-04-20T21:21:07.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.963+0000 7f488b200640 1 -- start start 2024-04-20T21:21:07.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f488b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c073c70 0x7f488c144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f488b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c0c75a0 0x7f488c146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f488b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f488c153180 0x7f488c146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f488b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f488c159550 con 0x7f488c073c70 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f488b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f488c15ca90 con 0x7f488c0c75a0 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f488b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f488c15d290 con 0x7f488c153180 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f4889400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c0c75a0 0x7f488c146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f4889e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c073c70 0x7f488c144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f4889400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c0c75a0 0x7f488c146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:50204/0 (socket says 172.21.15.102:50204) 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f488a800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f488c153180 0x7f488c146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.964+0000 7f4889400640 1 -- 172.21.15.102:0/630539729 learned_addr learned my addr 172.21.15.102:0/630539729 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4889400640 1 -- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f488c153180 msgr2=0x7f488c146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4889400640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f488c153180 0x7f488c146890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4889400640 1 -- 172.21.15.102:0/630539729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c073c70 msgr2=0x7f488c144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4889400640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c073c70 0x7f488c144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4889400640 1 -- 172.21.15.102:0/630539729 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c157680 con 0x7f488c0c75a0 2024-04-20T21:21:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4889400640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c0c75a0 0x7f488c146350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f487800e840 tx=0x7f487800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4882000640 1 -- 172.21.15.102:0/630539729 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878017070 con 0x7f488c0c75a0 2024-04-20T21:21:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4882000640 1 -- 172.21.15.102:0/630539729 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487800c8e0 con 0x7f488c0c75a0 2024-04-20T21:21:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f4882000640 1 -- 172.21.15.102:0/630539729 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487800ccb0 con 0x7f488c0c75a0 2024-04-20T21:21:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c0cac20 con 0x7f488c0c75a0 2024-04-20T21:21:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.965+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c0caea0 con 0x7f488c0c75a0 2024-04-20T21:21:07.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.967+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c04fd60 con 0x7f488c0c75a0 2024-04-20T21:21:07.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.968+0000 7f4882000640 1 -- 172.21.15.102:0/630539729 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f487801a070 con 0x7f488c0c75a0 2024-04-20T21:21:07.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.969+0000 7f4882000640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f486c06c8f0 0x7f486c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.969+0000 7f4882000640 1 -- 172.21.15.102:0/630539729 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6877+0+0 (secure 0 0 0) 0x7f487808c060 con 0x7f488c0c75a0 2024-04-20T21:21:07.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.969+0000 7f4889e00640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f486c06c8f0 0x7f486c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.970+0000 7f4889e00640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f486c06c8f0 0x7f486c06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f487c0011e0 tx=0x7f487c007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:07.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:07.973+0000 7f4882000640 1 -- 172.21.15.102:0/630539729 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f488c0cac20 con 0x7f488c0c75a0 2024-04-20T21:21:08.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.214+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f488c0c3690 con 0x7f488c0c75a0 2024-04-20T21:21:08.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.215+0000 7f4882000640 1 -- 172.21.15.102:0/630539729 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6862 (secure 0 0 0) 0x7f488c157680 con 0x7f488c0c75a0 2024-04-20T21:21:08.214 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:08.214 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:21:08.215 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 15 2024-04-20T21:21:08.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f486c06c8f0 msgr2=0x7f486c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f486c06c8f0 0x7f486c06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f487c0011e0 tx=0x7f487c007d30 comp rx=0 tx=0).stop 2024-04-20T21:21:08.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c0c75a0 msgr2=0x7f488c146350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c0c75a0 0x7f488c146350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f487800e840 tx=0x7f487800ed00 comp rx=0 tx=0).stop 2024-04-20T21:21:08.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 shutdown_connections 2024-04-20T21:21:08.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f488c153180 0x7f488c146890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f486c06c8f0 0x7f486c06edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f488c0c75a0 0x7f488c146350 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 --2- 172.21.15.102:0/630539729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c073c70 0x7f488c144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 >> 172.21.15.102:0/630539729 conn(0x7f488c0cc820 msgr2=0x7f488c0c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:08.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 shutdown_connections 2024-04-20T21:21:08.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:08.218+0000 7f488b200640 1 -- 172.21.15.102:0/630539729 wait complete. 2024-04-20T21:21:08.236 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:21:10.237 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:10.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.445+0000 7f7ac6c00640 1 Processor -- start 2024-04-20T21:21:10.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.445+0000 7f7ac6c00640 1 -- start start 2024-04-20T21:21:10.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.446+0000 7f7ac6c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7ac6c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac0153560 con 0x7f7ac00c75a0 2024-04-20T21:21:10.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7ac6c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac00c7a70 con 0x7f7ac0153180 2024-04-20T21:21:10.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7ac6c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac00c7d30 con 0x7f7ac0073c70 2024-04-20T21:21:10.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7ac4e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7ac00c75a0 0x7f7ac00c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37486/0 (socket says 172.21.15.102:37486) 2024-04-20T21:21:10.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7ac5800640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f7ac0153180 0x7f7ac0158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51584/0 (socket says 172.21.15.102:51584) 2024-04-20T21:21:10.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7ac4e00640 1 -- 172.21.15.102:0/3452492695 learned_addr learned my addr 172.21.15.102:0/3452492695 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:10.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7ac6200640 1 --2- 172.21.15.102:0/3452492695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7abfe00640 1 -- 172.21.15.102:0/3452492695 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135336039 0 0) 0x7f7ac00c7a70 con 0x7f7ac0153180 2024-04-20T21:21:10.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.447+0000 7f7ac6200640 1 -- 172.21.15.102:0/3452492695 >> v1:172.21.15.193:6789/0 conn(0x7f7ac0153180 legacy=0x7f7ac0158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.448+0000 7f7ac6200640 1 -- 172.21.15.102:0/3452492695 >> v1:172.21.15.102:6789/0 conn(0x7f7ac00c75a0 legacy=0x7f7ac00c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.448+0000 7f7ac6200640 1 -- 172.21.15.102:0/3452492695 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac015c4d0 con 0x7f7ac0073c70 2024-04-20T21:21:10.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.449+0000 7f7ac6200640 1 --2- 172.21.15.102:0/3452492695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01647d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009dc0 tx=0x7f7ab802bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf38aa8e85e5611 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:10.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.449+0000 7f7abfe00640 1 -- 172.21.15.102:0/3452492695 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8039070 con 0x7f7ac0073c70 2024-04-20T21:21:10.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.449+0000 7f7abfe00640 1 -- 172.21.15.102:0/3452492695 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab802ca80 con 0x7f7ac0073c70 2024-04-20T21:21:10.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.449+0000 7f7abfe00640 1 -- 172.21.15.102:0/3452492695 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab802ce50 con 0x7f7ac0073c70 2024-04-20T21:21:10.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.449+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3452492695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 msgr2=0x7f7ac01647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.449+0000 7f7ac6c00640 1 --2- 172.21.15.102:0/3452492695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01647d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009dc0 tx=0x7f7ab802bd50 comp rx=0 tx=0).stop 2024-04-20T21:21:10.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.450+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3452492695 shutdown_connections 2024-04-20T21:21:10.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.450+0000 7f7ac6c00640 1 --2- 172.21.15.102:0/3452492695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01647d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.450+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3452492695 >> 172.21.15.102:0/3452492695 conn(0x7f7ac00cc820 msgr2=0x7f7ac00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:10.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.450+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3452492695 shutdown_connections 2024-04-20T21:21:10.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.450+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3452492695 wait complete. 2024-04-20T21:21:10.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.451+0000 7f7ac6c00640 1 Processor -- start 2024-04-20T21:21:10.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.452+0000 7f7ac6c00640 1 -- start start 2024-04-20T21:21:10.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.452+0000 7f7ac6c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01e4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.452+0000 7f7ac6c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac00c75a0 0x7f7ac01e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.452+0000 7f7ac6c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ac0153180 0x7f7ac01e3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.452+0000 7f7ac6c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac0159550 con 0x7f7ac0153180 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.452+0000 7f7ac6c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac015ca90 con 0x7f7ac00c75a0 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.452+0000 7f7ac6c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac015d290 con 0x7f7ac0073c70 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac00c75a0 0x7f7ac01e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01e4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:42856/0 (socket says 172.21.15.102:42856) 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 -- 172.21.15.102:0/3442414304 learned_addr learned my addr 172.21.15.102:0/3442414304 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac6200640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ac0153180 0x7f7ac01e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 -- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac00c75a0 msgr2=0x7f7ac01e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac00c75a0 0x7f7ac01e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 -- 172.21.15.102:0/3442414304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ac0153180 msgr2=0x7f7ac01e3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ac0153180 0x7f7ac01e3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 -- 172.21.15.102:0/3442414304 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac01576f0 con 0x7f7ac0073c70 2024-04-20T21:21:10.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac4e00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac00c75a0 0x7f7ac01e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:10.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.453+0000 7f7ac5800640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01e4800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c4a0 tx=0x7f7ab400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:10.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.454+0000 7f7abd600640 1 -- 172.21.15.102:0/3442414304 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4018070 con 0x7f7ac0073c70 2024-04-20T21:21:10.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.454+0000 7f7abd600640 1 -- 172.21.15.102:0/3442414304 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab40074e0 con 0x7f7ac0073c70 2024-04-20T21:21:10.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.454+0000 7f7abd600640 1 -- 172.21.15.102:0/3442414304 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4004a20 con 0x7f7ac0073c70 2024-04-20T21:21:10.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.454+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac0156d50 con 0x7f7ac0073c70 2024-04-20T21:21:10.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.454+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac0152800 con 0x7f7ac0073c70 2024-04-20T21:21:10.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.455+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac004fd60 con 0x7f7ac0073c70 2024-04-20T21:21:10.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.456+0000 7f7abd600640 1 -- 172.21.15.102:0/3442414304 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ab4004bc0 con 0x7f7ac0073c70 2024-04-20T21:21:10.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.457+0000 7f7abd600640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7aa406c920 0x7f7aa406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.457+0000 7f7abd600640 1 -- 172.21.15.102:0/3442414304 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 6877+0+0 (secure 0 0 0) 0x7f7ab408b200 con 0x7f7ac0073c70 2024-04-20T21:21:10.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.457+0000 7f7ac4e00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7aa406c920 0x7f7aa406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.458+0000 7f7ac4e00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7aa406c920 0x7f7aa406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7ac014d0a0 tx=0x7f7ab0007e00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.462+0000 7f7abd600640 1 -- 172.21.15.102:0/3442414304 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7ab4058ab0 con 0x7f7ac0073c70 2024-04-20T21:21:10.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.708+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ac00c3690 con 0x7f7ac0073c70 2024-04-20T21:21:10.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.709+0000 7f7abd600640 1 -- 172.21.15.102:0/3442414304 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6862 (secure 0 0 0) 0x7f7ab405cdb0 con 0x7f7ac0073c70 2024-04-20T21:21:10.709 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:21:10.709 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 15 2024-04-20T21:21:10.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7aa406c920 msgr2=0x7f7aa406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7aa406c920 0x7f7aa406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7ac014d0a0 tx=0x7f7ab0007e00 comp rx=0 tx=0).stop 2024-04-20T21:21:10.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 msgr2=0x7f7ac01e4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01e4800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c4a0 tx=0x7f7ab400c960 comp rx=0 tx=0).stop 2024-04-20T21:21:10.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 shutdown_connections 2024-04-20T21:21:10.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ac0153180 0x7f7ac01e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7aa406c920 0x7f7aa406ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac00c75a0 0x7f7ac01e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 --2- 172.21.15.102:0/3442414304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac0073c70 0x7f7ac01e4800 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 >> 172.21.15.102:0/3442414304 conn(0x7f7ac00cc820 msgr2=0x7f7ac00c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:10.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 shutdown_connections 2024-04-20T21:21:10.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:10.712+0000 7f7ac6c00640 1 -- 172.21.15.102:0/3442414304 wait complete. 2024-04-20T21:21:10.730 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:21:12.732 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:12.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.941+0000 7fafe2a00640 1 Processor -- start 2024-04-20T21:21:12.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.941+0000 7fafe2a00640 1 -- start start 2024-04-20T21:21:12.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.942+0000 7fafe2a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fafdc1618a0 0x7fafdc163c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.942+0000 7fafe2a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafdc158e80 con 0x7fafdc152170 2024-04-20T21:21:12.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.942+0000 7fafe2a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafdc0c91c0 con 0x7fafdc0c8db0 2024-04-20T21:21:12.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.942+0000 7fafe2a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fafdc0c6dc0 con 0x7fafdc1618a0 2024-04-20T21:21:12.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafd3400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fafdc0c8db0 0x7fafdc070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51586/0 (socket says 172.21.15.102:51586) 2024-04-20T21:21:12.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafd3400640 1 -- 172.21.15.102:0/4193802020 learned_addr learned my addr 172.21.15.102:0/4193802020 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:12.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafe0a00640 1 --2- 172.21.15.102:0/4193802020 >> v2:172.21.15.102:3300/0 conn(0x7fafdc1618a0 0x7fafdc163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafdb400640 1 -- 172.21.15.102:0/4193802020 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694902482 0 0) 0x7fafdc0c91c0 con 0x7fafdc0c8db0 2024-04-20T21:21:12.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafdb400640 1 -- 172.21.15.102:0/4193802020 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc0005730 con 0x7fafdc0c8db0 2024-04-20T21:21:12.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafdb400640 1 -- 172.21.15.102:0/4193802020 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3433689384 0 0) 0x7fafdc158e80 con 0x7fafdc152170 2024-04-20T21:21:12.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafdb400640 1 -- 172.21.15.102:0/4193802020 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafdc0c91c0 con 0x7fafdc152170 2024-04-20T21:21:12.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafe0a00640 1 -- 172.21.15.102:0/4193802020 >> v1:172.21.15.193:6789/0 conn(0x7fafdc0c8db0 legacy=0x7fafdc070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafe0a00640 1 -- 172.21.15.102:0/4193802020 >> v1:172.21.15.102:6789/0 conn(0x7fafdc152170 legacy=0x7fafdc158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.943+0000 7fafe0a00640 1 -- 172.21.15.102:0/4193802020 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc071f70 con 0x7fafdc1618a0 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafe0a00640 1 --2- 172.21.15.102:0/4193802020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc1618a0 0x7fafdc163c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009d90 tx=0x7fafd402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa55779c78ec64a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafdb400640 1 -- 172.21.15.102:0/4193802020 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4039070 con 0x7fafdc1618a0 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafdb400640 1 -- 172.21.15.102:0/4193802020 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd402cd00 con 0x7fafdc1618a0 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafdb400640 1 -- 172.21.15.102:0/4193802020 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4034640 con 0x7fafdc1618a0 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafe2a00640 1 -- 172.21.15.102:0/4193802020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc1618a0 msgr2=0x7fafdc163c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafe2a00640 1 --2- 172.21.15.102:0/4193802020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc1618a0 0x7fafdc163c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009d90 tx=0x7fafd402bf50 comp rx=0 tx=0).stop 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafe2a00640 1 -- 172.21.15.102:0/4193802020 shutdown_connections 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafe2a00640 1 --2- 172.21.15.102:0/4193802020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc1618a0 0x7fafdc163c90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.945+0000 7fafe2a00640 1 -- 172.21.15.102:0/4193802020 >> 172.21.15.102:0/4193802020 conn(0x7fafdc0c9390 msgr2=0x7fafdc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:12.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.946+0000 7fafe2a00640 1 -- 172.21.15.102:0/4193802020 shutdown_connections 2024-04-20T21:21:12.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.946+0000 7fafe2a00640 1 -- 172.21.15.102:0/4193802020 wait complete. 2024-04-20T21:21:12.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.947+0000 7fafe2a00640 1 Processor -- start 2024-04-20T21:21:12.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.947+0000 7fafe2a00640 1 -- start start 2024-04-20T21:21:12.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafe2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafdc0c8db0 0x7fafdc156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafe2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc152170 0x7fafdc15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafe2a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafdc1618a0 0x7fafdc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafe2a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc0724b0 con 0x7fafdc152170 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafe2a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc072210 con 0x7fafdc0c8db0 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafe2a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fafdc158250 con 0x7fafdc1618a0 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafd3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc152170 0x7fafdc15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafd3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc152170 0x7fafdc15a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34020/0 (socket says 172.21.15.102:34020) 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.948+0000 7fafd3400640 1 -- 172.21.15.102:0/1072536251 learned_addr learned my addr 172.21.15.102:0/1072536251 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafe0a00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafdc1618a0 0x7fafdc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafdbe00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafdc0c8db0 0x7fafdc156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafd3400640 1 -- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafdc1618a0 msgr2=0x7fafdc1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafd3400640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafdc1618a0 0x7fafdc1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafd3400640 1 -- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafdc0c8db0 msgr2=0x7fafdc156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafd3400640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafdc0c8db0 0x7fafdc156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafd3400640 1 -- 172.21.15.102:0/1072536251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc14cb80 con 0x7fafdc152170 2024-04-20T21:21:12.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafd3400640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc152170 0x7fafdc15a980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c410 tx=0x7fafc400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.949+0000 7fafd8c00640 1 -- 172.21.15.102:0/1072536251 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4017070 con 0x7fafdc152170 2024-04-20T21:21:12.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.950+0000 7fafd8c00640 1 -- 172.21.15.102:0/1072536251 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc4005ce0 con 0x7fafdc152170 2024-04-20T21:21:12.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.950+0000 7fafd8c00640 1 -- 172.21.15.102:0/1072536251 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc40072d0 con 0x7fafdc152170 2024-04-20T21:21:12.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.950+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc0cc1b0 con 0x7fafdc152170 2024-04-20T21:21:12.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.950+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1eff60 con 0x7fafdc152170 2024-04-20T21:21:12.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.951+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc074340 con 0x7fafdc152170 2024-04-20T21:21:12.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.952+0000 7fafd8c00640 1 -- 172.21.15.102:0/1072536251 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafc401a070 con 0x7fafdc152170 2024-04-20T21:21:12.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.953+0000 7fafd8c00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafb806c920 0x7fafb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.953+0000 7fafd8c00640 1 -- 172.21.15.102:0/1072536251 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6877+0+0 (secure 0 0 0) 0x7fafc408b3c0 con 0x7fafdc152170 2024-04-20T21:21:12.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.953+0000 7fafdbe00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafb806c920 0x7fafb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.954+0000 7fafdbe00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafb806c920 0x7fafb806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fafcc007fd0 tx=0x7fafcc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:12.957+0000 7fafd8c00640 1 -- 172.21.15.102:0/1072536251 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fafc4058be0 con 0x7fafdc152170 2024-04-20T21:21:13.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.198+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafdc0c3460 con 0x7fafdc152170 2024-04-20T21:21:13.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.199+0000 7fafd8c00640 1 -- 172.21.15.102:0/1072536251 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6862 (secure 0 0 0) 0x7fafc405cee0 con 0x7fafdc152170 2024-04-20T21:21:13.199 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:21:13.199 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 15 2024-04-20T21:21:13.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafb806c920 msgr2=0x7fafb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafb806c920 0x7fafb806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fafcc007fd0 tx=0x7fafcc009040 comp rx=0 tx=0).stop 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc152170 msgr2=0x7fafdc15a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc152170 0x7fafdc15a980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c410 tx=0x7fafc400c8d0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 shutdown_connections 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafdc1618a0 0x7fafdc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc152170 0x7fafdc15a980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafb806c920 0x7fafb806ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 --2- 172.21.15.102:0/1072536251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafdc0c8db0 0x7fafdc156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 >> 172.21.15.102:0/1072536251 conn(0x7fafdc0c9390 msgr2=0x7fafdc06e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 shutdown_connections 2024-04-20T21:21:13.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:13.202+0000 7fafe2a00640 1 -- 172.21.15.102:0/1072536251 wait complete. 2024-04-20T21:21:13.220 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:21:15.221 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:15.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.429+0000 7f36cd000640 1 Processor -- start 2024-04-20T21:21:15.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.429+0000 7f36cd000640 1 -- start start 2024-04-20T21:21:15.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36cd000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f36c80cc390 0x7f36c806b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:15.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36cd000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f36c80b41e0 0x7f36c8070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:15.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36cd000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c80cb440 con 0x7f36c8155560 2024-04-20T21:21:15.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36cd000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f36c80ca140 con 0x7f36c80cc390 2024-04-20T21:21:15.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36cd000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f36c80c9de0 con 0x7f36c80b41e0 2024-04-20T21:21:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36c7400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f36c8155560 0x7f36c8073890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51592/0 (socket says 172.21.15.102:51592) 2024-04-20T21:21:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36c7400640 1 -- 172.21.15.102:0/458030295 learned_addr learned my addr 172.21.15.102:0/458030295 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36c7e00640 1 --2- 172.21.15.102:0/458030295 >> v2:172.21.15.193:3300/0 conn(0x7f36c80b41e0 0x7f36c8070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36c6a00640 1 --2- 172.21.15.102:0/458030295 >> v2:172.21.15.102:3300/0 conn(0x7f36c80cc390 0x7f36c806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36c6000640 1 -- 172.21.15.102:0/458030295 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3454430598 0 0) 0x7f36c80cb440 con 0x7f36c8155560 2024-04-20T21:21:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36c7e00640 1 -- 172.21.15.102:0/458030295 >> v2:172.21.15.102:3300/0 conn(0x7f36c80cc390 msgr2=0x7f36c806b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36c7e00640 1 --2- 172.21.15.102:0/458030295 >> v2:172.21.15.102:3300/0 conn(0x7f36c80cc390 0x7f36c806b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.431+0000 7f36c7e00640 1 -- 172.21.15.102:0/458030295 >> v1:172.21.15.193:6789/0 conn(0x7f36c8155560 legacy=0x7f36c8073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:15.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.432+0000 7f36c7e00640 1 -- 172.21.15.102:0/458030295 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c80c96a0 con 0x7f36c80b41e0 2024-04-20T21:21:15.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.433+0000 7f36c7e00640 1 --2- 172.21.15.102:0/458030295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c80b41e0 0x7f36c8070d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f36c80b39f0 tx=0x7f36c002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d8020ca17dab55 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:15.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.433+0000 7f36c6000640 1 -- 172.21.15.102:0/458030295 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0039070 con 0x7f36c80b41e0 2024-04-20T21:21:15.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.433+0000 7f36c6000640 1 -- 172.21.15.102:0/458030295 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c002cd00 con 0x7f36c80b41e0 2024-04-20T21:21:15.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.433+0000 7f36c6000640 1 -- 172.21.15.102:0/458030295 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0034660 con 0x7f36c80b41e0 2024-04-20T21:21:15.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.433+0000 7f36cd000640 1 -- 172.21.15.102:0/458030295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c80b41e0 msgr2=0x7f36c8070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:15.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.433+0000 7f36cd000640 1 --2- 172.21.15.102:0/458030295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c80b41e0 0x7f36c8070d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f36c80b39f0 tx=0x7f36c002bf50 comp rx=0 tx=0).stop 2024-04-20T21:21:15.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.434+0000 7f36cd000640 1 -- 172.21.15.102:0/458030295 shutdown_connections 2024-04-20T21:21:15.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.434+0000 7f36cd000640 1 --2- 172.21.15.102:0/458030295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c80b41e0 0x7f36c8070d00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.434+0000 7f36cd000640 1 --2- 172.21.15.102:0/458030295 >> v2:172.21.15.102:3300/0 conn(0x7f36c80cc390 0x7f36c806b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.434+0000 7f36cd000640 1 -- 172.21.15.102:0/458030295 >> 172.21.15.102:0/458030295 conn(0x7f36c8075080 msgr2=0x7f36c8075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:15.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.434+0000 7f36cd000640 1 -- 172.21.15.102:0/458030295 shutdown_connections 2024-04-20T21:21:15.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.434+0000 7f36cd000640 1 -- 172.21.15.102:0/458030295 wait complete. 2024-04-20T21:21:15.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.435+0000 7f36cd000640 1 Processor -- start 2024-04-20T21:21:15.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.436+0000 7f36cd000640 1 -- start start 2024-04-20T21:21:15.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36cd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36c80b41e0 0x7f36c80c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:15.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36cd000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36c80cc390 0x7f36c80c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:15.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36cd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c8155560 0x7f36c80c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:15.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36cd000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f36c80ca3e0 con 0x7f36c80b41e0 2024-04-20T21:21:15.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36cd000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36c80c9980 con 0x7f36c8155560 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36cd000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f36c806d1e0 con 0x7f36c80cc390 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c8155560 0x7f36c80c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c8155560 0x7f36c80c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:50254/0 (socket says 172.21.15.102:50254) 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36c7e00640 1 -- 172.21.15.102:0/1826508949 learned_addr learned my addr 172.21.15.102:0/1826508949 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36c7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36c80b41e0 0x7f36c80c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.437+0000 7f36c6a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36c80cc390 0x7f36c80c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36c7e00640 1 -- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36c80cc390 msgr2=0x7f36c80c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36c7e00640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36c80cc390 0x7f36c80c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36c7e00640 1 -- 172.21.15.102:0/1826508949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36c80b41e0 msgr2=0x7f36c80c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36c7e00640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36c80b41e0 0x7f36c80c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36c7e00640 1 -- 172.21.15.102:0/1826508949 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c81f1290 con 0x7f36c8155560 2024-04-20T21:21:15.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36c7e00640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c8155560 0x7f36c80c8440 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36c002c560 tx=0x7f36c0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:15.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36bf400640 1 -- 172.21.15.102:0/1826508949 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0039050 con 0x7f36c8155560 2024-04-20T21:21:15.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36bf400640 1 -- 172.21.15.102:0/1826508949 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c0045070 con 0x7f36c8155560 2024-04-20T21:21:15.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36bf400640 1 -- 172.21.15.102:0/1826508949 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0009450 con 0x7f36c8155560 2024-04-20T21:21:15.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.438+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c81f1570 con 0x7f36c8155560 2024-04-20T21:21:15.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.439+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c81f1b30 con 0x7f36c8155560 2024-04-20T21:21:15.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.439+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c80cbc00 con 0x7f36c8155560 2024-04-20T21:21:15.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.440+0000 7f36bf400640 1 -- 172.21.15.102:0/1826508949 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36c003e850 con 0x7f36c8155560 2024-04-20T21:21:15.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.441+0000 7f36bf400640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f36a806c8f0 0x7f36a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:15.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.441+0000 7f36bf400640 1 -- 172.21.15.102:0/1826508949 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6877+0+0 (secure 0 0 0) 0x7f36c00aedd0 con 0x7f36c8155560 2024-04-20T21:21:15.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.442+0000 7f36c7400640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f36a806c8f0 0x7f36a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:15.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.442+0000 7f36c7400640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f36a806c8f0 0x7f36a806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f36b8007fd0 tx=0x7f36b8007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:15.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.446+0000 7f36bf400640 1 -- 172.21.15.102:0/1826508949 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f36c80cbc00 con 0x7f36c8155560 2024-04-20T21:21:15.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.705+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36c804fd60 con 0x7f36c8155560 2024-04-20T21:21:15.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.706+0000 7f36bf400640 1 -- 172.21.15.102:0/1826508949 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6862 (secure 0 0 0) 0x7f36c007c710 con 0x7f36c8155560 2024-04-20T21:21:15.705 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:21:15.706 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 15 2024-04-20T21:21:15.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f36a806c8f0 msgr2=0x7f36a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:15.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f36a806c8f0 0x7f36a806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f36b8007fd0 tx=0x7f36b8007de0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c8155560 msgr2=0x7f36c80c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:15.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c8155560 0x7f36c80c8440 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36c002c560 tx=0x7f36c0004a40 comp rx=0 tx=0).stop 2024-04-20T21:21:15.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 shutdown_connections 2024-04-20T21:21:15.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f36a806c8f0 0x7f36a806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36c8155560 0x7f36c80c8440 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36c80cc390 0x7f36c80c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 --2- 172.21.15.102:0/1826508949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36c80b41e0 0x7f36c80c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.708+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 >> 172.21.15.102:0/1826508949 conn(0x7f36c8075080 msgr2=0x7f36c8075460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:15.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.709+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 shutdown_connections 2024-04-20T21:21:15.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:15.709+0000 7f36cd000640 1 -- 172.21.15.102:0/1826508949 wait complete. 2024-04-20T21:21:15.728 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:21:17.729 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:17.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.941+0000 7f102de00640 1 Processor -- start 2024-04-20T21:21:17.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.941+0000 7f102de00640 1 -- start start 2024-04-20T21:21:17.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.942+0000 7f102de00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f102814fa50 0x7f10280c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:17.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.942+0000 7f102de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10280c9eb0 0x7f1028158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:17.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.942+0000 7f102de00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1028073a10 0x7f10280c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:17.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.942+0000 7f102de00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1028159500 con 0x7f102814fa50 2024-04-20T21:21:17.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.942+0000 7f102de00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f10281536d0 con 0x7f1028073a10 2024-04-20T21:21:17.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.942+0000 7f102de00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f10280c76c0 con 0x7f10280c9eb0 2024-04-20T21:21:17.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1026a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f102814fa50 0x7f10280c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:17.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1027400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1028073a10 0x7f10280c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:17.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1026a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f102814fa50 0x7f10280c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45292/0 (socket says 172.21.15.102:45292) 2024-04-20T21:21:17.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f101e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10280c9eb0 0x7f1028158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:17.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1026a00640 1 -- 172.21.15.102:0/3469157998 learned_addr learned my addr 172.21.15.102:0/3469157998 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1026a00640 1 -- 172.21.15.102:0/3469157998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10280c9eb0 msgr2=0x7f1028158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1026a00640 1 --2- 172.21.15.102:0/3469157998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10280c9eb0 0x7f1028158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1026a00640 1 -- 172.21.15.102:0/3469157998 >> v2:172.21.15.193:3300/0 conn(0x7f1028073a10 msgr2=0x7f10280c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1026a00640 1 --2- 172.21.15.102:0/3469157998 >> v2:172.21.15.193:3300/0 conn(0x7f1028073a10 0x7f10280c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1026a00640 1 -- 172.21.15.102:0/3469157998 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10280c7960 con 0x7f102814fa50 2024-04-20T21:21:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.943+0000 7f1027400640 1 --2- 172.21.15.102:0/3469157998 >> v2:172.21.15.193:3300/0 conn(0x7f1028073a10 0x7f10280c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.944+0000 7f1026a00640 1 --2- 172.21.15.102:0/3469157998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f102814fa50 0x7f10280c4830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f102806c190 tx=0x7f101802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136125005007e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.945+0000 7f1026000640 1 -- 172.21.15.102:0/3469157998 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018034070 con 0x7f102814fa50 2024-04-20T21:21:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.945+0000 7f1026000640 1 -- 172.21.15.102:0/3469157998 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1018030860 con 0x7f102814fa50 2024-04-20T21:21:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.945+0000 7f1026000640 1 -- 172.21.15.102:0/3469157998 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018030c30 con 0x7f102814fa50 2024-04-20T21:21:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.945+0000 7f102de00640 1 -- 172.21.15.102:0/3469157998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f102814fa50 msgr2=0x7f10280c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.945+0000 7f102de00640 1 --2- 172.21.15.102:0/3469157998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f102814fa50 0x7f10280c4830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f102806c190 tx=0x7f101802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:21:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.946+0000 7f102de00640 1 -- 172.21.15.102:0/3469157998 shutdown_connections 2024-04-20T21:21:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.946+0000 7f102de00640 1 --2- 172.21.15.102:0/3469157998 >> v2:172.21.15.193:3300/0 conn(0x7f1028073a10 0x7f10280c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.946+0000 7f102de00640 1 --2- 172.21.15.102:0/3469157998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10280c9eb0 0x7f1028158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.946+0000 7f102de00640 1 --2- 172.21.15.102:0/3469157998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f102814fa50 0x7f10280c4830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.946+0000 7f102de00640 1 -- 172.21.15.102:0/3469157998 >> 172.21.15.102:0/3469157998 conn(0x7f10280c9390 msgr2=0x7f10280c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.946+0000 7f102de00640 1 -- 172.21.15.102:0/3469157998 shutdown_connections 2024-04-20T21:21:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.946+0000 7f102de00640 1 -- 172.21.15.102:0/3469157998 wait complete. 2024-04-20T21:21:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.947+0000 7f102de00640 1 Processor -- start 2024-04-20T21:21:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.948+0000 7f102de00640 1 -- start start 2024-04-20T21:21:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.948+0000 7f102de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028073a10 0x7f1028071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f102de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10280c9eb0 0x7f1028154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f102de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f102814fa50 0x7f10280cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f102de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f10280c7c00 con 0x7f10280c9eb0 2024-04-20T21:21:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f102de00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10280c3d90 con 0x7f1028073a10 2024-04-20T21:21:17.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f102de00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f10280c4030 con 0x7f102814fa50 2024-04-20T21:21:17.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f101e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10280c9eb0 0x7f1028154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:17.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f1026a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028073a10 0x7f1028071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:17.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f101e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10280c9eb0 0x7f1028154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45302/0 (socket says 172.21.15.102:45302) 2024-04-20T21:21:17.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f101e000640 1 -- 172.21.15.102:0/3869432998 learned_addr learned my addr 172.21.15.102:0/3869432998 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:17.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f1027400640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f102814fa50 0x7f10280cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f101e000640 1 -- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f102814fa50 msgr2=0x7f10280cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f101e000640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f102814fa50 0x7f10280cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f101e000640 1 -- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028073a10 msgr2=0x7f1028071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f101e000640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028073a10 0x7f1028071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.949+0000 7f101e000640 1 -- 172.21.15.102:0/3869432998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10281f4e50 con 0x7f10280c9eb0 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.950+0000 7f101e000640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10280c9eb0 0x7f1028154780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f101000e9e0 tx=0x7f101000ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.950+0000 7f101fe00640 1 -- 172.21.15.102:0/3869432998 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010017070 con 0x7f10280c9eb0 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.950+0000 7f101fe00640 1 -- 172.21.15.102:0/3869432998 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101000c940 con 0x7f10280c9eb0 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.950+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281f4b70 con 0x7f10280c9eb0 2024-04-20T21:21:17.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.950+0000 7f101fe00640 1 -- 172.21.15.102:0/3869432998 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101000cd30 con 0x7f10280c9eb0 2024-04-20T21:21:17.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.950+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1028153ca0 con 0x7f10280c9eb0 2024-04-20T21:21:17.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.951+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1028074340 con 0x7f10280c9eb0 2024-04-20T21:21:17.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.952+0000 7f101fe00640 1 -- 172.21.15.102:0/3869432998 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f101001a070 con 0x7f10280c9eb0 2024-04-20T21:21:17.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.953+0000 7f101fe00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f100406c920 0x7f100406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:17.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.953+0000 7f101fe00640 1 -- 172.21.15.102:0/3869432998 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6877+0+0 (secure 0 0 0) 0x7f101008bfc0 con 0x7f10280c9eb0 2024-04-20T21:21:17.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.953+0000 7f1026a00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f100406c920 0x7f100406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:17.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.954+0000 7f1026a00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f100406c920 0x7f100406ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1018036000 tx=0x7f101803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:17.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:17.958+0000 7f101fe00640 1 -- 172.21.15.102:0/3869432998 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f10100597e0 con 0x7f10280c9eb0 2024-04-20T21:21:18.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.197+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10280c9980 con 0x7f10280c9eb0 2024-04-20T21:21:18.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.199+0000 7f101fe00640 1 -- 172.21.15.102:0/3869432998 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6862 (secure 0 0 0) 0x7f101005dae0 con 0x7f10280c9eb0 2024-04-20T21:21:18.198 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:18.198 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:20:03.569583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":52,"addr":"172.21.15.102:6833/2795763369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2795763369},{"type":"v1","addr":"172.21.15.102:6833","nonce":2795763369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-20T21:21:18.199 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 15 2024-04-20T21:21:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f100406c920 msgr2=0x7f100406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f100406c920 0x7f100406ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1018036000 tx=0x7f101803b040 comp rx=0 tx=0).stop 2024-04-20T21:21:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10280c9eb0 msgr2=0x7f1028154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10280c9eb0 0x7f1028154780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f101000e9e0 tx=0x7f101000ede0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 shutdown_connections 2024-04-20T21:21:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f102814fa50 0x7f10280cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10280c9eb0 0x7f1028154780 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f100406c920 0x7f100406ede0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 --2- 172.21.15.102:0/3869432998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028073a10 0x7f1028071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.201+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 >> 172.21.15.102:0/3869432998 conn(0x7f10280c9390 msgr2=0x7f102814e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.202+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 shutdown_connections 2024-04-20T21:21:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:18.202+0000 7f102de00640 1 -- 172.21.15.102:0/3869432998 wait complete. 2024-04-20T21:21:18.220 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:21:20.222 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:20.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.431+0000 7f8974e00640 1 Processor -- start 2024-04-20T21:21:20.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.432+0000 7f8974e00640 1 -- start start 2024-04-20T21:21:20.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.433+0000 7f8974e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8970073a10 0x7f89700c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.433+0000 7f8974e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89701536d0 con 0x7f89700c9eb0 2024-04-20T21:21:20.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.433+0000 7f8974e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89700c76c0 con 0x7f897014fa50 2024-04-20T21:21:20.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.433+0000 7f8974e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f89700c7960 con 0x7f8970073a10 2024-04-20T21:21:20.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.433+0000 7f8967e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f89700c9eb0 0x7f8970161930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34830/0 (socket says 172.21.15.102:34830) 2024-04-20T21:21:20.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.433+0000 7f8967e00640 1 -- 172.21.15.102:0/3807796902 learned_addr learned my addr 172.21.15.102:0/3807796902 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:20.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.434+0000 7f896e800640 1 --2- 172.21.15.102:0/3807796902 >> v2:172.21.15.102:3300/0 conn(0x7f8970073a10 0x7f89700c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.434+0000 7f896d400640 1 -- 172.21.15.102:0/3807796902 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945734010 0 0) 0x7f8958001110 con 0x7f89700c9eb0 2024-04-20T21:21:20.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.434+0000 7f896d400640 1 -- 172.21.15.102:0/3807796902 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8954003520 con 0x7f89700c9eb0 2024-04-20T21:21:20.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.434+0000 7f896d400640 1 -- 172.21.15.102:0/3807796902 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3369432402 0 0) 0x7f89700c76c0 con 0x7f897014fa50 2024-04-20T21:21:20.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.434+0000 7f896d400640 1 -- 172.21.15.102:0/3807796902 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8958001110 con 0x7f897014fa50 2024-04-20T21:21:20.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.434+0000 7f896e800640 1 -- 172.21.15.102:0/3807796902 >> v1:172.21.15.193:6789/0 conn(0x7f897014fa50 legacy=0x7f89700c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.434+0000 7f896e800640 1 -- 172.21.15.102:0/3807796902 >> v1:172.21.15.102:6789/0 conn(0x7f89700c9eb0 legacy=0x7f8970161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.434+0000 7f896e800640 1 -- 172.21.15.102:0/3807796902 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89700c3d90 con 0x7f8970073a10 2024-04-20T21:21:20.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.436+0000 7f896e800640 1 --2- 172.21.15.102:0/3807796902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8970073a10 0x7f89700c4830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89700c7890 tx=0x7f896802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee8518e10e5491d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:20.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.436+0000 7f896d400640 1 -- 172.21.15.102:0/3807796902 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968039070 con 0x7f8970073a10 2024-04-20T21:21:20.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.436+0000 7f896d400640 1 -- 172.21.15.102:0/3807796902 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f896802cd80 con 0x7f8970073a10 2024-04-20T21:21:20.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.436+0000 7f896d400640 1 -- 172.21.15.102:0/3807796902 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968034660 con 0x7f8970073a10 2024-04-20T21:21:20.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.437+0000 7f8974e00640 1 -- 172.21.15.102:0/3807796902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8970073a10 msgr2=0x7f89700c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.437+0000 7f8974e00640 1 --2- 172.21.15.102:0/3807796902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8970073a10 0x7f89700c4830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89700c7890 tx=0x7f896802bfd0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.437+0000 7f8974e00640 1 -- 172.21.15.102:0/3807796902 shutdown_connections 2024-04-20T21:21:20.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.437+0000 7f8974e00640 1 --2- 172.21.15.102:0/3807796902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8970073a10 0x7f89700c4830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.437+0000 7f8974e00640 1 -- 172.21.15.102:0/3807796902 >> 172.21.15.102:0/3807796902 conn(0x7f89700c9390 msgr2=0x7f89700c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:20.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.437+0000 7f8974e00640 1 -- 172.21.15.102:0/3807796902 shutdown_connections 2024-04-20T21:21:20.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.437+0000 7f8974e00640 1 -- 172.21.15.102:0/3807796902 wait complete. 2024-04-20T21:21:20.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.438+0000 7f8974e00640 1 Processor -- start 2024-04-20T21:21:20.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.439+0000 7f8974e00640 1 -- start start 2024-04-20T21:21:20.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f8974e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8970073a10 0x7f89701f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f8974e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89700c9eb0 0x7f8970154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f8974e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f897014fa50 0x7f8970158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f8974e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f89700c4030 con 0x7f89700c9eb0 2024-04-20T21:21:20.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f8974e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89700c7c00 con 0x7f8970073a10 2024-04-20T21:21:20.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f8974e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f89700c4310 con 0x7f897014fa50 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f8967e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89700c9eb0 0x7f8970154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f896de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8970073a10 0x7f89701f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f896e800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f897014fa50 0x7f8970158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f896e800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f897014fa50 0x7f8970158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56358/0 (socket says 172.21.15.102:56358) 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f896de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8970073a10 0x7f89701f3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41824/0 (socket says 172.21.15.102:41824) 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.440+0000 7f896e800640 1 -- 172.21.15.102:0/3698274787 learned_addr learned my addr 172.21.15.102:0/3698274787 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f896e800640 1 -- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8970073a10 msgr2=0x7f89701f3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f896e800640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8970073a10 0x7f89701f3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f896e800640 1 -- 172.21.15.102:0/3698274787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89700c9eb0 msgr2=0x7f8970154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f896e800640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89700c9eb0 0x7f8970154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f896e800640 1 -- 172.21.15.102:0/3698274787 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89701f3940 con 0x7f897014fa50 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f8967e00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89700c9eb0 0x7f8970154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f896de00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8970073a10 0x7f89701f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:20.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f896e800640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f897014fa50 0x7f8970158480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f896802c5e0 tx=0x7f8968004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:20.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f8966000640 1 -- 172.21.15.102:0/3698274787 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968039050 con 0x7f897014fa50 2024-04-20T21:21:20.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f8966000640 1 -- 172.21.15.102:0/3698274787 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8968045070 con 0x7f897014fa50 2024-04-20T21:21:20.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f8966000640 1 -- 172.21.15.102:0/3698274787 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968009450 con 0x7f897014fa50 2024-04-20T21:21:20.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89700c9b00 con 0x7f897014fa50 2024-04-20T21:21:20.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.441+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f897015cfc0 con 0x7f897014fa50 2024-04-20T21:21:20.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.443+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8970073800 con 0x7f897014fa50 2024-04-20T21:21:20.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.443+0000 7f8966000640 1 -- 172.21.15.102:0/3698274787 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f896803e850 con 0x7f897014fa50 2024-04-20T21:21:20.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.444+0000 7f8966000640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f894c06c8f0 0x7f894c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.444+0000 7f8966000640 1 -- 172.21.15.102:0/3698274787 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 6877+0+0 (secure 0 0 0) 0x7f89680aec90 con 0x7f897014fa50 2024-04-20T21:21:20.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.445+0000 7f896de00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f894c06c8f0 0x7f894c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.445+0000 7f896de00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f894c06c8f0 0x7f894c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8960007fd0 tx=0x7f8960009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:20.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.449+0000 7f8966000640 1 -- 172.21.15.102:0/3698274787 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8970073800 con 0x7f897014fa50 2024-04-20T21:21:20.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.695+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8970074340 con 0x7f897014fa50 2024-04-20T21:21:20.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.697+0000 7f8966000640 1 -- 172.21.15.102:0/3698274787 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6026 (secure 0 0 0) 0x7f896807c6a0 con 0x7f897014fa50 2024-04-20T21:21:20.697 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:20.697 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:20.486187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:20.697 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 16 2024-04-20T21:21:20.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.699+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f894c06c8f0 msgr2=0x7f894c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.699+0000 7f8974e00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f894c06c8f0 0x7f894c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8960007fd0 tx=0x7f8960009040 comp rx=0 tx=0).stop 2024-04-20T21:21:20.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.699+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f897014fa50 msgr2=0x7f8970158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.699+0000 7f8974e00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f897014fa50 0x7f8970158480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f896802c5e0 tx=0x7f8968004a40 comp rx=0 tx=0).stop 2024-04-20T21:21:20.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.700+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 shutdown_connections 2024-04-20T21:21:20.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.700+0000 7f8974e00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f897014fa50 0x7f8970158480 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.700+0000 7f8974e00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89700c9eb0 0x7f8970154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.700+0000 7f8974e00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f894c06c8f0 0x7f894c06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.700+0000 7f8974e00640 1 --2- 172.21.15.102:0/3698274787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8970073a10 0x7f89701f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.700+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 >> 172.21.15.102:0/3698274787 conn(0x7f89700c9390 msgr2=0x7f897014dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:20.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.700+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 shutdown_connections 2024-04-20T21:21:20.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.700+0000 7f8974e00640 1 -- 172.21.15.102:0/3698274787 wait complete. 2024-04-20T21:21:20.718 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-04-20T21:21:20.718 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:21:20.718 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:20.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.911+0000 7f6986c00640 1 Processor -- start 2024-04-20T21:21:20.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.911+0000 7f6986c00640 1 -- start start 2024-04-20T21:21:20.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f6986c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f698015ea20 0x7f698006d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f6986c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f69800710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f6986c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69800c8f20 con 0x7f698015ee00 2024-04-20T21:21:20.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f6986c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f698006cb80 con 0x7f698015ea20 2024-04-20T21:21:20.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f6986c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f698006ce60 con 0x7f698015f2b0 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f697fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f698015ea20 0x7f698006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f697fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f698015ea20 0x7f698006d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45332/0 (socket says 172.21.15.102:45332) 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f697fe00640 1 -- 172.21.15.102:0/2338238780 learned_addr learned my addr 172.21.15.102:0/2338238780 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.913+0000 7f6984c00640 1 --2- 172.21.15.102:0/2338238780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f69800710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.914+0000 7f697ea00640 1 -- 172.21.15.102:0/2338238780 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2969504156 0 0) 0x7f69800c8f20 con 0x7f698015ee00 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.914+0000 7f697ea00640 1 -- 172.21.15.102:0/2338238780 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6964003520 con 0x7f698015ee00 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.914+0000 7f6984c00640 1 -- 172.21.15.102:0/2338238780 >> v2:172.21.15.102:3300/0 conn(0x7f698015ea20 msgr2=0x7f698006d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.914+0000 7f6984c00640 1 --2- 172.21.15.102:0/2338238780 >> v2:172.21.15.102:3300/0 conn(0x7f698015ea20 0x7f698006d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.914+0000 7f6984c00640 1 -- 172.21.15.102:0/2338238780 >> v1:172.21.15.193:6789/0 conn(0x7f698015ee00 legacy=0x7f69800c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.914+0000 7f6984c00640 1 -- 172.21.15.102:0/2338238780 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698006d140 con 0x7f698015f2b0 2024-04-20T21:21:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.914+0000 7f697fe00640 1 --2- 172.21.15.102:0/2338238780 >> v2:172.21.15.102:3300/0 conn(0x7f698015ea20 0x7f698006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:20.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.915+0000 7f6984c00640 1 --2- 172.21.15.102:0/2338238780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f69800710a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6978009d90 tx=0x7f697802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1bb2bed6798a2c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:20.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.915+0000 7f697ea00640 1 -- 172.21.15.102:0/2338238780 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978039070 con 0x7f698015f2b0 2024-04-20T21:21:20.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.915+0000 7f697ea00640 1 -- 172.21.15.102:0/2338238780 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697802cce0 con 0x7f698015f2b0 2024-04-20T21:21:20.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.915+0000 7f697ea00640 1 -- 172.21.15.102:0/2338238780 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978034660 con 0x7f698015f2b0 2024-04-20T21:21:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.916+0000 7f6986c00640 1 -- 172.21.15.102:0/2338238780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 msgr2=0x7f69800710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.916+0000 7f6986c00640 1 --2- 172.21.15.102:0/2338238780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f69800710a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6978009d90 tx=0x7f697802bf30 comp rx=0 tx=0).stop 2024-04-20T21:21:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.916+0000 7f6986c00640 1 -- 172.21.15.102:0/2338238780 shutdown_connections 2024-04-20T21:21:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.916+0000 7f6986c00640 1 --2- 172.21.15.102:0/2338238780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f69800710a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.916+0000 7f6986c00640 1 --2- 172.21.15.102:0/2338238780 >> v2:172.21.15.102:3300/0 conn(0x7f698015ea20 0x7f698006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.916+0000 7f6986c00640 1 -- 172.21.15.102:0/2338238780 >> 172.21.15.102:0/2338238780 conn(0x7f69800c9390 msgr2=0x7f69800c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.916+0000 7f6986c00640 1 -- 172.21.15.102:0/2338238780 shutdown_connections 2024-04-20T21:21:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.916+0000 7f6986c00640 1 -- 172.21.15.102:0/2338238780 wait complete. 2024-04-20T21:21:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.918+0000 7f6986c00640 1 Processor -- start 2024-04-20T21:21:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.918+0000 7f6986c00640 1 -- start start 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f6986c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f698015ea20 0x7f69800cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f6986c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f698015ee00 0x7f698006e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f6986c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f698015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f6986c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f69800c8c80 con 0x7f698015ee00 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f6986c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f69800c8b00 con 0x7f698015ea20 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f6986c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6980154e80 con 0x7f698015f2b0 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f697f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f698015ee00 0x7f698006e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f697f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f698015ee00 0x7f698006e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45344/0 (socket says 172.21.15.102:45344) 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f697f400640 1 -- 172.21.15.102:0/2474646143 learned_addr learned my addr 172.21.15.102:0/2474646143 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f697fe00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f698015ea20 0x7f69800cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.919+0000 7f6984c00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f698015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f697f400640 1 -- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 msgr2=0x7f698015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f697f400640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f698015de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f697f400640 1 -- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f698015ea20 msgr2=0x7f69800cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f697f400640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f698015ea20 0x7f69800cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f697f400640 1 -- 172.21.15.102:0/2474646143 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698015c8d0 con 0x7f698015ee00 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f697f400640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f698015ee00 0x7f698006e7b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f696800c430 tx=0x7f696800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f6977e00640 1 -- 172.21.15.102:0/2474646143 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968017070 con 0x7f698015ee00 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f6977e00640 1 -- 172.21.15.102:0/2474646143 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69680052d0 con 0x7f698015ee00 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698015c030 con 0x7f698015ee00 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f6977e00640 1 -- 172.21.15.102:0/2474646143 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968003810 con 0x7f698015ee00 2024-04-20T21:21:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.920+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698015c5f0 con 0x7f698015ee00 2024-04-20T21:21:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.922+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6980074340 con 0x7f698015ee00 2024-04-20T21:21:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.923+0000 7f6977e00640 1 -- 172.21.15.102:0/2474646143 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f696801a070 con 0x7f698015ee00 2024-04-20T21:21:20.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.924+0000 7f6977e00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:20.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.924+0000 7f6977e00640 1 -- 172.21.15.102:0/2474646143 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f696808b460 con 0x7f698015ee00 2024-04-20T21:21:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.924+0000 7f697fe00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.925+0000 7f697fe00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f695c06c8f0 0x7f695c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6970006ca0 tx=0x7f6970009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:20.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:20.929+0000 7f6977e00640 1 -- 172.21.15.102:0/2474646143 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6968058ca0 con 0x7f698015ee00 2024-04-20T21:21:21.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.170+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69800c78d0 con 0x7f698015ee00 2024-04-20T21:21:21.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.171+0000 7f6977e00640 1 -- 172.21.15.102:0/2474646143 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6026 (secure 0 0 0) 0x7f696805cfa0 con 0x7f698015ee00 2024-04-20T21:21:21.171 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:21.171 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:20.486187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:21.171 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 16 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f695c06c8f0 msgr2=0x7f695c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f695c06c8f0 0x7f695c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6970006ca0 tx=0x7f6970009240 comp rx=0 tx=0).stop 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f698015ee00 msgr2=0x7f698006e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f698015ee00 0x7f698006e7b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f696800c430 tx=0x7f696800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 shutdown_connections 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f698015f2b0 0x7f698015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f698015ee00 0x7f698006e7b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 --2- 172.21.15.102:0/2474646143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f698015ea20 0x7f69800cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 >> 172.21.15.102:0/2474646143 conn(0x7f69800c9390 msgr2=0x7f698014a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 shutdown_connections 2024-04-20T21:21:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:21.174+0000 7f6986c00640 1 -- 172.21.15.102:0/2474646143 wait complete. 2024-04-20T21:21:21.193 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 16, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4847": { "addr": "172.21.15.102:6835/1017522496", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 1017522496, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 1017522496, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4847, "incarnation": 16, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:21:20.486187+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4847 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:21:22.073 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:22.081 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:23.194 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.413+0000 7fc382200640 1 Processor -- start 2024-04-20T21:21:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.413+0000 7fc382200640 1 -- start start 2024-04-20T21:21:23.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.414+0000 7fc382200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc37c153180 0x7fc37c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:23.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.414+0000 7fc382200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c159380 0x7fc37c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:23.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.414+0000 7fc382200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc37c0c75a0 0x7fc37c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:23.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.414+0000 7fc382200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc37c153560 con 0x7fc37c0c75a0 2024-04-20T21:21:23.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.414+0000 7fc382200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc37c0c7a70 con 0x7fc37c153180 2024-04-20T21:21:23.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.414+0000 7fc382200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c0c7d30 con 0x7fc37c159380 2024-04-20T21:21:23.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc381800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc37c0c75a0 0x7fc37c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:23.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc381800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc37c0c75a0 0x7fc37c0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45352/0 (socket says 172.21.15.102:45352) 2024-04-20T21:21:23.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc381800640 1 -- 172.21.15.102:0/3458693995 learned_addr learned my addr 172.21.15.102:0/3458693995 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:23.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc380e00640 1 --2- 172.21.15.102:0/3458693995 >> v2:172.21.15.193:3300/0 conn(0x7fc37c153180 0x7fc37c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:23.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc37be00640 1 --2- 172.21.15.102:0/3458693995 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c159380 0x7fc37c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:23.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc381800640 1 -- 172.21.15.102:0/3458693995 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c159380 msgr2=0x7fc37c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc381800640 1 --2- 172.21.15.102:0/3458693995 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c159380 0x7fc37c0c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc381800640 1 -- 172.21.15.102:0/3458693995 >> v2:172.21.15.193:3300/0 conn(0x7fc37c153180 msgr2=0x7fc37c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc381800640 1 --2- 172.21.15.102:0/3458693995 >> v2:172.21.15.193:3300/0 conn(0x7fc37c153180 0x7fc37c158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.415+0000 7fc381800640 1 -- 172.21.15.102:0/3458693995 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c15c4d0 con 0x7fc37c0c75a0 2024-04-20T21:21:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.416+0000 7fc381800640 1 --2- 172.21.15.102:0/3458693995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c0c6470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0cd860 tx=0x7fc37402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a533748915a6de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc37b400640 1 -- 172.21.15.102:0/3458693995 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37402ca80 con 0x7fc37c0c75a0 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc37b400640 1 -- 172.21.15.102:0/3458693995 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37402cc20 con 0x7fc37c0c75a0 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc37b400640 1 -- 172.21.15.102:0/3458693995 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3740343f0 con 0x7fc37c0c75a0 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc382200640 1 -- 172.21.15.102:0/3458693995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 msgr2=0x7fc37c0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc382200640 1 --2- 172.21.15.102:0/3458693995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c0c6470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0cd860 tx=0x7fc37402bee0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc382200640 1 -- 172.21.15.102:0/3458693995 shutdown_connections 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc382200640 1 --2- 172.21.15.102:0/3458693995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c0c6470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc382200640 1 --2- 172.21.15.102:0/3458693995 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c159380 0x7fc37c0c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc382200640 1 --2- 172.21.15.102:0/3458693995 >> v2:172.21.15.193:3300/0 conn(0x7fc37c153180 0x7fc37c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.417+0000 7fc382200640 1 -- 172.21.15.102:0/3458693995 >> 172.21.15.102:0/3458693995 conn(0x7fc37c0cc820 msgr2=0x7fc37c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.418+0000 7fc382200640 1 -- 172.21.15.102:0/3458693995 shutdown_connections 2024-04-20T21:21:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.418+0000 7fc382200640 1 -- 172.21.15.102:0/3458693995 wait complete. 2024-04-20T21:21:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.420+0000 7fc382200640 1 Processor -- start 2024-04-20T21:21:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.420+0000 7fc382200640 1 -- start start 2024-04-20T21:21:23.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc382200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c1e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:23.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc382200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c153180 0x7fc37c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc382200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c159380 0x7fc37c06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc382200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c15d290 con 0x7fc37c0c75a0 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc382200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c15ca90 con 0x7fc37c159380 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc382200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c0c6be0 con 0x7fc37c153180 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc380e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c1e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc380e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c1e4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45366/0 (socket says 172.21.15.102:45366) 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.421+0000 7fc380e00640 1 -- 172.21.15.102:0/1916529413 learned_addr learned my addr 172.21.15.102:0/1916529413 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc381800640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c159380 0x7fc37c06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc37be00640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c153180 0x7fc37c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc380e00640 1 -- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c153180 msgr2=0x7fc37c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc380e00640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c153180 0x7fc37c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc380e00640 1 -- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c159380 msgr2=0x7fc37c06e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc380e00640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c159380 0x7fc37c06e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc380e00640 1 -- 172.21.15.102:0/1916529413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c156f00 con 0x7fc37c0c75a0 2024-04-20T21:21:23.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc380e00640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c1e4aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc37000e8c0 tx=0x7fc37000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:23.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.422+0000 7fc378c00640 1 -- 172.21.15.102:0/1916529413 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370017070 con 0x7fc37c0c75a0 2024-04-20T21:21:23.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.423+0000 7fc378c00640 1 -- 172.21.15.102:0/1916529413 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37000c940 con 0x7fc37c0c75a0 2024-04-20T21:21:23.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.423+0000 7fc378c00640 1 -- 172.21.15.102:0/1916529413 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37000cd10 con 0x7fc37c0c75a0 2024-04-20T21:21:23.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.423+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c1571e0 con 0x7fc37c0c75a0 2024-04-20T21:21:23.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.423+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c0cb520 con 0x7fc37c0c75a0 2024-04-20T21:21:23.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.424+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37c04fd60 con 0x7fc37c0c75a0 2024-04-20T21:21:23.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.425+0000 7fc378c00640 1 -- 172.21.15.102:0/1916529413 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc37001a070 con 0x7fc37c0c75a0 2024-04-20T21:21:23.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.426+0000 7fc378c00640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:23.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.426+0000 7fc378c00640 1 -- 172.21.15.102:0/1916529413 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7fc37008c2b0 con 0x7fc37c0c75a0 2024-04-20T21:21:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.426+0000 7fc37be00640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.427+0000 7fc37be00640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc35c06c8f0 0x7fc35c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0cc320 tx=0x7fc368009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:23.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.430+0000 7fc378c00640 1 -- 172.21.15.102:0/1916529413 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc370059bd0 con 0x7fc37c0c75a0 2024-04-20T21:21:23.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.670+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc37c0cc350 con 0x7fc37c0c75a0 2024-04-20T21:21:23.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.671+0000 7fc378c00640 1 -- 172.21.15.102:0/1916529413 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6026 (secure 0 0 0) 0x7fc37005ded0 con 0x7fc37c0c75a0 2024-04-20T21:21:23.670 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:23.671 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:20.486187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:23.671 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 16 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc35c06c8f0 msgr2=0x7fc35c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc35c06c8f0 0x7fc35c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0cc320 tx=0x7fc368009190 comp rx=0 tx=0).stop 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 msgr2=0x7fc37c1e4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c1e4aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc37000e8c0 tx=0x7fc37000ed80 comp rx=0 tx=0).stop 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 shutdown_connections 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c159380 0x7fc37c06e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc37c153180 0x7fc37c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 --2- 172.21.15.102:0/1916529413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc37c0c75a0 0x7fc37c1e4aa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:23.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 >> 172.21.15.102:0/1916529413 conn(0x7fc37c0cc820 msgr2=0x7fc37c148dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:23.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 shutdown_connections 2024-04-20T21:21:23.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:23.674+0000 7fc382200640 1 -- 172.21.15.102:0/1916529413 wait complete. 2024-04-20T21:21:23.692 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:21:25.693 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:25.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.876+0000 7f3e47200640 1 Processor -- start 2024-04-20T21:21:25.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.876+0000 7f3e47200640 1 -- start start 2024-04-20T21:21:25.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.877+0000 7f3e47200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3e48155530 0x7f3e4815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.877+0000 7f3e47200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e4815ef60 con 0x7f3e480cc470 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.877+0000 7f3e47200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e4815f240 con 0x7f3e480cb6a0 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.877+0000 7f3e47200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3e480c5ca0 con 0x7f3e48155530 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.878+0000 7f3e45e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3e48155530 0x7f3e4815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.878+0000 7f3e45400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3e480cc470 0x7f3e480cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34844/0 (socket says 172.21.15.102:34844) 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.878+0000 7f3e45400640 1 -- 172.21.15.102:0/2240017524 learned_addr learned my addr 172.21.15.102:0/2240017524 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.878+0000 7f3e45e00640 1 -- 172.21.15.102:0/2240017524 >> v1:172.21.15.193:6789/0 conn(0x7f3e480cb6a0 legacy=0x7f3e481659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.878+0000 7f3e44a00640 1 -- 172.21.15.102:0/2240017524 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519725536 0 0) 0x7f3e4815f240 con 0x7f3e480cb6a0 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.878+0000 7f3e45e00640 1 -- 172.21.15.102:0/2240017524 >> v1:172.21.15.102:6789/0 conn(0x7f3e480cc470 legacy=0x7f3e480cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:25.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.878+0000 7f3e45e00640 1 -- 172.21.15.102:0/2240017524 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4815f520 con 0x7f3e48155530 2024-04-20T21:21:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.880+0000 7f3e45e00640 1 --2- 172.21.15.102:0/2240017524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e48155530 0x7f3e4815e640 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e380097a0 tx=0x7f3e3802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c087743c77a5a351 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.880+0000 7f3e44a00640 1 -- 172.21.15.102:0/2240017524 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38037070 con 0x7f3e48155530 2024-04-20T21:21:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.880+0000 7f3e44a00640 1 -- 172.21.15.102:0/2240017524 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e38030710 con 0x7f3e48155530 2024-04-20T21:21:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.880+0000 7f3e44a00640 1 -- 172.21.15.102:0/2240017524 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38030ae0 con 0x7f3e48155530 2024-04-20T21:21:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.881+0000 7f3e47200640 1 -- 172.21.15.102:0/2240017524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e48155530 msgr2=0x7f3e4815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.881+0000 7f3e47200640 1 --2- 172.21.15.102:0/2240017524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e48155530 0x7f3e4815e640 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e380097a0 tx=0x7f3e3802b790 comp rx=0 tx=0).stop 2024-04-20T21:21:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.881+0000 7f3e47200640 1 -- 172.21.15.102:0/2240017524 shutdown_connections 2024-04-20T21:21:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.881+0000 7f3e47200640 1 --2- 172.21.15.102:0/2240017524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e48155530 0x7f3e4815e640 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.881+0000 7f3e47200640 1 -- 172.21.15.102:0/2240017524 >> 172.21.15.102:0/2240017524 conn(0x7f3e48075080 msgr2=0x7f3e48075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.881+0000 7f3e47200640 1 -- 172.21.15.102:0/2240017524 shutdown_connections 2024-04-20T21:21:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.881+0000 7f3e47200640 1 -- 172.21.15.102:0/2240017524 wait complete. 2024-04-20T21:21:25.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.883+0000 7f3e47200640 1 Processor -- start 2024-04-20T21:21:25.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.883+0000 7f3e47200640 1 -- start start 2024-04-20T21:21:25.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e47200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e480cb6a0 0x7f3e480ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:25.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e47200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e480cc470 0x7f3e480c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:25.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e47200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e48155530 0x7f3e480c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:25.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e47200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3e480cd280 con 0x7f3e480cc470 2024-04-20T21:21:25.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e47200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e480cd5f0 con 0x7f3e480cb6a0 2024-04-20T21:21:25.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e47200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3e480ccbe0 con 0x7f3e48155530 2024-04-20T21:21:25.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e45e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e480cb6a0 0x7f3e480ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:25.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e45400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e480cc470 0x7f3e480c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e45e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e480cb6a0 0x7f3e480ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41864/0 (socket says 172.21.15.102:41864) 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e45e00640 1 -- 172.21.15.102:0/1359103817 learned_addr learned my addr 172.21.15.102:0/1359103817 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.884+0000 7f3e46800640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e48155530 0x7f3e480c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.885+0000 7f3e45e00640 1 -- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e48155530 msgr2=0x7f3e480c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.885+0000 7f3e45e00640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e48155530 0x7f3e480c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.885+0000 7f3e45e00640 1 -- 172.21.15.102:0/1359103817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e480cc470 msgr2=0x7f3e480c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.885+0000 7f3e45e00640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e480cc470 0x7f3e480c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.885+0000 7f3e45e00640 1 -- 172.21.15.102:0/1359103817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4815be30 con 0x7f3e480cb6a0 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.885+0000 7f3e45e00640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e480cb6a0 0x7f3e480ceb70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e38035040 tx=0x7f3e380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:25.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.886+0000 7f3e3e000640 1 -- 172.21.15.102:0/1359103817 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38037040 con 0x7f3e480cb6a0 2024-04-20T21:21:25.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.886+0000 7f3e3e000640 1 -- 172.21.15.102:0/1359103817 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e380329c0 con 0x7f3e480cb6a0 2024-04-20T21:21:25.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.886+0000 7f3e3e000640 1 -- 172.21.15.102:0/1359103817 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38032d90 con 0x7f3e480cb6a0 2024-04-20T21:21:25.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.886+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481e6b30 con 0x7f3e480cb6a0 2024-04-20T21:21:25.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.886+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481e63c0 con 0x7f3e480cb6a0 2024-04-20T21:21:25.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.887+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e48072420 con 0x7f3e480cb6a0 2024-04-20T21:21:25.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.888+0000 7f3e3e000640 1 -- 172.21.15.102:0/1359103817 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e38004030 con 0x7f3e480cb6a0 2024-04-20T21:21:25.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.889+0000 7f3e3e000640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e2806c8f0 0x7f3e2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:25.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.889+0000 7f3e3e000640 1 -- 172.21.15.102:0/1359103817 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f3e380add00 con 0x7f3e480cb6a0 2024-04-20T21:21:25.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.889+0000 7f3e45400640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e2806c8f0 0x7f3e2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:25.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.890+0000 7f3e45400640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e2806c8f0 0x7f3e2806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3e481455e0 tx=0x7f3e34007e20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:25.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:25.893+0000 7f3e3e000640 1 -- 172.21.15.102:0/1359103817 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3e48072420 con 0x7f3e480cb6a0 2024-04-20T21:21:26.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.134+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e48148410 con 0x7f3e480cb6a0 2024-04-20T21:21:26.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.135+0000 7f3e3e000640 1 -- 172.21.15.102:0/1359103817 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6026 (secure 0 0 0) 0x7f3e38042020 con 0x7f3e480cb6a0 2024-04-20T21:21:26.134 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:26.134 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:20.486187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:26.135 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 16 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e2806c8f0 msgr2=0x7f3e2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e2806c8f0 0x7f3e2806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3e481455e0 tx=0x7f3e34007e20 comp rx=0 tx=0).stop 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e480cb6a0 msgr2=0x7f3e480ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e480cb6a0 0x7f3e480ceb70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e38035040 tx=0x7f3e380046c0 comp rx=0 tx=0).stop 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 shutdown_connections 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e48155530 0x7f3e480c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e480cc470 0x7f3e480c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e2806c8f0 0x7f3e2806edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 --2- 172.21.15.102:0/1359103817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e480cb6a0 0x7f3e480ceb70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:26.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.138+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 >> 172.21.15.102:0/1359103817 conn(0x7f3e48075080 msgr2=0x7f3e480c5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:26.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.139+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 shutdown_connections 2024-04-20T21:21:26.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:26.139+0000 7f3e47200640 1 -- 172.21.15.102:0/1359103817 wait complete. 2024-04-20T21:21:26.162 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:21:28.163 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:28.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.374+0000 7fb57b200640 1 Processor -- start 2024-04-20T21:21:28.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.374+0000 7fb57b200640 1 -- start start 2024-04-20T21:21:28.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.375+0000 7fb57b200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb57c153180 0x7fb57c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:28.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.375+0000 7fb57b200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb57c159380 0x7fb57c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:28.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb57b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c0c75a0 0x7fb57c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb57b200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb57c153560 con 0x7fb57c153180 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb57b200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb57c0c7a70 con 0x7fb57c159380 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb57b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c0c7d30 con 0x7fb57c0c75a0 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb579e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb57c153180 0x7fb57c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb579e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb57c153180 0x7fb57c158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48090/0 (socket says 172.21.15.102:48090) 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb579e00640 1 -- 172.21.15.102:0/3694861464 learned_addr learned my addr 172.21.15.102:0/3694861464 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb57a800640 1 --2- 172.21.15.102:0/3694861464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c0c75a0 0x7fb57c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.376+0000 7fb579400640 1 --2- 172.21.15.102:0/3694861464 >> v2:172.21.15.193:3300/0 conn(0x7fb57c159380 0x7fb57c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.377+0000 7fb57a800640 1 -- 172.21.15.102:0/3694861464 >> v2:172.21.15.193:3300/0 conn(0x7fb57c159380 msgr2=0x7fb57c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:28.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.377+0000 7fb57a800640 1 --2- 172.21.15.102:0/3694861464 >> v2:172.21.15.193:3300/0 conn(0x7fb57c159380 0x7fb57c0c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.377+0000 7fb57a800640 1 -- 172.21.15.102:0/3694861464 >> v2:172.21.15.102:3300/0 conn(0x7fb57c153180 msgr2=0x7fb57c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:28.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.377+0000 7fb57a800640 1 --2- 172.21.15.102:0/3694861464 >> v2:172.21.15.102:3300/0 conn(0x7fb57c153180 0x7fb57c158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.377+0000 7fb57a800640 1 -- 172.21.15.102:0/3694861464 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c15c4d0 con 0x7fb57c0c75a0 2024-04-20T21:21:28.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.378+0000 7fb57a800640 1 --2- 172.21.15.102:0/3694861464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c0c75a0 0x7fb57c0c6470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb57c06f8d0 tx=0x7fb57402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b566e1281509cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:28.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.378+0000 7fb578a00640 1 -- 172.21.15.102:0/3694861464 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5740315a0 con 0x7fb57c0c75a0 2024-04-20T21:21:28.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.378+0000 7fb578a00640 1 -- 172.21.15.102:0/3694861464 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb574037070 con 0x7fb57c0c75a0 2024-04-20T21:21:28.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.378+0000 7fb578a00640 1 -- 172.21.15.102:0/3694861464 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574031960 con 0x7fb57c0c75a0 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 -- 172.21.15.102:0/3694861464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c0c75a0 msgr2=0x7fb57c0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 --2- 172.21.15.102:0/3694861464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c0c75a0 0x7fb57c0c6470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb57c06f8d0 tx=0x7fb57402b840 comp rx=0 tx=0).stop 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 -- 172.21.15.102:0/3694861464 shutdown_connections 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 --2- 172.21.15.102:0/3694861464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c0c75a0 0x7fb57c0c6470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 --2- 172.21.15.102:0/3694861464 >> v2:172.21.15.193:3300/0 conn(0x7fb57c159380 0x7fb57c0c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 --2- 172.21.15.102:0/3694861464 >> v2:172.21.15.102:3300/0 conn(0x7fb57c153180 0x7fb57c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 -- 172.21.15.102:0/3694861464 >> 172.21.15.102:0/3694861464 conn(0x7fb57c0cc820 msgr2=0x7fb57c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 -- 172.21.15.102:0/3694861464 shutdown_connections 2024-04-20T21:21:28.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.379+0000 7fb57b200640 1 -- 172.21.15.102:0/3694861464 wait complete. 2024-04-20T21:21:28.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.381+0000 7fb57b200640 1 Processor -- start 2024-04-20T21:21:28.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.381+0000 7fb57b200640 1 -- start start 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb57b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb57c0c75a0 0x7fb57c142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb57b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c153180 0x7fb57c1ec910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb57b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57c159380 0x7fb57c0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb57b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c15d290 con 0x7fb57c0c75a0 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb57b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c15ca90 con 0x7fb57c159380 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb57b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c0c6be0 con 0x7fb57c153180 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb579e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb57c0c75a0 0x7fb57c142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb579400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c153180 0x7fb57c1ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb579400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c153180 0x7fb57c1ec910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:39652/0 (socket says 172.21.15.102:39652) 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb57a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57c159380 0x7fb57c0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:28.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.382+0000 7fb579400640 1 -- 172.21.15.102:0/724178341 learned_addr learned my addr 172.21.15.102:0/724178341 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb579400640 1 -- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57c159380 msgr2=0x7fb57c0c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb579400640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57c159380 0x7fb57c0c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb579400640 1 -- 172.21.15.102:0/724178341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb57c0c75a0 msgr2=0x7fb57c142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb579400640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb57c0c75a0 0x7fb57c142000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb579400640 1 -- 172.21.15.102:0/724178341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c145930 con 0x7fb57c153180 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb579e00640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb57c0c75a0 0x7fb57c142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb579400640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c153180 0x7fb57c1ec910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e9b0 tx=0x7fb56800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb572000640 1 -- 172.21.15.102:0/724178341 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568017070 con 0x7fb57c153180 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb572000640 1 -- 172.21.15.102:0/724178341 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56800c9a0 con 0x7fb57c153180 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c157550 con 0x7fb57c153180 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.383+0000 7fb572000640 1 -- 172.21.15.102:0/724178341 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56800cd70 con 0x7fb57c153180 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.384+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c1effb0 con 0x7fb57c153180 2024-04-20T21:21:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.385+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57c04fd60 con 0x7fb57c153180 2024-04-20T21:21:28.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.386+0000 7fb572000640 1 -- 172.21.15.102:0/724178341 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb56801a070 con 0x7fb57c153180 2024-04-20T21:21:28.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.386+0000 7fb572000640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb55c06c8f0 0x7fb55c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:28.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.387+0000 7fb572000640 1 -- 172.21.15.102:0/724178341 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7fb56808bff0 con 0x7fb57c153180 2024-04-20T21:21:28.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.387+0000 7fb579e00640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb55c06c8f0 0x7fb55c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:28.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.387+0000 7fb579e00640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb55c06c8f0 0x7fb55c06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0092a0 tx=0x7fb56c00d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.392+0000 7fb572000640 1 -- 172.21.15.102:0/724178341 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb568059880 con 0x7fb57c153180 2024-04-20T21:21:28.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.633+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb57c0c3690 con 0x7fb57c153180 2024-04-20T21:21:28.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.634+0000 7fb572000640 1 -- 172.21.15.102:0/724178341 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6030 (secure 0 0 0) 0x7fb56805db80 con 0x7fb57c153180 2024-04-20T21:21:28.633 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:28.634 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:26.428918+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:28.634 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-04-20T21:21:28.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.636+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb55c06c8f0 msgr2=0x7fb55c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:28.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.636+0000 7fb57b200640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb55c06c8f0 0x7fb55c06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0092a0 tx=0x7fb56c00d040 comp rx=0 tx=0).stop 2024-04-20T21:21:28.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.636+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c153180 msgr2=0x7fb57c1ec910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:28.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.636+0000 7fb57b200640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c153180 0x7fb57c1ec910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e9b0 tx=0x7fb56800ee70 comp rx=0 tx=0).stop 2024-04-20T21:21:28.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.637+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 shutdown_connections 2024-04-20T21:21:28.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.637+0000 7fb57b200640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb55c06c8f0 0x7fb55c06edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.637+0000 7fb57b200640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57c159380 0x7fb57c0c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.637+0000 7fb57b200640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb57c153180 0x7fb57c1ec910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.637+0000 7fb57b200640 1 --2- 172.21.15.102:0/724178341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb57c0c75a0 0x7fb57c142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:28.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.637+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 >> 172.21.15.102:0/724178341 conn(0x7fb57c0cc820 msgr2=0x7fb57c151370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:28.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.637+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 shutdown_connections 2024-04-20T21:21:28.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:28.637+0000 7fb57b200640 1 -- 172.21.15.102:0/724178341 wait complete. 2024-04-20T21:21:28.655 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:21:30.656 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:30.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.868+0000 7f5ca3a00640 1 Processor -- start 2024-04-20T21:21:30.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.868+0000 7f5ca3a00640 1 -- start start 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.869+0000 7f5ca3a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c9c14fa50 0x7f5c9c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.869+0000 7f5ca3a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c073a10 0x7f5c9c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.869+0000 7f5ca3a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c9c1536d0 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.869+0000 7f5ca3a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9c159500 con 0x7f5c9c14fa50 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.869+0000 7f5ca3a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c0c76c0 con 0x7f5c9c073a10 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5ca1000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c9c14fa50 0x7f5c9c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5ca1000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c9c14fa50 0x7f5c9c0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48100/0 (socket says 172.21.15.102:48100) 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5ca1000640 1 -- 172.21.15.102:0/1308256062 learned_addr learned my addr 172.21.15.102:0/1308256062 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5c93e00640 1 --1- 172.21.15.102:0/1308256062 >> v1:172.21.15.193:6789/0 conn(0x7f5c9c0c9eb0 0x7f5c9c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:37672/0 (socket says 172.21.15.102:37672) 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5ca1a00640 1 --2- 172.21.15.102:0/1308256062 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c073a10 0x7f5c9c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5c9be00640 1 -- 172.21.15.102:0/1308256062 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2772519462 0 0) 0x7f5c84001110 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5ca1000640 1 -- 172.21.15.102:0/1308256062 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c073a10 msgr2=0x7f5c9c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:30.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5ca1000640 1 --2- 172.21.15.102:0/1308256062 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c073a10 0x7f5c9c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:30.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5ca1000640 1 -- 172.21.15.102:0/1308256062 >> v1:172.21.15.193:6789/0 conn(0x7f5c9c0c9eb0 legacy=0x7f5c9c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:30.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.870+0000 7f5ca1000640 1 -- 172.21.15.102:0/1308256062 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c0c7960 con 0x7f5c9c14fa50 2024-04-20T21:21:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.871+0000 7f5ca1000640 1 --2- 172.21.15.102:0/1308256062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c14fa50 0x7f5c9c0c4830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009870 tx=0x7f5c8c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d53f4dcf093c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5c9be00640 1 -- 172.21.15.102:0/1308256062 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c02f840 con 0x7f5c9c14fa50 2024-04-20T21:21:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5c9be00640 1 -- 172.21.15.102:0/1308256062 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c02f9e0 con 0x7f5c9c14fa50 2024-04-20T21:21:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5c9be00640 1 -- 172.21.15.102:0/1308256062 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c02fdb0 con 0x7f5c9c14fa50 2024-04-20T21:21:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5ca3a00640 1 -- 172.21.15.102:0/1308256062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c14fa50 msgr2=0x7f5c9c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/1308256062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c14fa50 0x7f5c9c0c4830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009870 tx=0x7f5c8c02ba20 comp rx=0 tx=0).stop 2024-04-20T21:21:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5ca3a00640 1 -- 172.21.15.102:0/1308256062 shutdown_connections 2024-04-20T21:21:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/1308256062 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c073a10 0x7f5c9c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/1308256062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c14fa50 0x7f5c9c0c4830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5ca3a00640 1 -- 172.21.15.102:0/1308256062 >> 172.21.15.102:0/1308256062 conn(0x7f5c9c0c9390 msgr2=0x7f5c9c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5ca3a00640 1 -- 172.21.15.102:0/1308256062 shutdown_connections 2024-04-20T21:21:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.872+0000 7f5ca3a00640 1 -- 172.21.15.102:0/1308256062 wait complete. 2024-04-20T21:21:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.874+0000 7f5ca3a00640 1 Processor -- start 2024-04-20T21:21:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.874+0000 7f5ca3a00640 1 -- start start 2024-04-20T21:21:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5ca3a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c073a10 0x7f5c9c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5ca3a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c0c9eb0 0x7f5c9c072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5ca3a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c14fa50 0x7f5c9c14a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5ca3a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c0c7c00 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5ca3a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c0c3d90 con 0x7f5c9c073a10 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5ca3a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c0c4030 con 0x7f5c9c14fa50 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c0c9eb0 0x7f5c9c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c0c9eb0 0x7f5c9c072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48102/0 (socket says 172.21.15.102:48102) 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5c93e00640 1 -- 172.21.15.102:0/3618643414 learned_addr learned my addr 172.21.15.102:0/3618643414 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5ca1000640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c073a10 0x7f5c9c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.875+0000 7f5ca1a00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c14fa50 0x7f5c9c14a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c93e00640 1 -- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c14fa50 msgr2=0x7f5c9c14a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c93e00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c14fa50 0x7f5c9c14a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c93e00640 1 -- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c073a10 msgr2=0x7f5c9c0cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c93e00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c073a10 0x7f5c9c0cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c93e00640 1 -- 172.21.15.102:0/3618643414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c070df0 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c93e00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c0c9eb0 0x7f5c9c072130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400c780 tx=0x7f5c8400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:30.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c99600640 1 -- 172.21.15.102:0/3618643414 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84018070 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c99600640 1 -- 172.21.15.102:0/3618643414 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c840072d0 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5c99600640 1 -- 172.21.15.102:0/3618643414 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840047e0 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.876+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c071090 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.877+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c071650 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.878+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c074340 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.879+0000 7f5c99600640 1 -- 172.21.15.102:0/3618643414 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c8401b070 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.879+0000 7f5c99600640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c7806c8f0 0x7f5c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.880+0000 7f5c99600640 1 -- 172.21.15.102:0/3618643414 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f5c8408b310 con 0x7f5c9c0c9eb0 2024-04-20T21:21:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.880+0000 7f5ca1000640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c7806c8f0 0x7f5c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:30.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.880+0000 7f5ca1000640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c7806c8f0 0x7f5c7806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c036000 tx=0x7f5c8c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:30.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:30.884+0000 7f5c99600640 1 -- 172.21.15.102:0/3618643414 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5c9c1536d0 con 0x7f5c9c0c9eb0 2024-04-20T21:21:31.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.121+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c9c1568e0 con 0x7f5c9c0c9eb0 2024-04-20T21:21:31.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.122+0000 7f5c99600640 1 -- 172.21.15.102:0/3618643414 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6030 (secure 0 0 0) 0x7f5c84059e20 con 0x7f5c9c0c9eb0 2024-04-20T21:21:31.122 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:31.123 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:26.428918+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:31.123 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c7806c8f0 msgr2=0x7f5c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c7806c8f0 0x7f5c7806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c036000 tx=0x7f5c8c008030 comp rx=0 tx=0).stop 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c0c9eb0 msgr2=0x7f5c9c072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c0c9eb0 0x7f5c9c072130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400c780 tx=0x7f5c8400cc40 comp rx=0 tx=0).stop 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 shutdown_connections 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c9c14fa50 0x7f5c9c14a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c0c9eb0 0x7f5c9c072130 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c7806c8f0 0x7f5c7806edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 --2- 172.21.15.102:0/3618643414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c073a10 0x7f5c9c0cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:31.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.125+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 >> 172.21.15.102:0/3618643414 conn(0x7f5c9c0c9390 msgr2=0x7f5c9c06ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:31.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.126+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 shutdown_connections 2024-04-20T21:21:31.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:31.126+0000 7f5ca3a00640 1 -- 172.21.15.102:0/3618643414 wait complete. 2024-04-20T21:21:31.143 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:21:33.144 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:33.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.360+0000 7f1394c00640 1 Processor -- start 2024-04-20T21:21:33.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.360+0000 7f1394c00640 1 -- start start 2024-04-20T21:21:33.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f1394c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1390153200 0x7f1390159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:33.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f1394c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13900cf730 0x7f13900708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:33.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f1394c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1390070df0 0x7f13900c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:33.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f1394c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1390071280 con 0x7f1390153200 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f1394c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1390072f90 con 0x7f1390070df0 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f1394c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1390073250 con 0x7f13900cf730 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f138de00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1390153200 0x7f1390159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f138de00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1390153200 0x7f1390159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48116/0 (socket says 172.21.15.102:48116) 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.362+0000 7f138de00640 1 -- 172.21.15.102:0/1910482659 learned_addr learned my addr 172.21.15.102:0/1910482659 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.363+0000 7f138d400640 1 --2- 172.21.15.102:0/1910482659 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13900cf730 0x7f13900708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.363+0000 7f138e800640 1 --2- 172.21.15.102:0/1910482659 >> v2:172.21.15.193:3300/0 conn(0x7f1390070df0 0x7f13900c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.363+0000 7f138de00640 1 -- 172.21.15.102:0/1910482659 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13900cf730 msgr2=0x7f13900708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.363+0000 7f138de00640 1 --2- 172.21.15.102:0/1910482659 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13900cf730 0x7f13900708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.363+0000 7f138de00640 1 -- 172.21.15.102:0/1910482659 >> v2:172.21.15.193:3300/0 conn(0x7f1390070df0 msgr2=0x7f13900c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.363+0000 7f138de00640 1 --2- 172.21.15.102:0/1910482659 >> v2:172.21.15.193:3300/0 conn(0x7f1390070df0 0x7f13900c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.363+0000 7f138de00640 1 -- 172.21.15.102:0/1910482659 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1390073530 con 0x7f1390153200 2024-04-20T21:21:33.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.364+0000 7f138de00640 1 --2- 172.21.15.102:0/1910482659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1390153200 0x7f1390159a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f13900c5b30 tx=0x7f138002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a281e6037259ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:33.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.364+0000 7f138ca00640 1 -- 172.21.15.102:0/1910482659 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380034070 con 0x7f1390153200 2024-04-20T21:21:33.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.364+0000 7f138ca00640 1 -- 172.21.15.102:0/1910482659 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1380030860 con 0x7f1390153200 2024-04-20T21:21:33.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.364+0000 7f138ca00640 1 -- 172.21.15.102:0/1910482659 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380030c30 con 0x7f1390153200 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 -- 172.21.15.102:0/1910482659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1390153200 msgr2=0x7f1390159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 --2- 172.21.15.102:0/1910482659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1390153200 0x7f1390159a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f13900c5b30 tx=0x7f138002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 -- 172.21.15.102:0/1910482659 shutdown_connections 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 --2- 172.21.15.102:0/1910482659 >> v2:172.21.15.193:3300/0 conn(0x7f1390070df0 0x7f13900c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 --2- 172.21.15.102:0/1910482659 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13900cf730 0x7f13900708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 --2- 172.21.15.102:0/1910482659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1390153200 0x7f1390159a70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 -- 172.21.15.102:0/1910482659 >> 172.21.15.102:0/1910482659 conn(0x7f13900ceb60 msgr2=0x7f13900cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 -- 172.21.15.102:0/1910482659 shutdown_connections 2024-04-20T21:21:33.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.365+0000 7f1394c00640 1 -- 172.21.15.102:0/1910482659 wait complete. 2024-04-20T21:21:33.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.367+0000 7f1394c00640 1 Processor -- start 2024-04-20T21:21:33.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.367+0000 7f1394c00640 1 -- start start 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.368+0000 7f1394c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1390070df0 0x7f139014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.368+0000 7f1394c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13900cf730 0x7f13901483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.368+0000 7f1394c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1390153200 0x7f1390144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.368+0000 7f1394c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1390073dd0 con 0x7f13900cf730 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.368+0000 7f1394c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1390073af0 con 0x7f1390153200 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.368+0000 7f1394c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f13900c4400 con 0x7f1390070df0 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.368+0000 7f138d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13900cf730 0x7f13901483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1390153200 0x7f1390144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13900cf730 0x7f13901483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48132/0 (socket says 172.21.15.102:48132) 2024-04-20T21:21:33.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138d400640 1 -- 172.21.15.102:0/3844125131 learned_addr learned my addr 172.21.15.102:0/3844125131 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:33.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138de00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1390070df0 0x7f139014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:33.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138e800640 1 -- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1390070df0 msgr2=0x7f139014dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138e800640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1390070df0 0x7f139014dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138e800640 1 -- 172.21.15.102:0/3844125131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13900cf730 msgr2=0x7f13901483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138e800640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13900cf730 0x7f13901483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138e800640 1 -- 172.21.15.102:0/3844125131 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1390148b20 con 0x7f1390153200 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138d400640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13900cf730 0x7f13901483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.369+0000 7f138e800640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1390153200 0x7f1390144110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f138800efc0 tx=0x7f138800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.370+0000 7f1386000640 1 -- 172.21.15.102:0/3844125131 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388018020 con 0x7f1390153200 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.370+0000 7f1386000640 1 -- 172.21.15.102:0/3844125131 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1388014480 con 0x7f1390153200 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.370+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1390148e00 con 0x7f1390153200 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.370+0000 7f1386000640 1 -- 172.21.15.102:0/3844125131 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388014870 con 0x7f1390153200 2024-04-20T21:21:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.370+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139014ab20 con 0x7f1390153200 2024-04-20T21:21:33.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.371+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1390068c30 con 0x7f1390153200 2024-04-20T21:21:33.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.372+0000 7f1386000640 1 -- 172.21.15.102:0/3844125131 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1388005d10 con 0x7f1390153200 2024-04-20T21:21:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.373+0000 7f1386000640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f136c06c920 0x7f136c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.373+0000 7f1386000640 1 -- 172.21.15.102:0/3844125131 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f138808cd20 con 0x7f1390153200 2024-04-20T21:21:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.373+0000 7f138de00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f136c06c920 0x7f136c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.374+0000 7f138de00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f136c06c920 0x7f136c06ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1380036000 tx=0x7f138003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:33.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.378+0000 7f1386000640 1 -- 172.21.15.102:0/3844125131 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1388022030 con 0x7f1390153200 2024-04-20T21:21:33.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.622+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13900c9220 con 0x7f1390153200 2024-04-20T21:21:33.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.623+0000 7f1386000640 1 -- 172.21.15.102:0/3844125131 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6027 (secure 0 0 0) 0x7f1388010040 con 0x7f1390153200 2024-04-20T21:21:33.623 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:33.623 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:31.863718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:33.623 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 18 2024-04-20T21:21:33.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.626+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f136c06c920 msgr2=0x7f136c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:33.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.626+0000 7f1394c00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f136c06c920 0x7f136c06ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1380036000 tx=0x7f138003b040 comp rx=0 tx=0).stop 2024-04-20T21:21:33.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.626+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1390153200 msgr2=0x7f1390144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:33.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.626+0000 7f1394c00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1390153200 0x7f1390144110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f138800efc0 tx=0x7f138800c490 comp rx=0 tx=0).stop 2024-04-20T21:21:33.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.627+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 shutdown_connections 2024-04-20T21:21:33.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.627+0000 7f1394c00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f136c06c920 0x7f136c06ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.627+0000 7f1394c00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1390153200 0x7f1390144110 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.627+0000 7f1394c00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13900cf730 0x7f13901483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.627+0000 7f1394c00640 1 --2- 172.21.15.102:0/3844125131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1390070df0 0x7f139014dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:33.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.627+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 >> 172.21.15.102:0/3844125131 conn(0x7f13900ceb60 msgr2=0x7f139006d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:33.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.627+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 shutdown_connections 2024-04-20T21:21:33.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:33.627+0000 7f1394c00640 1 -- 172.21.15.102:0/3844125131 wait complete. 2024-04-20T21:21:33.646 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 18, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4847": { "addr": "172.21.15.102:6835/1017522496", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 1017522496, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 1017522496, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4847, "incarnation": 16, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:rejoin", "state_seq": 23 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:21:31.863718+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4847 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:21:35.647 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:35.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.857+0000 7f87e4600640 1 Processor -- start 2024-04-20T21:21:35.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.857+0000 7f87e4600640 1 -- start start 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.858+0000 7f87e4600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f87dc152170 0x7f87dc0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.858+0000 7f87e4600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f87dc158770 0x7f87dc070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.858+0000 7f87e4600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.858+0000 7f87e4600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f87dc0c6dc0 con 0x7f87dc152170 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.858+0000 7f87e4600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f87dc071f70 con 0x7f87dc158770 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.858+0000 7f87e4600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc072210 con 0x7f87dc158b80 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f87dc152170 0x7f87dc0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f87dc152170 0x7f87dc0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48140/0 (socket says 172.21.15.102:48140) 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1c00640 1 -- 172.21.15.102:0/1039787711 learned_addr learned my addr 172.21.15.102:0/1039787711 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e2600640 1 --2- 172.21.15.102:0/1039787711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:35.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1200640 1 --2- 172.21.15.102:0/1039787711 >> v2:172.21.15.193:3300/0 conn(0x7f87dc158770 0x7f87dc070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:35.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1c00640 1 -- 172.21.15.102:0/1039787711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 msgr2=0x7f87dc071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:35.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1c00640 1 --2- 172.21.15.102:0/1039787711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:35.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1c00640 1 -- 172.21.15.102:0/1039787711 >> v2:172.21.15.193:3300/0 conn(0x7f87dc158770 msgr2=0x7f87dc070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:35.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1c00640 1 --2- 172.21.15.102:0/1039787711 >> v2:172.21.15.193:3300/0 conn(0x7f87dc158770 0x7f87dc070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:35.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.859+0000 7f87e1c00640 1 -- 172.21.15.102:0/1039787711 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc0724b0 con 0x7f87dc152170 2024-04-20T21:21:35.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.860+0000 7f87e1c00640 1 --2- 172.21.15.102:0/1039787711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc152170 0x7f87dc0c8db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87dc072680 tx=0x7f87d002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2f2e53f45bc33d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:35.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87dbe00640 1 -- 172.21.15.102:0/1039787711 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0034070 con 0x7f87dc152170 2024-04-20T21:21:35.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87dbe00640 1 -- 172.21.15.102:0/1039787711 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d0030840 con 0x7f87dc152170 2024-04-20T21:21:35.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87dbe00640 1 -- 172.21.15.102:0/1039787711 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0030c10 con 0x7f87dc152170 2024-04-20T21:21:35.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 -- 172.21.15.102:0/1039787711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc152170 msgr2=0x7f87dc0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:35.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 --2- 172.21.15.102:0/1039787711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc152170 0x7f87dc0c8db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87dc072680 tx=0x7f87d002b980 comp rx=0 tx=0).stop 2024-04-20T21:21:35.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 -- 172.21.15.102:0/1039787711 shutdown_connections 2024-04-20T21:21:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 --2- 172.21.15.102:0/1039787711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 --2- 172.21.15.102:0/1039787711 >> v2:172.21.15.193:3300/0 conn(0x7f87dc158770 0x7f87dc070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 --2- 172.21.15.102:0/1039787711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc152170 0x7f87dc0c8db0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 -- 172.21.15.102:0/1039787711 >> 172.21.15.102:0/1039787711 conn(0x7f87dc0c9390 msgr2=0x7f87dc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 -- 172.21.15.102:0/1039787711 shutdown_connections 2024-04-20T21:21:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.861+0000 7f87e4600640 1 -- 172.21.15.102:0/1039787711 wait complete. 2024-04-20T21:21:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.863+0000 7f87e4600640 1 Processor -- start 2024-04-20T21:21:35.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.863+0000 7f87e4600640 1 -- start start 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e4600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87dc152170 0x7f87dc156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e4600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc158770 0x7f87dc1f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e4600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc1f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e4600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0c9f20 con 0x7f87dc158770 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e4600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc158250 con 0x7f87dc152170 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e4600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc157f20 con 0x7f87dc158b80 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc158770 0x7f87dc1f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc158770 0x7f87dc1f0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48154/0 (socket says 172.21.15.102:48154) 2024-04-20T21:21:35.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e1200640 1 -- 172.21.15.102:0/1731122193 learned_addr learned my addr 172.21.15.102:0/1731122193 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:35.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.864+0000 7f87e2600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc1f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:35.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e1c00640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87dc152170 0x7f87dc156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e2600640 1 -- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87dc152170 msgr2=0x7f87dc156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e2600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87dc152170 0x7f87dc156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e2600640 1 -- 172.21.15.102:0/1731122193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc158770 msgr2=0x7f87dc1f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e2600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc158770 0x7f87dc1f0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e2600640 1 -- 172.21.15.102:0/1731122193 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc0cc470 con 0x7f87dc158b80 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e1200640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc158770 0x7f87dc1f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e1c00640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87dc152170 0x7f87dc156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.865+0000 7f87e2600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc1f1450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f87d400eba0 tx=0x7f87d400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.866+0000 7f87d9600640 1 -- 172.21.15.102:0/1731122193 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4018020 con 0x7f87dc158b80 2024-04-20T21:21:35.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.866+0000 7f87d9600640 1 -- 172.21.15.102:0/1731122193 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d400ccc0 con 0x7f87dc158b80 2024-04-20T21:21:35.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.866+0000 7f87d9600640 1 -- 172.21.15.102:0/1731122193 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d40146c0 con 0x7f87dc158b80 2024-04-20T21:21:35.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.866+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc150a30 con 0x7f87dc158b80 2024-04-20T21:21:35.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.866+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc1573b0 con 0x7f87dc158b80 2024-04-20T21:21:35.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.867+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87dc074340 con 0x7f87dc158b80 2024-04-20T21:21:35.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.868+0000 7f87d9600640 1 -- 172.21.15.102:0/1731122193 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87d401b020 con 0x7f87dc158b80 2024-04-20T21:21:35.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.869+0000 7f87d9600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f87bc06c8f0 0x7f87bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:35.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.869+0000 7f87d9600640 1 -- 172.21.15.102:0/1731122193 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f87d408cb70 con 0x7f87dc158b80 2024-04-20T21:21:35.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.869+0000 7f87e1c00640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f87bc06c8f0 0x7f87bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:35.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.870+0000 7f87e1c00640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f87bc06c8f0 0x7f87bc06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f87d0036000 tx=0x7f87d0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:35.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:35.874+0000 7f87d9600640 1 -- 172.21.15.102:0/1731122193 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87d4022020 con 0x7f87dc158b80 2024-04-20T21:21:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.115+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87dc0c3460 con 0x7f87dc158b80 2024-04-20T21:21:36.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.117+0000 7f87d9600640 1 -- 172.21.15.102:0/1731122193 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6027 (secure 0 0 0) 0x7f87d4010040 con 0x7f87dc158b80 2024-04-20T21:21:36.116 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:36.116 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:31.863718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:36.117 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 18 2024-04-20T21:21:36.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f87bc06c8f0 msgr2=0x7f87bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f87bc06c8f0 0x7f87bc06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f87d0036000 tx=0x7f87d0008030 comp rx=0 tx=0).stop 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 msgr2=0x7f87dc1f1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc1f1450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f87d400eba0 tx=0x7f87d400c490 comp rx=0 tx=0).stop 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 shutdown_connections 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87dc158b80 0x7f87dc1f1450 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87dc158770 0x7f87dc1f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f87bc06c8f0 0x7f87bc06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 --2- 172.21.15.102:0/1731122193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87dc152170 0x7f87dc156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 >> 172.21.15.102:0/1731122193 conn(0x7f87dc0c9390 msgr2=0x7f87dc072ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 shutdown_connections 2024-04-20T21:21:36.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:36.119+0000 7f87e4600640 1 -- 172.21.15.102:0/1731122193 wait complete. 2024-04-20T21:21:36.137 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:21:38.139 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:38.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.345+0000 7f596b800640 1 Processor -- start 2024-04-20T21:21:38.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.346+0000 7f596b800640 1 -- start start 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f596b800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f596415ea20 0x7f596406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f596b800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415f2b0 0x7f59640710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f596b800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59640c8f20 con 0x7f596415ee00 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f596b800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f596406cb80 con 0x7f596415ea20 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f596b800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f596406ce60 con 0x7f596415f2b0 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f5969800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415f2b0 0x7f59640710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f5963e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f596415ee00 0x7f59640c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55998/0 (socket says 172.21.15.102:55998) 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f5963e00640 1 -- 172.21.15.102:0/2228487144 learned_addr learned my addr 172.21.15.102:0/2228487144 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.347+0000 7f5968e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f596415ea20 0x7f596406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.348+0000 7f5963400640 1 -- 172.21.15.102:0/2228487144 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3108583345 0 0) 0x7f59640c8f20 con 0x7f596415ee00 2024-04-20T21:21:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.348+0000 7f5963400640 1 -- 172.21.15.102:0/2228487144 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f594c003520 con 0x7f596415ee00 2024-04-20T21:21:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.348+0000 7f5969800640 1 -- 172.21.15.102:0/2228487144 >> v2:172.21.15.193:3300/0 conn(0x7f596415ea20 msgr2=0x7f596406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.348+0000 7f5969800640 1 --2- 172.21.15.102:0/2228487144 >> v2:172.21.15.193:3300/0 conn(0x7f596415ea20 0x7f596406d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.348+0000 7f5969800640 1 -- 172.21.15.102:0/2228487144 >> v1:172.21.15.193:6789/0 conn(0x7f596415ee00 legacy=0x7f59640c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.348+0000 7f5969800640 1 -- 172.21.15.102:0/2228487144 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596406d140 con 0x7f596415f2b0 2024-04-20T21:21:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.348+0000 7f5968e00640 1 --2- 172.21.15.102:0/2228487144 >> v2:172.21.15.193:3300/0 conn(0x7f596415ea20 0x7f596406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:38.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.349+0000 7f5969800640 1 --2- 172.21.15.102:0/2228487144 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415f2b0 0x7f59640710a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f595c009d90 tx=0x7f595c02bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d10035f185d44c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:38.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f5963400640 1 -- 172.21.15.102:0/2228487144 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c039070 con 0x7f596415f2b0 2024-04-20T21:21:38.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f5963400640 1 -- 172.21.15.102:0/2228487144 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595c02cce0 con 0x7f596415f2b0 2024-04-20T21:21:38.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f5963400640 1 -- 172.21.15.102:0/2228487144 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c034640 con 0x7f596415f2b0 2024-04-20T21:21:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f596b800640 1 -- 172.21.15.102:0/2228487144 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415f2b0 msgr2=0x7f59640710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f596b800640 1 --2- 172.21.15.102:0/2228487144 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415f2b0 0x7f59640710a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f595c009d90 tx=0x7f595c02bf30 comp rx=0 tx=0).stop 2024-04-20T21:21:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f596b800640 1 -- 172.21.15.102:0/2228487144 shutdown_connections 2024-04-20T21:21:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f596b800640 1 --2- 172.21.15.102:0/2228487144 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415f2b0 0x7f59640710a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f596b800640 1 --2- 172.21.15.102:0/2228487144 >> v2:172.21.15.193:3300/0 conn(0x7f596415ea20 0x7f596406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f596b800640 1 -- 172.21.15.102:0/2228487144 >> 172.21.15.102:0/2228487144 conn(0x7f59640c9390 msgr2=0x7f59640c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f596b800640 1 -- 172.21.15.102:0/2228487144 shutdown_connections 2024-04-20T21:21:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.350+0000 7f596b800640 1 -- 172.21.15.102:0/2228487144 wait complete. 2024-04-20T21:21:38.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.352+0000 7f596b800640 1 Processor -- start 2024-04-20T21:21:38.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.352+0000 7f596b800640 1 -- start start 2024-04-20T21:21:38.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f596b800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f596415ea20 0x7f59640cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:38.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f596b800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415ee00 0x7f59641f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:38.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f596b800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f596415f2b0 0x7f59641f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:38.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f596b800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f59640c8c80 con 0x7f596415ea20 2024-04-20T21:21:38.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f596b800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f59640c8b00 con 0x7f596415f2b0 2024-04-20T21:21:38.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f596b800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5964154e80 con 0x7f596415ee00 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f5968e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f596415ea20 0x7f59640cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f5968e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f596415ea20 0x7f59640cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56086/0 (socket says 172.21.15.102:56086) 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.353+0000 7f5968e00640 1 -- 172.21.15.102:0/344396587 learned_addr learned my addr 172.21.15.102:0/344396587 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5969800640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f596415f2b0 0x7f59641f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5963e00640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415ee00 0x7f59641f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5968e00640 1 -- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415ee00 msgr2=0x7f59641f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5968e00640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415ee00 0x7f59641f0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5968e00640 1 -- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f596415f2b0 msgr2=0x7f59641f1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5968e00640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f596415f2b0 0x7f59641f1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5968e00640 1 -- 172.21.15.102:0/344396587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59641f48d0 con 0x7f596415ea20 2024-04-20T21:21:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5968e00640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f596415ea20 0x7f59640cc020 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f595800ec10 tx=0x7f595800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.354+0000 7f5960c00640 1 -- 172.21.15.102:0/344396587 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958017070 con 0x7f596415ea20 2024-04-20T21:21:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.355+0000 7f5960c00640 1 -- 172.21.15.102:0/344396587 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595800cc30 con 0x7f596415ea20 2024-04-20T21:21:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.355+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59641f4bb0 con 0x7f596415ea20 2024-04-20T21:21:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.355+0000 7f5960c00640 1 -- 172.21.15.102:0/344396587 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958013470 con 0x7f596415ea20 2024-04-20T21:21:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.355+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59640c69c0 con 0x7f596415ea20 2024-04-20T21:21:38.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.356+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5964074340 con 0x7f596415ea20 2024-04-20T21:21:38.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.357+0000 7f5960c00640 1 -- 172.21.15.102:0/344396587 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f595801a070 con 0x7f596415ea20 2024-04-20T21:21:38.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.358+0000 7f5960c00640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f594406c920 0x7f594406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:38.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.358+0000 7f5960c00640 1 -- 172.21.15.102:0/344396587 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f595808c330 con 0x7f596415ea20 2024-04-20T21:21:38.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.358+0000 7f5963e00640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f594406c920 0x7f594406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:38.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.359+0000 7f5963e00640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f594406c920 0x7f594406ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f59640c65a0 tx=0x7f5950009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.363+0000 7f5960c00640 1 -- 172.21.15.102:0/344396587 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5958059b30 con 0x7f596415ea20 2024-04-20T21:21:38.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.603+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5964152780 con 0x7f596415ea20 2024-04-20T21:21:38.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.603+0000 7f5960c00640 1 -- 172.21.15.102:0/344396587 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6027 (secure 0 0 0) 0x7f595805de30 con 0x7f596415ea20 2024-04-20T21:21:38.603 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:38.603 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:31.863718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:38.603 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 18 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f594406c920 msgr2=0x7f594406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f594406c920 0x7f594406ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f59640c65a0 tx=0x7f5950009040 comp rx=0 tx=0).stop 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f596415ea20 msgr2=0x7f59640cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f596415ea20 0x7f59640cc020 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f595800ec10 tx=0x7f595800c520 comp rx=0 tx=0).stop 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 shutdown_connections 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f594406c920 0x7f594406ede0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f596415f2b0 0x7f59641f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f596415ee00 0x7f59641f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 --2- 172.21.15.102:0/344396587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f596415ea20 0x7f59640cc020 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:38.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.606+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 >> 172.21.15.102:0/344396587 conn(0x7f59640c9390 msgr2=0x7f59641533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:38.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.607+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 shutdown_connections 2024-04-20T21:21:38.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:38.607+0000 7f596b800640 1 -- 172.21.15.102:0/344396587 wait complete. 2024-04-20T21:21:38.626 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:21:40.627 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:40.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.834+0000 7f6904600640 1 Processor -- start 2024-04-20T21:21:40.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.834+0000 7f6904600640 1 -- start start 2024-04-20T21:21:40.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.835+0000 7f6904600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f68fc14fa50 0x7f68fc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:40.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.835+0000 7f6904600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f68fc0c9eb0 0x7f68fc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.835+0000 7f6904600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68fc1536d0 con 0x7f68fc073a10 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.835+0000 7f6904600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f68fc159500 con 0x7f68fc14fa50 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.835+0000 7f6904600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f68fc0c76c0 con 0x7f68fc0c9eb0 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.836+0000 7f6901c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f68fc14fa50 0x7f68fc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.836+0000 7f6901c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f68fc14fa50 0x7f68fc0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56102/0 (socket says 172.21.15.102:56102) 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.836+0000 7f6901c00640 1 -- 172.21.15.102:0/1785078026 learned_addr learned my addr 172.21.15.102:0/1785078026 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.836+0000 7f6901200640 1 --2- 172.21.15.102:0/1785078026 >> v2:172.21.15.193:3300/0 conn(0x7f68fc0c9eb0 0x7f68fc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.836+0000 7f6901c00640 1 -- 172.21.15.102:0/1785078026 >> v2:172.21.15.193:3300/0 conn(0x7f68fc0c9eb0 msgr2=0x7f68fc158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.836+0000 7f6901c00640 1 --2- 172.21.15.102:0/1785078026 >> v2:172.21.15.193:3300/0 conn(0x7f68fc0c9eb0 0x7f68fc158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.836+0000 7f6901c00640 1 -- 172.21.15.102:0/1785078026 >> v1:172.21.15.193:6789/0 conn(0x7f68fc073a10 legacy=0x7f68fc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:40.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.836+0000 7f6901c00640 1 -- 172.21.15.102:0/1785078026 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc0c7960 con 0x7f68fc14fa50 2024-04-20T21:21:40.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.837+0000 7f6901c00640 1 --2- 172.21.15.102:0/1785078026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc14fa50 0x7f68fc0c4830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68fc06a220 tx=0x7f68f002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744175f99a4c861d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:40.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f68fbe00640 1 -- 172.21.15.102:0/1785078026 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f002f850 con 0x7f68fc14fa50 2024-04-20T21:21:40.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f68fbe00640 1 -- 172.21.15.102:0/1785078026 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f002f9f0 con 0x7f68fc14fa50 2024-04-20T21:21:40.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f68fbe00640 1 -- 172.21.15.102:0/1785078026 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f002fdc0 con 0x7f68fc14fa50 2024-04-20T21:21:40.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f6904600640 1 -- 172.21.15.102:0/1785078026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc14fa50 msgr2=0x7f68fc0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:40.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f6904600640 1 --2- 172.21.15.102:0/1785078026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc14fa50 0x7f68fc0c4830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68fc06a220 tx=0x7f68f002ba30 comp rx=0 tx=0).stop 2024-04-20T21:21:40.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f6904600640 1 -- 172.21.15.102:0/1785078026 shutdown_connections 2024-04-20T21:21:40.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f6904600640 1 --2- 172.21.15.102:0/1785078026 >> v2:172.21.15.193:3300/0 conn(0x7f68fc0c9eb0 0x7f68fc158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:40.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f6904600640 1 --2- 172.21.15.102:0/1785078026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc14fa50 0x7f68fc0c4830 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:40.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f6904600640 1 -- 172.21.15.102:0/1785078026 >> 172.21.15.102:0/1785078026 conn(0x7f68fc0c9390 msgr2=0x7f68fc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:40.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f6904600640 1 -- 172.21.15.102:0/1785078026 shutdown_connections 2024-04-20T21:21:40.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.838+0000 7f6904600640 1 -- 172.21.15.102:0/1785078026 wait complete. 2024-04-20T21:21:40.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.840+0000 7f6904600640 1 Processor -- start 2024-04-20T21:21:40.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.840+0000 7f6904600640 1 -- start start 2024-04-20T21:21:40.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.841+0000 7f6904600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc073a10 0x7f68fc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:40.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.841+0000 7f6904600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68fc0c9eb0 0x7f68fc14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.841+0000 7f6904600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68fc14fa50 0x7f68fc1f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.841+0000 7f6904600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0c7c00 con 0x7f68fc073a10 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.841+0000 7f6904600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0c3d90 con 0x7f68fc14fa50 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.841+0000 7f6904600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc0c4030 con 0x7f68fc0c9eb0 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.841+0000 7f6901c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc073a10 0x7f68fc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc073a10 0x7f68fc071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56104/0 (socket says 172.21.15.102:56104) 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6902600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68fc14fa50 0x7f68fc1f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901c00640 1 -- 172.21.15.102:0/1004578378 learned_addr learned my addr 172.21.15.102:0/1004578378 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901200640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68fc0c9eb0 0x7f68fc14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901c00640 1 -- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68fc0c9eb0 msgr2=0x7f68fc14ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901c00640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68fc0c9eb0 0x7f68fc14ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901c00640 1 -- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68fc14fa50 msgr2=0x7f68fc1f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:40.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901c00640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68fc14fa50 0x7f68fc1f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:40.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901c00640 1 -- 172.21.15.102:0/1004578378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc1f4df0 con 0x7f68fc073a10 2024-04-20T21:21:40.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.842+0000 7f6901c00640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc073a10 0x7f68fc071cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68f002ba70 tx=0x7f68f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:40.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.843+0000 7f68f9600640 1 -- 172.21.15.102:0/1004578378 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0037d10 con 0x7f68fc073a10 2024-04-20T21:21:40.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.843+0000 7f68f9600640 1 -- 172.21.15.102:0/1004578378 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f0046070 con 0x7f68fc073a10 2024-04-20T21:21:40.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.843+0000 7f68f9600640 1 -- 172.21.15.102:0/1004578378 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0041640 con 0x7f68fc073a10 2024-04-20T21:21:40.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.843+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc071700 con 0x7f68fc073a10 2024-04-20T21:21:40.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.843+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc1f0f30 con 0x7f68fc073a10 2024-04-20T21:21:40.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.844+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc1f0d40 con 0x7f68fc073a10 2024-04-20T21:21:40.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.845+0000 7f68f9600640 1 -- 172.21.15.102:0/1004578378 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68f0040070 con 0x7f68fc073a10 2024-04-20T21:21:40.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.846+0000 7f68f9600640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f68dc06c8f0 0x7f68dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:40.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.846+0000 7f68f9600640 1 -- 172.21.15.102:0/1004578378 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f68f00ae5a0 con 0x7f68fc073a10 2024-04-20T21:21:40.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.846+0000 7f6901200640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f68dc06c8f0 0x7f68dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:40.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.847+0000 7f6901200640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f68dc06c8f0 0x7f68dc06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68fc06e0e0 tx=0x7f68e8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:40.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:40.850+0000 7f68f9600640 1 -- 172.21.15.102:0/1004578378 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f68f0033030 con 0x7f68fc073a10 2024-04-20T21:21:41.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.090+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68fc074340 con 0x7f68fc073a10 2024-04-20T21:21:41.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.092+0000 7f68f9600640 1 -- 172.21.15.102:0/1004578378 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+6033 (secure 0 0 0) 0x7f68f007be30 con 0x7f68fc073a10 2024-04-20T21:21:41.091 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:41.092 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:38.058571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:clientreplay","state_seq":25,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:41.092 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 19 2024-04-20T21:21:41.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f68dc06c8f0 msgr2=0x7f68dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:41.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f68dc06c8f0 0x7f68dc06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68fc06e0e0 tx=0x7f68e8009190 comp rx=0 tx=0).stop 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc073a10 msgr2=0x7f68fc071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc073a10 0x7f68fc071cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68f002ba70 tx=0x7f68f0004730 comp rx=0 tx=0).stop 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 shutdown_connections 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f68dc06c8f0 0x7f68dc06edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68fc14fa50 0x7f68fc1f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68fc0c9eb0 0x7f68fc14ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 --2- 172.21.15.102:0/1004578378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68fc073a10 0x7f68fc071cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 >> 172.21.15.102:0/1004578378 conn(0x7f68fc0c9390 msgr2=0x7f68fc06c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 shutdown_connections 2024-04-20T21:21:41.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:41.095+0000 7f6904600640 1 -- 172.21.15.102:0/1004578378 wait complete. 2024-04-20T21:21:41.113 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:21:43.114 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:43.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.326+0000 7f7592e00640 1 Processor -- start 2024-04-20T21:21:43.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.326+0000 7f7592e00640 1 -- start start 2024-04-20T21:21:43.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.327+0000 7f7592e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15ea20 0x7f758c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:43.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.328+0000 7f7592e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758c0c8f20 con 0x7f758c15f2b0 2024-04-20T21:21:43.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.328+0000 7f7592e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758c06cb80 con 0x7f758c15ee00 2024-04-20T21:21:43.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.328+0000 7f7592e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f758c06ce60 con 0x7f758c15ea20 2024-04-20T21:21:43.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.328+0000 7f7590e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f758c15f2b0 0x7f758c154e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40554/0 (socket says 172.21.15.102:40554) 2024-04-20T21:21:43.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.328+0000 7f7590e00640 1 -- 172.21.15.102:0/3021905273 learned_addr learned my addr 172.21.15.102:0/3021905273 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:43.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.328+0000 7f758be00640 1 --2- 172.21.15.102:0/3021905273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15ea20 0x7f758c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:43.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.328+0000 7f758b400640 1 -- 172.21.15.102:0/3021905273 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237337709 0 0) 0x7f758c0c8f20 con 0x7f758c15f2b0 2024-04-20T21:21:43.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.328+0000 7f758b400640 1 -- 172.21.15.102:0/3021905273 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7570003520 con 0x7f758c15f2b0 2024-04-20T21:21:43.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.329+0000 7f758be00640 1 -- 172.21.15.102:0/3021905273 >> v1:172.21.15.193:6789/0 conn(0x7f758c15ee00 legacy=0x7f758c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:43.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.329+0000 7f758be00640 1 -- 172.21.15.102:0/3021905273 >> v1:172.21.15.102:6789/0 conn(0x7f758c15f2b0 legacy=0x7f758c154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:43.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.329+0000 7f758be00640 1 -- 172.21.15.102:0/3021905273 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c06d140 con 0x7f758c15ea20 2024-04-20T21:21:43.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.330+0000 7f758be00640 1 --2- 172.21.15.102:0/3021905273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15ea20 0x7f758c06d940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f757c009750 tx=0x7f757c02b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd7619ead3a1864 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:43.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.330+0000 7f758b400640 1 -- 172.21.15.102:0/3021905273 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c038070 con 0x7f758c15ea20 2024-04-20T21:21:43.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.330+0000 7f758b400640 1 -- 172.21.15.102:0/3021905273 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c031710 con 0x7f758c15ea20 2024-04-20T21:21:43.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.330+0000 7f758b400640 1 -- 172.21.15.102:0/3021905273 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c031ae0 con 0x7f758c15ea20 2024-04-20T21:21:43.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.330+0000 7f7592e00640 1 -- 172.21.15.102:0/3021905273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15ea20 msgr2=0x7f758c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:43.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.330+0000 7f7592e00640 1 --2- 172.21.15.102:0/3021905273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15ea20 0x7f758c06d940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f757c009750 tx=0x7f757c02b7b0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.331+0000 7f7592e00640 1 -- 172.21.15.102:0/3021905273 shutdown_connections 2024-04-20T21:21:43.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.331+0000 7f7592e00640 1 --2- 172.21.15.102:0/3021905273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15ea20 0x7f758c06d940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.331+0000 7f7592e00640 1 -- 172.21.15.102:0/3021905273 >> 172.21.15.102:0/3021905273 conn(0x7f758c0c9390 msgr2=0x7f758c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:43.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.331+0000 7f7592e00640 1 -- 172.21.15.102:0/3021905273 shutdown_connections 2024-04-20T21:21:43.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.331+0000 7f7592e00640 1 -- 172.21.15.102:0/3021905273 wait complete. 2024-04-20T21:21:43.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.333+0000 7f7592e00640 1 Processor -- start 2024-04-20T21:21:43.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.333+0000 7f7592e00640 1 -- start start 2024-04-20T21:21:43.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7592e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f758c15ea20 0x7f758c071c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7592e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f758c15ee00 0x7f758c1f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7592e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15f2b0 0x7f758c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7592e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f758c0c8c80 con 0x7f758c15ea20 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7592e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f758c0c8b00 con 0x7f758c15ee00 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7592e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f758c155590 con 0x7f758c15f2b0 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f758be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f758c15ea20 0x7f758c071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f758be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f758c15ea20 0x7f758c071c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56120/0 (socket says 172.21.15.102:56120) 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f758be00640 1 -- 172.21.15.102:0/769550101 learned_addr learned my addr 172.21.15.102:0/769550101 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7590e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15f2b0 0x7f758c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7583e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f758c15ee00 0x7f758c1f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f758be00640 1 -- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15f2b0 msgr2=0x7f758c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f758be00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15f2b0 0x7f758c15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f758be00640 1 -- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f758c15ee00 msgr2=0x7f758c1f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f758be00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f758c15ee00 0x7f758c1f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f758be00640 1 -- 172.21.15.102:0/769550101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c072250 con 0x7f758c15ea20 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.334+0000 7f7583e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f758c15ee00 0x7f758c1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.335+0000 7f758be00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f758c15ea20 0x7f758c071c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f757c009870 tx=0x7f757c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.335+0000 7f7588c00640 1 -- 172.21.15.102:0/769550101 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c042070 con 0x7f758c15ea20 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.335+0000 7f7588c00640 1 -- 172.21.15.102:0/769550101 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c03d890 con 0x7f758c15ea20 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.335+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c072530 con 0x7f758c15ea20 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.335+0000 7f7588c00640 1 -- 172.21.15.102:0/769550101 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c038040 con 0x7f758c15ea20 2024-04-20T21:21:43.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.335+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c1f13e0 con 0x7f758c15ea20 2024-04-20T21:21:43.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.336+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c074340 con 0x7f758c15ea20 2024-04-20T21:21:43.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.337+0000 7f7588c00640 1 -- 172.21.15.102:0/769550101 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f757c045070 con 0x7f758c15ea20 2024-04-20T21:21:43.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.338+0000 7f7588c00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f756806c8f0 0x7f756806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:43.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.338+0000 7f7588c00640 1 -- 172.21.15.102:0/769550101 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f757c0add10 con 0x7f758c15ea20 2024-04-20T21:21:43.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.339+0000 7f7583e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f756806c8f0 0x7f756806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:43.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.339+0000 7f7583e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f756806c8f0 0x7f756806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f758c158ce0 tx=0x7f7574009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:43.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.342+0000 7f7588c00640 1 -- 172.21.15.102:0/769550101 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f757c0b3020 con 0x7f758c15ea20 2024-04-20T21:21:43.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.583+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f758c15c040 con 0x7f758c15ea20 2024-04-20T21:21:43.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.584+0000 7f7588c00640 1 -- 172.21.15.102:0/769550101 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+6033 (secure 0 0 0) 0x7f757c07b630 con 0x7f758c15ea20 2024-04-20T21:21:43.584 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:43.584 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:38.058571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:clientreplay","state_seq":25,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:21:43.584 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 19 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f756806c8f0 msgr2=0x7f756806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f756806c8f0 0x7f756806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f758c158ce0 tx=0x7f7574009040 comp rx=0 tx=0).stop 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f758c15ea20 msgr2=0x7f758c071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f758c15ea20 0x7f758c071c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f757c009870 tx=0x7f757c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 shutdown_connections 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f758c15f2b0 0x7f758c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f756806c8f0 0x7f756806edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f758c15ee00 0x7f758c1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 --2- 172.21.15.102:0/769550101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f758c15ea20 0x7f758c071c30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 >> 172.21.15.102:0/769550101 conn(0x7f758c0c9390 msgr2=0x7f758c153150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:43.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 shutdown_connections 2024-04-20T21:21:43.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:43.587+0000 7f7592e00640 1 -- 172.21.15.102:0/769550101 wait complete. 2024-04-20T21:21:43.605 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:21:45.606 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:45.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.811+0000 7f2074600640 1 Processor -- start 2024-04-20T21:21:45.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.811+0000 7f2074600640 1 -- start start 2024-04-20T21:21:45.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.813+0000 7f2074600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f206c14fa50 0x7f206c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:45.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.813+0000 7f2074600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f206c1536d0 con 0x7f206c0c9eb0 2024-04-20T21:21:45.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.813+0000 7f2074600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f206c0c76c0 con 0x7f206c073a10 2024-04-20T21:21:45.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.813+0000 7f2074600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f206c0c7960 con 0x7f206c14fa50 2024-04-20T21:21:45.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.813+0000 7f2071200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f206c0c9eb0 0x7f206c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40564/0 (socket says 172.21.15.102:40564) 2024-04-20T21:21:45.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.813+0000 7f2071200640 1 -- 172.21.15.102:0/2612205886 learned_addr learned my addr 172.21.15.102:0/2612205886 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:21:45.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.813+0000 7f2071c00640 1 --2- 172.21.15.102:0/2612205886 >> v2:172.21.15.102:3300/0 conn(0x7f206c14fa50 0x7f206c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:45.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.813+0000 7f206be00640 1 -- 172.21.15.102:0/2612205886 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833094973 0 0) 0x7f2058001110 con 0x7f206c0c9eb0 2024-04-20T21:21:45.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.814+0000 7f206be00640 1 -- 172.21.15.102:0/2612205886 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2054003520 con 0x7f206c0c9eb0 2024-04-20T21:21:45.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.814+0000 7f206be00640 1 -- 172.21.15.102:0/2612205886 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3655684399 0 0) 0x7f206c0c76c0 con 0x7f206c073a10 2024-04-20T21:21:45.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.814+0000 7f206be00640 1 -- 172.21.15.102:0/2612205886 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2058001110 con 0x7f206c073a10 2024-04-20T21:21:45.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.814+0000 7f2071c00640 1 -- 172.21.15.102:0/2612205886 >> v1:172.21.15.193:6789/0 conn(0x7f206c073a10 legacy=0x7f206c163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:45.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.814+0000 7f2071c00640 1 -- 172.21.15.102:0/2612205886 >> v1:172.21.15.102:6789/0 conn(0x7f206c0c9eb0 legacy=0x7f206c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:45.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.814+0000 7f2071c00640 1 -- 172.21.15.102:0/2612205886 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c0c3d90 con 0x7f206c14fa50 2024-04-20T21:21:45.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.815+0000 7f2071c00640 1 --2- 172.21.15.102:0/2612205886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c4830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f206c0c7890 tx=0x7f206002b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a881bdd31a222d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:45.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.815+0000 7f206be00640 1 -- 172.21.15.102:0/2612205886 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060034070 con 0x7f206c14fa50 2024-04-20T21:21:45.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.815+0000 7f206be00640 1 -- 172.21.15.102:0/2612205886 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2060030890 con 0x7f206c14fa50 2024-04-20T21:21:45.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.815+0000 7f206be00640 1 -- 172.21.15.102:0/2612205886 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060030c60 con 0x7f206c14fa50 2024-04-20T21:21:45.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.816+0000 7f2074600640 1 -- 172.21.15.102:0/2612205886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 msgr2=0x7f206c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:45.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.816+0000 7f2074600640 1 --2- 172.21.15.102:0/2612205886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c4830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f206c0c7890 tx=0x7f206002b9d0 comp rx=0 tx=0).stop 2024-04-20T21:21:45.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.816+0000 7f2074600640 1 -- 172.21.15.102:0/2612205886 shutdown_connections 2024-04-20T21:21:45.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.816+0000 7f2074600640 1 --2- 172.21.15.102:0/2612205886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c4830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:45.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.816+0000 7f2074600640 1 -- 172.21.15.102:0/2612205886 >> 172.21.15.102:0/2612205886 conn(0x7f206c0c9390 msgr2=0x7f206c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:45.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.816+0000 7f2074600640 1 -- 172.21.15.102:0/2612205886 shutdown_connections 2024-04-20T21:21:45.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.816+0000 7f2074600640 1 -- 172.21.15.102:0/2612205886 wait complete. 2024-04-20T21:21:45.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.818+0000 7f2074600640 1 Processor -- start 2024-04-20T21:21:45.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.818+0000 7f2074600640 1 -- start start 2024-04-20T21:21:45.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2074600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f206c073a10 0x7f206c1f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:45.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2074600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f206c0c9eb0 0x7f206c1e2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:45.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2074600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:45.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2074600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f206c0c4030 con 0x7f206c14fa50 2024-04-20T21:21:45.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2074600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f206c0c7c00 con 0x7f206c0c9eb0 2024-04-20T21:21:45.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2074600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f206c0c4310 con 0x7f206c073a10 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2072600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2072600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56152/0 (socket says 172.21.15.102:56152) 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2072600640 1 -- 172.21.15.102:0/1282391988 learned_addr learned my addr 172.21.15.102:0/1282391988 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2071200640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f206c0c9eb0 0x7f206c1e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.819+0000 7f2071c00640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f206c073a10 0x7f206c1f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2072600640 1 -- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f206c073a10 msgr2=0x7f206c1f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2072600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f206c073a10 0x7f206c1f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2072600640 1 -- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f206c0c9eb0 msgr2=0x7f206c1e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2072600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f206c0c9eb0 0x7f206c1e2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2072600640 1 -- 172.21.15.102:0/1282391988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c1e2c40 con 0x7f206c14fa50 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2071200640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f206c0c9eb0 0x7f206c1e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:45.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2071c00640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f206c073a10 0x7f206c1f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:21:45.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2072600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c8aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f206400c7c0 tx=0x7f206400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:45.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2069600640 1 -- 172.21.15.102:0/1282391988 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064003b70 con 0x7f206c14fa50 2024-04-20T21:21:45.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2069600640 1 -- 172.21.15.102:0/1282391988 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2064003d10 con 0x7f206c14fa50 2024-04-20T21:21:45.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.820+0000 7f2069600640 1 -- 172.21.15.102:0/1282391988 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206400aa50 con 0x7f206c14fa50 2024-04-20T21:21:45.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.821+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206c1e2f00 con 0x7f206c14fa50 2024-04-20T21:21:45.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.821+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f206c072390 con 0x7f206c14fa50 2024-04-20T21:21:45.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.822+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206c0c56b0 con 0x7f206c14fa50 2024-04-20T21:21:45.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.823+0000 7f2069600640 1 -- 172.21.15.102:0/1282391988 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f206400abf0 con 0x7f206c14fa50 2024-04-20T21:21:45.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.824+0000 7f2069600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204c06c920 0x7f204c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:45.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.824+0000 7f2069600640 1 -- 172.21.15.102:0/1282391988 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f206408be40 con 0x7f206c14fa50 2024-04-20T21:21:45.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.824+0000 7f2071c00640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204c06c920 0x7f204c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:45.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.825+0000 7f2071c00640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204c06c920 0x7f204c06ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2060036000 tx=0x7f206003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:45.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:45.829+0000 7f2069600640 1 -- 172.21.15.102:0/1282391988 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f206c0c56b0 con 0x7f206c14fa50 2024-04-20T21:21:46.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.067+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f206c069340 con 0x7f206c14fa50 2024-04-20T21:21:46.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.068+0000 7f2069600640 1 -- 172.21.15.102:0/1282391988 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+6034 (secure 0 0 0) 0x7f206c069340 con 0x7f206c14fa50 2024-04-20T21:21:46.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:46.068 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:21:46.069 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-20T21:21:46.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204c06c920 msgr2=0x7f204c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:46.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204c06c920 0x7f204c06ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2060036000 tx=0x7f206003b040 comp rx=0 tx=0).stop 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 msgr2=0x7f206c0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c8aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f206400c7c0 tx=0x7f206400cc80 comp rx=0 tx=0).stop 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 shutdown_connections 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f206c14fa50 0x7f206c0c8aa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204c06c920 0x7f204c06ede0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f206c0c9eb0 0x7f206c1e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 --2- 172.21.15.102:0/1282391988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f206c073a10 0x7f206c1f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 >> 172.21.15.102:0/1282391988 conn(0x7f206c0c9390 msgr2=0x7f206c06c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 shutdown_connections 2024-04-20T21:21:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:46.071+0000 7f2074600640 1 -- 172.21.15.102:0/1282391988 wait complete. 2024-04-20T21:21:46.089 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=4297 2024-04-20T21:21:46.089 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.a 2024-04-20T21:21:50.090 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-04-20T21:21:50.090 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:21:50.090 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:21:50.099 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:21:50.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:50.267 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-20T21:21:50.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.351+0000 7f21ea800640 1 Processor -- start 2024-04-20T21:21:50.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.351+0000 7f21ea800640 1 -- start start 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21ea800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f21e40cabd0 0x7f21e4159030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21ea800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e40c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21ea800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e4073320 con 0x7f21e4153490 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21ea800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f21e4075e10 con 0x7f21e40cabd0 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21ea800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f21e4073900 con 0x7f21e4159570 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21e3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e40c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21e3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e40c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:46160/0 (socket says 172.21.15.102:46160) 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21e3e00640 1 -- 172.21.15.102:0/1385837015 learned_addr learned my addr 172.21.15.102:0/1385837015 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:50.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.353+0000 7f21daa00640 1 --2- 172.21.15.102:0/1385837015 >> v2:172.21.15.102:3300/0 conn(0x7f21e40cabd0 0x7f21e4159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.354+0000 7f21e3e00640 1 -- 172.21.15.102:0/1385837015 >> v2:172.21.15.102:3300/0 conn(0x7f21e40cabd0 msgr2=0x7f21e4159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.354+0000 7f21e3e00640 1 --2- 172.21.15.102:0/1385837015 >> v2:172.21.15.102:3300/0 conn(0x7f21e40cabd0 0x7f21e4159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.354+0000 7f21e3e00640 1 -- 172.21.15.102:0/1385837015 >> v1:172.21.15.102:6789/0 conn(0x7f21e4153490 legacy=0x7f21e40c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.354+0000 7f21e3e00640 1 -- 172.21.15.102:0/1385837015 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e4073d90 con 0x7f21e4159570 2024-04-20T21:21:50.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.355+0000 7f21e3e00640 1 --2- 172.21.15.102:0/1385837015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e40c7050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009870 tx=0x7f21dc02ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76e68946571f7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.355+0000 7f21e2a00640 1 -- 172.21.15.102:0/1385837015 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc02f830 con 0x7f21e4159570 2024-04-20T21:21:50.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.356+0000 7f21e2a00640 1 -- 172.21.15.102:0/1385837015 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc02f9d0 con 0x7f21e4159570 2024-04-20T21:21:50.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.356+0000 7f21e2a00640 1 -- 172.21.15.102:0/1385837015 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc02fda0 con 0x7f21e4159570 2024-04-20T21:21:50.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.356+0000 7f21ea800640 1 -- 172.21.15.102:0/1385837015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 msgr2=0x7f21e40c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.356+0000 7f21ea800640 1 --2- 172.21.15.102:0/1385837015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e40c7050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009870 tx=0x7f21dc02ba10 comp rx=0 tx=0).stop 2024-04-20T21:21:50.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.357+0000 7f21ea800640 1 -- 172.21.15.102:0/1385837015 shutdown_connections 2024-04-20T21:21:50.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.357+0000 7f21ea800640 1 --2- 172.21.15.102:0/1385837015 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e40c7050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.357+0000 7f21ea800640 1 --2- 172.21.15.102:0/1385837015 >> v2:172.21.15.102:3300/0 conn(0x7f21e40cabd0 0x7f21e4159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.357+0000 7f21ea800640 1 -- 172.21.15.102:0/1385837015 >> 172.21.15.102:0/1385837015 conn(0x7f21e406fa00 msgr2=0x7f21e406fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.357+0000 7f21ea800640 1 -- 172.21.15.102:0/1385837015 shutdown_connections 2024-04-20T21:21:50.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.357+0000 7f21ea800640 1 -- 172.21.15.102:0/1385837015 wait complete. 2024-04-20T21:21:50.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.359+0000 7f21ea800640 1 Processor -- start 2024-04-20T21:21:50.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.359+0000 7f21ea800640 1 -- start start 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21ea800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21e40cabd0 0x7f21e40ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21e4153490 0x7f21e41422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21ea800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e41442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21ea800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f21e40c8600 con 0x7f21e4153490 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21ea800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21e40cb040 con 0x7f21e40cabd0 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21ea800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f21e4073590 con 0x7f21e4159570 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21e40cabd0 0x7f21e40ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21daa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21e4153490 0x7f21e41422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21e40cabd0 0x7f21e40ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58020/0 (socket says 172.21.15.102:58020) 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21e3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e41442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.360+0000 7f21e3400640 1 -- 172.21.15.102:0/4165346988 learned_addr learned my addr 172.21.15.102:0/4165346988 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:50.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.361+0000 7f21e3e00640 1 -- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21e40cabd0 msgr2=0x7f21e40ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.361+0000 7f21e3e00640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21e40cabd0 0x7f21e40ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.361+0000 7f21e3e00640 1 -- 172.21.15.102:0/4165346988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21e4153490 msgr2=0x7f21e41422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.361+0000 7f21e3e00640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21e4153490 0x7f21e41422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.361+0000 7f21e3e00640 1 -- 172.21.15.102:0/4165346988 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e40cd0c0 con 0x7f21e4159570 2024-04-20T21:21:50.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.361+0000 7f21e3e00640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e41442d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21dc033000 tx=0x7f21dc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.362+0000 7f21dbe00640 1 -- 172.21.15.102:0/4165346988 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc037dd0 con 0x7f21e4159570 2024-04-20T21:21:50.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.362+0000 7f21dbe00640 1 -- 172.21.15.102:0/4165346988 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc046070 con 0x7f21e4159570 2024-04-20T21:21:50.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.362+0000 7f21dbe00640 1 -- 172.21.15.102:0/4165346988 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc041640 con 0x7f21e4159570 2024-04-20T21:21:50.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.362+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e4142830 con 0x7f21e4159570 2024-04-20T21:21:50.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.362+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e4142ac0 con 0x7f21e4159570 2024-04-20T21:21:50.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.364+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e4068c30 con 0x7f21e4159570 2024-04-20T21:21:50.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.364+0000 7f21dbe00640 1 -- 172.21.15.102:0/4165346988 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21dc040070 con 0x7f21e4159570 2024-04-20T21:21:50.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.365+0000 7f21dbe00640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21c006c940 0x7f21c006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.365+0000 7f21dbe00640 1 -- 172.21.15.102:0/4165346988 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f21dc0ae360 con 0x7f21e4159570 2024-04-20T21:21:50.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.365+0000 7f21e3400640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21c006c940 0x7f21c006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.366+0000 7f21e3400640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21c006c940 0x7f21c006ee00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21d4007fd0 tx=0x7f21d4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.369+0000 7f21dbe00640 1 -- 172.21.15.102:0/4165346988 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f21e40cd0c0 con 0x7f21e4159570 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2024-04-20T21:21:50.415 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2024-04-20T21:21:50.416 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2024-04-20T21:21:50.417 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2024-04-20T21:21:50.418 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2024-04-20T21:21:50.419 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2024-04-20T21:21:50.420 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2024-04-20T21:21:50.614 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:21:50.614+0000 7fcab4200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:21:50.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.646+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21e414d510 con 0x7f21e4159570 2024-04-20T21:21:50.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.647+0000 7f21dbe00640 1 -- 172.21.15.102:0/4165346988 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+6034 (secure 0 0 0) 0x7f21dc033030 con 0x7f21e4159570 2024-04-20T21:21:50.646 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:50.647 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:21:50.647 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-20T21:21:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.649+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21c006c940 msgr2=0x7f21c006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.649+0000 7f21ea800640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21c006c940 0x7f21c006ee00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21d4007fd0 tx=0x7f21d4009040 comp rx=0 tx=0).stop 2024-04-20T21:21:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.649+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 msgr2=0x7f21e41442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.649+0000 7f21ea800640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e41442d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21dc033000 tx=0x7f21dc004730 comp rx=0 tx=0).stop 2024-04-20T21:21:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.650+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 shutdown_connections 2024-04-20T21:21:50.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.650+0000 7f21ea800640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21e4159570 0x7f21e41442d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.650+0000 7f21ea800640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21e4153490 0x7f21e41422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.650+0000 7f21ea800640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21c006c940 0x7f21c006ee00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.650+0000 7f21ea800640 1 --2- 172.21.15.102:0/4165346988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21e40cabd0 0x7f21e40ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.650+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 >> 172.21.15.102:0/4165346988 conn(0x7f21e406fa00 msgr2=0x7f21e414f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.650+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 shutdown_connections 2024-04-20T21:21:50.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:50.650+0000 7f21ea800640 1 -- 172.21.15.102:0/4165346988 wait complete. 2024-04-20T21:21:50.667 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-04-20T21:21:51.882 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2024-04-20T21:21:51.882 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2024-04-20T21:21:51.883 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2024-04-20T21:21:51.884 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2024-04-20T21:21:51.885 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2024-04-20T21:21:51.886 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2024-04-20T21:21:51.887 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2024-04-20T21:21:51.888 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2024-04-20T21:21:51.888 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2024-04-20T21:21:51.888 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2024-04-20T21:21:51.888 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2024-04-20T21:21:51.888 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2024-04-20T21:21:51.888 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2024-04-20T21:21:52.295 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:52.303 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:52.668 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2024-04-20T21:21:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2024-04-20T21:21:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2024-04-20T21:21:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2024-04-20T21:21:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2024-04-20T21:21:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2024-04-20T21:21:52.872 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2024-04-20T21:21:52.873 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2024-04-20T21:21:52.873 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2024-04-20T21:21:52.873 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2024-04-20T21:21:52.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.884+0000 7f13e9200640 1 Processor -- start 2024-04-20T21:21:52.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.884+0000 7f13e9200640 1 -- start start 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e9200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f13e4152320 0x7f13e40c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e9200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f13e40ca720 0x7f13e415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e9200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e4072be0 con 0x7f13e415c2a0 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e9200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f13e40ca340 con 0x7f13e4152320 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e9200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f13e415b5f0 con 0x7f13e40ca720 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f13e4152320 0x7f13e40c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f13e4152320 0x7f13e40c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50568/0 (socket says 172.21.15.102:50568) 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e3400640 1 -- 172.21.15.102:0/1046661364 learned_addr learned my addr 172.21.15.102:0/1046661364 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:52.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.886+0000 7f13e2a00640 1 --2- 172.21.15.102:0/1046661364 >> v2:172.21.15.193:3300/0 conn(0x7f13e40ca720 0x7f13e415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:52.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.887+0000 7f13e2000640 1 -- 172.21.15.102:0/1046661364 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3636622478 0 0) 0x7f13e4072be0 con 0x7f13e415c2a0 2024-04-20T21:21:52.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.887+0000 7f13e3400640 1 -- 172.21.15.102:0/1046661364 >> v2:172.21.15.193:3300/0 conn(0x7f13e40ca720 msgr2=0x7f13e415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:52.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.887+0000 7f13e3400640 1 --2- 172.21.15.102:0/1046661364 >> v2:172.21.15.193:3300/0 conn(0x7f13e40ca720 0x7f13e415bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:52.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.887+0000 7f13e3400640 1 -- 172.21.15.102:0/1046661364 >> v1:172.21.15.193:6789/0 conn(0x7f13e415c2a0 legacy=0x7f13e40c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:52.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.887+0000 7f13e3400640 1 -- 172.21.15.102:0/1046661364 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e4072e50 con 0x7f13e4152320 2024-04-20T21:21:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.888+0000 7f13e3400640 1 --2- 172.21.15.102:0/1046661364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 0x7f13e40c9e00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13e40c7910 tx=0x7f13dc02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a2a12111783358 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.888+0000 7f13e2000640 1 -- 172.21.15.102:0/1046661364 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc02cb10 con 0x7f13e4152320 2024-04-20T21:21:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e2000640 1 -- 172.21.15.102:0/1046661364 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13dc02ccb0 con 0x7f13e4152320 2024-04-20T21:21:52.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e2000640 1 -- 172.21.15.102:0/1046661364 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc02ce60 con 0x7f13e4152320 2024-04-20T21:21:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e9200640 1 -- 172.21.15.102:0/1046661364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 msgr2=0x7f13e40c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e9200640 1 --2- 172.21.15.102:0/1046661364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 0x7f13e40c9e00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13e40c7910 tx=0x7f13dc02bf70 comp rx=0 tx=0).stop 2024-04-20T21:21:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e9200640 1 -- 172.21.15.102:0/1046661364 shutdown_connections 2024-04-20T21:21:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e9200640 1 --2- 172.21.15.102:0/1046661364 >> v2:172.21.15.193:3300/0 conn(0x7f13e40ca720 0x7f13e415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:52.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e9200640 1 --2- 172.21.15.102:0/1046661364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 0x7f13e40c9e00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e9200640 1 -- 172.21.15.102:0/1046661364 >> 172.21.15.102:0/1046661364 conn(0x7f13e406b940 msgr2=0x7f13e406bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e9200640 1 -- 172.21.15.102:0/1046661364 shutdown_connections 2024-04-20T21:21:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.889+0000 7f13e9200640 1 -- 172.21.15.102:0/1046661364 wait complete. 2024-04-20T21:21:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.891+0000 7f13e9200640 1 Processor -- start 2024-04-20T21:21:52.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.891+0000 7f13e9200640 1 -- start start 2024-04-20T21:21:52.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.892+0000 7f13e9200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e40ca720 0x7f13e40cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.892+0000 7f13e9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 0x7f13e40cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.892+0000 7f13e9200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13e415c2a0 0x7f13e40c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.892+0000 7f13e9200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4073410 con 0x7f13e4152320 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.892+0000 7f13e9200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13e415b8d0 con 0x7f13e40ca720 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.892+0000 7f13e9200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f13e40736f0 con 0x7f13e415c2a0 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e40ca720 0x7f13e40cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e40ca720 0x7f13e40cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58042/0 (socket says 172.21.15.102:58042) 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 -- 172.21.15.102:0/3475186322 learned_addr learned my addr 172.21.15.102:0/3475186322 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3e00640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13e415c2a0 0x7f13e40c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e2a00640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 0x7f13e40cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 -- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13e415c2a0 msgr2=0x7f13e40c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13e415c2a0 0x7f13e40c2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 -- 172.21.15.102:0/3475186322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 msgr2=0x7f13e40cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:52.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 0x7f13e40cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:52.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 -- 172.21.15.102:0/3475186322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e4147d50 con 0x7f13e40ca720 2024-04-20T21:21:52.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.893+0000 7f13e3400640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e40ca720 0x7f13e40cd4d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13dc02c580 tx=0x7f13dc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:52.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.894+0000 7f13db400640 1 -- 172.21.15.102:0/3475186322 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc043020 con 0x7f13e40ca720 2024-04-20T21:21:52.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.894+0000 7f13db400640 1 -- 172.21.15.102:0/3475186322 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13dc03e450 con 0x7f13e40ca720 2024-04-20T21:21:52.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.894+0000 7f13db400640 1 -- 172.21.15.102:0/3475186322 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc02ce60 con 0x7f13e40ca720 2024-04-20T21:21:52.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.894+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e4148030 con 0x7f13e40ca720 2024-04-20T21:21:52.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.894+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e41f1950 con 0x7f13e40ca720 2024-04-20T21:21:52.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.895+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e404fd60 con 0x7f13e40ca720 2024-04-20T21:21:52.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.896+0000 7f13db400640 1 -- 172.21.15.102:0/3475186322 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13dc046020 con 0x7f13e40ca720 2024-04-20T21:21:52.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.897+0000 7f13db400640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f13c406c920 0x7f13c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:52.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.897+0000 7f13db400640 1 -- 172.21.15.102:0/3475186322 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f13dc0ae360 con 0x7f13e40ca720 2024-04-20T21:21:52.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.898+0000 7f13e2a00640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f13c406c920 0x7f13c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:52.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.898+0000 7f13e2a00640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f13c406c920 0x7f13c406ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f13e40afb00 tx=0x7f13d0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:52.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:52.902+0000 7f13db400640 1 -- 172.21.15.102:0/3475186322 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f13dc04d070 con 0x7f13e40ca720 2024-04-20T21:21:53.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.164+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13e40ca500 con 0x7f13e40ca720 2024-04-20T21:21:53.164 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2024-04-20T21:21:53.164 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2024-04-20T21:21:53.164 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2024-04-20T21:21:53.164 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2024-04-20T21:21:53.164 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2024-04-20T21:21:53.164 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2024-04-20T21:21:53.165 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2024-04-20T21:21:53.166 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2024-04-20T21:21:53.167 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2024-04-20T21:21:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2024-04-20T21:21:53.169 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2024-04-20T21:21:53.170 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2024-04-20T21:21:53.170 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2024-04-20T21:21:53.170 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2024-04-20T21:21:53.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.165+0000 7f13db400640 1 -- 172.21.15.102:0/3475186322 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6859 (secure 0 0 0) 0x7f13dc07ce20 con 0x7f13e40ca720 2024-04-20T21:21:53.170 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:21:53.170 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:21:53.171 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-04-20T21:21:53.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f13c406c920 msgr2=0x7f13c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:53.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f13c406c920 0x7f13c406ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f13e40afb00 tx=0x7f13d0009190 comp rx=0 tx=0).stop 2024-04-20T21:21:53.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e40ca720 msgr2=0x7f13e40cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e40ca720 0x7f13e40cd4d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13dc02c580 tx=0x7f13dc004a40 comp rx=0 tx=0).stop 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 shutdown_connections 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13e415c2a0 0x7f13e40c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13e4152320 0x7f13e40cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f13c406c920 0x7f13c406ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 --2- 172.21.15.102:0/3475186322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e40ca720 0x7f13e40cd4d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.168+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 >> 172.21.15.102:0/3475186322 conn(0x7f13e406b940 msgr2=0x7f13e414d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.169+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 shutdown_connections 2024-04-20T21:21:53.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:21:53.169+0000 7f13e9200640 1 -- 172.21.15.102:0/3475186322 wait complete. 2024-04-20T21:21:53.189 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-04-20T21:21:53.190 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 12 secs before thrashing 2024-04-20T21:21:53.609 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2024-04-20T21:21:53.610 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2024-04-20T21:21:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2024-04-20T21:21:53.612 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2024-04-20T21:21:53.613 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2024-04-20T21:21:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2024-04-20T21:21:53.615 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2024-04-20T21:21:53.615 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2024-04-20T21:21:53.615 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2024-04-20T21:21:53.615 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2024-04-20T21:21:54.947 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2024-04-20T21:21:54.947 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2024-04-20T21:21:54.947 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2024-04-20T21:21:54.948 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2024-04-20T21:21:54.949 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2024-04-20T21:21:54.950 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2024-04-20T21:21:54.951 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2024-04-20T21:21:54.952 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2024-04-20T21:21:54.953 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2024-04-20T21:21:56.811 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2024-04-20T21:21:56.811 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2024-04-20T21:21:56.811 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2024-04-20T21:21:56.812 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2024-04-20T21:21:56.813 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2024-04-20T21:21:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2024-04-20T21:21:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2024-04-20T21:21:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:Take third snapshot .snap/test3 2024-04-20T21:21:56.980 INFO:tasks.workunit.client.0.smithi102.stdout:Delete the snapshots... 2024-04-20T21:21:59.556 INFO:tasks.workunit.client.0.smithi102.stdout:.//.snap/test1 2024-04-20T21:21:59.556 INFO:tasks.workunit.client.0.smithi102.stdout:.//.snap/test2 2024-04-20T21:21:59.557 INFO:tasks.workunit.client.0.smithi102.stdout:.//.snap/test3 2024-04-20T21:21:59.570 INFO:tasks.workunit.client.0.smithi102.stdout:./274/.snap/snap-subdir-test 2024-04-20T21:21:59.584 INFO:tasks.workunit.client.0.smithi102.stdout:./166/.snap/snap-subdir-test 2024-04-20T21:21:59.595 INFO:tasks.workunit.client.0.smithi102.stdout:./256/.snap/snap-subdir-test 2024-04-20T21:21:59.608 INFO:tasks.workunit.client.0.smithi102.stdout:./247/.snap/snap-subdir-test 2024-04-20T21:21:59.621 INFO:tasks.workunit.client.0.smithi102.stdout:./248/.snap/snap-subdir-test 2024-04-20T21:21:59.637 INFO:tasks.workunit.client.0.smithi102.stdout:./147/.snap/snap-subdir-test 2024-04-20T21:21:59.653 INFO:tasks.workunit.client.0.smithi102.stdout:./151/.snap/snap-subdir-test 2024-04-20T21:21:59.669 INFO:tasks.workunit.client.0.smithi102.stdout:./170/.snap/snap-subdir-test 2024-04-20T21:21:59.684 INFO:tasks.workunit.client.0.smithi102.stdout:./167/.snap/snap-subdir-test 2024-04-20T21:21:59.702 INFO:tasks.workunit.client.0.smithi102.stdout:./140/.snap/snap-subdir-test 2024-04-20T21:21:59.716 INFO:tasks.workunit.client.0.smithi102.stdout:./142/.snap/snap-subdir-test 2024-04-20T21:21:59.733 INFO:tasks.workunit.client.0.smithi102.stdout:./150/.snap/snap-subdir-test 2024-04-20T21:21:59.747 INFO:tasks.workunit.client.0.smithi102.stdout:./273/.snap/snap-subdir-test 2024-04-20T21:21:59.760 INFO:tasks.workunit.client.0.smithi102.stdout:./269/.snap/snap-subdir-test 2024-04-20T21:21:59.778 INFO:tasks.workunit.client.0.smithi102.stdout:./105/.snap/snap-subdir-test 2024-04-20T21:22:00.087 INFO:tasks.workunit.client.0.smithi102.stdout:./210/.snap/snap-subdir-test 2024-04-20T21:22:00.103 INFO:tasks.workunit.client.0.smithi102.stdout:./265/.snap/snap-subdir-test 2024-04-20T21:22:00.119 INFO:tasks.workunit.client.0.smithi102.stdout:./218/.snap/snap-subdir-test 2024-04-20T21:22:00.130 INFO:tasks.workunit.client.0.smithi102.stdout:./290/.snap/snap-subdir-test 2024-04-20T21:22:00.142 INFO:tasks.workunit.client.0.smithi102.stdout:./149/.snap/snap-subdir-test 2024-04-20T21:22:00.155 INFO:tasks.workunit.client.0.smithi102.stdout:./109/.snap/snap-subdir-test 2024-04-20T21:22:00.167 INFO:tasks.workunit.client.0.smithi102.stdout:./292/.snap/snap-subdir-test 2024-04-20T21:22:00.178 INFO:tasks.workunit.client.0.smithi102.stdout:./242/.snap/snap-subdir-test 2024-04-20T21:22:00.192 INFO:tasks.workunit.client.0.smithi102.stdout:./158/.snap/snap-subdir-test 2024-04-20T21:22:00.205 INFO:tasks.workunit.client.0.smithi102.stdout:./243/.snap/snap-subdir-test 2024-04-20T21:22:00.223 INFO:tasks.workunit.client.0.smithi102.stdout:./106/.snap/snap-subdir-test 2024-04-20T21:22:00.237 INFO:tasks.workunit.client.0.smithi102.stdout:./250/.snap/snap-subdir-test 2024-04-20T21:22:00.252 INFO:tasks.workunit.client.0.smithi102.stdout:./221/.snap/snap-subdir-test 2024-04-20T21:22:00.267 INFO:tasks.workunit.client.0.smithi102.stdout:./244/.snap/snap-subdir-test 2024-04-20T21:22:00.281 INFO:tasks.workunit.client.0.smithi102.stdout:./258/.snap/snap-subdir-test 2024-04-20T21:22:00.294 INFO:tasks.workunit.client.0.smithi102.stdout:./268/.snap/snap-subdir-test 2024-04-20T21:22:00.304 INFO:tasks.workunit.client.0.smithi102.stdout:./189/.snap/snap-subdir-test 2024-04-20T21:22:00.965 INFO:tasks.workunit.client.0.smithi102.stdout:./285/.snap/snap-subdir-test 2024-04-20T21:22:00.981 INFO:tasks.workunit.client.0.smithi102.stdout:./208/.snap/snap-subdir-test 2024-04-20T21:22:00.997 INFO:tasks.workunit.client.0.smithi102.stdout:./288/.snap/snap-subdir-test 2024-04-20T21:22:01.012 INFO:tasks.workunit.client.0.smithi102.stdout:./209/.snap/snap-subdir-test 2024-04-20T21:22:01.355 INFO:tasks.workunit.client.0.smithi102.stdout:./253/.snap/snap-subdir-test 2024-04-20T21:22:01.369 INFO:tasks.workunit.client.0.smithi102.stdout:./249/.snap/snap-subdir-test 2024-04-20T21:22:01.387 INFO:tasks.workunit.client.0.smithi102.stdout:./133/.snap/snap-subdir-test 2024-04-20T21:22:01.402 INFO:tasks.workunit.client.0.smithi102.stdout:./291/.snap/snap-subdir-test 2024-04-20T21:22:01.415 INFO:tasks.workunit.client.0.smithi102.stdout:./226/.snap/snap-subdir-test 2024-04-20T21:22:01.428 INFO:tasks.workunit.client.0.smithi102.stdout:./173/.snap/snap-subdir-test 2024-04-20T21:22:01.442 INFO:tasks.workunit.client.0.smithi102.stdout:./174/.snap/snap-subdir-test 2024-04-20T21:22:01.456 INFO:tasks.workunit.client.0.smithi102.stdout:./299/.snap/snap-subdir-test 2024-04-20T21:22:01.734 INFO:tasks.workunit.client.0.smithi102.stdout:./262/.snap/snap-subdir-test 2024-04-20T21:22:01.749 INFO:tasks.workunit.client.0.smithi102.stdout:./231/.snap/snap-subdir-test 2024-04-20T21:22:01.764 INFO:tasks.workunit.client.0.smithi102.stdout:./214/.snap/snap-subdir-test 2024-04-20T21:22:01.779 INFO:tasks.workunit.client.0.smithi102.stdout:./281/.snap/snap-subdir-test 2024-04-20T21:22:01.793 INFO:tasks.workunit.client.0.smithi102.stdout:./289/.snap/snap-subdir-test 2024-04-20T21:22:01.807 INFO:tasks.workunit.client.0.smithi102.stdout:./283/.snap/snap-subdir-test 2024-04-20T21:22:01.820 INFO:tasks.workunit.client.0.smithi102.stdout:./232/.snap/snap-subdir-test 2024-04-20T21:22:01.835 INFO:tasks.workunit.client.0.smithi102.stdout:./298/.snap/snap-subdir-test 2024-04-20T21:22:01.849 INFO:tasks.workunit.client.0.smithi102.stdout:./233/.snap/snap-subdir-test 2024-04-20T21:22:01.863 INFO:tasks.workunit.client.0.smithi102.stdout:./155/.snap/snap-subdir-test 2024-04-20T21:22:01.880 INFO:tasks.workunit.client.0.smithi102.stdout:./114/.snap/snap-subdir-test 2024-04-20T21:22:01.895 INFO:tasks.workunit.client.0.smithi102.stdout:./159/.snap/snap-subdir-test 2024-04-20T21:22:01.911 INFO:tasks.workunit.client.0.smithi102.stdout:./187/.snap/snap-subdir-test 2024-04-20T21:22:02.669 INFO:tasks.workunit.client.0.smithi102.stdout:./185/.snap/snap-subdir-test 2024-04-20T21:22:02.684 INFO:tasks.workunit.client.0.smithi102.stdout:./293/.snap/snap-subdir-test 2024-04-20T21:22:02.698 INFO:tasks.workunit.client.0.smithi102.stdout:./228/.snap/snap-subdir-test 2024-04-20T21:22:02.715 INFO:tasks.workunit.client.0.smithi102.stdout:./120/.snap/snap-subdir-test 2024-04-20T21:22:02.732 INFO:tasks.workunit.client.0.smithi102.stdout:./110/.snap/snap-subdir-test 2024-04-20T21:22:02.745 INFO:tasks.workunit.client.0.smithi102.stdout:./267/.snap/snap-subdir-test 2024-04-20T21:22:02.758 INFO:tasks.workunit.client.0.smithi102.stdout:./202/.snap/snap-subdir-test 2024-04-20T21:22:02.771 INFO:tasks.workunit.client.0.smithi102.stdout:./137/.snap/snap-subdir-test 2024-04-20T21:22:02.783 INFO:tasks.workunit.client.0.smithi102.stdout:./246/.snap/snap-subdir-test 2024-04-20T21:22:02.795 INFO:tasks.workunit.client.0.smithi102.stdout:./219/.snap/snap-subdir-test 2024-04-20T21:22:02.810 INFO:tasks.workunit.client.0.smithi102.stdout:./198/.snap/snap-subdir-test 2024-04-20T21:22:02.824 INFO:tasks.workunit.client.0.smithi102.stdout:./136/.snap/snap-subdir-test 2024-04-20T21:22:02.838 INFO:tasks.workunit.client.0.smithi102.stdout:./270/.snap/snap-subdir-test 2024-04-20T21:22:02.849 INFO:tasks.workunit.client.0.smithi102.stdout:./125/.snap/snap-subdir-test 2024-04-20T21:22:02.864 INFO:tasks.workunit.client.0.smithi102.stdout:./194/.snap/snap-subdir-test 2024-04-20T21:22:02.881 INFO:tasks.workunit.client.0.smithi102.stdout:./119/.snap/snap-subdir-test 2024-04-20T21:22:02.893 INFO:tasks.workunit.client.0.smithi102.stdout:./297/.snap/snap-subdir-test 2024-04-20T21:22:02.903 INFO:tasks.workunit.client.0.smithi102.stdout:./230/.snap/snap-subdir-test 2024-04-20T21:22:02.914 INFO:tasks.workunit.client.0.smithi102.stdout:./260/.snap/snap-subdir-test 2024-04-20T21:22:02.923 INFO:tasks.workunit.client.0.smithi102.stdout:./157/.snap/snap-subdir-test 2024-04-20T21:22:03.453 INFO:tasks.workunit.client.0.smithi102.stdout:./172/.snap/snap-subdir-test 2024-04-20T21:22:03.469 INFO:tasks.workunit.client.0.smithi102.stdout:./277/.snap/snap-subdir-test 2024-04-20T21:22:03.484 INFO:tasks.workunit.client.0.smithi102.stdout:./186/.snap/snap-subdir-test 2024-04-20T21:22:03.499 INFO:tasks.workunit.client.0.smithi102.stdout:./237/.snap/snap-subdir-test 2024-04-20T21:22:03.511 INFO:tasks.workunit.client.0.smithi102.stdout:./190/.snap/snap-subdir-test 2024-04-20T21:22:03.523 INFO:tasks.workunit.client.0.smithi102.stdout:./213/.snap/snap-subdir-test 2024-04-20T21:22:03.540 INFO:tasks.workunit.client.0.smithi102.stdout:./104/.snap/snap-subdir-test 2024-04-20T21:22:03.556 INFO:tasks.workunit.client.0.smithi102.stdout:./164/.snap/snap-subdir-test 2024-04-20T21:22:04.329 INFO:tasks.workunit.client.0.smithi102.stdout:./112/.snap/snap-subdir-test 2024-04-20T21:22:04.343 INFO:tasks.workunit.client.0.smithi102.stdout:./295/.snap/snap-subdir-test 2024-04-20T21:22:04.357 INFO:tasks.workunit.client.0.smithi102.stdout:./272/.snap/snap-subdir-test 2024-04-20T21:22:04.375 INFO:tasks.workunit.client.0.smithi102.stdout:./100/.snap/snap-subdir-test 2024-04-20T21:22:04.394 INFO:tasks.workunit.client.0.smithi102.stdout:./134/.snap/snap-subdir-test 2024-04-20T21:22:04.410 INFO:tasks.workunit.client.0.smithi102.stdout:./200/.snap/snap-subdir-test 2024-04-20T21:22:04.428 INFO:tasks.workunit.client.0.smithi102.stdout:./115/.snap/snap-subdir-test 2024-04-20T21:22:04.444 INFO:tasks.workunit.client.0.smithi102.stdout:./284/.snap/snap-subdir-test 2024-04-20T21:22:04.460 INFO:tasks.workunit.client.0.smithi102.stdout:./280/.snap/snap-subdir-test 2024-04-20T21:22:04.479 INFO:tasks.workunit.client.0.smithi102.stdout:./116/.snap/snap-subdir-test 2024-04-20T21:22:04.494 INFO:tasks.workunit.client.0.smithi102.stdout:./245/.snap/snap-subdir-test 2024-04-20T21:22:04.511 INFO:tasks.workunit.client.0.smithi102.stdout:./195/.snap/snap-subdir-test 2024-04-20T21:22:04.530 INFO:tasks.workunit.client.0.smithi102.stdout:./138/.snap/snap-subdir-test 2024-04-20T21:22:04.545 INFO:tasks.workunit.client.0.smithi102.stdout:./135/.snap/snap-subdir-test 2024-04-20T21:22:04.563 INFO:tasks.workunit.client.0.smithi102.stdout:./117/.snap/snap-subdir-test 2024-04-20T21:22:04.581 INFO:tasks.workunit.client.0.smithi102.stdout:./241/.snap/snap-subdir-test 2024-04-20T21:22:04.602 INFO:tasks.workunit.client.0.smithi102.stdout:./143/.snap/snap-subdir-test 2024-04-20T21:22:04.890 INFO:tasks.workunit.client.0.smithi102.stdout:./180/.snap/snap-subdir-test 2024-04-20T21:22:04.912 INFO:tasks.workunit.client.0.smithi102.stdout:./126/.snap/snap-subdir-test 2024-04-20T21:22:04.929 INFO:tasks.workunit.client.0.smithi102.stdout:./161/.snap/snap-subdir-test 2024-04-20T21:22:04.947 INFO:tasks.workunit.client.0.smithi102.stdout:./204/.snap/snap-subdir-test 2024-04-20T21:22:04.965 INFO:tasks.workunit.client.0.smithi102.stdout:./176/.snap/snap-subdir-test 2024-04-20T21:22:04.980 INFO:tasks.workunit.client.0.smithi102.stdout:./255/.snap/snap-subdir-test 2024-04-20T21:22:04.997 INFO:tasks.workunit.client.0.smithi102.stdout:./215/.snap/snap-subdir-test 2024-04-20T21:22:05.015 INFO:tasks.workunit.client.0.smithi102.stdout:./111/.snap/snap-subdir-test 2024-04-20T21:22:05.030 INFO:tasks.workunit.client.0.smithi102.stdout:./238/.snap/snap-subdir-test 2024-04-20T21:22:05.046 INFO:tasks.workunit.client.0.smithi102.stdout:./191/.snap/snap-subdir-test 2024-04-20T21:22:05.062 INFO:tasks.workunit.client.0.smithi102.stdout:./184/.snap/snap-subdir-test 2024-04-20T21:22:05.075 INFO:tasks.workunit.client.0.smithi102.stdout:./192/.snap/snap-subdir-test 2024-04-20T21:22:05.095 INFO:tasks.workunit.client.0.smithi102.stdout:./103/.snap/snap-subdir-test 2024-04-20T21:22:05.117 INFO:tasks.workunit.client.0.smithi102.stdout:./128/.snap/snap-subdir-test 2024-04-20T21:22:05.136 INFO:tasks.workunit.client.0.smithi102.stdout:./229/.snap/snap-subdir-test 2024-04-20T21:22:05.157 INFO:tasks.workunit.client.0.smithi102.stdout:./144/.snap/snap-subdir-test 2024-04-20T21:22:05.178 INFO:tasks.workunit.client.0.smithi102.stdout:./146/.snap/snap-subdir-test 2024-04-20T21:22:05.191 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:05.199 INFO:tasks.workunit.client.0.smithi102.stdout:./227/.snap/snap-subdir-test 2024-04-20T21:22:05.210 INFO:tasks.workunit.client.0.smithi102.stdout:./278/.snap/snap-subdir-test 2024-04-20T21:22:05.221 INFO:tasks.workunit.client.0.smithi102.stdout:./141/.snap/snap-subdir-test 2024-04-20T21:22:05.229 INFO:tasks.workunit.client.0.smithi102.stdout:./234/.snap/snap-subdir-test 2024-04-20T21:22:05.239 INFO:tasks.workunit.client.0.smithi102.stdout:./225/.snap/snap-subdir-test 2024-04-20T21:22:05.250 INFO:tasks.workunit.client.0.smithi102.stdout:./182/.snap/snap-subdir-test 2024-04-20T21:22:05.263 INFO:tasks.workunit.client.0.smithi102.stdout:./107/.snap/snap-subdir-test 2024-04-20T21:22:05.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.384+0000 7fb063600640 1 Processor -- start 2024-04-20T21:22:05.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.384+0000 7fb063600640 1 -- start start 2024-04-20T21:22:05.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb063600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ee00 0x7fb05c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb063600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb05c15f2b0 0x7fb05c0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb063600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb05c0c8f20 con 0x7fb05c15ea20 2024-04-20T21:22:05.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb063600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb05c06cb80 con 0x7fb05c15f2b0 2024-04-20T21:22:05.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb063600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb05c06ce60 con 0x7fb05c15ee00 2024-04-20T21:22:05.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb060c00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fb05c15ea20 0x7fb05c0c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:37956/0 (socket says 172.21.15.102:37956) 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb060c00640 1 -- 172.21.15.102:0/1024472307 learned_addr learned my addr 172.21.15.102:0/1024472307 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb05be00640 1 --2- 172.21.15.102:0/1024472307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ee00 0x7fb05c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.386+0000 7fb061600640 1 --2- 172.21.15.102:0/1024472307 >> v2:172.21.15.102:3300/0 conn(0x7fb05c15f2b0 0x7fb05c0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.387+0000 7fb05b400640 1 -- 172.21.15.102:0/1024472307 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341882330 0 0) 0x7fb05c0c8f20 con 0x7fb05c15ea20 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.387+0000 7fb05b400640 1 -- 172.21.15.102:0/1024472307 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb03c003520 con 0x7fb05c15ea20 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.387+0000 7fb05be00640 1 -- 172.21.15.102:0/1024472307 >> v2:172.21.15.102:3300/0 conn(0x7fb05c15f2b0 msgr2=0x7fb05c0710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.387+0000 7fb05be00640 1 --2- 172.21.15.102:0/1024472307 >> v2:172.21.15.102:3300/0 conn(0x7fb05c15f2b0 0x7fb05c0710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.387+0000 7fb05be00640 1 -- 172.21.15.102:0/1024472307 >> v1:172.21.15.193:6789/0 conn(0x7fb05c15ea20 legacy=0x7fb05c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.387+0000 7fb05be00640 1 -- 172.21.15.102:0/1024472307 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c06d140 con 0x7fb05c15ee00 2024-04-20T21:22:05.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.387+0000 7fb061600640 1 --2- 172.21.15.102:0/1024472307 >> v2:172.21.15.102:3300/0 conn(0x7fb05c15f2b0 0x7fb05c0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:05.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.388+0000 7fb05be00640 1 --2- 172.21.15.102:0/1024472307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ee00 0x7fb05c06d940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb044009d90 tx=0x7fb04402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a08e053518f599 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:05.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.388+0000 7fb05b400640 1 -- 172.21.15.102:0/1024472307 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044039070 con 0x7fb05c15ee00 2024-04-20T21:22:05.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.388+0000 7fb05b400640 1 -- 172.21.15.102:0/1024472307 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04402cce0 con 0x7fb05c15ee00 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.388+0000 7fb05b400640 1 -- 172.21.15.102:0/1024472307 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044034660 con 0x7fb05c15ee00 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.389+0000 7fb063600640 1 -- 172.21.15.102:0/1024472307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ee00 msgr2=0x7fb05c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.389+0000 7fb063600640 1 --2- 172.21.15.102:0/1024472307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ee00 0x7fb05c06d940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb044009d90 tx=0x7fb04402bf30 comp rx=0 tx=0).stop 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.389+0000 7fb063600640 1 -- 172.21.15.102:0/1024472307 shutdown_connections 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.389+0000 7fb063600640 1 --2- 172.21.15.102:0/1024472307 >> v2:172.21.15.102:3300/0 conn(0x7fb05c15f2b0 0x7fb05c0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.389+0000 7fb063600640 1 --2- 172.21.15.102:0/1024472307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ee00 0x7fb05c06d940 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.389+0000 7fb063600640 1 -- 172.21.15.102:0/1024472307 >> 172.21.15.102:0/1024472307 conn(0x7fb05c0c9390 msgr2=0x7fb05c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.389+0000 7fb063600640 1 -- 172.21.15.102:0/1024472307 shutdown_connections 2024-04-20T21:22:05.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.389+0000 7fb063600640 1 -- 172.21.15.102:0/1024472307 wait complete. 2024-04-20T21:22:05.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.391+0000 7fb063600640 1 Processor -- start 2024-04-20T21:22:05.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.391+0000 7fb063600640 1 -- start start 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb063600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ea20 0x7fb05c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb063600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb05c15ee00 0x7fb05c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb063600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c15f2b0 0x7fb05c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb063600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c0c8c80 con 0x7fb05c15ee00 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb063600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c0c8b00 con 0x7fb05c15f2b0 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb063600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb05c154e80 con 0x7fb05c15ea20 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb05be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb05c15ee00 0x7fb05c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb060c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ea20 0x7fb05c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb061600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c15f2b0 0x7fb05c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb060c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ea20 0x7fb05c15aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:42694/0 (socket says 172.21.15.102:42694) 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb061600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c15f2b0 0x7fb05c15d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:34450/0 (socket says 172.21.15.102:34450) 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb05be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb05c15ee00 0x7fb05c15b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50206/0 (socket says 172.21.15.102:50206) 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb060c00640 1 -- 172.21.15.102:0/2789507064 learned_addr learned my addr 172.21.15.102:0/2789507064 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb061600640 1 -- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ea20 msgr2=0x7fb05c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb061600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ea20 0x7fb05c15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb061600640 1 -- 172.21.15.102:0/2789507064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb05c15ee00 msgr2=0x7fb05c15b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb061600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb05c15ee00 0x7fb05c15b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.392+0000 7fb061600640 1 -- 172.21.15.102:0/2789507064 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c159a70 con 0x7fb05c15f2b0 2024-04-20T21:22:05.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.393+0000 7fb061600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c15f2b0 0x7fb05c15d5e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e7a0 tx=0x7fb05400ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:05.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.393+0000 7fb058c00640 1 -- 172.21.15.102:0/2789507064 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054017070 con 0x7fb05c15f2b0 2024-04-20T21:22:05.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.393+0000 7fb058c00640 1 -- 172.21.15.102:0/2789507064 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb054004020 con 0x7fb05c15f2b0 2024-04-20T21:22:05.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.393+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c1588a0 con 0x7fb05c15f2b0 2024-04-20T21:22:05.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.393+0000 7fb058c00640 1 -- 172.21.15.102:0/2789507064 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540043f0 con 0x7fb05c15f2b0 2024-04-20T21:22:05.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.393+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c159d10 con 0x7fb05c15f2b0 2024-04-20T21:22:05.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.394+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05c074340 con 0x7fb05c15f2b0 2024-04-20T21:22:05.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.395+0000 7fb058c00640 1 -- 172.21.15.102:0/2789507064 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb05400c940 con 0x7fb05c15f2b0 2024-04-20T21:22:05.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.396+0000 7fb058c00640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c920 0x7fb03806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.396+0000 7fb058c00640 1 -- 172.21.15.102:0/2789507064 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7fb05408bc80 con 0x7fb05c15f2b0 2024-04-20T21:22:05.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.397+0000 7fb060c00640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c920 0x7fb03806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.397+0000 7fb060c00640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c920 0x7fb03806ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0011e0 tx=0x7fb04c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:05.399 INFO:tasks.workunit.client.0.smithi102.stdout:./178/.snap/snap-subdir-test 2024-04-20T21:22:05.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.401+0000 7fb058c00640 1 -- 172.21.15.102:0/2789507064 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb054059510 con 0x7fb05c15f2b0 2024-04-20T21:22:05.407 INFO:tasks.workunit.client.0.smithi102.stdout:./266/.snap/snap-subdir-test 2024-04-20T21:22:05.415 INFO:tasks.workunit.client.0.smithi102.stdout:./240/.snap/snap-subdir-test 2024-04-20T21:22:05.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.647+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb05c0ca180 con 0x7fb05c15f2b0 2024-04-20T21:22:05.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.648+0000 7fb058c00640 1 -- 172.21.15.102:0/2789507064 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6859 (secure 0 0 0) 0x7fb05405d810 con 0x7fb05c15f2b0 2024-04-20T21:22:05.648 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:05.648 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:22:05.648 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-04-20T21:22:05.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c920 msgr2=0x7fb03806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c920 0x7fb03806ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0011e0 tx=0x7fb04c009040 comp rx=0 tx=0).stop 2024-04-20T21:22:05.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c15f2b0 msgr2=0x7fb05c15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c15f2b0 0x7fb05c15d5e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e7a0 tx=0x7fb05400ec60 comp rx=0 tx=0).stop 2024-04-20T21:22:05.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 shutdown_connections 2024-04-20T21:22:05.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c920 0x7fb03806ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c15f2b0 0x7fb05c15d5e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb05c15ee00 0x7fb05c15b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 --2- 172.21.15.102:0/2789507064 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb05c15ea20 0x7fb05c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.650+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 >> 172.21.15.102:0/2789507064 conn(0x7fb05c0c9390 msgr2=0x7fb05c0c31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:05.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.651+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 shutdown_connections 2024-04-20T21:22:05.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.651+0000 7fb063600640 1 -- 172.21.15.102:0/2789507064 wait complete. 2024-04-20T21:22:05.669 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:22:05.669 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:22:05.669 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:22:05.737 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:22:05.738 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:05.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.984+0000 7f136fe00640 1 Processor -- start 2024-04-20T21:22:05.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.984+0000 7f136fe00640 1 -- start start 2024-04-20T21:22:05.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.985+0000 7f136fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f13700c8b10 0x7f137013e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136fe00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13700c8fd0 con 0x7f13700caf70 2024-04-20T21:22:05.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136fe00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13700c6100 con 0x7f1370073550 2024-04-20T21:22:05.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136fe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f13700c6d10 con 0x7f13700c8b10 2024-04-20T21:22:05.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136ea00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f13700caf70 0x7f13700ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34212/0 (socket says 172.21.15.102:34212) 2024-04-20T21:22:05.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136ea00640 1 -- 172.21.15.102:0/714439751 learned_addr learned my addr 172.21.15.102:0/714439751 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:05.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136f400640 1 --2- 172.21.15.102:0/714439751 >> v2:172.21.15.102:3300/0 conn(0x7f13700c8b10 0x7f137013e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136d600640 1 -- 172.21.15.102:0/714439751 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3015118494 0 0) 0x7f13700c8fd0 con 0x7f13700caf70 2024-04-20T21:22:05.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136d600640 1 -- 172.21.15.102:0/714439751 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1354003520 con 0x7f13700caf70 2024-04-20T21:22:05.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136d600640 1 -- 172.21.15.102:0/714439751 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2543195407 0 0) 0x7f13700c6100 con 0x7f1370073550 2024-04-20T21:22:05.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136d600640 1 -- 172.21.15.102:0/714439751 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13700c8fd0 con 0x7f1370073550 2024-04-20T21:22:05.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136f400640 1 -- 172.21.15.102:0/714439751 >> v1:172.21.15.193:6789/0 conn(0x7f1370073550 legacy=0x7f13700c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136f400640 1 -- 172.21.15.102:0/714439751 >> v1:172.21.15.102:6789/0 conn(0x7f13700caf70 legacy=0x7f13700ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.986+0000 7f136f400640 1 -- 172.21.15.102:0/714439751 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13700c63e0 con 0x7f13700c8b10 2024-04-20T21:22:05.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.987+0000 7f136f400640 1 --2- 172.21.15.102:0/714439751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700c8b10 0x7f137013e200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1368009d90 tx=0x7f136802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9083a84ea53131f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:05.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.988+0000 7f136d600640 1 -- 172.21.15.102:0/714439751 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368039070 con 0x7f13700c8b10 2024-04-20T21:22:05.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.988+0000 7f136d600640 1 -- 172.21.15.102:0/714439751 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136802cd00 con 0x7f13700c8b10 2024-04-20T21:22:05.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.988+0000 7f136d600640 1 -- 172.21.15.102:0/714439751 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368034660 con 0x7f13700c8b10 2024-04-20T21:22:05.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.988+0000 7f136fe00640 1 -- 172.21.15.102:0/714439751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700c8b10 msgr2=0x7f137013e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.988+0000 7f136fe00640 1 --2- 172.21.15.102:0/714439751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700c8b10 0x7f137013e200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1368009d90 tx=0x7f136802bf50 comp rx=0 tx=0).stop 2024-04-20T21:22:05.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.988+0000 7f136fe00640 1 -- 172.21.15.102:0/714439751 shutdown_connections 2024-04-20T21:22:05.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.988+0000 7f136fe00640 1 --2- 172.21.15.102:0/714439751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700c8b10 0x7f137013e200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.988+0000 7f136fe00640 1 -- 172.21.15.102:0/714439751 >> 172.21.15.102:0/714439751 conn(0x7f137006eaa0 msgr2=0x7f137006ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:05.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.989+0000 7f136fe00640 1 -- 172.21.15.102:0/714439751 shutdown_connections 2024-04-20T21:22:05.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.989+0000 7f136fe00640 1 -- 172.21.15.102:0/714439751 wait complete. 2024-04-20T21:22:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.990+0000 7f136fe00640 1 Processor -- start 2024-04-20T21:22:05.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.990+0000 7f136fe00640 1 -- start start 2024-04-20T21:22:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.991+0000 7f136fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1370073550 0x7f1370158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.991+0000 7f136fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13700c8b10 0x7f13701f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.991+0000 7f136fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700caf70 0x7f13701f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.991+0000 7f136fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13700cb350 con 0x7f13700caf70 2024-04-20T21:22:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.991+0000 7f136fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13700c69a0 con 0x7f13700c8b10 2024-04-20T21:22:05.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.991+0000 7f136fe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f13700cf260 con 0x7f1370073550 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1370073550 0x7f1370158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700caf70 0x7f13701f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1370073550 0x7f1370158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:42702/0 (socket says 172.21.15.102:42702) 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 -- 172.21.15.102:0/1572137708 learned_addr learned my addr 172.21.15.102:0/1572137708 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136e000640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13700c8b10 0x7f13701f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 -- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13700c8b10 msgr2=0x7f13701f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13700c8b10 0x7f13701f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 -- 172.21.15.102:0/1572137708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700caf70 msgr2=0x7f13701f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:05.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700caf70 0x7f13701f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 -- 172.21.15.102:0/1572137708 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13701f22f0 con 0x7f1370073550 2024-04-20T21:22:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136f400640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700caf70 0x7f13701f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136e000640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13700c8b10 0x7f13701f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.992+0000 7f136ea00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1370073550 0x7f1370158b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f136000c410 tx=0x7f136000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.993+0000 7f1366a00640 1 -- 172.21.15.102:0/1572137708 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360017070 con 0x7f1370073550 2024-04-20T21:22:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.993+0000 7f1366a00640 1 -- 172.21.15.102:0/1572137708 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1360005ce0 con 0x7f1370073550 2024-04-20T21:22:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.993+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1370156ef0 con 0x7f1370073550 2024-04-20T21:22:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.993+0000 7f1366a00640 1 -- 172.21.15.102:0/1572137708 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360007310 con 0x7f1370073550 2024-04-20T21:22:05.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.993+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1370158060 con 0x7f1370073550 2024-04-20T21:22:05.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.994+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137004fd60 con 0x7f1370073550 2024-04-20T21:22:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.995+0000 7f1366a00640 1 -- 172.21.15.102:0/1572137708 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f136001a070 con 0x7f1370073550 2024-04-20T21:22:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.996+0000 7f1366a00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f135006c970 0x7f135006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.996+0000 7f1366a00640 1 -- 172.21.15.102:0/1572137708 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f136008b2b0 con 0x7f1370073550 2024-04-20T21:22:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.996+0000 7f136e000640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f135006c970 0x7f135006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:05.997+0000 7f136e000640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f135006c970 0x7f135006ee30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f135c007fd0 tx=0x7f135c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:06.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.001+0000 7f1366a00640 1 -- 172.21.15.102:0/1572137708 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1360058d10 con 0x7f1370073550 2024-04-20T21:22:06.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.243+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1370156870 con 0x7f1370073550 2024-04-20T21:22:06.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.244+0000 7f1366a00640 1 -- 172.21.15.102:0/1572137708 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6859 (secure 0 0 0) 0x7f136005d010 con 0x7f1370073550 2024-04-20T21:22:06.244 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:06.244 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:22:06.244 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-04-20T21:22:06.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f135006c970 msgr2=0x7f135006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f135006c970 0x7f135006ee30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f135c007fd0 tx=0x7f135c009040 comp rx=0 tx=0).stop 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1370073550 msgr2=0x7f1370158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1370073550 0x7f1370158b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f136000c410 tx=0x7f136000c8d0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 shutdown_connections 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13700caf70 0x7f13701f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f135006c970 0x7f135006ee30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13700c8b10 0x7f13701f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 --2- 172.21.15.102:0/1572137708 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1370073550 0x7f1370158b40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 >> 172.21.15.102:0/1572137708 conn(0x7f137006eaa0 msgr2=0x7f13700cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 shutdown_connections 2024-04-20T21:22:06.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:06.247+0000 7f136fe00640 1 -- 172.21.15.102:0/1572137708 wait complete. 2024-04-20T21:22:06.265 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:22:08.266 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:08.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.469+0000 7f7316e00640 1 Processor -- start 2024-04-20T21:22:08.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.469+0000 7f7316e00640 1 -- start start 2024-04-20T21:22:08.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f7316e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f731014fa50 0x7f73100c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f7316e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f73100c9eb0 0x7f7310158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f7316e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73101536d0 con 0x7f7310073a10 2024-04-20T21:22:08.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f7316e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7310159500 con 0x7f73100c9eb0 2024-04-20T21:22:08.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f7316e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f73100c76c0 con 0x7f731014fa50 2024-04-20T21:22:08.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f730f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f73100c9eb0 0x7f7310158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:08.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f7314e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f7310073a10 0x7f73100c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:42986/0 (socket says 172.21.15.102:42986) 2024-04-20T21:22:08.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f7314e00640 1 -- 172.21.15.102:0/3310635787 learned_addr learned my addr 172.21.15.102:0/3310635787 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:08.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.471+0000 7f730fe00640 1 --2- 172.21.15.102:0/3310635787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f731014fa50 0x7f73100c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:08.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.472+0000 7f730ea00640 1 -- 172.21.15.102:0/3310635787 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3608656814 0 0) 0x7f7308001490 con 0x7f7310073a10 2024-04-20T21:22:08.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.472+0000 7f730ea00640 1 -- 172.21.15.102:0/3310635787 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72f4003520 con 0x7f7310073a10 2024-04-20T21:22:08.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.472+0000 7f730f400640 1 -- 172.21.15.102:0/3310635787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f731014fa50 msgr2=0x7f73100c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:08.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.472+0000 7f730f400640 1 --2- 172.21.15.102:0/3310635787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f731014fa50 0x7f73100c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.472+0000 7f730f400640 1 -- 172.21.15.102:0/3310635787 >> v1:172.21.15.193:6789/0 conn(0x7f7310073a10 legacy=0x7f73100c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:08.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.472+0000 7f730f400640 1 -- 172.21.15.102:0/3310635787 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73100c7960 con 0x7f73100c9eb0 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.474+0000 7f730f400640 1 --2- 172.21.15.102:0/3310635787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73100c9eb0 0x7f7310158fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f73100c7b30 tx=0x7f72fc02b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71bb25f124850791 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.474+0000 7f730ea00640 1 -- 172.21.15.102:0/3310635787 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc035020 con 0x7f73100c9eb0 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.474+0000 7f730ea00640 1 -- 172.21.15.102:0/3310635787 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc031890 con 0x7f73100c9eb0 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.474+0000 7f730ea00640 1 -- 172.21.15.102:0/3310635787 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc031c40 con 0x7f73100c9eb0 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.475+0000 7f7316e00640 1 -- 172.21.15.102:0/3310635787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73100c9eb0 msgr2=0x7f7310158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.475+0000 7f7316e00640 1 --2- 172.21.15.102:0/3310635787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73100c9eb0 0x7f7310158fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f73100c7b30 tx=0x7f72fc02b930 comp rx=0 tx=0).stop 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.475+0000 7f7316e00640 1 -- 172.21.15.102:0/3310635787 shutdown_connections 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.475+0000 7f7316e00640 1 --2- 172.21.15.102:0/3310635787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73100c9eb0 0x7f7310158fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.475+0000 7f7316e00640 1 --2- 172.21.15.102:0/3310635787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f731014fa50 0x7f73100c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.475+0000 7f7316e00640 1 -- 172.21.15.102:0/3310635787 >> 172.21.15.102:0/3310635787 conn(0x7f73100c9390 msgr2=0x7f73100c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:08.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.475+0000 7f7316e00640 1 -- 172.21.15.102:0/3310635787 shutdown_connections 2024-04-20T21:22:08.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.475+0000 7f7316e00640 1 -- 172.21.15.102:0/3310635787 wait complete. 2024-04-20T21:22:08.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.477+0000 7f7316e00640 1 Processor -- start 2024-04-20T21:22:08.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.477+0000 7f7316e00640 1 -- start start 2024-04-20T21:22:08.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7316e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7310073a10 0x7f73101e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7316e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73100c9eb0 0x7f73101efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7316e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f731014fa50 0x7f73101451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7316e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73100c7c00 con 0x7f731014fa50 2024-04-20T21:22:08.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7316e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73100c3d90 con 0x7f7310073a10 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7316e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f73100c4030 con 0x7f73100c9eb0 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7314e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f731014fa50 0x7f73101451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f730f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73100c9eb0 0x7f73101efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7314e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f731014fa50 0x7f73101451f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57892/0 (socket says 172.21.15.102:57892) 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f730f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73100c9eb0 0x7f73101efcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:37188/0 (socket says 172.21.15.102:37188) 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f730fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7310073a10 0x7f73101e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7314e00640 1 -- 172.21.15.102:0/2041159972 learned_addr learned my addr 172.21.15.102:0/2041159972 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f730f400640 1 -- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7310073a10 msgr2=0x7f73101e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f730f400640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7310073a10 0x7f73101e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f730f400640 1 -- 172.21.15.102:0/2041159972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f731014fa50 msgr2=0x7f73101451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f730f400640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f731014fa50 0x7f73101451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f730f400640 1 -- 172.21.15.102:0/2041159972 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7310154bd0 con 0x7f73100c9eb0 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.478+0000 7f7314e00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f731014fa50 0x7f73101451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.479+0000 7f730f400640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73100c9eb0 0x7f73101efcf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f72fc033000 tx=0x7f72fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.479+0000 7f7307e00640 1 -- 172.21.15.102:0/2041159972 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc035050 con 0x7f73100c9eb0 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.479+0000 7f7307e00640 1 -- 172.21.15.102:0/2041159972 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc003820 con 0x7f73100c9eb0 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.479+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73101ef510 con 0x7f73100c9eb0 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.479+0000 7f7307e00640 1 -- 172.21.15.102:0/2041159972 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc003bd0 con 0x7f73100c9eb0 2024-04-20T21:22:08.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.479+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73101f3250 con 0x7f73100c9eb0 2024-04-20T21:22:08.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.481+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731014c830 con 0x7f73100c9eb0 2024-04-20T21:22:08.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.481+0000 7f7307e00640 1 -- 172.21.15.102:0/2041159972 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72fc038020 con 0x7f73100c9eb0 2024-04-20T21:22:08.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.482+0000 7f7307e00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f72f006c920 0x7f72f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.482+0000 7f7307e00640 1 -- 172.21.15.102:0/2041159972 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f72fc0adca0 con 0x7f73100c9eb0 2024-04-20T21:22:08.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.483+0000 7f730fe00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f72f006c920 0x7f72f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:08.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.483+0000 7f730fe00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f72f006c920 0x7f72f006ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7300009b40 tx=0x7f7300009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:08.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.487+0000 7f7307e00640 1 -- 172.21.15.102:0/2041159972 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f731014c830 con 0x7f73100c9eb0 2024-04-20T21:22:08.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.728+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73101599b0 con 0x7f73100c9eb0 2024-04-20T21:22:08.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.729+0000 7f7307e00640 1 -- 172.21.15.102:0/2041159972 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6859 (secure 0 0 0) 0x7f73101599b0 con 0x7f73100c9eb0 2024-04-20T21:22:08.729 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:22:08.729 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f72f006c920 msgr2=0x7f72f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f72f006c920 0x7f72f006ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7300009b40 tx=0x7f7300009190 comp rx=0 tx=0).stop 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73100c9eb0 msgr2=0x7f73101efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73100c9eb0 0x7f73101efcf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f72fc033000 tx=0x7f72fc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 shutdown_connections 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f731014fa50 0x7f73101451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73100c9eb0 0x7f73101efcf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f72f006c920 0x7f72f006ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 --2- 172.21.15.102:0/2041159972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7310073a10 0x7f73101e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 >> 172.21.15.102:0/2041159972 conn(0x7f73100c9390 msgr2=0x7f731006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 shutdown_connections 2024-04-20T21:22:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:08.732+0000 7f7316e00640 1 -- 172.21.15.102:0/2041159972 wait complete. 2024-04-20T21:22:08.752 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:22:10.753 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:10.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.963+0000 7fa803200640 1 Processor -- start 2024-04-20T21:22:10.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.963+0000 7fa803200640 1 -- start start 2024-04-20T21:22:10.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa803200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa8040c75a0 0x7fa8040c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:10.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa803200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa804073c70 0x7fa8040c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:10.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa803200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa804153560 con 0x7fa804153180 2024-04-20T21:22:10.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa803200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa8040c7a70 con 0x7fa804073c70 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa803200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa8040c7d30 con 0x7fa8040c75a0 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa802800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa804073c70 0x7fa8040c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa802800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa804073c70 0x7fa8040c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57902/0 (socket says 172.21.15.102:57902) 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa802800640 1 -- 172.21.15.102:0/1101772455 learned_addr learned my addr 172.21.15.102:0/1101772455 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa801400640 1 --2- 172.21.15.102:0/1101772455 >> v2:172.21.15.193:3300/0 conn(0x7fa8040c75a0 0x7fa8040c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa802800640 1 -- 172.21.15.102:0/1101772455 >> v2:172.21.15.193:3300/0 conn(0x7fa8040c75a0 msgr2=0x7fa8040c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa802800640 1 --2- 172.21.15.102:0/1101772455 >> v2:172.21.15.193:3300/0 conn(0x7fa8040c75a0 0x7fa8040c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa802800640 1 -- 172.21.15.102:0/1101772455 >> v1:172.21.15.193:6789/0 conn(0x7fa804153180 legacy=0x7fa804158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:10.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.965+0000 7fa802800640 1 -- 172.21.15.102:0/1101772455 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80415c4d0 con 0x7fa804073c70 2024-04-20T21:22:10.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.967+0000 7fa802800640 1 --2- 172.21.15.102:0/1101772455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c6470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa80406f8d0 tx=0x7fa7fc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ecb76896483b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:10.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.967+0000 7fa800a00640 1 -- 172.21.15.102:0/1101772455 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc02cb20 con 0x7fa804073c70 2024-04-20T21:22:10.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.967+0000 7fa800a00640 1 -- 172.21.15.102:0/1101772455 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7fc02ccc0 con 0x7fa804073c70 2024-04-20T21:22:10.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.967+0000 7fa800a00640 1 -- 172.21.15.102:0/1101772455 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc034610 con 0x7fa804073c70 2024-04-20T21:22:10.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.967+0000 7fa803200640 1 -- 172.21.15.102:0/1101772455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 msgr2=0x7fa8040c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:10.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.967+0000 7fa803200640 1 --2- 172.21.15.102:0/1101772455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c6470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa80406f8d0 tx=0x7fa7fc02bf80 comp rx=0 tx=0).stop 2024-04-20T21:22:10.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.968+0000 7fa803200640 1 -- 172.21.15.102:0/1101772455 shutdown_connections 2024-04-20T21:22:10.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.968+0000 7fa803200640 1 --2- 172.21.15.102:0/1101772455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c6470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:10.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.968+0000 7fa803200640 1 --2- 172.21.15.102:0/1101772455 >> v2:172.21.15.193:3300/0 conn(0x7fa8040c75a0 0x7fa8040c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:10.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.968+0000 7fa803200640 1 -- 172.21.15.102:0/1101772455 >> 172.21.15.102:0/1101772455 conn(0x7fa8040cc820 msgr2=0x7fa8040ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:10.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.968+0000 7fa803200640 1 -- 172.21.15.102:0/1101772455 shutdown_connections 2024-04-20T21:22:10.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.968+0000 7fa803200640 1 -- 172.21.15.102:0/1101772455 wait complete. 2024-04-20T21:22:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.969+0000 7fa803200640 1 Processor -- start 2024-04-20T21:22:10.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.969+0000 7fa803200640 1 -- start start 2024-04-20T21:22:10.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.970+0000 7fa803200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:10.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.970+0000 7fa803200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8040c75a0 0x7fa8041e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:10.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.970+0000 7fa803200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa804153180 0x7fa804144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:10.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.970+0000 7fa803200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa804159550 con 0x7fa804073c70 2024-04-20T21:22:10.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.970+0000 7fa803200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa80415ca90 con 0x7fa804153180 2024-04-20T21:22:10.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.970+0000 7fa803200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa80415d290 con 0x7fa8040c75a0 2024-04-20T21:22:10.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:10.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57904/0 (socket says 172.21.15.102:57904) 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 -- 172.21.15.102:0/2721749204 learned_addr learned my addr 172.21.15.102:0/2721749204 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801400640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8040c75a0 0x7fa8041e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa802800640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa804153180 0x7fa804144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 -- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8040c75a0 msgr2=0x7fa8041e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8040c75a0 0x7fa8041e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 -- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa804153180 msgr2=0x7fa804144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa804153180 0x7fa804144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 -- 172.21.15.102:0/2721749204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8040c4260 con 0x7fa804073c70 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.971+0000 7fa801e00640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c3d20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c3f0 tx=0x7fa7f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.972+0000 7fa7fa000640 1 -- 172.21.15.102:0/2721749204 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40052d0 con 0x7fa804073c70 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.972+0000 7fa7fa000640 1 -- 172.21.15.102:0/2721749204 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f4005490 con 0x7fa804073c70 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.972+0000 7fa7fa000640 1 -- 172.21.15.102:0/2721749204 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4007260 con 0x7fa804073c70 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.972+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8040cab50 con 0x7fa804073c70 2024-04-20T21:22:10.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.972+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8040cb110 con 0x7fa804073c70 2024-04-20T21:22:10.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.973+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80404fd60 con 0x7fa804073c70 2024-04-20T21:22:10.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.974+0000 7fa7fa000640 1 -- 172.21.15.102:0/2721749204 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7f4007400 con 0x7fa804073c70 2024-04-20T21:22:10.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.975+0000 7fa7fa000640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa7e406c920 0x7fa7e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:10.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.975+0000 7fa7fa000640 1 -- 172.21.15.102:0/2721749204 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7fa7f408b3e0 con 0x7fa804073c70 2024-04-20T21:22:10.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.975+0000 7fa801400640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa7e406c920 0x7fa7e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:10.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.976+0000 7fa801400640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa7e406c920 0x7fa7e406ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8041e6460 tx=0x7fa7f0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:10.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:10.980+0000 7fa7fa000640 1 -- 172.21.15.102:0/2721749204 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7f4058d00 con 0x7fa804073c70 2024-04-20T21:22:11.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.224+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8040c3690 con 0x7fa804073c70 2024-04-20T21:22:11.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.226+0000 7fa7fa000640 1 -- 172.21.15.102:0/2721749204 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6859 (secure 0 0 0) 0x7fa7f405d000 con 0x7fa804073c70 2024-04-20T21:22:11.226 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:22:11.226 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa7e406c920 msgr2=0x7fa7e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa7e406c920 0x7fa7e406ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8041e6460 tx=0x7fa7f0009190 comp rx=0 tx=0).stop 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 msgr2=0x7fa8040c3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c3d20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c3f0 tx=0x7fa7f400c8b0 comp rx=0 tx=0).stop 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 shutdown_connections 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa7e406c920 0x7fa7e406ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa804153180 0x7fa804144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8040c75a0 0x7fa8041e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 --2- 172.21.15.102:0/2721749204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa804073c70 0x7fa8040c3d20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 >> 172.21.15.102:0/2721749204 conn(0x7fa8040cc820 msgr2=0x7fa8040c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:11.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 shutdown_connections 2024-04-20T21:22:11.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:11.229+0000 7fa803200640 1 -- 172.21.15.102:0/2721749204 wait complete. 2024-04-20T21:22:11.247 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:22:13.248 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:13.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.457+0000 7f129e000640 1 Processor -- start 2024-04-20T21:22:13.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.457+0000 7f129e000640 1 -- start start 2024-04-20T21:22:13.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.458+0000 7f129e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12980c8db0 0x7f1298070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:13.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.458+0000 7f129e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1298071330 0x7f12981628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:13.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.458+0000 7f129e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1298158e80 con 0x7f1298152170 2024-04-20T21:22:13.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.458+0000 7f129e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f12980c6dc0 con 0x7f1298071330 2024-04-20T21:22:13.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.458+0000 7f129e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1298071f70 con 0x7f12980c8db0 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1296a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12980c8db0 0x7f1298070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1297e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1298071330 0x7f12981628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1297400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f1298152170 0x7f1298158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43004/0 (socket says 172.21.15.102:43004) 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1297400640 1 -- 172.21.15.102:0/1001518696 learned_addr learned my addr 172.21.15.102:0/1001518696 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1296000640 1 -- 172.21.15.102:0/1001518696 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4105039259 0 0) 0x7f1298158e80 con 0x7f1298152170 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1296a00640 1 -- 172.21.15.102:0/1001518696 >> v2:172.21.15.193:3300/0 conn(0x7f1298071330 msgr2=0x7f12981628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1296a00640 1 --2- 172.21.15.102:0/1001518696 >> v2:172.21.15.193:3300/0 conn(0x7f1298071330 0x7f12981628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1296a00640 1 -- 172.21.15.102:0/1001518696 >> v1:172.21.15.193:6789/0 conn(0x7f1298152170 legacy=0x7f1298158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:13.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.459+0000 7f1296a00640 1 -- 172.21.15.102:0/1001518696 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1298072210 con 0x7f12980c8db0 2024-04-20T21:22:13.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.460+0000 7f1296a00640 1 --2- 172.21.15.102:0/1001518696 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12980c8db0 0x7f1298070df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12840097a0 tx=0x7f128402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5736366ff3c3eae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:13.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f1296000640 1 -- 172.21.15.102:0/1001518696 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284035020 con 0x7f12980c8db0 2024-04-20T21:22:13.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f1296000640 1 -- 172.21.15.102:0/1001518696 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1284031830 con 0x7f12980c8db0 2024-04-20T21:22:13.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f1296000640 1 -- 172.21.15.102:0/1001518696 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284031c00 con 0x7f12980c8db0 2024-04-20T21:22:13.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f129e000640 1 -- 172.21.15.102:0/1001518696 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12980c8db0 msgr2=0x7f1298070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:13.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f129e000640 1 --2- 172.21.15.102:0/1001518696 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12980c8db0 0x7f1298070df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12840097a0 tx=0x7f128402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f129e000640 1 -- 172.21.15.102:0/1001518696 shutdown_connections 2024-04-20T21:22:13.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f129e000640 1 --2- 172.21.15.102:0/1001518696 >> v2:172.21.15.193:3300/0 conn(0x7f1298071330 0x7f12981628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f129e000640 1 --2- 172.21.15.102:0/1001518696 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12980c8db0 0x7f1298070df0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f129e000640 1 -- 172.21.15.102:0/1001518696 >> 172.21.15.102:0/1001518696 conn(0x7f12980c9390 msgr2=0x7f12980c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:13.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f129e000640 1 -- 172.21.15.102:0/1001518696 shutdown_connections 2024-04-20T21:22:13.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.461+0000 7f129e000640 1 -- 172.21.15.102:0/1001518696 wait complete. 2024-04-20T21:22:13.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.463+0000 7f129e000640 1 Processor -- start 2024-04-20T21:22:13.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.463+0000 7f129e000640 1 -- start start 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.464+0000 7f129e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1298071330 0x7f12981e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.464+0000 7f129e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12980c8db0 0x7f129814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.464+0000 7f129e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1298152170 0x7f129814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.464+0000 7f129e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12980c9190 con 0x7f12980c8db0 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.464+0000 7f129e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12980724b0 con 0x7f1298071330 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.464+0000 7f129e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1298158250 con 0x7f1298152170 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1296a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12980c8db0 0x7f129814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1298071330 0x7f12981e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1298152170 0x7f129814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1296a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12980c8db0 0x7f129814f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57914/0 (socket says 172.21.15.102:57914) 2024-04-20T21:22:13.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1296a00640 1 -- 172.21.15.102:0/852454426 learned_addr learned my addr 172.21.15.102:0/852454426 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297e00640 1 -- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1298071330 msgr2=0x7f12981e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297e00640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1298071330 0x7f12981e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297e00640 1 -- 172.21.15.102:0/852454426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12980c8db0 msgr2=0x7f129814f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297e00640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12980c8db0 0x7f129814f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297e00640 1 -- 172.21.15.102:0/852454426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1298156240 con 0x7f1298152170 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297400640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1298071330 0x7f12981e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.465+0000 7f1297e00640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1298152170 0x7f129814e200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1290009d90 tx=0x7f129000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.466+0000 7f128f400640 1 -- 172.21.15.102:0/852454426 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290018020 con 0x7f1298152170 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.466+0000 7f128f400640 1 -- 172.21.15.102:0/852454426 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1290014480 con 0x7f1298152170 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.466+0000 7f128f400640 1 -- 172.21.15.102:0/852454426 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290014870 con 0x7f1298152170 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.466+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12981f0ca0 con 0x7f1298152170 2024-04-20T21:22:13.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.466+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12981f1260 con 0x7f1298152170 2024-04-20T21:22:13.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.467+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129806f2d0 con 0x7f1298152170 2024-04-20T21:22:13.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.468+0000 7f128f400640 1 -- 172.21.15.102:0/852454426 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1290005d10 con 0x7f1298152170 2024-04-20T21:22:13.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.469+0000 7f128f400640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f127806c8f0 0x7f127806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:13.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.469+0000 7f128f400640 1 -- 172.21.15.102:0/852454426 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f129008cd40 con 0x7f1298152170 2024-04-20T21:22:13.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.469+0000 7f1297400640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f127806c8f0 0x7f127806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:13.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.470+0000 7f1297400640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f127806c8f0 0x7f127806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1288007fd0 tx=0x7f1288007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:13.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.474+0000 7f128f400640 1 -- 172.21.15.102:0/852454426 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f129806f2d0 con 0x7f1298152170 2024-04-20T21:22:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.708+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1298074340 con 0x7f1298152170 2024-04-20T21:22:13.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.709+0000 7f128f400640 1 -- 172.21.15.102:0/852454426 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6859 (secure 0 0 0) 0x7f1290022030 con 0x7f1298152170 2024-04-20T21:22:13.709 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:22:13.709 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-04-20T21:22:13.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.711+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f127806c8f0 msgr2=0x7f127806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:13.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f127806c8f0 0x7f127806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1288007fd0 tx=0x7f1288007de0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1298152170 msgr2=0x7f129814e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1298152170 0x7f129814e200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1290009d90 tx=0x7f129000c490 comp rx=0 tx=0).stop 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 shutdown_connections 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1298152170 0x7f129814e200 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12980c8db0 0x7f129814f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f127806c8f0 0x7f127806edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 --2- 172.21.15.102:0/852454426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1298071330 0x7f12981e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 >> 172.21.15.102:0/852454426 conn(0x7f12980c9390 msgr2=0x7f129806e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 shutdown_connections 2024-04-20T21:22:13.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:13.712+0000 7f129e000640 1 -- 172.21.15.102:0/852454426 wait complete. 2024-04-20T21:22:13.730 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:22:15.731 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:15.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.938+0000 7fa6e5200640 1 Processor -- start 2024-04-20T21:22:15.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.938+0000 7fa6e5200640 1 -- start start 2024-04-20T21:22:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.940+0000 7fa6e5200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e014fa50 0x7fa6e00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.940+0000 7fa6e5200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa6e0073a10 0x7fa6e0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.940+0000 7fa6e5200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e01536d0 con 0x7fa6e00c9eb0 2024-04-20T21:22:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.940+0000 7fa6e5200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa6e0159500 con 0x7fa6e0073a10 2024-04-20T21:22:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.940+0000 7fa6e5200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e00c76c0 con 0x7fa6e014fa50 2024-04-20T21:22:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.940+0000 7fa6d5400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa6e00c9eb0 0x7fa6e00c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:51164/0 (socket says 172.21.15.102:51164) 2024-04-20T21:22:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6d5400640 1 -- 172.21.15.102:0/4239715031 learned_addr learned my addr 172.21.15.102:0/4239715031 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6dde00640 1 --2- 172.21.15.102:0/4239715031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e014fa50 0x7fa6e00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6de800640 1 --2- 172.21.15.102:0/4239715031 >> v2:172.21.15.102:3300/0 conn(0x7fa6e0073a10 0x7fa6e0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6dde00640 1 -- 172.21.15.102:0/4239715031 >> v2:172.21.15.102:3300/0 conn(0x7fa6e0073a10 msgr2=0x7fa6e0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6dde00640 1 --2- 172.21.15.102:0/4239715031 >> v2:172.21.15.102:3300/0 conn(0x7fa6e0073a10 0x7fa6e0158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6dde00640 1 -- 172.21.15.102:0/4239715031 >> v1:172.21.15.102:6789/0 conn(0x7fa6e00c9eb0 legacy=0x7fa6e00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6dd400640 1 -- 172.21.15.102:0/4239715031 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3518110517 0 0) 0x7fa6c8001110 con 0x7fa6e00c9eb0 2024-04-20T21:22:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6dde00640 1 -- 172.21.15.102:0/4239715031 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e00c7960 con 0x7fa6e014fa50 2024-04-20T21:22:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.941+0000 7fa6de800640 1 --2- 172.21.15.102:0/4239715031 >> v2:172.21.15.102:3300/0 conn(0x7fa6e0073a10 0x7fa6e0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:22:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.943+0000 7fa6dde00640 1 --2- 172.21.15.102:0/4239715031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e014fa50 0x7fa6e00c4830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00097a0 tx=0x7fa6d002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d68a0066a3e1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.943+0000 7fa6dd400640 1 -- 172.21.15.102:0/4239715031 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0034070 con 0x7fa6e014fa50 2024-04-20T21:22:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.943+0000 7fa6dd400640 1 -- 172.21.15.102:0/4239715031 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d0030800 con 0x7fa6e014fa50 2024-04-20T21:22:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.943+0000 7fa6dd400640 1 -- 172.21.15.102:0/4239715031 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0030bd0 con 0x7fa6e014fa50 2024-04-20T21:22:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.944+0000 7fa6e5200640 1 -- 172.21.15.102:0/4239715031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e014fa50 msgr2=0x7fa6e00c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.944+0000 7fa6e5200640 1 --2- 172.21.15.102:0/4239715031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e014fa50 0x7fa6e00c4830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00097a0 tx=0x7fa6d002b940 comp rx=0 tx=0).stop 2024-04-20T21:22:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.944+0000 7fa6e5200640 1 -- 172.21.15.102:0/4239715031 shutdown_connections 2024-04-20T21:22:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.944+0000 7fa6e5200640 1 --2- 172.21.15.102:0/4239715031 >> v2:172.21.15.102:3300/0 conn(0x7fa6e0073a10 0x7fa6e0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.944+0000 7fa6e5200640 1 --2- 172.21.15.102:0/4239715031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e014fa50 0x7fa6e00c4830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.944+0000 7fa6e5200640 1 -- 172.21.15.102:0/4239715031 >> 172.21.15.102:0/4239715031 conn(0x7fa6e00c9390 msgr2=0x7fa6e00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.944+0000 7fa6e5200640 1 -- 172.21.15.102:0/4239715031 shutdown_connections 2024-04-20T21:22:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.944+0000 7fa6e5200640 1 -- 172.21.15.102:0/4239715031 wait complete. 2024-04-20T21:22:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.946+0000 7fa6e5200640 1 Processor -- start 2024-04-20T21:22:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.946+0000 7fa6e5200640 1 -- start start 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6e5200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e0073a10 0x7fa6e00cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6e5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6e00c9eb0 0x7fa6e01f1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6e5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6e014fa50 0x7fa6e0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6e5200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e00c7c00 con 0x7fa6e00c9eb0 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6e5200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e00c3d90 con 0x7fa6e014fa50 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6e5200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e00c4030 con 0x7fa6e0073a10 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6e00c9eb0 0x7fa6e01f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6dde00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e0073a10 0x7fa6e00cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6e00c9eb0 0x7fa6e01f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57936/0 (socket says 172.21.15.102:57936) 2024-04-20T21:22:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6d5400640 1 -- 172.21.15.102:0/1984693934 learned_addr learned my addr 172.21.15.102:0/1984693934 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:15.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.947+0000 7fa6de800640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6e014fa50 0x7fa6e0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:15.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6dde00640 1 -- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6e014fa50 msgr2=0x7fa6e0070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:15.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6dde00640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6e014fa50 0x7fa6e0070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:15.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6dde00640 1 -- 172.21.15.102:0/1984693934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6e00c9eb0 msgr2=0x7fa6e01f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:15.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6dde00640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6e00c9eb0 0x7fa6e01f1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:15.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6dde00640 1 -- 172.21.15.102:0/1984693934 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e00cc600 con 0x7fa6e0073a10 2024-04-20T21:22:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6dde00640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e0073a10 0x7fa6e00cb790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0033000 tx=0x7fa6d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6d6a00640 1 -- 172.21.15.102:0/1984693934 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0034040 con 0x7fa6e0073a10 2024-04-20T21:22:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6d6a00640 1 -- 172.21.15.102:0/1984693934 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d003ada0 con 0x7fa6e0073a10 2024-04-20T21:22:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e0071330 con 0x7fa6e0073a10 2024-04-20T21:22:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6d6a00640 1 -- 172.21.15.102:0/1984693934 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0044630 con 0x7fa6e0073a10 2024-04-20T21:22:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.948+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e00cc320 con 0x7fa6e0073a10 2024-04-20T21:22:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.950+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e00c6f80 con 0x7fa6e0073a10 2024-04-20T21:22:15.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.951+0000 7fa6d6a00640 1 -- 172.21.15.102:0/1984693934 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6d004b060 con 0x7fa6e0073a10 2024-04-20T21:22:15.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.951+0000 7fa6d6a00640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6bc06c920 0x7fa6bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:15.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.952+0000 7fa6d6a00640 1 -- 172.21.15.102:0/1984693934 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7fa6d00ade60 con 0x7fa6e0073a10 2024-04-20T21:22:15.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.952+0000 7fa6d5400640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6bc06c920 0x7fa6bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:15.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.952+0000 7fa6d5400640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6bc06c920 0x7fa6bc06ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa6e01f23b0 tx=0x7fa6c8007c10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:15.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:15.956+0000 7fa6d6a00640 1 -- 172.21.15.102:0/1984693934 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa6e00c6f80 con 0x7fa6e0073a10 2024-04-20T21:22:16.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.193+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6e014e980 con 0x7fa6e0073a10 2024-04-20T21:22:16.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.195+0000 7fa6d6a00640 1 -- 172.21.15.102:0/1984693934 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6859 (secure 0 0 0) 0x7fa6e014e980 con 0x7fa6e0073a10 2024-04-20T21:22:16.194 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:22:16.195 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6bc06c920 msgr2=0x7fa6bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6bc06c920 0x7fa6bc06ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa6e01f23b0 tx=0x7fa6c8007c10 comp rx=0 tx=0).stop 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e0073a10 msgr2=0x7fa6e00cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e0073a10 0x7fa6e00cb790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0033000 tx=0x7fa6d00046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 shutdown_connections 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6bc06c920 0x7fa6bc06ede0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6e014fa50 0x7fa6e0070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6e00c9eb0 0x7fa6e01f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 --2- 172.21.15.102:0/1984693934 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6e0073a10 0x7fa6e00cb790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.197+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 >> 172.21.15.102:0/1984693934 conn(0x7fa6e00c9390 msgr2=0x7fa6e006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:16.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.198+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 shutdown_connections 2024-04-20T21:22:16.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:16.198+0000 7fa6e5200640 1 -- 172.21.15.102:0/1984693934 wait complete. 2024-04-20T21:22:16.218 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:22:18.219 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:18.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.422+0000 7f5d43200640 1 Processor -- start 2024-04-20T21:22:18.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.422+0000 7f5d43200640 1 -- start start 2024-04-20T21:22:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d43200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5d440c4040 0x7f5d4415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d43200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d4415eac0 0x7f5d44165910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d43200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4415f3c0 con 0x7f5d44155110 2024-04-20T21:22:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d43200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5d440c7c30 con 0x7f5d440c4040 2024-04-20T21:22:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d43200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4406ea00 con 0x7f5d4415eac0 2024-04-20T21:22:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d39400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5d440c4040 0x7f5d4415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d39400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5d440c4040 0x7f5d4415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44412/0 (socket says 172.21.15.102:44412) 2024-04-20T21:22:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d42800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d4415eac0 0x7f5d44165910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d39400640 1 -- 172.21.15.102:0/1530243306 learned_addr learned my addr 172.21.15.102:0/1530243306 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d39400640 1 -- 172.21.15.102:0/1530243306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d4415eac0 msgr2=0x7f5d44165910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:18.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d39400640 1 --2- 172.21.15.102:0/1530243306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d4415eac0 0x7f5d44165910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d39400640 1 -- 172.21.15.102:0/1530243306 >> v1:172.21.15.193:6789/0 conn(0x7f5d44155110 legacy=0x7f5d4406ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.424+0000 7f5d39400640 1 -- 172.21.15.102:0/1530243306 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4406e6d0 con 0x7f5d440c4040 2024-04-20T21:22:18.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.425+0000 7f5d39400640 1 --2- 172.21.15.102:0/1530243306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d440c4040 0x7f5d4415e580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5d440c7db0 tx=0x7f5d2c02ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fbf1187d359ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.426+0000 7f5d41400640 1 -- 172.21.15.102:0/1530243306 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c02f820 con 0x7f5d440c4040 2024-04-20T21:22:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.426+0000 7f5d41400640 1 -- 172.21.15.102:0/1530243306 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2c02f9c0 con 0x7f5d440c4040 2024-04-20T21:22:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.426+0000 7f5d41400640 1 -- 172.21.15.102:0/1530243306 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c02fd90 con 0x7f5d440c4040 2024-04-20T21:22:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.426+0000 7f5d43200640 1 -- 172.21.15.102:0/1530243306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d440c4040 msgr2=0x7f5d4415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.426+0000 7f5d43200640 1 --2- 172.21.15.102:0/1530243306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d440c4040 0x7f5d4415e580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5d440c7db0 tx=0x7f5d2c02ba00 comp rx=0 tx=0).stop 2024-04-20T21:22:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.426+0000 7f5d43200640 1 -- 172.21.15.102:0/1530243306 shutdown_connections 2024-04-20T21:22:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.427+0000 7f5d43200640 1 --2- 172.21.15.102:0/1530243306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d4415eac0 0x7f5d44165910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.427+0000 7f5d43200640 1 --2- 172.21.15.102:0/1530243306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d440c4040 0x7f5d4415e580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.427+0000 7f5d43200640 1 -- 172.21.15.102:0/1530243306 >> 172.21.15.102:0/1530243306 conn(0x7f5d440cc820 msgr2=0x7f5d440ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:18.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.427+0000 7f5d43200640 1 -- 172.21.15.102:0/1530243306 shutdown_connections 2024-04-20T21:22:18.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.427+0000 7f5d43200640 1 -- 172.21.15.102:0/1530243306 wait complete. 2024-04-20T21:22:18.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.428+0000 7f5d43200640 1 Processor -- start 2024-04-20T21:22:18.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.428+0000 7f5d43200640 1 -- start start 2024-04-20T21:22:18.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.429+0000 7f5d43200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d440c4040 0x7f5d44159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.429+0000 7f5d43200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d44155110 0x7f5d440c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.429+0000 7f5d43200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4415eac0 0x7f5d441e6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.429+0000 7f5d43200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4406f5a0 con 0x7f5d44155110 2024-04-20T21:22:18.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.429+0000 7f5d43200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440c8160 con 0x7f5d4415eac0 2024-04-20T21:22:18.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.429+0000 7f5d43200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4406f720 con 0x7f5d440c4040 2024-04-20T21:22:18.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d39400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d44155110 0x7f5d440c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d41e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d440c4040 0x7f5d44159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d42800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4415eac0 0x7f5d441e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d41e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d440c4040 0x7f5d44159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:46810/0 (socket says 172.21.15.102:46810) 2024-04-20T21:22:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d39400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d44155110 0x7f5d440c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44418/0 (socket says 172.21.15.102:44418) 2024-04-20T21:22:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d41e00640 1 -- 172.21.15.102:0/2003265072 learned_addr learned my addr 172.21.15.102:0/2003265072 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d41e00640 1 -- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4415eac0 msgr2=0x7f5d441e6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d41e00640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4415eac0 0x7f5d441e6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d41e00640 1 -- 172.21.15.102:0/2003265072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d44155110 msgr2=0x7f5d440c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d41e00640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d44155110 0x7f5d440c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d41e00640 1 -- 172.21.15.102:0/2003265072 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d441e7050 con 0x7f5d440c4040 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d42800640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4415eac0 0x7f5d441e6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.430+0000 7f5d39400640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d44155110 0x7f5d440c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.431+0000 7f5d41e00640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d440c4040 0x7f5d44159e80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400c460 tx=0x7f5d3400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.431+0000 7f5d3aa00640 1 -- 172.21.15.102:0/2003265072 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340052d0 con 0x7f5d440c4040 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.431+0000 7f5d3aa00640 1 -- 172.21.15.102:0/2003265072 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d34012630 con 0x7f5d440c4040 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.431+0000 7f5d3aa00640 1 -- 172.21.15.102:0/2003265072 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340054b0 con 0x7f5d440c4040 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.431+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d441e45c0 con 0x7f5d440c4040 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.431+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d441e4b80 con 0x7f5d440c4040 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.432+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4406a0c0 con 0x7f5d440c4040 2024-04-20T21:22:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.433+0000 7f5d3aa00640 1 -- 172.21.15.102:0/2003265072 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d34012a20 con 0x7f5d440c4040 2024-04-20T21:22:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.434+0000 7f5d3aa00640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.434+0000 7f5d3aa00640 1 -- 172.21.15.102:0/2003265072 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6907+0+0 (secure 0 0 0) 0x7f5d3408bbc0 con 0x7f5d440c4040 2024-04-20T21:22:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.434+0000 7f5d39400640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.435+0000 7f5d39400640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5d2006c8f0 0x7f5d2006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c036000 tx=0x7f5d2c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:18.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.438+0000 7f5d3aa00640 1 -- 172.21.15.102:0/2003265072 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5d4406a0c0 con 0x7f5d440c4040 2024-04-20T21:22:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.680+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d441554f0 con 0x7f5d440c4040 2024-04-20T21:22:18.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.682+0000 7f5d3aa00640 1 -- 172.21.15.102:0/2003265072 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6859 (secure 0 0 0) 0x7f5d441554f0 con 0x7f5d440c4040 2024-04-20T21:22:18.681 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:18.682 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:21:44.909333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4847},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4847":{"gid":4847,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6835/1017522496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1017522496},{"type":"v1","addr":"172.21.15.102:6835","nonce":1017522496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4847,"qdb_cluster":[4847]},"id":1}]} 2024-04-20T21:22:18.682 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-04-20T21:22:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5d2006c8f0 msgr2=0x7f5d2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5d2006c8f0 0x7f5d2006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c036000 tx=0x7f5d2c008030 comp rx=0 tx=0).stop 2024-04-20T21:22:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d440c4040 msgr2=0x7f5d44159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d440c4040 0x7f5d44159e80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400c460 tx=0x7f5d3400c920 comp rx=0 tx=0).stop 2024-04-20T21:22:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 shutdown_connections 2024-04-20T21:22:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4415eac0 0x7f5d441e6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d44155110 0x7f5d440c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 --2- 172.21.15.102:0/2003265072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5d440c4040 0x7f5d44159e80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.684+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 >> 172.21.15.102:0/2003265072 conn(0x7f5d440cc820 msgr2=0x7f5d4406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:18.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.685+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 shutdown_connections 2024-04-20T21:22:18.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:18.685+0000 7f5d43200640 1 -- 172.21.15.102:0/2003265072 wait complete. 2024-04-20T21:22:18.702 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:22:20.703 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.914+0000 7ff803c00640 1 Processor -- start 2024-04-20T21:22:20.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.915+0000 7ff803c00640 1 -- start start 2024-04-20T21:22:20.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.916+0000 7ff803c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff7fc163220 0x7ff7fc159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.916+0000 7ff803c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7fc06d2e0 con 0x7ff7fc162e40 2024-04-20T21:22:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.916+0000 7ff803c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7fc06d5c0 con 0x7ff7fc163660 2024-04-20T21:22:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.916+0000 7ff803c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff7fc0c7290 con 0x7ff7fc163220 2024-04-20T21:22:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.916+0000 7ff801200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff7fc162e40 0x7ff7fc06c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33320/0 (socket says 172.21.15.102:33320) 2024-04-20T21:22:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.916+0000 7ff801200640 1 -- 172.21.15.102:0/135178740 learned_addr learned my addr 172.21.15.102:0/135178740 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.916+0000 7ff7fbe00640 1 --2- 172.21.15.102:0/135178740 >> v2:172.21.15.193:3300/0 conn(0x7ff7fc163220 0x7ff7fc159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.917+0000 7ff7fb400640 1 -- 172.21.15.102:0/135178740 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1497165568 0 0) 0x7ff7fc06d5c0 con 0x7ff7fc163660 2024-04-20T21:22:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.917+0000 7ff7fb400640 1 -- 172.21.15.102:0/135178740 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7e0003520 con 0x7ff7fc163660 2024-04-20T21:22:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.917+0000 7ff7fbe00640 1 -- 172.21.15.102:0/135178740 >> v1:172.21.15.193:6789/0 conn(0x7ff7fc163660 legacy=0x7ff7fc0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.917+0000 7ff7fbe00640 1 -- 172.21.15.102:0/135178740 >> v1:172.21.15.102:6789/0 conn(0x7ff7fc162e40 legacy=0x7ff7fc06c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.917+0000 7ff7fb400640 1 -- 172.21.15.102:0/135178740 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3875782907 0 0) 0x7ff7fc06d2e0 con 0x7ff7fc162e40 2024-04-20T21:22:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.917+0000 7ff7fbe00640 1 -- 172.21.15.102:0/135178740 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc06d8a0 con 0x7ff7fc163220 2024-04-20T21:22:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.918+0000 7ff7fbe00640 1 --2- 172.21.15.102:0/135178740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163220 0x7ff7fc159440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009d90 tx=0x7ff7e802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232ce53b24481b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.918+0000 7ff7fb400640 1 -- 172.21.15.102:0/135178740 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8039070 con 0x7ff7fc163220 2024-04-20T21:22:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.918+0000 7ff7fb400640 1 -- 172.21.15.102:0/135178740 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e802cd80 con 0x7ff7fc163220 2024-04-20T21:22:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.918+0000 7ff7fb400640 1 -- 172.21.15.102:0/135178740 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8034660 con 0x7ff7fc163220 2024-04-20T21:22:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.919+0000 7ff803c00640 1 -- 172.21.15.102:0/135178740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163220 msgr2=0x7ff7fc159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.919+0000 7ff803c00640 1 --2- 172.21.15.102:0/135178740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163220 0x7ff7fc159440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009d90 tx=0x7ff7e802bfd0 comp rx=0 tx=0).stop 2024-04-20T21:22:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.919+0000 7ff803c00640 1 -- 172.21.15.102:0/135178740 shutdown_connections 2024-04-20T21:22:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.919+0000 7ff803c00640 1 --2- 172.21.15.102:0/135178740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163220 0x7ff7fc159440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.919+0000 7ff803c00640 1 -- 172.21.15.102:0/135178740 >> 172.21.15.102:0/135178740 conn(0x7ff7fc0c9390 msgr2=0x7ff7fc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.919+0000 7ff803c00640 1 -- 172.21.15.102:0/135178740 shutdown_connections 2024-04-20T21:22:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.919+0000 7ff803c00640 1 -- 172.21.15.102:0/135178740 wait complete. 2024-04-20T21:22:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.921+0000 7ff803c00640 1 Processor -- start 2024-04-20T21:22:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.921+0000 7ff803c00640 1 -- start start 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff803c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7fc162e40 0x7ff7fc0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff803c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7fc163220 0x7ff7fc0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff803c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163660 0x7ff7fc1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff803c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc0c9080 con 0x7ff7fc162e40 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff803c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc0c7570 con 0x7ff7fc163660 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff803c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff7fc0c8110 con 0x7ff7fc163220 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff801200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7fc162e40 0x7ff7fc0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff801200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7fc162e40 0x7ff7fc0c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44420/0 (socket says 172.21.15.102:44420) 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff801200640 1 -- 172.21.15.102:0/1157462603 learned_addr learned my addr 172.21.15.102:0/1157462603 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff7fbe00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7fc163220 0x7ff7fc0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:20.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.922+0000 7ff801c00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163660 0x7ff7fc1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:20.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff801200640 1 -- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7fc163220 msgr2=0x7ff7fc0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:20.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff801200640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7fc163220 0x7ff7fc0c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:20.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff801200640 1 -- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163660 msgr2=0x7ff7fc1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:20.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff801200640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163660 0x7ff7fc1584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:20.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff801200640 1 -- 172.21.15.102:0/1157462603 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc1f8370 con 0x7ff7fc162e40 2024-04-20T21:22:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff801200640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7fc162e40 0x7ff7fc0c6710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c550 tx=0x7ff7ec00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff7f8c00640 1 -- 172.21.15.102:0/1157462603 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec018070 con 0x7ff7fc162e40 2024-04-20T21:22:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff7f8c00640 1 -- 172.21.15.102:0/1157462603 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec0074e0 con 0x7ff7fc162e40 2024-04-20T21:22:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7fc1f8650 con 0x7ff7fc162e40 2024-04-20T21:22:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.923+0000 7ff7f8c00640 1 -- 172.21.15.102:0/1157462603 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec004a00 con 0x7ff7fc162e40 2024-04-20T21:22:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.924+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7fc1f8c10 con 0x7ff7fc162e40 2024-04-20T21:22:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.925+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7fc074340 con 0x7ff7fc162e40 2024-04-20T21:22:20.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.926+0000 7ff7f8c00640 1 -- 172.21.15.102:0/1157462603 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7ec01b070 con 0x7ff7fc162e40 2024-04-20T21:22:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.927+0000 7ff7f8c00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff7dc06c8f0 0x7ff7dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.927+0000 7ff7f8c00640 1 -- 172.21.15.102:0/1157462603 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7ff7ec08b270 con 0x7ff7fc162e40 2024-04-20T21:22:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.927+0000 7ff7fbe00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff7dc06c8f0 0x7ff7dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.928+0000 7ff7fbe00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff7dc06c8f0 0x7ff7dc06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8036000 tx=0x7ff7e8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:20.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:20.931+0000 7ff7f8c00640 1 -- 172.21.15.102:0/1157462603 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff7ec058b30 con 0x7ff7fc162e40 2024-04-20T21:22:21.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.171+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7fc15cf20 con 0x7ff7fc162e40 2024-04-20T21:22:21.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.172+0000 7ff7f8c00640 1 -- 172.21.15.102:0/1157462603 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6023 (secure 0 0 0) 0x7ff7ec05ce30 con 0x7ff7fc162e40 2024-04-20T21:22:21.172 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:21.172 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:20.501968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:22:21.172 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 22 2024-04-20T21:22:21.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff7dc06c8f0 msgr2=0x7ff7dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff7dc06c8f0 0x7ff7dc06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8036000 tx=0x7ff7e8030000 comp rx=0 tx=0).stop 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7fc162e40 msgr2=0x7ff7fc0c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7fc162e40 0x7ff7fc0c6710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c550 tx=0x7ff7ec00ca10 comp rx=0 tx=0).stop 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 shutdown_connections 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff7dc06c8f0 0x7ff7dc06edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7fc163660 0x7ff7fc1584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7fc163220 0x7ff7fc0c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 --2- 172.21.15.102:0/1157462603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7fc162e40 0x7ff7fc0c6710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 >> 172.21.15.102:0/1157462603 conn(0x7ff7fc0c9390 msgr2=0x7ff7fc0c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 shutdown_connections 2024-04-20T21:22:21.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.175+0000 7ff803c00640 1 -- 172.21.15.102:0/1157462603 wait complete. 2024-04-20T21:22:21.192 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:22:21.192 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:22:21.192 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:21.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.391+0000 7f9048c00640 1 Processor -- start 2024-04-20T21:22:21.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.391+0000 7f9048c00640 1 -- start start 2024-04-20T21:22:21.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9048c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9044155110 0x7f904406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9048c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f904406f3d0 0x7f904415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9048c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904415f3c0 con 0x7f90440c4040 2024-04-20T21:22:21.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9048c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f90440c7c30 con 0x7f904406f3d0 2024-04-20T21:22:21.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9048c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f904406ea00 con 0x7f9044155110 2024-04-20T21:22:21.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9042a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f904406f3d0 0x7f904415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9043e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f90440c4040 0x7f904415eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33322/0 (socket says 172.21.15.102:33322) 2024-04-20T21:22:21.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9043e00640 1 -- 172.21.15.102:0/144224262 learned_addr learned my addr 172.21.15.102:0/144224262 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:21.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.393+0000 7f9043400640 1 --2- 172.21.15.102:0/144224262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9044155110 0x7f904406ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.394+0000 7f9042a00640 1 -- 172.21.15.102:0/144224262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9044155110 msgr2=0x7f904406ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:21.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.394+0000 7f9042a00640 1 --2- 172.21.15.102:0/144224262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9044155110 0x7f904406ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.394+0000 7f9042a00640 1 -- 172.21.15.102:0/144224262 >> v1:172.21.15.102:6789/0 conn(0x7f90440c4040 legacy=0x7f904415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.394+0000 7f9042a00640 1 -- 172.21.15.102:0/144224262 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904406e6d0 con 0x7f904406f3d0 2024-04-20T21:22:21.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.394+0000 7f9043400640 1 --2- 172.21.15.102:0/144224262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9044155110 0x7f904406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:21.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.394+0000 7f9042000640 1 -- 172.21.15.102:0/144224262 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268454556 0 0) 0x7f904415f3c0 con 0x7f90440c4040 2024-04-20T21:22:21.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.395+0000 7f9042a00640 1 --2- 172.21.15.102:0/144224262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904406f3d0 0x7f904415e580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f90440c7db0 tx=0x7f903002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecd58f11be2cfc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.395+0000 7f9042000640 1 -- 172.21.15.102:0/144224262 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030035020 con 0x7f904406f3d0 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.395+0000 7f9042000640 1 -- 172.21.15.102:0/144224262 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030031890 con 0x7f904406f3d0 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.395+0000 7f9042000640 1 -- 172.21.15.102:0/144224262 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030031c60 con 0x7f904406f3d0 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.395+0000 7f9048c00640 1 -- 172.21.15.102:0/144224262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904406f3d0 msgr2=0x7f904415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.395+0000 7f9048c00640 1 --2- 172.21.15.102:0/144224262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904406f3d0 0x7f904415e580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f90440c7db0 tx=0x7f903002b930 comp rx=0 tx=0).stop 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.396+0000 7f9048c00640 1 -- 172.21.15.102:0/144224262 shutdown_connections 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.396+0000 7f9048c00640 1 --2- 172.21.15.102:0/144224262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904406f3d0 0x7f904415e580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.396+0000 7f9048c00640 1 --2- 172.21.15.102:0/144224262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9044155110 0x7f904406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.396+0000 7f9048c00640 1 -- 172.21.15.102:0/144224262 >> 172.21.15.102:0/144224262 conn(0x7f90440cc820 msgr2=0x7f90440ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:21.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.396+0000 7f9048c00640 1 -- 172.21.15.102:0/144224262 shutdown_connections 2024-04-20T21:22:21.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.396+0000 7f9048c00640 1 -- 172.21.15.102:0/144224262 wait complete. 2024-04-20T21:22:21.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.397+0000 7f9048c00640 1 Processor -- start 2024-04-20T21:22:21.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.397+0000 7f9048c00640 1 -- start start 2024-04-20T21:22:21.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.398+0000 7f9048c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904406f3d0 0x7f90441f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.398+0000 7f9048c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90440c4040 0x7f90440ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.398+0000 7f9048c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9044155110 0x7f90440cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.398+0000 7f9048c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90441554f0 con 0x7f90440c4040 2024-04-20T21:22:21.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.398+0000 7f9048c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90440c8160 con 0x7f9044155110 2024-04-20T21:22:21.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.398+0000 7f9048c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9044155670 con 0x7f904406f3d0 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.398+0000 7f9042a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90440c4040 0x7f90440ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9042a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90440c4040 0x7f90440ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44438/0 (socket says 172.21.15.102:44438) 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9042a00640 1 -- 172.21.15.102:0/287723520 learned_addr learned my addr 172.21.15.102:0/287723520 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9043400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904406f3d0 0x7f90441f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9043e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9044155110 0x7f90440cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9043400640 1 -- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9044155110 msgr2=0x7f90440cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9043400640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9044155110 0x7f90440cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9043400640 1 -- 172.21.15.102:0/287723520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90440c4040 msgr2=0x7f90440ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9043400640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90440c4040 0x7f90440ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9043400640 1 -- 172.21.15.102:0/287723520 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9044158220 con 0x7f904406f3d0 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9042a00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90440c4040 0x7f90440ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:21.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.399+0000 7f9043400640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904406f3d0 0x7f90441f4500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f903400e8a0 tx=0x7f903400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:21.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.400+0000 7f903b400640 1 -- 172.21.15.102:0/287723520 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034017070 con 0x7f904406f3d0 2024-04-20T21:22:21.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.400+0000 7f903b400640 1 -- 172.21.15.102:0/287723520 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903400c940 con 0x7f904406f3d0 2024-04-20T21:22:21.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.400+0000 7f903b400640 1 -- 172.21.15.102:0/287723520 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903400cd30 con 0x7f904406f3d0 2024-04-20T21:22:21.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.400+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90441f4fa0 con 0x7f904406f3d0 2024-04-20T21:22:21.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.400+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9044156610 con 0x7f904406f3d0 2024-04-20T21:22:21.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.401+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904415ca70 con 0x7f904406f3d0 2024-04-20T21:22:21.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.402+0000 7f903b400640 1 -- 172.21.15.102:0/287723520 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f903401a070 con 0x7f904406f3d0 2024-04-20T21:22:21.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.403+0000 7f903b400640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f902406c8f0 0x7f902406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.403+0000 7f903b400640 1 -- 172.21.15.102:0/287723520 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f903408bec0 con 0x7f904406f3d0 2024-04-20T21:22:21.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.403+0000 7f9042a00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f902406c8f0 0x7f902406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.404+0000 7f9042a00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f902406c8f0 0x7f902406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90440c3950 tx=0x7f903003d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:21.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.407+0000 7f903b400640 1 -- 172.21.15.102:0/287723520 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f904415ca70 con 0x7f904406f3d0 2024-04-20T21:22:21.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.649+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f904404fd60 con 0x7f904406f3d0 2024-04-20T21:22:21.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.650+0000 7f903b400640 1 -- 172.21.15.102:0/287723520 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6023 (secure 0 0 0) 0x7f9044158220 con 0x7f904406f3d0 2024-04-20T21:22:21.650 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:21.650 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:20.501968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:22:21.651 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 22 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f902406c8f0 msgr2=0x7f902406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f902406c8f0 0x7f902406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90440c3950 tx=0x7f903003d040 comp rx=0 tx=0).stop 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904406f3d0 msgr2=0x7f90441f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904406f3d0 0x7f90441f4500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f903400e8a0 tx=0x7f903400ed60 comp rx=0 tx=0).stop 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 shutdown_connections 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f902406c8f0 0x7f902406edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9044155110 0x7f90440cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90440c4040 0x7f90440ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 --2- 172.21.15.102:0/287723520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904406f3d0 0x7f90441f4500 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f903400e8a0 tx=0x7f903400ed60 comp rx=0 tx=0).stop 2024-04-20T21:22:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 >> 172.21.15.102:0/287723520 conn(0x7f90440cc820 msgr2=0x7f904415c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 shutdown_connections 2024-04-20T21:22:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:21.653+0000 7f9048c00640 1 -- 172.21.15.102:0/287723520 wait complete. 2024-04-20T21:22:21.671 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5266": { "addr": "172.21.15.102:6833/713712414", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 713712414, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 713712414, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5266, "incarnation": 22, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:22:20.501968+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5266 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:22:22.433 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:22.441 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:23.673 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:23.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.880+0000 7f5403600640 1 Processor -- start 2024-04-20T21:22:23.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.880+0000 7f5403600640 1 -- start start 2024-04-20T21:22:23.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.881+0000 7f5403600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.881+0000 7f5403600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f53fc15bd60 0x7f53fc15f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.881+0000 7f5403600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f53fc073950 0x7f53fc0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.881+0000 7f5403600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f53fc15b8d0 con 0x7f53fc073950 2024-04-20T21:22:23.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.881+0000 7f5403600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f53fc15f6f0 con 0x7f53fc15bd60 2024-04-20T21:22:23.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.881+0000 7f5403600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f53fc15c180 con 0x7f53fc155610 2024-04-20T21:22:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5400c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5401600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f53fc073950 0x7f53fc0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f53fbe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f53fc15bd60 0x7f53fc15f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5400c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc15c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:46832/0 (socket says 172.21.15.102:46832) 2024-04-20T21:22:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5400c00640 1 -- 172.21.15.102:0/1734701097 learned_addr learned my addr 172.21.15.102:0/1734701097 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5400c00640 1 -- 172.21.15.102:0/1734701097 >> v2:172.21.15.193:3300/0 conn(0x7f53fc15bd60 msgr2=0x7f53fc15f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5400c00640 1 --2- 172.21.15.102:0/1734701097 >> v2:172.21.15.193:3300/0 conn(0x7f53fc15bd60 0x7f53fc15f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5400c00640 1 -- 172.21.15.102:0/1734701097 >> v2:172.21.15.102:3300/0 conn(0x7f53fc073950 msgr2=0x7f53fc0ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:23.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5400c00640 1 --2- 172.21.15.102:0/1734701097 >> v2:172.21.15.102:3300/0 conn(0x7f53fc073950 0x7f53fc0ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.882+0000 7f5400c00640 1 -- 172.21.15.102:0/1734701097 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc15b2c0 con 0x7f53fc155610 2024-04-20T21:22:23.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.883+0000 7f5400c00640 1 --2- 172.21.15.102:0/1734701097 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc15c6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f53fc06a010 tx=0x7f53f002b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4876e28bc84c2fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f53fb400640 1 -- 172.21.15.102:0/1734701097 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0034070 con 0x7f53fc155610 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f53fb400640 1 -- 172.21.15.102:0/1734701097 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f0030850 con 0x7f53fc155610 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f53fb400640 1 -- 172.21.15.102:0/1734701097 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0030c20 con 0x7f53fc155610 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f5403600640 1 -- 172.21.15.102:0/1734701097 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 msgr2=0x7f53fc15c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f5403600640 1 --2- 172.21.15.102:0/1734701097 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc15c6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f53fc06a010 tx=0x7f53f002b990 comp rx=0 tx=0).stop 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f5403600640 1 -- 172.21.15.102:0/1734701097 shutdown_connections 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f5403600640 1 --2- 172.21.15.102:0/1734701097 >> v2:172.21.15.102:3300/0 conn(0x7f53fc073950 0x7f53fc0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f5403600640 1 --2- 172.21.15.102:0/1734701097 >> v2:172.21.15.193:3300/0 conn(0x7f53fc15bd60 0x7f53fc15f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f5403600640 1 --2- 172.21.15.102:0/1734701097 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc15c6a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.884+0000 7f5403600640 1 -- 172.21.15.102:0/1734701097 >> 172.21.15.102:0/1734701097 conn(0x7f53fc06ebf0 msgr2=0x7f53fc06efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.885+0000 7f5403600640 1 -- 172.21.15.102:0/1734701097 shutdown_connections 2024-04-20T21:22:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.885+0000 7f5403600640 1 -- 172.21.15.102:0/1734701097 wait complete. 2024-04-20T21:22:23.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.886+0000 7f5403600640 1 Processor -- start 2024-04-20T21:22:23.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.886+0000 7f5403600640 1 -- start start 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.887+0000 7f5403600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53fc073950 0x7f53fc1e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.887+0000 7f5403600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc1e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.887+0000 7f5403600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc15bd60 0x7f53fc1e7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.887+0000 7f5403600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc15b560 con 0x7f53fc15bd60 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.887+0000 7f5403600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc0c9d50 con 0x7f53fc073950 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.887+0000 7f5403600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f53fc0ca030 con 0x7f53fc155610 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53fc073950 0x7f53fc1e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f53fbe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc1e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53fc073950 0x7f53fc1e8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:49376/0 (socket says 172.21.15.102:49376) 2024-04-20T21:22:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 -- 172.21.15.102:0/2145559609 learned_addr learned my addr 172.21.15.102:0/2145559609 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:23.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5401600640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc15bd60 0x7f53fc1e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 -- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 msgr2=0x7f53fc1e6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc1e6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 -- 172.21.15.102:0/2145559609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc15bd60 msgr2=0x7f53fc1e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc15bd60 0x7f53fc1e7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 -- 172.21.15.102:0/2145559609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc0cbf50 con 0x7f53fc073950 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f53fbe00640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc1e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.888+0000 7f5400c00640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53fc073950 0x7f53fc1e8170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53f00097c0 tx=0x7f53f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.889+0000 7f53f8c00640 1 -- 172.21.15.102:0/2145559609 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0048070 con 0x7f53fc073950 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.889+0000 7f53f8c00640 1 -- 172.21.15.102:0/2145559609 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f0030670 con 0x7f53fc073950 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.889+0000 7f53f8c00640 1 -- 172.21.15.102:0/2145559609 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0034040 con 0x7f53fc073950 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.889+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53fc0ca540 con 0x7f53fc073950 2024-04-20T21:22:23.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.889+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53fc0cc1c0 con 0x7f53fc073950 2024-04-20T21:22:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.890+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53fc071a70 con 0x7f53fc073950 2024-04-20T21:22:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.891+0000 7f53f8c00640 1 -- 172.21.15.102:0/2145559609 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53f0037070 con 0x7f53fc073950 2024-04-20T21:22:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.892+0000 7f53f8c00640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53dc06c8f0 0x7f53dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.892+0000 7f53f8c00640 1 -- 172.21.15.102:0/2145559609 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f53f00ade10 con 0x7f53fc073950 2024-04-20T21:22:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.892+0000 7f53fbe00640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53dc06c8f0 0x7f53dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.893+0000 7f53fbe00640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53dc06c8f0 0x7f53dc06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f53fc159d50 tx=0x7f53e8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:23.897+0000 7f53f8c00640 1 -- 172.21.15.102:0/2145559609 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f53fc071a70 con 0x7f53fc073950 2024-04-20T21:22:24.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.135+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53fc074340 con 0x7f53fc073950 2024-04-20T21:22:24.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.137+0000 7f53f8c00640 1 -- 172.21.15.102:0/2145559609 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6023 (secure 0 0 0) 0x7f53f00ad070 con 0x7f53fc073950 2024-04-20T21:22:24.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:24.136 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:20.501968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:22:24.137 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 22 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53dc06c8f0 msgr2=0x7f53dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53dc06c8f0 0x7f53dc06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f53fc159d50 tx=0x7f53e8009190 comp rx=0 tx=0).stop 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53fc073950 msgr2=0x7f53fc1e8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53fc073950 0x7f53fc1e8170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53f00097c0 tx=0x7f53f0004730 comp rx=0 tx=0).stop 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 shutdown_connections 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc15bd60 0x7f53fc1e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53fc155610 0x7f53fc1e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53dc06c8f0 0x7f53dc06edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 --2- 172.21.15.102:0/2145559609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53fc073950 0x7f53fc1e8170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 >> 172.21.15.102:0/2145559609 conn(0x7f53fc06ebf0 msgr2=0x7f53fc06c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 shutdown_connections 2024-04-20T21:22:24.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:24.139+0000 7f5403600640 1 -- 172.21.15.102:0/2145559609 wait complete. 2024-04-20T21:22:24.157 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:22:26.158 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:26.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.342+0000 7f41dde00640 1 Processor -- start 2024-04-20T21:22:26.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.342+0000 7f41dde00640 1 -- start start 2024-04-20T21:22:26.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.344+0000 7f41dde00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f41d8155530 0x7f41d815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.344+0000 7f41dde00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d815ef60 con 0x7f41d80cc470 2024-04-20T21:22:26.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.345+0000 7f41dde00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d815f240 con 0x7f41d80cb6a0 2024-04-20T21:22:26.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.345+0000 7f41dde00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80c5ca0 con 0x7f41d8155530 2024-04-20T21:22:26.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.345+0000 7f41dd400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f41d80cb6a0 0x7f41d81659d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38698/0 (socket says 172.21.15.102:38698) 2024-04-20T21:22:26.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.345+0000 7f41d7e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f41d80cc470 0x7f41d80cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33326/0 (socket says 172.21.15.102:33326) 2024-04-20T21:22:26.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.345+0000 7f41dca00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f41d8155530 0x7f41d815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.345+0000 7f41dd400640 1 -- 172.21.15.102:0/845721991 learned_addr learned my addr 172.21.15.102:0/845721991 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:26.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.345+0000 7f41d7400640 1 -- 172.21.15.102:0/845721991 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192717870 0 0) 0x7f41d815f240 con 0x7f41d80cb6a0 2024-04-20T21:22:26.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.346+0000 7f41d7400640 1 -- 172.21.15.102:0/845721991 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41c0003520 con 0x7f41d80cb6a0 2024-04-20T21:22:26.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.346+0000 7f41dca00640 1 -- 172.21.15.102:0/845721991 >> v1:172.21.15.193:6789/0 conn(0x7f41d80cb6a0 legacy=0x7f41d81659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.346+0000 7f41dca00640 1 -- 172.21.15.102:0/845721991 >> v1:172.21.15.102:6789/0 conn(0x7f41d80cc470 legacy=0x7f41d80cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.346+0000 7f41d7400640 1 -- 172.21.15.102:0/845721991 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2617217404 0 0) 0x7f41d815ef60 con 0x7f41d80cc470 2024-04-20T21:22:26.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.346+0000 7f41dca00640 1 -- 172.21.15.102:0/845721991 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d815f520 con 0x7f41d8155530 2024-04-20T21:22:26.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.347+0000 7f41dca00640 1 --2- 172.21.15.102:0/845721991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d8155530 0x7f41d815e640 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0097a0 tx=0x7f41cc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4512b8818bd4e762 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:26.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.348+0000 7f41d7400640 1 -- 172.21.15.102:0/845721991 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc034070 con 0x7f41d8155530 2024-04-20T21:22:26.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.348+0000 7f41d7400640 1 -- 172.21.15.102:0/845721991 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc030610 con 0x7f41d8155530 2024-04-20T21:22:26.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.348+0000 7f41d7400640 1 -- 172.21.15.102:0/845721991 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0309c0 con 0x7f41d8155530 2024-04-20T21:22:26.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.349+0000 7f41dde00640 1 -- 172.21.15.102:0/845721991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d8155530 msgr2=0x7f41d815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.349+0000 7f41dde00640 1 --2- 172.21.15.102:0/845721991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d8155530 0x7f41d815e640 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0097a0 tx=0x7f41cc02b960 comp rx=0 tx=0).stop 2024-04-20T21:22:26.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.349+0000 7f41dde00640 1 -- 172.21.15.102:0/845721991 shutdown_connections 2024-04-20T21:22:26.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.349+0000 7f41dde00640 1 --2- 172.21.15.102:0/845721991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d8155530 0x7f41d815e640 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.349+0000 7f41dde00640 1 -- 172.21.15.102:0/845721991 >> 172.21.15.102:0/845721991 conn(0x7f41d8075080 msgr2=0x7f41d8075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:26.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.349+0000 7f41dde00640 1 -- 172.21.15.102:0/845721991 shutdown_connections 2024-04-20T21:22:26.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.349+0000 7f41dde00640 1 -- 172.21.15.102:0/845721991 wait complete. 2024-04-20T21:22:26.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.351+0000 7f41dde00640 1 Processor -- start 2024-04-20T21:22:26.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.351+0000 7f41dde00640 1 -- start start 2024-04-20T21:22:26.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d80cb6a0 0x7f41d80b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dde00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41d80cc470 0x7f41d81520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d8155530 0x7f41d8152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dde00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80cd280 con 0x7f41d80cb6a0 2024-04-20T21:22:26.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dde00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80cd5f0 con 0x7f41d8155530 2024-04-20T21:22:26.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dde00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f41d80ccbe0 con 0x7f41d80cc470 2024-04-20T21:22:26.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d80cb6a0 0x7f41d80b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d80cb6a0 0x7f41d80b6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44472/0 (socket says 172.21.15.102:44472) 2024-04-20T21:22:26.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dca00640 1 -- 172.21.15.102:0/3691424280 learned_addr learned my addr 172.21.15.102:0/3691424280 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:26.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41d7e00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41d80cc470 0x7f41d81520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dd400640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d8155530 0x7f41d8152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.352+0000 7f41dca00640 1 -- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41d80cc470 msgr2=0x7f41d81520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41dca00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41d80cc470 0x7f41d81520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41dca00640 1 -- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d8155530 msgr2=0x7f41d8152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41dca00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d8155530 0x7f41d8152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41dca00640 1 -- 172.21.15.102:0/3691424280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d81f0dd0 con 0x7f41d80cb6a0 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41dd400640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d8155530 0x7f41d8152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41dca00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d80cb6a0 0x7f41d80b6910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f41cc02b9a0 tx=0x7f41cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41d4c00640 1 -- 172.21.15.102:0/3691424280 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc034040 con 0x7f41d80cb6a0 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41d4c00640 1 -- 172.21.15.102:0/3691424280 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc03aaa0 con 0x7f41d80cb6a0 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41d4c00640 1 -- 172.21.15.102:0/3691424280 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0443f0 con 0x7f41d80cb6a0 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d81f10b0 con 0x7f41d80cb6a0 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.353+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81f1670 con 0x7f41d80cb6a0 2024-04-20T21:22:26.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.355+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d81f1980 con 0x7f41d80cb6a0 2024-04-20T21:22:26.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.356+0000 7f41d4c00640 1 -- 172.21.15.102:0/3691424280 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41cc04b060 con 0x7f41d80cb6a0 2024-04-20T21:22:26.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.357+0000 7f41d4c00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.357+0000 7f41d4c00640 1 -- 172.21.15.102:0/3691424280 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f41cc0addf0 con 0x7f41d80cb6a0 2024-04-20T21:22:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.357+0000 7f41d7e00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.358+0000 7f41d7e00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41b806c8f0 0x7f41b806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f41d815c410 tx=0x7f41c4007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:26.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.362+0000 7f41d4c00640 1 -- 172.21.15.102:0/3691424280 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f41cc03f070 con 0x7f41d80cb6a0 2024-04-20T21:22:26.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.619+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41d804fd60 con 0x7f41d80cb6a0 2024-04-20T21:22:26.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.620+0000 7f41d4c00640 1 -- 172.21.15.102:0/3691424280 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6023 (secure 0 0 0) 0x7f41cc07b630 con 0x7f41d80cb6a0 2024-04-20T21:22:26.619 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:26.619 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:20.501968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:22:26.619 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 22 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41b806c8f0 msgr2=0x7f41b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41b806c8f0 0x7f41b806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f41d815c410 tx=0x7f41c4007d50 comp rx=0 tx=0).stop 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d80cb6a0 msgr2=0x7f41d80b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d80cb6a0 0x7f41d80b6910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f41cc02b9a0 tx=0x7f41cc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 shutdown_connections 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d8155530 0x7f41d8152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41d80cc470 0x7f41d81520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 --2- 172.21.15.102:0/3691424280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41d80cb6a0 0x7f41d80b6910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.622+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 >> 172.21.15.102:0/3691424280 conn(0x7f41d8075080 msgr2=0x7f41d80cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.623+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 shutdown_connections 2024-04-20T21:22:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:26.623+0000 7f41dde00640 1 -- 172.21.15.102:0/3691424280 wait complete. 2024-04-20T21:22:26.647 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:22:28.648 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:28.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.855+0000 7f560b200640 1 Processor -- start 2024-04-20T21:22:28.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.855+0000 7f560b200640 1 -- start start 2024-04-20T21:22:28.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.856+0000 7f560b200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f560c152290 0x7f560c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.856+0000 7f560b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c0c3700 0x7f560c06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.856+0000 7f560b200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f560c15c220 con 0x7f560c06eb70 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.856+0000 7f560b200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f560c0c9180 con 0x7f560c152290 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.856+0000 7f560b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f560c0c4840 con 0x7f560c0c3700 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f560a800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f560c06eb70 0x7f560c0c8330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55400/0 (socket says 172.21.15.102:55400) 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f560a800640 1 -- 172.21.15.102:0/523187585 learned_addr learned my addr 172.21.15.102:0/523187585 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5609e00640 1 --2- 172.21.15.102:0/523187585 >> v2:172.21.15.193:3300/0 conn(0x7f560c152290 0x7f560c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5603e00640 1 --2- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c0c3700 0x7f560c06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5609400640 1 -- 172.21.15.102:0/523187585 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601406416 0 0) 0x7f560c15c220 con 0x7f560c06eb70 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5609400640 1 -- 172.21.15.102:0/523187585 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f0003520 con 0x7f560c06eb70 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5609e00640 1 -- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c0c3700 msgr2=0x7f560c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5609e00640 1 --2- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c0c3700 0x7f560c06e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5609e00640 1 -- 172.21.15.102:0/523187585 >> v1:172.21.15.102:6789/0 conn(0x7f560c06eb70 legacy=0x7f560c0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5609e00640 1 -- 172.21.15.102:0/523187585 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c0c4560 con 0x7f560c152290 2024-04-20T21:22:28.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.857+0000 7f5603e00640 1 --2- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c0c3700 0x7f560c06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:28.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.858+0000 7f5609e00640 1 --2- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c152290 0x7f560c0cd310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0097a0 tx=0x7f55fc02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1161f47aa66bae64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.859+0000 7f5609400640 1 -- 172.21.15.102:0/523187585 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc034070 con 0x7f560c152290 2024-04-20T21:22:28.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.859+0000 7f5609400640 1 -- 172.21.15.102:0/523187585 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc030800 con 0x7f560c152290 2024-04-20T21:22:28.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.859+0000 7f5609400640 1 -- 172.21.15.102:0/523187585 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc030bb0 con 0x7f560c152290 2024-04-20T21:22:28.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.859+0000 7f560b200640 1 -- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c152290 msgr2=0x7f560c0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.859+0000 7f560b200640 1 --2- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c152290 0x7f560c0cd310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0097a0 tx=0x7f55fc02b940 comp rx=0 tx=0).stop 2024-04-20T21:22:28.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.860+0000 7f560b200640 1 -- 172.21.15.102:0/523187585 shutdown_connections 2024-04-20T21:22:28.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.860+0000 7f560b200640 1 --2- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c0c3700 0x7f560c06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.860+0000 7f560b200640 1 --2- 172.21.15.102:0/523187585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c152290 0x7f560c0cd310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.860+0000 7f560b200640 1 -- 172.21.15.102:0/523187585 >> 172.21.15.102:0/523187585 conn(0x7f560c0cc820 msgr2=0x7f560c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:28.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.860+0000 7f560b200640 1 -- 172.21.15.102:0/523187585 shutdown_connections 2024-04-20T21:22:28.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.860+0000 7f560b200640 1 -- 172.21.15.102:0/523187585 wait complete. 2024-04-20T21:22:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.862+0000 7f560b200640 1 Processor -- start 2024-04-20T21:22:28.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.862+0000 7f560b200640 1 -- start start 2024-04-20T21:22:28.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f560b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c06eb70 0x7f560c1f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f560b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0c3700 0x7f560c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f560b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c152290 0x7f560c151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f560b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f560c15ba20 con 0x7f560c0c3700 2024-04-20T21:22:28.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f560b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f560c15b740 con 0x7f560c06eb70 2024-04-20T21:22:28.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f560b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f560c15bd00 con 0x7f560c152290 2024-04-20T21:22:28.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f5603e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0c3700 0x7f560c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f5603e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0c3700 0x7f560c150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34986/0 (socket says 172.21.15.102:34986) 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f5603e00640 1 -- 172.21.15.102:0/2669880511 learned_addr learned my addr 172.21.15.102:0/2669880511 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f560a800640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c152290 0x7f560c151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.863+0000 7f5609e00640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c06eb70 0x7f560c1f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f5603e00640 1 -- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c152290 msgr2=0x7f560c151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f5603e00640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c152290 0x7f560c151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f5603e00640 1 -- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c06eb70 msgr2=0x7f560c1f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f5603e00640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c06eb70 0x7f560c1f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f5603e00640 1 -- 172.21.15.102:0/2669880511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c13b130 con 0x7f560c0c3700 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f560a800640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c152290 0x7f560c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f5609e00640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c06eb70 0x7f560c1f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f5603e00640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0c3700 0x7f560c150b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e960 tx=0x7f55f400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.864+0000 7f5602000640 1 -- 172.21.15.102:0/2669880511 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4017070 con 0x7f560c0c3700 2024-04-20T21:22:28.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.865+0000 7f5602000640 1 -- 172.21.15.102:0/2669880511 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f400ca00 con 0x7f560c0c3700 2024-04-20T21:22:28.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.865+0000 7f5602000640 1 -- 172.21.15.102:0/2669880511 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400cdd0 con 0x7f560c0c3700 2024-04-20T21:22:28.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.865+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c155720 con 0x7f560c0c3700 2024-04-20T21:22:28.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.865+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c13ae50 con 0x7f560c0c3700 2024-04-20T21:22:28.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.866+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c0cae40 con 0x7f560c0c3700 2024-04-20T21:22:28.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.867+0000 7f5602000640 1 -- 172.21.15.102:0/2669880511 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55f401a070 con 0x7f560c0c3700 2024-04-20T21:22:28.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.868+0000 7f5602000640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f55e806c8f0 0x7f55e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.868+0000 7f5602000640 1 -- 172.21.15.102:0/2669880511 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f55f408bf30 con 0x7f560c0c3700 2024-04-20T21:22:28.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.868+0000 7f5609e00640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f55e806c8f0 0x7f55e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.869+0000 7f5609e00640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f55e806c8f0 0x7f55e806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55fc036000 tx=0x7f55fc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:28.872+0000 7f5602000640 1 -- 172.21.15.102:0/2669880511 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f560c0cae40 con 0x7f560c0c3700 2024-04-20T21:22:29.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.112+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f560c04fd60 con 0x7f560c0c3700 2024-04-20T21:22:29.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.113+0000 7f5602000640 1 -- 172.21.15.102:0/2669880511 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6027 (secure 0 0 0) 0x7f55f40596d0 con 0x7f560c0c3700 2024-04-20T21:22:29.112 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:29.112 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:26.782787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:22:29.112 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 23 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f55e806c8f0 msgr2=0x7f55e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f55e806c8f0 0x7f55e806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55fc036000 tx=0x7f55fc008030 comp rx=0 tx=0).stop 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0c3700 msgr2=0x7f560c150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0c3700 0x7f560c150b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e960 tx=0x7f55f400ee20 comp rx=0 tx=0).stop 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 shutdown_connections 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f560c152290 0x7f560c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0c3700 0x7f560c150b00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f55e806c8f0 0x7f55e806edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 --2- 172.21.15.102:0/2669880511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c06eb70 0x7f560c1f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 >> 172.21.15.102:0/2669880511 conn(0x7f560c0cc820 msgr2=0x7f560c0c61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 shutdown_connections 2024-04-20T21:22:29.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:29.116+0000 7f560b200640 1 -- 172.21.15.102:0/2669880511 wait complete. 2024-04-20T21:22:29.134 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:22:31.136 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:31.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.343+0000 7f110c800640 1 Processor -- start 2024-04-20T21:22:31.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.343+0000 7f110c800640 1 -- start start 2024-04-20T21:22:31.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.345+0000 7f110c800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f11040c8db0 0x7f1104070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.345+0000 7f110c800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1104158e80 con 0x7f1104071330 2024-04-20T21:22:31.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.345+0000 7f110c800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11040c91c0 con 0x7f1104152170 2024-04-20T21:22:31.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.345+0000 7f110c800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f11040c6dc0 con 0x7f11040c8db0 2024-04-20T21:22:31.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.345+0000 7f1109400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f11040c8db0 0x7f1104070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.345+0000 7f110a800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1104071330 0x7f11041628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55406/0 (socket says 172.21.15.102:55406) 2024-04-20T21:22:31.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.345+0000 7f110a800640 1 -- 172.21.15.102:0/652670376 learned_addr learned my addr 172.21.15.102:0/652670376 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:31.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.345+0000 7f1108a00640 1 -- 172.21.15.102:0/652670376 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2267626138 0 0) 0x7f1104158e80 con 0x7f1104071330 2024-04-20T21:22:31.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.346+0000 7f1109400640 1 -- 172.21.15.102:0/652670376 >> v1:172.21.15.193:6789/0 conn(0x7f1104152170 legacy=0x7f1104158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.346+0000 7f1109400640 1 -- 172.21.15.102:0/652670376 >> v1:172.21.15.102:6789/0 conn(0x7f1104071330 legacy=0x7f11041628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.346+0000 7f1109400640 1 -- 172.21.15.102:0/652670376 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1104071f70 con 0x7f11040c8db0 2024-04-20T21:22:31.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.347+0000 7f1109400640 1 --2- 172.21.15.102:0/652670376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104070df0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f802db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbdb201e0eee6e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.347+0000 7f1108a00640 1 -- 172.21.15.102:0/652670376 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f803b070 con 0x7f11040c8db0 2024-04-20T21:22:31.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.347+0000 7f1108a00640 1 -- 172.21.15.102:0/652670376 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f802e920 con 0x7f11040c8db0 2024-04-20T21:22:31.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.347+0000 7f1108a00640 1 -- 172.21.15.102:0/652670376 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f802ecf0 con 0x7f11040c8db0 2024-04-20T21:22:31.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.348+0000 7f110c800640 1 -- 172.21.15.102:0/652670376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 msgr2=0x7f1104070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.348+0000 7f110c800640 1 --2- 172.21.15.102:0/652670376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104070df0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f802db70 comp rx=0 tx=0).stop 2024-04-20T21:22:31.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.348+0000 7f110c800640 1 -- 172.21.15.102:0/652670376 shutdown_connections 2024-04-20T21:22:31.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.348+0000 7f110c800640 1 --2- 172.21.15.102:0/652670376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104070df0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.348+0000 7f110c800640 1 -- 172.21.15.102:0/652670376 >> 172.21.15.102:0/652670376 conn(0x7f11040c9390 msgr2=0x7f11040c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:31.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.348+0000 7f110c800640 1 -- 172.21.15.102:0/652670376 shutdown_connections 2024-04-20T21:22:31.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.348+0000 7f110c800640 1 -- 172.21.15.102:0/652670376 wait complete. 2024-04-20T21:22:31.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.349+0000 7f110c800640 1 Processor -- start 2024-04-20T21:22:31.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.350+0000 7f110c800640 1 -- start start 2024-04-20T21:22:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.350+0000 7f110c800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1104071330 0x7f11041529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.350+0000 7f110c800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.350+0000 7f110c800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152170 0x7f1104151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.350+0000 7f110c800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11040724b0 con 0x7f11040c8db0 2024-04-20T21:22:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.350+0000 7f110c800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1104072210 con 0x7f1104152170 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.350+0000 7f110c800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1104158250 con 0x7f1104071330 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f110a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152170 0x7f1104151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1104071330 0x7f11041529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35006/0 (socket says 172.21.15.102:35006) 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f110a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152170 0x7f1104151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:42436/0 (socket says 172.21.15.102:42436) 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109400640 1 -- 172.21.15.102:0/4223256106 learned_addr learned my addr 172.21.15.102:0/4223256106 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109400640 1 -- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1104071330 msgr2=0x7f11041529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109400640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1104071330 0x7f11041529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109400640 1 -- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152170 msgr2=0x7f1104151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109400640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152170 0x7f1104151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.351+0000 7f1109400640 1 -- 172.21.15.102:0/4223256106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11041536f0 con 0x7f11040c8db0 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.352+0000 7f1109400640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104151210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f10f8000c00 tx=0x7f10f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.352+0000 7f10fe000640 1 -- 172.21.15.102:0/4223256106 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f803b040 con 0x7f11040c8db0 2024-04-20T21:22:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.352+0000 7f10fe000640 1 -- 172.21.15.102:0/4223256106 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f8046070 con 0x7f11040c8db0 2024-04-20T21:22:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.352+0000 7f10fe000640 1 -- 172.21.15.102:0/4223256106 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8041440 con 0x7f11040c8db0 2024-04-20T21:22:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.352+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11041539d0 con 0x7f11040c8db0 2024-04-20T21:22:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.352+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1104153f90 con 0x7f11040c8db0 2024-04-20T21:22:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.353+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1104074340 con 0x7f11040c8db0 2024-04-20T21:22:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.354+0000 7f10fe000640 1 -- 172.21.15.102:0/4223256106 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10f802e7e0 con 0x7f11040c8db0 2024-04-20T21:22:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.355+0000 7f10fe000640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e406c8f0 0x7f10e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.355+0000 7f10fe000640 1 -- 172.21.15.102:0/4223256106 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f10f80b0190 con 0x7f11040c8db0 2024-04-20T21:22:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.355+0000 7f1109e00640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e406c8f0 0x7f10e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.356+0000 7f1109e00640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e406c8f0 0x7f10e406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10f0001780 tx=0x7f10f0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.360+0000 7f10fe000640 1 -- 172.21.15.102:0/4223256106 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f10f807d9c0 con 0x7f11040c8db0 2024-04-20T21:22:31.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.598+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11040cba60 con 0x7f11040c8db0 2024-04-20T21:22:31.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.599+0000 7f10fe000640 1 -- 172.21.15.102:0/4223256106 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6027 (secure 0 0 0) 0x7f10f8081cc0 con 0x7f11040c8db0 2024-04-20T21:22:31.599 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:31.599 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:26.782787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:22:31.599 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 23 2024-04-20T21:22:31.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.601+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e406c8f0 msgr2=0x7f10e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.601+0000 7f110c800640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e406c8f0 0x7f10e406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10f0001780 tx=0x7f10f0009040 comp rx=0 tx=0).stop 2024-04-20T21:22:31.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.601+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 msgr2=0x7f1104151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.601+0000 7f110c800640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104151210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f10f8000c00 tx=0x7f10f80046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.602+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 shutdown_connections 2024-04-20T21:22:31.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.602+0000 7f110c800640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e406c8f0 0x7f10e406edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.602+0000 7f110c800640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152170 0x7f1104151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.602+0000 7f110c800640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11040c8db0 0x7f1104151210 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.602+0000 7f110c800640 1 --2- 172.21.15.102:0/4223256106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1104071330 0x7f11041529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.602+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 >> 172.21.15.102:0/4223256106 conn(0x7f11040c9390 msgr2=0x7f110406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:31.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.602+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 shutdown_connections 2024-04-20T21:22:31.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:31.602+0000 7f110c800640 1 -- 172.21.15.102:0/4223256106 wait complete. 2024-04-20T21:22:31.619 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:22:33.620 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:33.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.831+0000 7f910f200640 1 Processor -- start 2024-04-20T21:22:33.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.831+0000 7f910f200640 1 -- start start 2024-04-20T21:22:33.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.833+0000 7f910f200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f910815ee00 0x7f910806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:33.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.833+0000 7f910f200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f910815f2b0 0x7f91080710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:33.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.833+0000 7f910f200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91080c8f20 con 0x7f910815ea20 2024-04-20T21:22:33.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.833+0000 7f910f200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f910806cb80 con 0x7f910815ee00 2024-04-20T21:22:33.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.833+0000 7f910f200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f910806ce60 con 0x7f910815f2b0 2024-04-20T21:22:33.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f9107e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f910815ea20 0x7f91080c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55416/0 (socket says 172.21.15.102:55416) 2024-04-20T21:22:33.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f9107400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f910815ee00 0x7f910806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:33.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f9107e00640 1 -- 172.21.15.102:0/2013736900 learned_addr learned my addr 172.21.15.102:0/2013736900 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:33.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f910d200640 1 --2- 172.21.15.102:0/2013736900 >> v2:172.21.15.193:3300/0 conn(0x7f910815f2b0 0x7f91080710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:33.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f910d200640 1 -- 172.21.15.102:0/2013736900 >> v2:172.21.15.102:3300/0 conn(0x7f910815ee00 msgr2=0x7f910806d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:33.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f910d200640 1 --2- 172.21.15.102:0/2013736900 >> v2:172.21.15.102:3300/0 conn(0x7f910815ee00 0x7f910806d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f910d200640 1 -- 172.21.15.102:0/2013736900 >> v1:172.21.15.102:6789/0 conn(0x7f910815ea20 legacy=0x7f91080c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:33.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f910d200640 1 -- 172.21.15.102:0/2013736900 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910806d140 con 0x7f910815f2b0 2024-04-20T21:22:33.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.834+0000 7f9107400640 1 --2- 172.21.15.102:0/2013736900 >> v2:172.21.15.102:3300/0 conn(0x7f910815ee00 0x7f910806d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.836+0000 7f910d200640 1 --2- 172.21.15.102:0/2013736900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815f2b0 0x7f91080710a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f910806f970 tx=0x7f90f802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f1fed6195c8890 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.836+0000 7f9106a00640 1 -- 172.21.15.102:0/2013736900 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8034070 con 0x7f910815f2b0 2024-04-20T21:22:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.836+0000 7f9106a00640 1 -- 172.21.15.102:0/2013736900 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f8030610 con 0x7f910815f2b0 2024-04-20T21:22:33.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.836+0000 7f9106a00640 1 -- 172.21.15.102:0/2013736900 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f80309e0 con 0x7f910815f2b0 2024-04-20T21:22:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.837+0000 7f910f200640 1 -- 172.21.15.102:0/2013736900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815f2b0 msgr2=0x7f91080710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.837+0000 7f910f200640 1 --2- 172.21.15.102:0/2013736900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815f2b0 0x7f91080710a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f910806f970 tx=0x7f90f802b960 comp rx=0 tx=0).stop 2024-04-20T21:22:33.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.837+0000 7f910f200640 1 -- 172.21.15.102:0/2013736900 shutdown_connections 2024-04-20T21:22:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.837+0000 7f910f200640 1 --2- 172.21.15.102:0/2013736900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815f2b0 0x7f91080710a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.837+0000 7f910f200640 1 --2- 172.21.15.102:0/2013736900 >> v2:172.21.15.102:3300/0 conn(0x7f910815ee00 0x7f910806d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.837+0000 7f910f200640 1 -- 172.21.15.102:0/2013736900 >> 172.21.15.102:0/2013736900 conn(0x7f91080c9390 msgr2=0x7f91080c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.837+0000 7f910f200640 1 -- 172.21.15.102:0/2013736900 shutdown_connections 2024-04-20T21:22:33.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.837+0000 7f910f200640 1 -- 172.21.15.102:0/2013736900 wait complete. 2024-04-20T21:22:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.838+0000 7f910f200640 1 Processor -- start 2024-04-20T21:22:33.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.839+0000 7f910f200640 1 -- start start 2024-04-20T21:22:33.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.839+0000 7f910f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f910815ea20 0x7f910815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:33.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.839+0000 7f910f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815ee00 0x7f91080c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:33.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f910f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f910815f2b0 0x7f9108143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:33.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f910f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f91080c8c80 con 0x7f910815f2b0 2024-04-20T21:22:33.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f910f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91080c8b00 con 0x7f910815ee00 2024-04-20T21:22:33.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f910f200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9108154e80 con 0x7f910815ea20 2024-04-20T21:22:33.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f910d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f910815f2b0 0x7f9108143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:33.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f910815ea20 0x7f910815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:33.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f910815ea20 0x7f910815c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:57032/0 (socket says 172.21.15.102:57032) 2024-04-20T21:22:33.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815ee00 0x7f91080c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:33.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f910d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f910815f2b0 0x7f9108143610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35026/0 (socket says 172.21.15.102:35026) 2024-04-20T21:22:33.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107e00640 1 -- 172.21.15.102:0/1206999415 learned_addr learned my addr 172.21.15.102:0/1206999415 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107e00640 1 -- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815ee00 msgr2=0x7f91080c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107e00640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815ee00 0x7f91080c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107e00640 1 -- 172.21.15.102:0/1206999415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f910815f2b0 msgr2=0x7f9108143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107e00640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f910815f2b0 0x7f9108143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.840+0000 7f9107e00640 1 -- 172.21.15.102:0/1206999415 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910815c8c0 con 0x7f910815ea20 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.841+0000 7f9107e00640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f910815ea20 0x7f910815c380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f910000cb10 tx=0x7f910000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.841+0000 7f90ffe00640 1 -- 172.21.15.102:0/1206999415 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100017070 con 0x7f910815ea20 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.841+0000 7f90ffe00640 1 -- 172.21.15.102:0/1206999415 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910000ac70 con 0x7f910815ea20 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.841+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91081e4a40 con 0x7f910815ea20 2024-04-20T21:22:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.841+0000 7f90ffe00640 1 -- 172.21.15.102:0/1206999415 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910000ae60 con 0x7f910815ea20 2024-04-20T21:22:33.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.841+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91081f0df0 con 0x7f910815ea20 2024-04-20T21:22:33.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.842+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9108074340 con 0x7f910815ea20 2024-04-20T21:22:33.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.843+0000 7f90ffe00640 1 -- 172.21.15.102:0/1206999415 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f910001a070 con 0x7f910815ea20 2024-04-20T21:22:33.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.844+0000 7f90ffe00640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90e406c8f0 0x7f90e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:33.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.844+0000 7f90ffe00640 1 -- 172.21.15.102:0/1206999415 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f910008bd80 con 0x7f910815ea20 2024-04-20T21:22:33.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.844+0000 7f9107400640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90e406c8f0 0x7f90e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:33.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.845+0000 7f9107400640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90e406c8f0 0x7f90e406edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009560 tx=0x7f90f0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:33.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:33.849+0000 7f90ffe00640 1 -- 172.21.15.102:0/1206999415 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f910008f030 con 0x7f910815ea20 2024-04-20T21:22:34.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.090+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91080c99c0 con 0x7f910815ea20 2024-04-20T21:22:34.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.091+0000 7f90ffe00640 1 -- 172.21.15.102:0/1206999415 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6024 (secure 0 0 0) 0x7f9100059630 con 0x7f910815ea20 2024-04-20T21:22:34.090 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:34.091 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:32.068548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:22:34.091 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 24 2024-04-20T21:22:34.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90e406c8f0 msgr2=0x7f90e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90e406c8f0 0x7f90e406edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009560 tx=0x7f90f0009190 comp rx=0 tx=0).stop 2024-04-20T21:22:34.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f910815ea20 msgr2=0x7f910815c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f910815ea20 0x7f910815c380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f910000cb10 tx=0x7f910000a060 comp rx=0 tx=0).stop 2024-04-20T21:22:34.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 shutdown_connections 2024-04-20T21:22:34.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f910815f2b0 0x7f9108143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f90e406c8f0 0x7f90e406edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f910815ee00 0x7f91080c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 --2- 172.21.15.102:0/1206999415 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f910815ea20 0x7f910815c380 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.093+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 >> 172.21.15.102:0/1206999415 conn(0x7f91080c9390 msgr2=0x7f91081543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:34.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.094+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 shutdown_connections 2024-04-20T21:22:34.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:34.094+0000 7f910f200640 1 -- 172.21.15.102:0/1206999415 wait complete. 2024-04-20T21:22:34.112 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 24, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 24, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5266": { "addr": "172.21.15.102:6833/713712414", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 713712414, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 713712414, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5266, "incarnation": 22, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:22:32.068548+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5266 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:22:36.113 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:36.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.321+0000 7fa649e00640 1 Processor -- start 2024-04-20T21:22:36.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.321+0000 7fa649e00640 1 -- start start 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.322+0000 7fa649e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa644073c70 0x7fa6441647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.322+0000 7fa649e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa644153560 con 0x7fa644153180 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa649e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6440c7a70 con 0x7fa6440c75a0 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa649e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa6440c7d30 con 0x7fa644073c70 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa648a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa644153180 0x7fa644158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55430/0 (socket says 172.21.15.102:55430) 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa649400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa644073c70 0x7fa6441647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa648a00640 1 -- 172.21.15.102:0/178640323 learned_addr learned my addr 172.21.15.102:0/178640323 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:36.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa63be00640 1 --1- 172.21.15.102:0/178640323 >> v1:172.21.15.193:6789/0 conn(0x7fa6440c75a0 0x7fa6440c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:46848/0 (socket says 172.21.15.102:46848) 2024-04-20T21:22:36.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa643e00640 1 -- 172.21.15.102:0/178640323 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134523010 0 0) 0x7fa6440c7a70 con 0x7fa6440c75a0 2024-04-20T21:22:36.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa643e00640 1 -- 172.21.15.102:0/178640323 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa624003520 con 0x7fa6440c75a0 2024-04-20T21:22:36.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa649400640 1 -- 172.21.15.102:0/178640323 >> v1:172.21.15.193:6789/0 conn(0x7fa6440c75a0 legacy=0x7fa6440c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa649400640 1 -- 172.21.15.102:0/178640323 >> v1:172.21.15.102:6789/0 conn(0x7fa644153180 legacy=0x7fa644158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa643e00640 1 -- 172.21.15.102:0/178640323 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452394594 0 0) 0x7fa644153560 con 0x7fa644153180 2024-04-20T21:22:36.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.323+0000 7fa649400640 1 -- 172.21.15.102:0/178640323 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64415c4d0 con 0x7fa644073c70 2024-04-20T21:22:36.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.325+0000 7fa649400640 1 --2- 172.21.15.102:0/178640323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa644073c70 0x7fa6441647d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009d90 tx=0x7fa63c02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e826e279012b083 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.325+0000 7fa643e00640 1 -- 172.21.15.102:0/178640323 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c039070 con 0x7fa644073c70 2024-04-20T21:22:36.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.325+0000 7fa643e00640 1 -- 172.21.15.102:0/178640323 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63c02cd00 con 0x7fa644073c70 2024-04-20T21:22:36.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.325+0000 7fa643e00640 1 -- 172.21.15.102:0/178640323 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c034640 con 0x7fa644073c70 2024-04-20T21:22:36.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.326+0000 7fa649e00640 1 -- 172.21.15.102:0/178640323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa644073c70 msgr2=0x7fa6441647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.326+0000 7fa649e00640 1 --2- 172.21.15.102:0/178640323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa644073c70 0x7fa6441647d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009d90 tx=0x7fa63c02bf50 comp rx=0 tx=0).stop 2024-04-20T21:22:36.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.326+0000 7fa649e00640 1 -- 172.21.15.102:0/178640323 shutdown_connections 2024-04-20T21:22:36.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.326+0000 7fa649e00640 1 --2- 172.21.15.102:0/178640323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa644073c70 0x7fa6441647d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.326+0000 7fa649e00640 1 -- 172.21.15.102:0/178640323 >> 172.21.15.102:0/178640323 conn(0x7fa6440cc820 msgr2=0x7fa6440ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.326+0000 7fa649e00640 1 -- 172.21.15.102:0/178640323 shutdown_connections 2024-04-20T21:22:36.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.326+0000 7fa649e00640 1 -- 172.21.15.102:0/178640323 wait complete. 2024-04-20T21:22:36.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.328+0000 7fa649e00640 1 Processor -- start 2024-04-20T21:22:36.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.328+0000 7fa649e00640 1 -- start start 2024-04-20T21:22:36.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.329+0000 7fa649e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa644073c70 0x7fa6440cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.329+0000 7fa649e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6440c75a0 0x7fa644148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.329+0000 7fa649e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa644153180 0x7fa6441f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.329+0000 7fa649e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa644159550 con 0x7fa644073c70 2024-04-20T21:22:36.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.329+0000 7fa649e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa64415ca90 con 0x7fa6440c75a0 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.329+0000 7fa649e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa64415d290 con 0x7fa644153180 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6440c75a0 0x7fa644148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa648a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa644073c70 0x7fa6440cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6440c75a0 0x7fa644148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:42468/0 (socket says 172.21.15.102:42468) 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 -- 172.21.15.102:0/2280272201 learned_addr learned my addr 172.21.15.102:0/2280272201 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa649400640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa644153180 0x7fa6441f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 -- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa644153180 msgr2=0x7fa6441f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa644153180 0x7fa6441f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 -- 172.21.15.102:0/2280272201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa644073c70 msgr2=0x7fa6440cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa644073c70 0x7fa6440cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 -- 172.21.15.102:0/2280272201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6441f44b0 con 0x7fa6440c75a0 2024-04-20T21:22:36.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa63be00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6440c75a0 0x7fa644148410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c3f0 tx=0x7fa634001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.330+0000 7fa641600640 1 -- 172.21.15.102:0/2280272201 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340074e0 con 0x7fa6440c75a0 2024-04-20T21:22:36.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.331+0000 7fa641600640 1 -- 172.21.15.102:0/2280272201 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6340047e0 con 0x7fa6440c75a0 2024-04-20T21:22:36.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.331+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6440cbf60 con 0x7fa6440c75a0 2024-04-20T21:22:36.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.331+0000 7fa641600640 1 -- 172.21.15.102:0/2280272201 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634004bd0 con 0x7fa6440c75a0 2024-04-20T21:22:36.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.331+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6441f4750 con 0x7fa6440c75a0 2024-04-20T21:22:36.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.332+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa644068da0 con 0x7fa6440c75a0 2024-04-20T21:22:36.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.333+0000 7fa641600640 1 -- 172.21.15.102:0/2280272201 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa63401a020 con 0x7fa6440c75a0 2024-04-20T21:22:36.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.334+0000 7fa641600640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa62006c8f0 0x7fa62006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.334+0000 7fa641600640 1 -- 172.21.15.102:0/2280272201 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7fa63408b800 con 0x7fa6440c75a0 2024-04-20T21:22:36.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.334+0000 7fa648a00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa62006c8f0 0x7fa62006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.335+0000 7fa648a00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa62006c8f0 0x7fa62006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa62c000f60 tx=0x7fa62c007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.339+0000 7fa641600640 1 -- 172.21.15.102:0/2280272201 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa644068da0 con 0x7fa6440c75a0 2024-04-20T21:22:36.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.578+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6440c8230 con 0x7fa6440c75a0 2024-04-20T21:22:36.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.579+0000 7fa641600640 1 -- 172.21.15.102:0/2280272201 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6024 (secure 0 0 0) 0x7fa6440c8230 con 0x7fa6440c75a0 2024-04-20T21:22:36.579 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:36.579 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:32.068548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:22:36.579 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 24 2024-04-20T21:22:36.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa62006c8f0 msgr2=0x7fa62006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa62006c8f0 0x7fa62006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa62c000f60 tx=0x7fa62c007d50 comp rx=0 tx=0).stop 2024-04-20T21:22:36.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6440c75a0 msgr2=0x7fa644148410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6440c75a0 0x7fa644148410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c3f0 tx=0x7fa634001e70 comp rx=0 tx=0).stop 2024-04-20T21:22:36.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 shutdown_connections 2024-04-20T21:22:36.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa644153180 0x7fa6441f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa62006c8f0 0x7fa62006edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6440c75a0 0x7fa644148410 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 --2- 172.21.15.102:0/2280272201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa644073c70 0x7fa6440cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.582+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 >> 172.21.15.102:0/2280272201 conn(0x7fa6440cc820 msgr2=0x7fa64406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.583+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 shutdown_connections 2024-04-20T21:22:36.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:36.583+0000 7fa649e00640 1 -- 172.21.15.102:0/2280272201 wait complete. 2024-04-20T21:22:36.600 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:22:38.601 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:38.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.806+0000 7f8a1ea00640 1 Processor -- start 2024-04-20T21:22:38.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.806+0000 7f8a1ea00640 1 -- start start 2024-04-20T21:22:38.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.808+0000 7f8a1ea00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8a180c75a0 0x7f8a180c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.808+0000 7f8a1ea00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a18153560 con 0x7f8a18073c70 2024-04-20T21:22:38.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.808+0000 7f8a1ea00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a180c7a70 con 0x7f8a18153180 2024-04-20T21:22:38.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.808+0000 7f8a1ea00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8a180c7d30 con 0x7f8a180c75a0 2024-04-20T21:22:38.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.808+0000 7f8a1e000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8a18073c70 0x7f8a181637c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36548/0 (socket says 172.21.15.102:36548) 2024-04-20T21:22:38.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.808+0000 7f8a1e000640 1 -- 172.21.15.102:0/231070911 learned_addr learned my addr 172.21.15.102:0/231070911 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:38.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.808+0000 7f8a1cc00640 1 --2- 172.21.15.102:0/231070911 >> v2:172.21.15.193:3300/0 conn(0x7f8a180c75a0 0x7f8a180c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.809+0000 7f8a17e00640 1 -- 172.21.15.102:0/231070911 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3222050263 0 0) 0x7f8a18153560 con 0x7f8a18073c70 2024-04-20T21:22:38.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.809+0000 7f8a1cc00640 1 -- 172.21.15.102:0/231070911 >> v1:172.21.15.193:6789/0 conn(0x7f8a18153180 legacy=0x7f8a18158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.809+0000 7f8a1cc00640 1 -- 172.21.15.102:0/231070911 >> v1:172.21.15.102:6789/0 conn(0x7f8a18073c70 legacy=0x7f8a181637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.809+0000 7f8a1cc00640 1 -- 172.21.15.102:0/231070911 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1815c4d0 con 0x7f8a180c75a0 2024-04-20T21:22:38.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.810+0000 7f8a1cc00640 1 --2- 172.21.15.102:0/231070911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a180c75a0 0x7f8a180c5f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8a040097a0 tx=0x7f8a0402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=726751acf0aae09a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:38.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.810+0000 7f8a17e00640 1 -- 172.21.15.102:0/231070911 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04034070 con 0x7f8a180c75a0 2024-04-20T21:22:38.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.810+0000 7f8a17e00640 1 -- 172.21.15.102:0/231070911 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04030610 con 0x7f8a180c75a0 2024-04-20T21:22:38.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.810+0000 7f8a17e00640 1 -- 172.21.15.102:0/231070911 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040309e0 con 0x7f8a180c75a0 2024-04-20T21:22:38.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.811+0000 7f8a1ea00640 1 -- 172.21.15.102:0/231070911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a180c75a0 msgr2=0x7f8a180c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.811+0000 7f8a1ea00640 1 --2- 172.21.15.102:0/231070911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a180c75a0 0x7f8a180c5f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8a040097a0 tx=0x7f8a0402b960 comp rx=0 tx=0).stop 2024-04-20T21:22:38.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.811+0000 7f8a1ea00640 1 -- 172.21.15.102:0/231070911 shutdown_connections 2024-04-20T21:22:38.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.811+0000 7f8a1ea00640 1 --2- 172.21.15.102:0/231070911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a180c75a0 0x7f8a180c5f30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.811+0000 7f8a1ea00640 1 -- 172.21.15.102:0/231070911 >> 172.21.15.102:0/231070911 conn(0x7f8a180cc820 msgr2=0x7f8a180ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:38.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.811+0000 7f8a1ea00640 1 -- 172.21.15.102:0/231070911 shutdown_connections 2024-04-20T21:22:38.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.811+0000 7f8a1ea00640 1 -- 172.21.15.102:0/231070911 wait complete. 2024-04-20T21:22:38.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.813+0000 7f8a1ea00640 1 Processor -- start 2024-04-20T21:22:38.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.813+0000 7f8a1ea00640 1 -- start start 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.814+0000 7f8a1ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a18073c70 0x7f8a181403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.814+0000 7f8a1ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a180c75a0 0x7f8a181f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.814+0000 7f8a1ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18153180 0x7f8a181f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.814+0000 7f8a1ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18159550 con 0x7f8a180c75a0 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.814+0000 7f8a1ea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1815ca90 con 0x7f8a18153180 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.814+0000 7f8a1ea00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1815d290 con 0x7f8a18073c70 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a180c75a0 0x7f8a181f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a180c75a0 0x7f8a181f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54548/0 (socket says 172.21.15.102:54548) 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1cc00640 1 -- 172.21.15.102:0/1336643031 learned_addr learned my addr 172.21.15.102:0/1336643031 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1e000640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18153180 0x7f8a181f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1d600640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a18073c70 0x7f8a181403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1cc00640 1 -- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a18073c70 msgr2=0x7f8a181403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1cc00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a18073c70 0x7f8a181403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1cc00640 1 -- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18153180 msgr2=0x7f8a181f0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1cc00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18153180 0x7f8a181f0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.815+0000 7f8a1cc00640 1 -- 172.21.15.102:0/1336643031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a180c4260 con 0x7f8a180c75a0 2024-04-20T21:22:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.816+0000 7f8a1cc00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a180c75a0 0x7f8a181f0a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8a0402bf70 tx=0x7f8a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:38.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.816+0000 7f8a15600640 1 -- 172.21.15.102:0/1336643031 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04034040 con 0x7f8a180c75a0 2024-04-20T21:22:38.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.816+0000 7f8a15600640 1 -- 172.21.15.102:0/1336643031 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0403acc0 con 0x7f8a180c75a0 2024-04-20T21:22:38.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.816+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a181f4110 con 0x7f8a180c75a0 2024-04-20T21:22:38.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.816+0000 7f8a15600640 1 -- 172.21.15.102:0/1336643031 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04044630 con 0x7f8a180c75a0 2024-04-20T21:22:38.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.816+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181f46d0 con 0x7f8a180c75a0 2024-04-20T21:22:38.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.817+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a18070820 con 0x7f8a180c75a0 2024-04-20T21:22:38.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.818+0000 7f8a15600640 1 -- 172.21.15.102:0/1336643031 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a04037070 con 0x7f8a180c75a0 2024-04-20T21:22:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.819+0000 7f8a15600640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f89f806c8f0 0x7f89f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.820+0000 7f8a15600640 1 -- 172.21.15.102:0/1336643031 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f8a040adf40 con 0x7f8a180c75a0 2024-04-20T21:22:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.820+0000 7f8a1d600640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f89f806c8f0 0x7f89f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.820+0000 7f8a1d600640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f89f806c8f0 0x7f89f806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c000f60 tx=0x7f8a0c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:38.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:38.824+0000 7f8a15600640 1 -- 172.21.15.102:0/1336643031 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a18070820 con 0x7f8a180c75a0 2024-04-20T21:22:39.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.066+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a1804fd60 con 0x7f8a180c75a0 2024-04-20T21:22:39.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.067+0000 7f8a15600640 1 -- 172.21.15.102:0/1336643031 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+6031 (secure 0 0 0) 0x7f8a0403f070 con 0x7f8a180c75a0 2024-04-20T21:22:39.067 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:39.067 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:22:39.067 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 25 2024-04-20T21:22:39.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f89f806c8f0 msgr2=0x7f89f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f89f806c8f0 0x7f89f806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c000f60 tx=0x7f8a0c009040 comp rx=0 tx=0).stop 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a180c75a0 msgr2=0x7f8a181f0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a180c75a0 0x7f8a181f0a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8a0402bf70 tx=0x7f8a040046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 shutdown_connections 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f89f806c8f0 0x7f89f806edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18153180 0x7f8a181f0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a180c75a0 0x7f8a181f0a00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 --2- 172.21.15.102:0/1336643031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a18073c70 0x7f8a181403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 >> 172.21.15.102:0/1336643031 conn(0x7f8a180cc820 msgr2=0x7f8a180ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 shutdown_connections 2024-04-20T21:22:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:39.070+0000 7f8a1ea00640 1 -- 172.21.15.102:0/1336643031 wait complete. 2024-04-20T21:22:39.089 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=4847 2024-04-20T21:22:39.089 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.c 2024-04-20T21:22:43.313 INFO:tasks.workunit.client.0.smithi102.stdout:./156/.snap/snap-subdir-test 2024-04-20T21:22:43.323 INFO:tasks.workunit.client.0.smithi102.stdout:./131/.snap/snap-subdir-test 2024-04-20T21:22:43.331 INFO:tasks.workunit.client.0.smithi102.stdout:./123/.snap/snap-subdir-test 2024-04-20T21:22:43.340 INFO:tasks.workunit.client.0.smithi102.stdout:./160/.snap/snap-subdir-test 2024-04-20T21:22:43.347 INFO:tasks.workunit.client.0.smithi102.stdout:./165/.snap/snap-subdir-test 2024-04-20T21:22:43.355 INFO:tasks.workunit.client.0.smithi102.stdout:./101/.snap/snap-subdir-test 2024-04-20T21:22:43.361 INFO:tasks.workunit.client.0.smithi102.stdout:./148/.snap/snap-subdir-test 2024-04-20T21:22:43.369 INFO:tasks.workunit.client.0.smithi102.stdout:./168/.snap/snap-subdir-test 2024-04-20T21:22:43.377 INFO:tasks.workunit.client.0.smithi102.stdout:./179/.snap/snap-subdir-test 2024-04-20T21:22:43.385 INFO:tasks.workunit.client.0.smithi102.stdout:./257/.snap/snap-subdir-test 2024-04-20T21:22:43.392 INFO:tasks.workunit.client.0.smithi102.stdout:./252/.snap/snap-subdir-test 2024-04-20T21:22:43.401 INFO:tasks.workunit.client.0.smithi102.stdout:./132/.snap/snap-subdir-test 2024-04-20T21:22:43.409 INFO:tasks.workunit.client.0.smithi102.stdout:./236/.snap/snap-subdir-test 2024-04-20T21:22:43.416 INFO:tasks.workunit.client.0.smithi102.stdout:./211/.snap/snap-subdir-test 2024-04-20T21:22:43.425 INFO:tasks.workunit.client.0.smithi102.stdout:./199/.snap/snap-subdir-test 2024-04-20T21:22:43.431 INFO:tasks.workunit.client.0.smithi102.stdout:./205/.snap/snap-subdir-test 2024-04-20T21:22:43.439 INFO:tasks.workunit.client.0.smithi102.stdout:./163/.snap/snap-subdir-test 2024-04-20T21:22:43.446 INFO:tasks.workunit.client.0.smithi102.stdout:./261/.snap/snap-subdir-test 2024-04-20T21:22:43.454 INFO:tasks.workunit.client.0.smithi102.stdout:./201/.snap/snap-subdir-test 2024-04-20T21:22:43.462 INFO:tasks.workunit.client.0.smithi102.stdout:./282/.snap/snap-subdir-test 2024-04-20T21:22:43.470 INFO:tasks.workunit.client.0.smithi102.stdout:./296/.snap/snap-subdir-test 2024-04-20T21:22:43.479 INFO:tasks.workunit.client.0.smithi102.stdout:./217/.snap/snap-subdir-test 2024-04-20T21:22:43.489 INFO:tasks.workunit.client.0.smithi102.stdout:./153/.snap/snap-subdir-test 2024-04-20T21:22:43.498 INFO:tasks.workunit.client.0.smithi102.stdout:./206/.snap/snap-subdir-test 2024-04-20T21:22:43.508 INFO:tasks.workunit.client.0.smithi102.stdout:./122/.snap/snap-subdir-test 2024-04-20T21:22:43.516 INFO:tasks.workunit.client.0.smithi102.stdout:./162/.snap/snap-subdir-test 2024-04-20T21:22:43.524 INFO:tasks.workunit.client.0.smithi102.stdout:./197/.snap/snap-subdir-test 2024-04-20T21:22:43.535 INFO:tasks.workunit.client.0.smithi102.stdout:./216/.snap/snap-subdir-test 2024-04-20T21:22:44.090 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:22:44.090 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:22:44.090 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:22:44.099 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:22:44.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:44.250 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:22:44.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.285+0000 7f9ccb000640 1 Processor -- start 2024-04-20T21:22:44.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.285+0000 7f9ccb000640 1 -- start start 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.286+0000 7f9ccb000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.286+0000 7f9ccb000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9cc40c9eb0 0x7f9cc4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.286+0000 7f9ccb000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc41536d0 con 0x7f9cc4073a10 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.286+0000 7f9ccb000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc4159500 con 0x7f9cc40c9eb0 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.286+0000 7f9ccb000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc40c76c0 con 0x7f9cc414fa50 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.286+0000 7f9cc3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9cc40c9eb0 0x7f9cc4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.286+0000 7f9cc9000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9cc4073a10 0x7f9cc40c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36564/0 (socket says 172.21.15.102:36564) 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.287+0000 7f9cc9000640 1 -- 172.21.15.102:0/2938455447 learned_addr learned my addr 172.21.15.102:0/2938455447 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:22:44.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.287+0000 7f9cc3e00640 1 --2- 172.21.15.102:0/2938455447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.287+0000 7f9cc3400640 1 -- 172.21.15.102:0/2938455447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 msgr2=0x7f9cc40c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.287+0000 7f9cc3400640 1 --2- 172.21.15.102:0/2938455447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc40c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.287+0000 7f9cc3400640 1 -- 172.21.15.102:0/2938455447 >> v1:172.21.15.102:6789/0 conn(0x7f9cc4073a10 legacy=0x7f9cc40c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.287+0000 7f9cc3400640 1 -- 172.21.15.102:0/2938455447 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc40c7960 con 0x7f9cc40c9eb0 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.287+0000 7f9cc3e00640 1 --2- 172.21.15.102:0/2938455447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc40c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.288+0000 7f9cc3400640 1 --2- 172.21.15.102:0/2938455447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc40c9eb0 0x7f9cc4158fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40c7b30 tx=0x7f9cac02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ed154c326f0941 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.288+0000 7f9cc2a00640 1 -- 172.21.15.102:0/2938455447 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac034070 con 0x7f9cc40c9eb0 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.288+0000 7f9cc2a00640 1 -- 172.21.15.102:0/2938455447 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cac030800 con 0x7f9cc40c9eb0 2024-04-20T21:22:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.288+0000 7f9cc2a00640 1 -- 172.21.15.102:0/2938455447 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac030bd0 con 0x7f9cc40c9eb0 2024-04-20T21:22:44.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.289+0000 7f9ccb000640 1 -- 172.21.15.102:0/2938455447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc40c9eb0 msgr2=0x7f9cc4158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.289+0000 7f9ccb000640 1 --2- 172.21.15.102:0/2938455447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc40c9eb0 0x7f9cc4158fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40c7b30 tx=0x7f9cac02b940 comp rx=0 tx=0).stop 2024-04-20T21:22:44.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.289+0000 7f9ccb000640 1 -- 172.21.15.102:0/2938455447 shutdown_connections 2024-04-20T21:22:44.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.289+0000 7f9ccb000640 1 --2- 172.21.15.102:0/2938455447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc40c9eb0 0x7f9cc4158fc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.289+0000 7f9ccb000640 1 --2- 172.21.15.102:0/2938455447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc40c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.289+0000 7f9ccb000640 1 -- 172.21.15.102:0/2938455447 >> 172.21.15.102:0/2938455447 conn(0x7f9cc40c9390 msgr2=0x7f9cc40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:44.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.289+0000 7f9ccb000640 1 -- 172.21.15.102:0/2938455447 shutdown_connections 2024-04-20T21:22:44.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.289+0000 7f9ccb000640 1 -- 172.21.15.102:0/2938455447 wait complete. 2024-04-20T21:22:44.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.291+0000 7f9ccb000640 1 Processor -- start 2024-04-20T21:22:44.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.291+0000 7f9ccb000640 1 -- start start 2024-04-20T21:22:44.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9ccb000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc4073a10 0x7f9cc415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9ccb000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc40c9eb0 0x7f9cc41f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9ccb000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc41f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9ccb000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc40c7c00 con 0x7f9cc4073a10 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9ccb000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc40c3d90 con 0x7f9cc40c9eb0 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9ccb000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc40c4030 con 0x7f9cc414fa50 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc4073a10 0x7f9cc415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc4073a10 0x7f9cc415ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54584/0 (socket says 172.21.15.102:54584) 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3e00640 1 -- 172.21.15.102:0/1495722863 learned_addr learned my addr 172.21.15.102:0/1495722863 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3400640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc40c9eb0 0x7f9cc41f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc9000640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc41f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3e00640 1 -- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 msgr2=0x7f9cc41f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3e00640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc41f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3e00640 1 -- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc40c9eb0 msgr2=0x7f9cc41f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3e00640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc40c9eb0 0x7f9cc41f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.292+0000 7f9cc3e00640 1 -- 172.21.15.102:0/1495722863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc4158f00 con 0x7f9cc4073a10 2024-04-20T21:22:44.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.293+0000 7f9cc3e00640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc4073a10 0x7f9cc415ce30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009f60 tx=0x7f9cb4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.293+0000 7f9cbbe00640 1 -- 172.21.15.102:0/1495722863 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4018020 con 0x7f9cc4073a10 2024-04-20T21:22:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.293+0000 7f9cbbe00640 1 -- 172.21.15.102:0/1495722863 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb4014680 con 0x7f9cc4073a10 2024-04-20T21:22:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.293+0000 7f9cbbe00640 1 -- 172.21.15.102:0/1495722863 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4014a50 con 0x7f9cc4073a10 2024-04-20T21:22:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.293+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc41591a0 con 0x7f9cc4073a10 2024-04-20T21:22:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.293+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc41f4890 con 0x7f9cc4073a10 2024-04-20T21:22:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.294+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc406eef0 con 0x7f9cc4073a10 2024-04-20T21:22:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.295+0000 7f9cbbe00640 1 -- 172.21.15.102:0/1495722863 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cb4005d10 con 0x7f9cc4073a10 2024-04-20T21:22:44.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.296+0000 7f9cbbe00640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ca006c8f0 0x7f9ca006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.296+0000 7f9cbbe00640 1 -- 172.21.15.102:0/1495722863 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f9cb408ceb0 con 0x7f9cc4073a10 2024-04-20T21:22:44.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.296+0000 7f9cc3400640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ca006c8f0 0x7f9ca006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.297+0000 7f9cc3400640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ca006c8f0 0x7f9ca006edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9cac036000 tx=0x7f9cac03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:44.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.301+0000 7f9cbbe00640 1 -- 172.21.15.102:0/1495722863 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9cc406eef0 con 0x7f9cc4073a10 2024-04-20T21:22:44.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.546+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cc40c35b0 con 0x7f9cc4073a10 2024-04-20T21:22:44.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.547+0000 7f9cbbe00640 1 -- 172.21.15.102:0/1495722863 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+6031 (secure 0 0 0) 0x7f9cc40c35b0 con 0x7f9cc4073a10 2024-04-20T21:22:44.547 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:44.547 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:22:44.547 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 25 2024-04-20T21:22:44.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ca006c8f0 msgr2=0x7f9ca006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ca006c8f0 0x7f9ca006edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9cac036000 tx=0x7f9cac03b040 comp rx=0 tx=0).stop 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc4073a10 msgr2=0x7f9cc415ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc4073a10 0x7f9cc415ce30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009f60 tx=0x7f9cb4009f90 comp rx=0 tx=0).stop 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 shutdown_connections 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cc414fa50 0x7f9cc41f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ca006c8f0 0x7f9ca006edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc40c9eb0 0x7f9cc41f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 --2- 172.21.15.102:0/1495722863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cc4073a10 0x7f9cc415ce30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 >> 172.21.15.102:0/1495722863 conn(0x7f9cc40c9390 msgr2=0x7f9cc40c8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 shutdown_connections 2024-04-20T21:22:44.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:44.550+0000 7f9ccb000640 1 -- 172.21.15.102:0/1495722863 wait complete. 2024-04-20T21:22:44.568 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-20T21:22:45.954 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-04-20T21:22:45.955+0000 7f8bda600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:22:46.569 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.768+0000 7f16f7400640 1 Processor -- start 2024-04-20T21:22:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.768+0000 7f16f7400640 1 -- start start 2024-04-20T21:22:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.769+0000 7f16f7400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f00c7290 0x7f16f00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.769+0000 7f16f7400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f16f0071a70 0x7f16f0158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.769+0000 7f16f7400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f0158780 con 0x7f16f0151c90 2024-04-20T21:22:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.769+0000 7f16f7400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f16f00c9200 con 0x7f16f0071a70 2024-04-20T21:22:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.769+0000 7f16f7400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f16f0070df0 con 0x7f16f00c7290 2024-04-20T21:22:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16f4a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f00c7290 0x7f16f00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16efe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f16f0071a70 0x7f16f0158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16f4a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f00c7290 0x7f16f00c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:37712/0 (socket says 172.21.15.102:37712) 2024-04-20T21:22:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16f4a00640 1 -- 172.21.15.102:0/2533724076 learned_addr learned my addr 172.21.15.102:0/2533724076 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16f5400640 1 --1- 172.21.15.102:0/2533724076 >> v1:172.21.15.102:6789/0 conn(0x7f16f0151c90 0x7f16f015ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36574/0 (socket says 172.21.15.102:36574) 2024-04-20T21:22:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16f4a00640 1 -- 172.21.15.102:0/2533724076 >> v2:172.21.15.102:3300/0 conn(0x7f16f0071a70 msgr2=0x7f16f0158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16f4a00640 1 --2- 172.21.15.102:0/2533724076 >> v2:172.21.15.102:3300/0 conn(0x7f16f0071a70 0x7f16f0158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16f4a00640 1 -- 172.21.15.102:0/2533724076 >> v1:172.21.15.102:6789/0 conn(0x7f16f0151c90 legacy=0x7f16f015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.770+0000 7f16f4a00640 1 -- 172.21.15.102:0/2533724076 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f0071040 con 0x7f16f00c7290 2024-04-20T21:22:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.771+0000 7f16f4a00640 1 --2- 172.21.15.102:0/2533724076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f00c7290 0x7f16f00c8cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16e000bb10 tx=0x7f16e002e1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87675cc10859e939 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16ef400640 1 -- 172.21.15.102:0/2533724076 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e003b070 con 0x7f16f00c7290 2024-04-20T21:22:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16ef400640 1 -- 172.21.15.102:0/2533724076 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e0036440 con 0x7f16f00c7290 2024-04-20T21:22:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16ef400640 1 -- 172.21.15.102:0/2533724076 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0036810 con 0x7f16f00c7290 2024-04-20T21:22:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16f7400640 1 -- 172.21.15.102:0/2533724076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f00c7290 msgr2=0x7f16f00c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16f7400640 1 --2- 172.21.15.102:0/2533724076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f00c7290 0x7f16f00c8cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16e000bb10 tx=0x7f16e002e1e0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16f7400640 1 -- 172.21.15.102:0/2533724076 shutdown_connections 2024-04-20T21:22:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16f7400640 1 --2- 172.21.15.102:0/2533724076 >> v2:172.21.15.102:3300/0 conn(0x7f16f0071a70 0x7f16f0158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16f7400640 1 --2- 172.21.15.102:0/2533724076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f00c7290 0x7f16f00c8cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16f7400640 1 -- 172.21.15.102:0/2533724076 >> 172.21.15.102:0/2533724076 conn(0x7f16f00c9390 msgr2=0x7f16f00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:46.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16f7400640 1 -- 172.21.15.102:0/2533724076 shutdown_connections 2024-04-20T21:22:46.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.772+0000 7f16f7400640 1 -- 172.21.15.102:0/2533724076 wait complete. 2024-04-20T21:22:46.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.774+0000 7f16f7400640 1 Processor -- start 2024-04-20T21:22:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.774+0000 7f16f7400640 1 -- start start 2024-04-20T21:22:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.775+0000 7f16f7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f0071a70 0x7f16f00728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.775+0000 7f16f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f00c7290 0x7f16f0150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.775+0000 7f16f7400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f0151c90 0x7f16f00caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.775+0000 7f16f7400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0071600 con 0x7f16f0071a70 2024-04-20T21:22:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.775+0000 7f16f7400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0157db0 con 0x7f16f00c7290 2024-04-20T21:22:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.775+0000 7f16f7400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f16f0157a40 con 0x7f16f0151c90 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f0071a70 0x7f16f00728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f0071a70 0x7f16f00728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54608/0 (socket says 172.21.15.102:54608) 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 -- 172.21.15.102:0/1095336074 learned_addr learned my addr 172.21.15.102:0/1095336074 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f5400640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f0151c90 0x7f16f00caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16efe00640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f00c7290 0x7f16f0150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 -- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f0151c90 msgr2=0x7f16f00caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f0151c90 0x7f16f00caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 -- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f00c7290 msgr2=0x7f16f0150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f00c7290 0x7f16f0150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 -- 172.21.15.102:0/1095336074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f01e50b0 con 0x7f16f0071a70 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.776+0000 7f16f4a00640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f0071a70 0x7f16f00728c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000c00 tx=0x7f16e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.777+0000 7f16ecc00640 1 -- 172.21.15.102:0/1095336074 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0046070 con 0x7f16f0071a70 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.777+0000 7f16ecc00640 1 -- 172.21.15.102:0/1095336074 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e0034850 con 0x7f16f0071a70 2024-04-20T21:22:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.777+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f01e5390 con 0x7f16f0071a70 2024-04-20T21:22:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.777+0000 7f16ecc00640 1 -- 172.21.15.102:0/1095336074 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e003b070 con 0x7f16f0071a70 2024-04-20T21:22:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.777+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f01e5950 con 0x7f16f0071a70 2024-04-20T21:22:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.778+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f0069340 con 0x7f16f0071a70 2024-04-20T21:22:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.779+0000 7f16ecc00640 1 -- 172.21.15.102:0/1095336074 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16e0049070 con 0x7f16f0071a70 2024-04-20T21:22:46.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.780+0000 7f16ecc00640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f16d006c920 0x7f16d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.780+0000 7f16ecc00640 1 -- 172.21.15.102:0/1095336074 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6993+0+0 (secure 0 0 0) 0x7f16e00adfc0 con 0x7f16f0071a70 2024-04-20T21:22:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.781+0000 7f16efe00640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f16d006c920 0x7f16d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.781+0000 7f16efe00640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f16d006c920 0x7f16d006ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f16f01f5b70 tx=0x7f16dc00b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:46.785+0000 7f16ecc00640 1 -- 172.21.15.102:0/1095336074 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f16f0069340 con 0x7f16f0071a70 2024-04-20T21:22:47.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.026+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16f00c3c90 con 0x7f16f0071a70 2024-04-20T21:22:47.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.028+0000 7f16ecc00640 1 -- 172.21.15.102:0/1095336074 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6859 (secure 0 0 0) 0x7f16f00c3c90 con 0x7f16f0071a70 2024-04-20T21:22:47.026 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:22:47.027 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:22:47.027 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-04-20T21:22:47.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.030+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f16d006c920 msgr2=0x7f16d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:47.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.030+0000 7f16f7400640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f16d006c920 0x7f16d006ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f16f01f5b70 tx=0x7f16dc00b040 comp rx=0 tx=0).stop 2024-04-20T21:22:47.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f0071a70 msgr2=0x7f16f00728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:47.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f0071a70 0x7f16f00728c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000c00 tx=0x7f16e0004730 comp rx=0 tx=0).stop 2024-04-20T21:22:47.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 shutdown_connections 2024-04-20T21:22:47.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16f0151c90 0x7f16f00caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f16d006c920 0x7f16d006ede0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f00c7290 0x7f16f0150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 --2- 172.21.15.102:0/1095336074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f0071a70 0x7f16f00728c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 >> 172.21.15.102:0/1095336074 conn(0x7f16f00c9390 msgr2=0x7f16f014b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:47.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 shutdown_connections 2024-04-20T21:22:47.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:22:47.031+0000 7f16f7400640 1 -- 172.21.15.102:0/1095336074 wait complete. 2024-04-20T21:22:47.048 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:22:47.048 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 28 secs before thrashing 2024-04-20T21:22:48.101 INFO:tasks.workunit.client.0.smithi102.stdout:./254/.snap/snap-subdir-test 2024-04-20T21:22:48.374 INFO:tasks.workunit.client.0.smithi102.stdout:./113/.snap/snap-subdir-test 2024-04-20T21:22:48.386 INFO:tasks.workunit.client.0.smithi102.stdout:./152/.snap/snap-subdir-test 2024-04-20T21:22:48.397 INFO:tasks.workunit.client.0.smithi102.stdout:./121/.snap/snap-subdir-test 2024-04-20T21:22:48.408 INFO:tasks.workunit.client.0.smithi102.stdout:./118/.snap/snap-subdir-test 2024-04-20T21:22:48.418 INFO:tasks.workunit.client.0.smithi102.stdout:./196/.snap/snap-subdir-test 2024-04-20T21:22:48.431 INFO:tasks.workunit.client.0.smithi102.stdout:./124/.snap/snap-subdir-test 2024-04-20T21:22:48.443 INFO:tasks.workunit.client.0.smithi102.stdout:./239/.snap/snap-subdir-test 2024-04-20T21:22:48.455 INFO:tasks.workunit.client.0.smithi102.stdout:./188/.snap/snap-subdir-test 2024-04-20T21:22:48.795 INFO:tasks.workunit.client.0.smithi102.stdout:./287/.snap/snap-subdir-test 2024-04-20T21:22:49.127 INFO:tasks.workunit.client.0.smithi102.stdout:./275/.snap/snap-subdir-test 2024-04-20T21:22:49.140 INFO:tasks.workunit.client.0.smithi102.stdout:./259/.snap/snap-subdir-test 2024-04-20T21:22:49.152 INFO:tasks.workunit.client.0.smithi102.stdout:./224/.snap/snap-subdir-test 2024-04-20T21:22:49.165 INFO:tasks.workunit.client.0.smithi102.stdout:./264/.snap/snap-subdir-test 2024-04-20T21:22:49.178 INFO:tasks.workunit.client.0.smithi102.stdout:./145/.snap/snap-subdir-test 2024-04-20T21:22:49.189 INFO:tasks.workunit.client.0.smithi102.stdout:./279/.snap/snap-subdir-test 2024-04-20T21:22:49.201 INFO:tasks.workunit.client.0.smithi102.stdout:./263/.snap/snap-subdir-test 2024-04-20T21:22:49.213 INFO:tasks.workunit.client.0.smithi102.stdout:./130/.snap/snap-subdir-test 2024-04-20T21:22:49.224 INFO:tasks.workunit.client.0.smithi102.stdout:./193/.snap/snap-subdir-test 2024-04-20T21:22:49.234 INFO:tasks.workunit.client.0.smithi102.stdout:./223/.snap/snap-subdir-test 2024-04-20T21:22:49.245 INFO:tasks.workunit.client.0.smithi102.stdout:./207/.snap/snap-subdir-test 2024-04-20T21:22:49.256 INFO:tasks.workunit.client.0.smithi102.stdout:./276/.snap/snap-subdir-test 2024-04-20T21:22:49.268 INFO:tasks.workunit.client.0.smithi102.stdout:./271/.snap/snap-subdir-test 2024-04-20T21:22:49.279 INFO:tasks.workunit.client.0.smithi102.stdout:./251/.snap/snap-subdir-test 2024-04-20T21:22:49.289 INFO:tasks.workunit.client.0.smithi102.stdout:./175/.snap/snap-subdir-test 2024-04-20T21:22:49.301 INFO:tasks.workunit.client.0.smithi102.stdout:./222/.snap/snap-subdir-test 2024-04-20T21:22:49.313 INFO:tasks.workunit.client.0.smithi102.stdout:./220/.snap/snap-subdir-test 2024-04-20T21:22:49.325 INFO:tasks.workunit.client.0.smithi102.stdout:./203/.snap/snap-subdir-test 2024-04-20T21:22:49.337 INFO:tasks.workunit.client.0.smithi102.stdout:./181/.snap/snap-subdir-test 2024-04-20T21:22:49.348 INFO:tasks.workunit.client.0.smithi102.stdout:./286/.snap/snap-subdir-test 2024-04-20T21:22:49.361 INFO:tasks.workunit.client.0.smithi102.stdout:./127/.snap/snap-subdir-test 2024-04-20T21:22:49.375 INFO:tasks.workunit.client.0.smithi102.stdout:./108/.snap/snap-subdir-test 2024-04-20T21:22:49.386 INFO:tasks.workunit.client.0.smithi102.stdout:./183/.snap/snap-subdir-test 2024-04-20T21:22:49.395 INFO:tasks.workunit.client.0.smithi102.stdout:./294/.snap/snap-subdir-test 2024-04-20T21:22:49.404 INFO:tasks.workunit.client.0.smithi102.stdout:./235/.snap/snap-subdir-test 2024-04-20T21:22:49.413 INFO:tasks.workunit.client.0.smithi102.stdout:./177/.snap/snap-subdir-test 2024-04-20T21:22:49.425 INFO:tasks.workunit.client.0.smithi102.stdout:./169/.snap/snap-subdir-test 2024-04-20T21:22:49.437 INFO:tasks.workunit.client.0.smithi102.stdout:./212/.snap/snap-subdir-test 2024-04-20T21:22:49.449 INFO:tasks.workunit.client.0.smithi102.stdout:./154/.snap/snap-subdir-test 2024-04-20T21:22:49.460 INFO:tasks.workunit.client.0.smithi102.stdout:./171/.snap/snap-subdir-test 2024-04-20T21:22:50.261 INFO:tasks.workunit.client.0.smithi102.stdout:./139/.snap/snap-subdir-test 2024-04-20T21:22:50.273 INFO:tasks.workunit.client.0.smithi102.stdout:./102/.snap/snap-subdir-test 2024-04-20T21:22:50.283 INFO:tasks.workunit.client.0.smithi102.stdout:./129/.snap/snap-subdir-test 2024-04-20T21:22:52.570 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:52.578 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:58.928 INFO:tasks.workunit.client.0.smithi102.stdout:Delete all the files and directories ... 2024-04-20T21:23:01.073 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/19' 2024-04-20T21:23:01.073 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/15' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/31' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/36' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/20' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/12' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/39' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/10' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/34' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/38' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/11' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/16' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/32' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/30' 2024-04-20T21:23:01.074 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/14' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/18' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/13' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/33' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/17' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/35' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './100/37' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './100' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/19' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/15' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/31' 2024-04-20T21:23:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/36' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/20' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/12' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/39' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/10' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/34' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/38' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/11' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/16' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/32' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/30' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/14' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/18' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/13' 2024-04-20T21:23:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/33' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/17' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/35' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './101/37' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './101' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/19' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/15' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/31' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/36' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/20' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/12' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/39' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/10' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/34' 2024-04-20T21:23:01.077 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/38' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/11' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/16' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/32' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/30' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/14' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/18' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/13' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/33' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/17' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/35' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './102/37' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './102' 2024-04-20T21:23:01.078 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/19' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/15' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/31' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/36' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/20' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/12' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/39' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/10' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/34' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/38' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/11' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/16' 2024-04-20T21:23:01.079 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/32' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/30' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/14' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/18' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/13' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/33' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/17' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/35' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './103/37' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './103' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/19' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/15' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/31' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/36' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/20' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/12' 2024-04-20T21:23:01.080 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/39' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/10' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/34' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/38' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/11' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/16' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/32' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/30' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/14' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/18' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/13' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/33' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/17' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/35' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed './104/37' 2024-04-20T21:23:01.081 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './104' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/19' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/15' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/31' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/36' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/20' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/12' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/39' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/10' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/34' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/38' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/11' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/16' 2024-04-20T21:23:01.082 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/32' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/30' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/14' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/18' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/13' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/33' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/17' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/35' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './105/37' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './105' 2024-04-20T21:23:01.083 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/19' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/15' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/31' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/36' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/20' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/12' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/39' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/10' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/34' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/38' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/11' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/16' 2024-04-20T21:23:01.084 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/32' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/30' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/14' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/18' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/13' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/33' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/17' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/35' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './106/37' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './106' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/19' 2024-04-20T21:23:01.085 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/15' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/31' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/36' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/20' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/12' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/39' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/10' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/34' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/38' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/11' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/16' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/32' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/30' 2024-04-20T21:23:01.086 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/14' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/18' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/13' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/33' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/17' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/35' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './107/37' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './107' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/19' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/15' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/31' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/36' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/20' 2024-04-20T21:23:01.087 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/12' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/39' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/10' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/34' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/38' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/11' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/16' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/32' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/30' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/14' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/18' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/13' 2024-04-20T21:23:01.088 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/33' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/17' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/35' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './108/37' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './108' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/19' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/15' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/31' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/36' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/20' 2024-04-20T21:23:01.089 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/12' 2024-04-20T21:23:01.090 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/39' 2024-04-20T21:23:01.090 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/10' 2024-04-20T21:23:01.090 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/34' 2024-04-20T21:23:01.090 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/38' 2024-04-20T21:23:01.090 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/11' 2024-04-20T21:23:01.090 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/16' 2024-04-20T21:23:01.090 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/32' 2024-04-20T21:23:01.090 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/30' 2024-04-20T21:23:01.685 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/14' 2024-04-20T21:23:01.685 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/18' 2024-04-20T21:23:01.685 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/13' 2024-04-20T21:23:01.685 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/33' 2024-04-20T21:23:01.685 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/17' 2024-04-20T21:23:01.685 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/35' 2024-04-20T21:23:01.685 INFO:tasks.workunit.client.0.smithi102.stdout:removed './109/37' 2024-04-20T21:23:01.685 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './109' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/19' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/15' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/31' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/36' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/20' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/12' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/39' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/10' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/34' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/38' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/11' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/16' 2024-04-20T21:23:01.686 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/32' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/30' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/14' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/18' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/13' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/33' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/17' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/35' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './110/37' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './110' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/19' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/15' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/31' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/36' 2024-04-20T21:23:01.687 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/20' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/12' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/39' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/10' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/34' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/38' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/11' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/16' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/32' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/30' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/14' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/18' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/13' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/33' 2024-04-20T21:23:01.688 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/17' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/35' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './111/37' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './111' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/19' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/15' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/31' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/36' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/20' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/12' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/39' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/10' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/34' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/38' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/11' 2024-04-20T21:23:01.689 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/16' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/32' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/30' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/14' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/18' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/13' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/33' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/17' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/35' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './112/37' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './112' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/19' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/15' 2024-04-20T21:23:01.690 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/31' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/36' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/20' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/12' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/39' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/10' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/34' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/38' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/11' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/16' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/32' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/30' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/14' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/18' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/13' 2024-04-20T21:23:01.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/33' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/17' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/35' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './113/37' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './113' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/19' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/15' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/31' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/36' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/20' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/12' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/39' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/10' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/34' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/38' 2024-04-20T21:23:01.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/11' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/16' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/32' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/30' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/14' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/18' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/13' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/33' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/17' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/35' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './114/37' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './114' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/19' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/15' 2024-04-20T21:23:01.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/31' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/36' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/20' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/12' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/39' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/10' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/34' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/38' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/11' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/16' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/32' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/30' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/14' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/18' 2024-04-20T21:23:01.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/13' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/33' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/17' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/35' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './115/37' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './115' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/19' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/15' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/31' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/36' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/20' 2024-04-20T21:23:01.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/12' 2024-04-20T21:23:01.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/39' 2024-04-20T21:23:01.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/10' 2024-04-20T21:23:01.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/34' 2024-04-20T21:23:01.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/38' 2024-04-20T21:23:01.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/11' 2024-04-20T21:23:01.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/16' 2024-04-20T21:23:01.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/32' 2024-04-20T21:23:01.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/30' 2024-04-20T21:23:01.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/14' 2024-04-20T21:23:01.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/18' 2024-04-20T21:23:01.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/13' 2024-04-20T21:23:01.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/33' 2024-04-20T21:23:01.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/17' 2024-04-20T21:23:01.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/35' 2024-04-20T21:23:01.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './116/37' 2024-04-20T21:23:01.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './116' 2024-04-20T21:23:01.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/19' 2024-04-20T21:23:01.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/15' 2024-04-20T21:23:01.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/31' 2024-04-20T21:23:01.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/36' 2024-04-20T21:23:01.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/20' 2024-04-20T21:23:01.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/12' 2024-04-20T21:23:01.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/39' 2024-04-20T21:23:01.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/10' 2024-04-20T21:23:01.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/34' 2024-04-20T21:23:01.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/38' 2024-04-20T21:23:01.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/11' 2024-04-20T21:23:01.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/16' 2024-04-20T21:23:01.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/32' 2024-04-20T21:23:01.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/30' 2024-04-20T21:23:01.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/14' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/18' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/13' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/33' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/17' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/35' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './117/37' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './117' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/19' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/15' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/31' 2024-04-20T21:23:01.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/36' 2024-04-20T21:23:01.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/20' 2024-04-20T21:23:01.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/12' 2024-04-20T21:23:01.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/39' 2024-04-20T21:23:01.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/10' 2024-04-20T21:23:01.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/34' 2024-04-20T21:23:01.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/38' 2024-04-20T21:23:01.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/11' 2024-04-20T21:23:01.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/16' 2024-04-20T21:23:01.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/32' 2024-04-20T21:23:01.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/30' 2024-04-20T21:23:01.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/14' 2024-04-20T21:23:01.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/18' 2024-04-20T21:23:01.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/13' 2024-04-20T21:23:01.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/33' 2024-04-20T21:23:01.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/17' 2024-04-20T21:23:01.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/35' 2024-04-20T21:23:01.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './118/37' 2024-04-20T21:23:01.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './118' 2024-04-20T21:23:01.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/19' 2024-04-20T21:23:01.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/15' 2024-04-20T21:23:01.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/31' 2024-04-20T21:23:01.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/36' 2024-04-20T21:23:01.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/20' 2024-04-20T21:23:01.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/12' 2024-04-20T21:23:03.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/39' 2024-04-20T21:23:03.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/10' 2024-04-20T21:23:03.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/34' 2024-04-20T21:23:03.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/38' 2024-04-20T21:23:03.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/11' 2024-04-20T21:23:03.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/16' 2024-04-20T21:23:03.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/32' 2024-04-20T21:23:03.691 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/30' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/14' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/18' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/13' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/33' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/17' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/35' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './119/37' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './119' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/19' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/15' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/31' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/36' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/20' 2024-04-20T21:23:03.692 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/12' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/39' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/10' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/34' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/38' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/11' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/16' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/32' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/30' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/14' 2024-04-20T21:23:03.693 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/18' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/13' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/33' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/17' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/35' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './120/37' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './120' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/19' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/15' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/31' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/36' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/20' 2024-04-20T21:23:03.694 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/12' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/39' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/10' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/34' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/38' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/11' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/16' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/32' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/30' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/14' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/18' 2024-04-20T21:23:03.695 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/13' 2024-04-20T21:23:03.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/33' 2024-04-20T21:23:03.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/17' 2024-04-20T21:23:03.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/35' 2024-04-20T21:23:03.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './121/37' 2024-04-20T21:23:03.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './121' 2024-04-20T21:23:03.696 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/19' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/15' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/31' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/36' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/20' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/12' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/39' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/10' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/34' 2024-04-20T21:23:03.697 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/38' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/11' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/16' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/32' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/30' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/14' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/18' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/13' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/33' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/17' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/35' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './122/37' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './122' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/19' 2024-04-20T21:23:03.698 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/15' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/31' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/36' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/20' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/12' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/39' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/10' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/34' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/38' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/11' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/16' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/32' 2024-04-20T21:23:03.699 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/30' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/14' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/18' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/13' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/33' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/17' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/35' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './123/37' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './123' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/19' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/15' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/31' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/36' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/20' 2024-04-20T21:23:03.700 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/12' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/39' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/10' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/34' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/38' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/11' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/16' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/32' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/30' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/14' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/18' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/13' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/33' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/17' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/35' 2024-04-20T21:23:03.701 INFO:tasks.workunit.client.0.smithi102.stdout:removed './124/37' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './124' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/19' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/15' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/31' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/36' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/20' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/12' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/39' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/10' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/34' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/38' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/11' 2024-04-20T21:23:03.702 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/16' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/32' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/30' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/14' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/18' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/13' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/33' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/17' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/35' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './125/37' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './125' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/19' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/15' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/31' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/36' 2024-04-20T21:23:03.703 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/20' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/12' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/39' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/10' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/34' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/38' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/11' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/16' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/32' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/30' 2024-04-20T21:23:03.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/14' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/18' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/13' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/33' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/17' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/35' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './126/37' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './126' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/19' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/15' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/31' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/36' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/20' 2024-04-20T21:23:03.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/12' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/39' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/10' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/34' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/38' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/11' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/16' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/32' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/30' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/14' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/18' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/13' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/33' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/17' 2024-04-20T21:23:03.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/35' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './127/37' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './127' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/19' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/15' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/31' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/36' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/20' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/12' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/39' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/10' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/34' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/38' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/11' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/16' 2024-04-20T21:23:03.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/32' 2024-04-20T21:23:03.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/30' 2024-04-20T21:23:03.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/14' 2024-04-20T21:23:03.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/18' 2024-04-20T21:23:03.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/13' 2024-04-20T21:23:03.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/33' 2024-04-20T21:23:03.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/17' 2024-04-20T21:23:03.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/35' 2024-04-20T21:23:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './128/37' 2024-04-20T21:23:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './128' 2024-04-20T21:23:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/19' 2024-04-20T21:23:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/15' 2024-04-20T21:23:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/31' 2024-04-20T21:23:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/36' 2024-04-20T21:23:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/20' 2024-04-20T21:23:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/12' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/39' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/10' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/34' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/38' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/11' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/16' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/32' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/30' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/14' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/18' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/13' 2024-04-20T21:23:08.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/33' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/17' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/35' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './129/37' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './129' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/19' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/15' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/31' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/36' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/20' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/12' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/39' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/10' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/34' 2024-04-20T21:23:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/38' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/11' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/16' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/32' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/30' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/14' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/18' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/13' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/33' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/17' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/35' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './130/37' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './130' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/19' 2024-04-20T21:23:08.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/15' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/31' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/36' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/20' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/12' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/39' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/10' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/34' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/38' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/11' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/16' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/32' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/30' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/14' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/18' 2024-04-20T21:23:08.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/13' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/33' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/17' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/35' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './131/37' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './131' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/19' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/15' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/31' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/36' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/20' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/12' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/39' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/10' 2024-04-20T21:23:08.012 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/34' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/38' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/11' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/16' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/32' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/30' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/14' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/18' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/13' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/33' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/17' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/35' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './132/37' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './132' 2024-04-20T21:23:08.013 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/19' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/15' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/31' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/36' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/20' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/12' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/39' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/10' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/34' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/38' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/11' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/16' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/32' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/30' 2024-04-20T21:23:08.014 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/14' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/18' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/13' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/33' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/17' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/35' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './133/37' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './133' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/19' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/15' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/31' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/36' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/20' 2024-04-20T21:23:08.015 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/12' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/39' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/10' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/34' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/38' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/11' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/16' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/32' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/30' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/14' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/18' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/13' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/33' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/17' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/35' 2024-04-20T21:23:08.016 INFO:tasks.workunit.client.0.smithi102.stdout:removed './134/37' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './134' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/19' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/15' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/31' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/36' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/20' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/12' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/39' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/10' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/34' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/38' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/11' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/16' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/32' 2024-04-20T21:23:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/30' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/14' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/18' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/13' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/33' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/17' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/35' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './135/37' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './135' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/19' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/15' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/31' 2024-04-20T21:23:08.018 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/36' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/20' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/12' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/39' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/10' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/34' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/38' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/11' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/16' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/32' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/30' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/14' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/18' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/13' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/33' 2024-04-20T21:23:08.019 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/17' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/35' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './136/37' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './136' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/19' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/15' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/31' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/36' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/20' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/12' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/39' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/10' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/34' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/38' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/11' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/16' 2024-04-20T21:23:08.020 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/32' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/30' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/14' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/18' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/13' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/33' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/17' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/35' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './137/37' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './137' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/19' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/15' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/31' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/36' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/20' 2024-04-20T21:23:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/12' 2024-04-20T21:23:08.022 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/39' 2024-04-20T21:23:08.022 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/10' 2024-04-20T21:23:08.022 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/34' 2024-04-20T21:23:08.022 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/38' 2024-04-20T21:23:08.022 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/11' 2024-04-20T21:23:08.022 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/16' 2024-04-20T21:23:12.340 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/32' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/30' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/14' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/18' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/13' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/33' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/17' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/35' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './138/37' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './138' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/19' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/15' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/31' 2024-04-20T21:23:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/36' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/20' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/12' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/39' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/10' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/34' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/38' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/11' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/16' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/32' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/30' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/14' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/18' 2024-04-20T21:23:12.342 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/13' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/33' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/17' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/35' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './139/37' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './139' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/19' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/15' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/31' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/36' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/20' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/12' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/39' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/10' 2024-04-20T21:23:12.343 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/34' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/38' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/11' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/16' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/32' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/30' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/14' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/18' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/13' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/33' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/17' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/35' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './140/37' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './140' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/19' 2024-04-20T21:23:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/15' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/31' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/36' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/20' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/12' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/39' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/10' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/34' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/38' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/11' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/16' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/32' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/30' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/14' 2024-04-20T21:23:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/18' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/13' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/33' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/17' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/35' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './141/37' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './141' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/19' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/15' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/31' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/36' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/20' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/12' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/39' 2024-04-20T21:23:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/10' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/34' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/38' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/11' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/16' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/32' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/30' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/14' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/18' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/13' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/33' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/17' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/35' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed './142/37' 2024-04-20T21:23:12.347 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './142' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/19' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/15' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/31' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/36' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/20' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/12' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/39' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/10' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/34' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/38' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/11' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/16' 2024-04-20T21:23:12.348 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/32' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/30' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/14' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/18' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/13' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/33' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/17' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/35' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './143/37' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './143' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/19' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/15' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/31' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/36' 2024-04-20T21:23:12.349 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/20' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/12' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/39' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/10' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/34' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/38' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/11' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/16' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/32' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/30' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/14' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/18' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/13' 2024-04-20T21:23:12.350 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/33' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/17' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/35' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './144/37' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './144' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/19' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/15' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/31' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/36' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/20' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/12' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/39' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/10' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/34' 2024-04-20T21:23:12.351 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/38' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/11' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/16' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/32' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/30' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/14' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/18' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/13' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/33' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/17' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/35' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './145/37' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './145' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/19' 2024-04-20T21:23:12.352 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/15' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/31' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/36' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/20' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/12' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/39' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/10' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/34' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/38' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/11' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/16' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/32' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/30' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/14' 2024-04-20T21:23:12.353 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/18' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/13' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/33' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/17' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/35' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './146/37' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './146' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/19' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/15' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/31' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/36' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/20' 2024-04-20T21:23:12.354 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/12' 2024-04-20T21:23:12.355 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/39' 2024-04-20T21:23:12.355 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/10' 2024-04-20T21:23:12.355 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/34' 2024-04-20T21:23:12.355 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/38' 2024-04-20T21:23:12.355 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/11' 2024-04-20T21:23:12.355 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/16' 2024-04-20T21:23:12.355 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/32' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/30' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/14' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/18' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/13' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/33' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/17' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/35' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './147/37' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './147' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/19' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/15' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/31' 2024-04-20T21:23:12.358 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/36' 2024-04-20T21:23:13.446 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/20' 2024-04-20T21:23:13.446 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/12' 2024-04-20T21:23:13.446 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/39' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/10' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/34' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/38' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/11' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/16' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/32' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/30' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/14' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/18' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/13' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/33' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/17' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/35' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed './148/37' 2024-04-20T21:23:13.447 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './148' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/19' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/15' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/31' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/36' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/20' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/12' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/39' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/10' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/34' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/38' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/11' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/16' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/32' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/30' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/14' 2024-04-20T21:23:13.448 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/18' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/13' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/33' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/17' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/35' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './149/37' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './149' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/19' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/15' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/31' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/36' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/20' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/12' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/39' 2024-04-20T21:23:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/10' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/34' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/38' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/11' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/16' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/32' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/30' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/14' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/18' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/13' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/33' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/17' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/35' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed './150/37' 2024-04-20T21:23:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './150' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/19' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/15' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/31' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/36' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/20' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/12' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/39' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/10' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/34' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/38' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/11' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/16' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/32' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/30' 2024-04-20T21:23:13.451 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/14' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/18' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/13' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/33' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/17' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/35' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './151/37' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './151' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/19' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/15' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/31' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/36' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/20' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/12' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/39' 2024-04-20T21:23:13.452 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/10' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/34' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/38' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/11' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/16' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/32' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/30' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/14' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/18' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/13' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/33' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/17' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/35' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './152/37' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './152' 2024-04-20T21:23:13.453 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/19' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/15' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/31' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/36' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/20' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/12' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/39' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/10' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/34' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/38' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/11' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/16' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/32' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/30' 2024-04-20T21:23:13.454 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/14' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/18' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/13' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/33' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/17' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/35' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './153/37' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './153' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/19' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/15' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/31' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/36' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/20' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/12' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/39' 2024-04-20T21:23:13.455 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/10' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/34' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/38' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/11' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/16' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/32' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/30' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/14' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/18' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/13' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/33' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/17' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/35' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './154/37' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './154' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/19' 2024-04-20T21:23:13.456 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/15' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/31' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/36' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/20' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/12' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/39' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/10' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/34' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/38' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/11' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/16' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/32' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/30' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/14' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/18' 2024-04-20T21:23:13.457 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/13' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/33' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/17' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/35' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './155/37' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './155' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/19' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/15' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/31' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/36' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/20' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/12' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/39' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/10' 2024-04-20T21:23:13.458 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/34' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/38' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/11' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/16' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/32' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/30' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/14' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/18' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/13' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/33' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/17' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/35' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './156/37' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './156' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/19' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/15' 2024-04-20T21:23:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/31' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/36' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/20' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/12' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/39' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/10' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/34' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/38' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/11' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/16' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/32' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/30' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/14' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/18' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/13' 2024-04-20T21:23:13.460 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/33' 2024-04-20T21:23:14.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/17' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/35' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './157/37' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './157' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/19' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/15' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/31' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/36' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/20' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/12' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/39' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/10' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/34' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/38' 2024-04-20T21:23:14.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/11' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/16' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/32' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/30' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/14' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/18' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/13' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/33' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/17' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/35' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './158/37' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './158' 2024-04-20T21:23:14.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/19' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/15' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/31' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/36' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/20' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/12' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/39' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/10' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/34' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/38' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/11' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/16' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/32' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/30' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/14' 2024-04-20T21:23:14.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/18' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/13' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/33' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/17' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/35' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './159/37' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './159' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/19' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/15' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/31' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/36' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/20' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/12' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/39' 2024-04-20T21:23:14.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/10' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/34' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/38' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/11' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/16' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/32' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/30' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/14' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/18' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/13' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/33' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/17' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/35' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './160/37' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './160' 2024-04-20T21:23:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/19' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/15' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/31' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/36' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/20' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/12' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/39' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/10' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/34' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/38' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/11' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/16' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/32' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/30' 2024-04-20T21:23:14.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/14' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/18' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/13' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/33' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/17' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/35' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './161/37' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './161' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/19' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/15' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/31' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/36' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/20' 2024-04-20T21:23:14.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/12' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/39' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/10' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/34' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/38' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/11' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/16' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/32' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/30' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/14' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/18' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/13' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/33' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/17' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/35' 2024-04-20T21:23:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './162/37' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './162' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/19' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/15' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/31' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/36' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/20' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/12' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/39' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/10' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/34' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/38' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/11' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/16' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/32' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/30' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/14' 2024-04-20T21:23:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/18' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/13' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/33' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/17' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/35' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './163/37' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './163' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/19' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/15' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/31' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/36' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/20' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/12' 2024-04-20T21:23:14.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/39' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/10' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/34' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/38' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/11' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/16' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/32' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/30' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/14' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/18' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/13' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/33' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/17' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/35' 2024-04-20T21:23:14.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './164/37' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './164' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/19' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/15' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/31' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/36' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/20' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/12' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/39' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/10' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/34' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/38' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/11' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/16' 2024-04-20T21:23:14.208 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/32' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/30' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/14' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/18' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/13' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/33' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/17' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/35' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './165/37' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './165' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/19' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/15' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/31' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/36' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/20' 2024-04-20T21:23:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/12' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/39' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/10' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/34' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/38' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/11' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/16' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/32' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/30' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/14' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/18' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/13' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/33' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/17' 2024-04-20T21:23:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/35' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './166/37' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './166' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/19' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/15' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/31' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/36' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/20' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/12' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/39' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/10' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/34' 2024-04-20T21:23:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/38' 2024-04-20T21:23:15.049 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:15.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.287+0000 7fab24600640 1 Processor -- start 2024-04-20T21:23:15.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.287+0000 7fab24600640 1 -- start start 2024-04-20T21:23:15.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.288+0000 7fab24600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.288+0000 7fab24600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fab1c0cd940 0x7fab1c074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.288+0000 7fab24600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab1c15f940 con 0x7fab1c0ce310 2024-04-20T21:23:15.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.288+0000 7fab24600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fab1c15eb60 con 0x7fab1c0cd940 2024-04-20T21:23:15.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.288+0000 7fab24600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fab1c15ee00 con 0x7fab1c0c7900 2024-04-20T21:23:15.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fab1c0cd940 0x7fab1c074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fab1c0cd940 0x7fab1c074ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:49232/0 (socket says 172.21.15.102:49232) 2024-04-20T21:23:15.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21200640 1 -- 172.21.15.102:0/2447478892 learned_addr learned my addr 172.21.15.102:0/2447478892 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:15.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab22600640 1 --1- 172.21.15.102:0/2447478892 >> v1:172.21.15.193:6789/0 conn(0x7fab1c0ce310 0x7fab1c072230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:46076/0 (socket says 172.21.15.102:46076) 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21c00640 1 -- 172.21.15.102:0/2447478892 >> v2:172.21.15.193:3300/0 conn(0x7fab1c0cd940 msgr2=0x7fab1c074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21c00640 1 --2- 172.21.15.102:0/2447478892 >> v2:172.21.15.193:3300/0 conn(0x7fab1c0cd940 0x7fab1c074ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21c00640 1 -- 172.21.15.102:0/2447478892 >> v1:172.21.15.193:6789/0 conn(0x7fab1c0ce310 legacy=0x7fab1c072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21c00640 1 -- 172.21.15.102:0/2447478892 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c15f0a0 con 0x7fab1c0c7900 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.289+0000 7fab21200640 1 --2- 172.21.15.102:0/2447478892 >> v2:172.21.15.193:3300/0 conn(0x7fab1c0cd940 0x7fab1c074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.290+0000 7fab21c00640 1 --2- 172.21.15.102:0/2447478892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c15bee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab100097a0 tx=0x7fab1002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb07130baecd7281 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab1be00640 1 -- 172.21.15.102:0/2447478892 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10035020 con 0x7fab1c0c7900 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab1be00640 1 -- 172.21.15.102:0/2447478892 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab10031810 con 0x7fab1c0c7900 2024-04-20T21:23:15.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab1be00640 1 -- 172.21.15.102:0/2447478892 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10031be0 con 0x7fab1c0c7900 2024-04-20T21:23:15.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab24600640 1 -- 172.21.15.102:0/2447478892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 msgr2=0x7fab1c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab24600640 1 --2- 172.21.15.102:0/2447478892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c15bee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab100097a0 tx=0x7fab1002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab24600640 1 -- 172.21.15.102:0/2447478892 shutdown_connections 2024-04-20T21:23:15.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab24600640 1 --2- 172.21.15.102:0/2447478892 >> v2:172.21.15.193:3300/0 conn(0x7fab1c0cd940 0x7fab1c074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab24600640 1 --2- 172.21.15.102:0/2447478892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c15bee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab24600640 1 -- 172.21.15.102:0/2447478892 >> 172.21.15.102:0/2447478892 conn(0x7fab1c06dad0 msgr2=0x7fab1c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:15.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.291+0000 7fab24600640 1 -- 172.21.15.102:0/2447478892 shutdown_connections 2024-04-20T21:23:15.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.292+0000 7fab24600640 1 -- 172.21.15.102:0/2447478892 wait complete. 2024-04-20T21:23:15.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.293+0000 7fab24600640 1 Processor -- start 2024-04-20T21:23:15.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.293+0000 7fab24600640 1 -- start start 2024-04-20T21:23:15.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.294+0000 7fab24600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.294+0000 7fab24600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c0cd940 0x7fab1c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab24600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c0ce310 0x7fab1c151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab24600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c0c7ce0 con 0x7fab1c0cd940 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab24600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c15f660 con 0x7fab1c0ce310 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab24600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fab1c0c7e60 con 0x7fab1c0c7900 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c0cd940 0x7fab1c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c0cd940 0x7fab1c1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48816/0 (socket says 172.21.15.102:48816) 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21200640 1 -- 172.21.15.102:0/3051838623 learned_addr learned my addr 172.21.15.102:0/3051838623 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab22600640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c0ce310 0x7fab1c151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21c00640 1 -- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c0ce310 msgr2=0x7fab1c151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21c00640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c0ce310 0x7fab1c151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21c00640 1 -- 172.21.15.102:0/3051838623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c0cd940 msgr2=0x7fab1c1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21c00640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c0cd940 0x7fab1c1522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.295+0000 7fab21c00640 1 -- 172.21.15.102:0/3051838623 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c1f0990 con 0x7fab1c0c7900 2024-04-20T21:23:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.296+0000 7fab21c00640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c155db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab1c15ef80 tx=0x7fab100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.296+0000 7fab19600640 1 -- 172.21.15.102:0/3051838623 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10035040 con 0x7fab1c0c7900 2024-04-20T21:23:15.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.296+0000 7fab19600640 1 -- 172.21.15.102:0/3051838623 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab1003ad00 con 0x7fab1c0c7900 2024-04-20T21:23:15.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.296+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c1f0c70 con 0x7fab1c0c7900 2024-04-20T21:23:15.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.296+0000 7fab19600640 1 -- 172.21.15.102:0/3051838623 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab100466a0 con 0x7fab1c0c7900 2024-04-20T21:23:15.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.296+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c1f1230 con 0x7fab1c0c7900 2024-04-20T21:23:15.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.297+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1c074340 con 0x7fab1c0c7900 2024-04-20T21:23:15.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.299+0000 7fab19600640 1 -- 172.21.15.102:0/3051838623 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab10038020 con 0x7fab1c0c7900 2024-04-20T21:23:15.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.300+0000 7fab19600640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faafc06c8f0 0x7faafc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.300+0000 7fab19600640 1 -- 172.21.15.102:0/3051838623 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(41..41 src has 1..41) ==== 8181+0+0 (secure 0 0 0) 0x7fab100ae500 con 0x7fab1c0c7900 2024-04-20T21:23:15.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.300+0000 7fab21200640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faafc06c8f0 0x7faafc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.301+0000 7fab21200640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faafc06c8f0 0x7faafc06edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab08009630 tx=0x7fab08009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.305+0000 7fab19600640 1 -- 172.21.15.102:0/3051838623 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fab1007b8e0 con 0x7fab1c0c7900 2024-04-20T21:23:15.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.669+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab1c0c8560 con 0x7fab1c0c7900 2024-04-20T21:23:15.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.670+0000 7fab19600640 1 -- 172.21.15.102:0/3051838623 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6859 (secure 0 0 0) 0x7fab1007fbe0 con 0x7fab1c0c7900 2024-04-20T21:23:15.669 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:23:15.670 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faafc06c8f0 msgr2=0x7faafc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faafc06c8f0 0x7faafc06edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab08009630 tx=0x7fab08009240 comp rx=0 tx=0).stop 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 msgr2=0x7fab1c155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c155db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab1c15ef80 tx=0x7fab100046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 shutdown_connections 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faafc06c8f0 0x7faafc06edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c0ce310 0x7fab1c151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c0cd940 0x7fab1c1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 --2- 172.21.15.102:0/3051838623 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab1c0c7900 0x7fab1c155db0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 >> 172.21.15.102:0/3051838623 conn(0x7fab1c06dad0 msgr2=0x7fab1c0ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:15.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 shutdown_connections 2024-04-20T21:23:15.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.672+0000 7fab24600640 1 -- 172.21.15.102:0/3051838623 wait complete. 2024-04-20T21:23:15.687 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-04-20T21:23:15.687 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:23:15.687 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:23:15.711 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:23:15.711 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:15.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.937+0000 7f7055000640 1 Processor -- start 2024-04-20T21:23:15.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.937+0000 7f7055000640 1 -- start start 2024-04-20T21:23:15.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.939+0000 7f7055000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f705014fa50 0x7f70500c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.939+0000 7f7055000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70501536d0 con 0x7f7050073a10 2024-04-20T21:23:15.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.939+0000 7f7055000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70500c76c0 con 0x7f70500c9eb0 2024-04-20T21:23:15.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.939+0000 7f7055000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f70500c7960 con 0x7f705014fa50 2024-04-20T21:23:15.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.939+0000 7f704e800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7050073a10 0x7f7050163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45332/0 (socket says 172.21.15.102:45332) 2024-04-20T21:23:15.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.939+0000 7f704e800640 1 -- 172.21.15.102:0/3322776812 learned_addr learned my addr 172.21.15.102:0/3322776812 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:15.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.939+0000 7f704de00640 1 --2- 172.21.15.102:0/3322776812 >> v2:172.21.15.193:3300/0 conn(0x7f705014fa50 0x7f70500c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.940+0000 7f704d400640 1 -- 172.21.15.102:0/3322776812 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556894257 0 0) 0x7f70500c76c0 con 0x7f70500c9eb0 2024-04-20T21:23:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.940+0000 7f704d400640 1 -- 172.21.15.102:0/3322776812 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7034003520 con 0x7f70500c9eb0 2024-04-20T21:23:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.940+0000 7f704d400640 1 -- 172.21.15.102:0/3322776812 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263615463 0 0) 0x7f7048001490 con 0x7f7050073a10 2024-04-20T21:23:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.940+0000 7f704de00640 1 -- 172.21.15.102:0/3322776812 >> v1:172.21.15.193:6789/0 conn(0x7f70500c9eb0 legacy=0x7f70500c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.940+0000 7f704de00640 1 -- 172.21.15.102:0/3322776812 >> v1:172.21.15.102:6789/0 conn(0x7f7050073a10 legacy=0x7f7050163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.940+0000 7f704de00640 1 -- 172.21.15.102:0/3322776812 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70500c3d90 con 0x7f705014fa50 2024-04-20T21:23:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.941+0000 7f704de00640 1 --2- 172.21.15.102:0/3322776812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705014fa50 0x7f70500c4830 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f70500c7890 tx=0x7f704002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b83364a58b69161 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.941+0000 7f704d400640 1 -- 172.21.15.102:0/3322776812 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040034070 con 0x7f705014fa50 2024-04-20T21:23:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.941+0000 7f704d400640 1 -- 172.21.15.102:0/3322776812 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7040030610 con 0x7f705014fa50 2024-04-20T21:23:15.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.941+0000 7f704d400640 1 -- 172.21.15.102:0/3322776812 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70400309e0 con 0x7f705014fa50 2024-04-20T21:23:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.942+0000 7f7055000640 1 -- 172.21.15.102:0/3322776812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705014fa50 msgr2=0x7f70500c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.942+0000 7f7055000640 1 --2- 172.21.15.102:0/3322776812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705014fa50 0x7f70500c4830 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f70500c7890 tx=0x7f704002b960 comp rx=0 tx=0).stop 2024-04-20T21:23:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.942+0000 7f7055000640 1 -- 172.21.15.102:0/3322776812 shutdown_connections 2024-04-20T21:23:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.942+0000 7f7055000640 1 --2- 172.21.15.102:0/3322776812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705014fa50 0x7f70500c4830 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.942+0000 7f7055000640 1 -- 172.21.15.102:0/3322776812 >> 172.21.15.102:0/3322776812 conn(0x7f70500c9390 msgr2=0x7f70500c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.942+0000 7f7055000640 1 -- 172.21.15.102:0/3322776812 shutdown_connections 2024-04-20T21:23:15.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.942+0000 7f7055000640 1 -- 172.21.15.102:0/3322776812 wait complete. 2024-04-20T21:23:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.944+0000 7f7055000640 1 Processor -- start 2024-04-20T21:23:15.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.944+0000 7f7055000640 1 -- start start 2024-04-20T21:23:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f7055000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7050073a10 0x7f705006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f7055000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70500c9eb0 0x7f70501581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f7055000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f705014fa50 0x7f70501f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f7055000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f70500c4030 con 0x7f705014fa50 2024-04-20T21:23:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f7055000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70500c7c00 con 0x7f7050073a10 2024-04-20T21:23:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f7055000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f70500c4310 con 0x7f70500c9eb0 2024-04-20T21:23:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f704e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f705014fa50 0x7f70501f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f704de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7050073a10 0x7f705006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f7047e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70500c9eb0 0x7f70501581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f704de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7050073a10 0x7f705006d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:49250/0 (socket says 172.21.15.102:49250) 2024-04-20T21:23:15.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.945+0000 7f704de00640 1 -- 172.21.15.102:0/696231581 learned_addr learned my addr 172.21.15.102:0/696231581 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:15.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f704de00640 1 -- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70500c9eb0 msgr2=0x7f70501581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f704de00640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70500c9eb0 0x7f70501581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f704de00640 1 -- 172.21.15.102:0/696231581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f705014fa50 msgr2=0x7f70501f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f704de00640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f705014fa50 0x7f70501f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f704de00640 1 -- 172.21.15.102:0/696231581 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705015b0d0 con 0x7f7050073a10 2024-04-20T21:23:15.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f7047e00640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70500c9eb0 0x7f70501581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f704de00640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7050073a10 0x7f705006d160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7040000c00 tx=0x7f70400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f7046000640 1 -- 172.21.15.102:0/696231581 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040049070 con 0x7f7050073a10 2024-04-20T21:23:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f7046000640 1 -- 172.21.15.102:0/696231581 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704003a870 con 0x7f7050073a10 2024-04-20T21:23:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7050157130 con 0x7f7050073a10 2024-04-20T21:23:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f7046000640 1 -- 172.21.15.102:0/696231581 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040034040 con 0x7f7050073a10 2024-04-20T21:23:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.946+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70501579b0 con 0x7f7050073a10 2024-04-20T21:23:15.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.948+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7050074340 con 0x7f7050073a10 2024-04-20T21:23:15.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.949+0000 7f7046000640 1 -- 172.21.15.102:0/696231581 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f704004c070 con 0x7f7050073a10 2024-04-20T21:23:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.949+0000 7f7046000640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.950+0000 7f7046000640 1 -- 172.21.15.102:0/696231581 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 7641+0+0 (secure 0 0 0) 0x7f70400ade60 con 0x7f7050073a10 2024-04-20T21:23:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.950+0000 7f7047e00640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.950+0000 7f7047e00640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f702c06c920 0x7f702c06ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7050157090 tx=0x7f7038009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:15.954+0000 7f7046000640 1 -- 172.21.15.102:0/696231581 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f705015b0d0 con 0x7f7050073a10 2024-04-20T21:23:16.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.193+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70500c7320 con 0x7f7050073a10 2024-04-20T21:23:16.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.194+0000 7f7046000640 1 -- 172.21.15.102:0/696231581 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6859 (secure 0 0 0) 0x7f704003f070 con 0x7f7050073a10 2024-04-20T21:23:16.194 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:23:16.194 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f702c06c920 msgr2=0x7f702c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f702c06c920 0x7f702c06ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7050157090 tx=0x7f7038009040 comp rx=0 tx=0).stop 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7050073a10 msgr2=0x7f705006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7050073a10 0x7f705006d160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7040000c00 tx=0x7f70400046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 shutdown_connections 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f705014fa50 0x7f70501f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70500c9eb0 0x7f70501581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 --2- 172.21.15.102:0/696231581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7050073a10 0x7f705006d160 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.196+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 >> 172.21.15.102:0/696231581 conn(0x7f70500c9390 msgr2=0x7f705006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.197+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 shutdown_connections 2024-04-20T21:23:16.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:16.197+0000 7f7055000640 1 -- 172.21.15.102:0/696231581 wait complete. 2024-04-20T21:23:16.214 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:23:18.215 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:18.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.422+0000 7f27bb200640 1 Processor -- start 2024-04-20T21:23:18.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.422+0000 7f27bb200640 1 -- start start 2024-04-20T21:23:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.423+0000 7f27bb200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f27bc0c3700 0x7f27bc0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27bb200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc15c220 con 0x7f27bc152290 2024-04-20T21:23:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27bb200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc0c4560 con 0x7f27bc0c8330 2024-04-20T21:23:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27bb200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f27bc0c9180 con 0x7f27bc0c3700 2024-04-20T21:23:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27b9e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f27bc152290 0x7f27bc06e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52732/0 (socket says 172.21.15.102:52732) 2024-04-20T21:23:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27b9e00640 1 -- 172.21.15.102:0/2953653121 learned_addr learned my addr 172.21.15.102:0/2953653121 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:18.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27b9400640 1 --2- 172.21.15.102:0/2953653121 >> v2:172.21.15.102:3300/0 conn(0x7f27bc0c3700 0x7f27bc0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27b8a00640 1 -- 172.21.15.102:0/2953653121 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3760308285 0 0) 0x7f27bc15c220 con 0x7f27bc152290 2024-04-20T21:23:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27b8a00640 1 -- 172.21.15.102:0/2953653121 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27a0003520 con 0x7f27bc152290 2024-04-20T21:23:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27b8a00640 1 -- 172.21.15.102:0/2953653121 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973021651 0 0) 0x7f27bc0c4560 con 0x7f27bc0c8330 2024-04-20T21:23:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.424+0000 7f27b9400640 1 -- 172.21.15.102:0/2953653121 >> v1:172.21.15.193:6789/0 conn(0x7f27bc0c8330 legacy=0x7f27bc162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.425+0000 7f27b9400640 1 -- 172.21.15.102:0/2953653121 >> v1:172.21.15.102:6789/0 conn(0x7f27bc152290 legacy=0x7f27bc06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.425+0000 7f27b9400640 1 -- 172.21.15.102:0/2953653121 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc15b460 con 0x7f27bc0c3700 2024-04-20T21:23:18.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.426+0000 7f27b9400640 1 --2- 172.21.15.102:0/2953653121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc0c3700 0x7f27bc0cd310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27a80097a0 tx=0x7f27a802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b973c4ddd9374d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.426+0000 7f27b8a00640 1 -- 172.21.15.102:0/2953653121 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a802f810 con 0x7f27bc0c3700 2024-04-20T21:23:18.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.426+0000 7f27b8a00640 1 -- 172.21.15.102:0/2953653121 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a802f9b0 con 0x7f27bc0c3700 2024-04-20T21:23:18.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.426+0000 7f27b8a00640 1 -- 172.21.15.102:0/2953653121 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a802fd80 con 0x7f27bc0c3700 2024-04-20T21:23:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.427+0000 7f27bb200640 1 -- 172.21.15.102:0/2953653121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc0c3700 msgr2=0x7f27bc0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.427+0000 7f27bb200640 1 --2- 172.21.15.102:0/2953653121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc0c3700 0x7f27bc0cd310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27a80097a0 tx=0x7f27a802b9f0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.427+0000 7f27bb200640 1 -- 172.21.15.102:0/2953653121 shutdown_connections 2024-04-20T21:23:18.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.427+0000 7f27bb200640 1 --2- 172.21.15.102:0/2953653121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc0c3700 0x7f27bc0cd310 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.427+0000 7f27bb200640 1 -- 172.21.15.102:0/2953653121 >> 172.21.15.102:0/2953653121 conn(0x7f27bc0cc820 msgr2=0x7f27bc0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.427+0000 7f27bb200640 1 -- 172.21.15.102:0/2953653121 shutdown_connections 2024-04-20T21:23:18.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.427+0000 7f27bb200640 1 -- 172.21.15.102:0/2953653121 wait complete. 2024-04-20T21:23:18.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.429+0000 7f27bb200640 1 Processor -- start 2024-04-20T21:23:18.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.429+0000 7f27bb200640 1 -- start start 2024-04-20T21:23:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.430+0000 7f27bb200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27bc0c3700 0x7f27bc145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27bb200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc0c8330 0x7f27bc14d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27bb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc152290 0x7f27bc0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27bb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc15ba20 con 0x7f27bc152290 2024-04-20T21:23:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27bb200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc0c4840 con 0x7f27bc0c8330 2024-04-20T21:23:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27bb200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f27bc15bd00 con 0x7f27bc0c3700 2024-04-20T21:23:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27b9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc0c8330 0x7f27bc14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27b9e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27bc0c3700 0x7f27bc145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27b9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc0c8330 0x7f27bc14d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:53974/0 (socket says 172.21.15.102:53974) 2024-04-20T21:23:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27b9400640 1 -- 172.21.15.102:0/1265978635 learned_addr learned my addr 172.21.15.102:0/1265978635 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:18.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27ba800640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc152290 0x7f27bc0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27b9400640 1 -- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27bc0c3700 msgr2=0x7f27bc145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27b9400640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27bc0c3700 0x7f27bc145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.431+0000 7f27b9400640 1 -- 172.21.15.102:0/1265978635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc152290 msgr2=0x7f27bc0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27b9400640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc152290 0x7f27bc0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27b9400640 1 -- 172.21.15.102:0/1265978635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc1f3410 con 0x7f27bc0c8330 2024-04-20T21:23:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27ba800640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc152290 0x7f27bc0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:18.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27b9e00640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27bc0c3700 0x7f27bc145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27b9400640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc0c8330 0x7f27bc14d350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f27a80098c0 tx=0x7f27a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27b2000640 1 -- 172.21.15.102:0/1265978635 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8047070 con 0x7f27bc0c8330 2024-04-20T21:23:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27b2000640 1 -- 172.21.15.102:0/1265978635 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a8037810 con 0x7f27bc0c8330 2024-04-20T21:23:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc151ae0 con 0x7f27bc0c8330 2024-04-20T21:23:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27b2000640 1 -- 172.21.15.102:0/1265978635 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8037be0 con 0x7f27bc0c8330 2024-04-20T21:23:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.432+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc151800 con 0x7f27bc0c8330 2024-04-20T21:23:18.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.433+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc04fd60 con 0x7f27bc0c8330 2024-04-20T21:23:18.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.435+0000 7f27b2000640 1 -- 172.21.15.102:0/1265978635 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27a802d3a0 con 0x7f27bc0c8330 2024-04-20T21:23:18.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.436+0000 7f27b2000640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f279c06c8f0 0x7f279c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.436+0000 7f27b2000640 1 -- 172.21.15.102:0/1265978635 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 7641+0+0 (secure 0 0 0) 0x7f27a80adef0 con 0x7f27bc0c8330 2024-04-20T21:23:18.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.436+0000 7f27b9e00640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f279c06c8f0 0x7f279c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.437+0000 7f27b9e00640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f279c06c8f0 0x7f279c06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f27ac005880 tx=0x7f27ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.440+0000 7f27b2000640 1 -- 172.21.15.102:0/1265978635 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f27a8033030 con 0x7f27bc0c8330 2024-04-20T21:23:18.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.684+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27bc06f1d0 con 0x7f27bc0c8330 2024-04-20T21:23:18.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.685+0000 7f27b2000640 1 -- 172.21.15.102:0/1265978635 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6859 (secure 0 0 0) 0x7f27a807b420 con 0x7f27bc0c8330 2024-04-20T21:23:18.685 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:18.685 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:23:18.685 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-04-20T21:23:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f279c06c8f0 msgr2=0x7f279c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f279c06c8f0 0x7f279c06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f27ac005880 tx=0x7f27ac009040 comp rx=0 tx=0).stop 2024-04-20T21:23:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc0c8330 msgr2=0x7f27bc14d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc0c8330 0x7f27bc14d350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f27a80098c0 tx=0x7f27a8004730 comp rx=0 tx=0).stop 2024-04-20T21:23:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 shutdown_connections 2024-04-20T21:23:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27bc152290 0x7f27bc0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f279c06c8f0 0x7f279c06edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27bc0c8330 0x7f27bc14d350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 --2- 172.21.15.102:0/1265978635 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27bc0c3700 0x7f27bc145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 >> 172.21.15.102:0/1265978635 conn(0x7f27bc0cc820 msgr2=0x7f27bc0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 shutdown_connections 2024-04-20T21:23:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:18.688+0000 7f27bb200640 1 -- 172.21.15.102:0/1265978635 wait complete. 2024-04-20T21:23:18.706 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:23:20.707 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.917+0000 7febf5e00640 1 Processor -- start 2024-04-20T21:23:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.917+0000 7febf5e00640 1 -- start start 2024-04-20T21:23:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febf5e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 0x7febf006d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febf5e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7febf015f2b0 0x7febf00710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febf5e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf00c8f20 con 0x7febf015ea20 2024-04-20T21:23:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febf5e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7febf006cb80 con 0x7febf015f2b0 2024-04-20T21:23:20.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febf5e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7febf006ce60 con 0x7febf015ee00 2024-04-20T21:23:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febef400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7febf015f2b0 0x7febf00710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febeea00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7febf015ea20 0x7febf00c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52740/0 (socket says 172.21.15.102:52740) 2024-04-20T21:23:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febeea00640 1 -- 172.21.15.102:0/1185502302 learned_addr learned my addr 172.21.15.102:0/1185502302 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.919+0000 7febee000640 1 --2- 172.21.15.102:0/1185502302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 0x7febf006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.920+0000 7febef400640 1 -- 172.21.15.102:0/1185502302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 msgr2=0x7febf006d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.920+0000 7febef400640 1 --2- 172.21.15.102:0/1185502302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 0x7febf006d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.920+0000 7febef400640 1 -- 172.21.15.102:0/1185502302 >> v1:172.21.15.102:6789/0 conn(0x7febf015ea20 legacy=0x7febf00c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.920+0000 7febef400640 1 -- 172.21.15.102:0/1185502302 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf006d140 con 0x7febf015f2b0 2024-04-20T21:23:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.921+0000 7febef400640 1 --2- 172.21.15.102:0/1185502302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf00710a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7febe8009d90 tx=0x7febe802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b23e8785883242 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.921+0000 7febed600640 1 -- 172.21.15.102:0/1185502302 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8039070 con 0x7febf015f2b0 2024-04-20T21:23:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.921+0000 7febed600640 1 -- 172.21.15.102:0/1185502302 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe802cce0 con 0x7febf015f2b0 2024-04-20T21:23:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.921+0000 7febed600640 1 -- 172.21.15.102:0/1185502302 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8034660 con 0x7febf015f2b0 2024-04-20T21:23:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.922+0000 7febf5e00640 1 -- 172.21.15.102:0/1185502302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 msgr2=0x7febf00710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.922+0000 7febf5e00640 1 --2- 172.21.15.102:0/1185502302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf00710a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7febe8009d90 tx=0x7febe802bf30 comp rx=0 tx=0).stop 2024-04-20T21:23:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.922+0000 7febf5e00640 1 -- 172.21.15.102:0/1185502302 shutdown_connections 2024-04-20T21:23:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.922+0000 7febf5e00640 1 --2- 172.21.15.102:0/1185502302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf00710a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.922+0000 7febf5e00640 1 --2- 172.21.15.102:0/1185502302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 0x7febf006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.922+0000 7febf5e00640 1 -- 172.21.15.102:0/1185502302 >> 172.21.15.102:0/1185502302 conn(0x7febf00c9390 msgr2=0x7febf00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.922+0000 7febf5e00640 1 -- 172.21.15.102:0/1185502302 shutdown_connections 2024-04-20T21:23:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.922+0000 7febf5e00640 1 -- 172.21.15.102:0/1185502302 wait complete. 2024-04-20T21:23:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.924+0000 7febf5e00640 1 Processor -- start 2024-04-20T21:23:20.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.925+0000 7febf5e00640 1 -- start start 2024-04-20T21:23:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febf5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf015ea20 0x7febf0071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febf5e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 0x7febf01f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febf5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf01f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febf5e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7febf00c8c80 con 0x7febf015f2b0 2024-04-20T21:23:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febf5e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febf00c8b00 con 0x7febf015ea20 2024-04-20T21:23:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febf5e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7febf0154e80 con 0x7febf015ee00 2024-04-20T21:23:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf01f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febeea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf015ea20 0x7febf0071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf01f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55670/0 (socket says 172.21.15.102:55670) 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febef400640 1 -- 172.21.15.102:0/3337200418 learned_addr learned my addr 172.21.15.102:0/3337200418 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febee000640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 0x7febf01f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febef400640 1 -- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 msgr2=0x7febf01f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febef400640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 0x7febf01f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.926+0000 7febef400640 1 -- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf015ea20 msgr2=0x7febf0071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febef400640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf015ea20 0x7febf0071060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febef400640 1 -- 172.21.15.102:0/3337200418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf015ce30 con 0x7febf015f2b0 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febeea00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf015ea20 0x7febf0071060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:20.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febef400640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf01f4d00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7febe8009eb0 tx=0x7febe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febe7400640 1 -- 172.21.15.102:0/3337200418 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8039050 con 0x7febf015f2b0 2024-04-20T21:23:20.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febe7400640 1 -- 172.21.15.102:0/3337200418 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe8045070 con 0x7febf015f2b0 2024-04-20T21:23:20.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febe7400640 1 -- 172.21.15.102:0/3337200418 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8009450 con 0x7febf015f2b0 2024-04-20T21:23:20.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf015d110 con 0x7febf015f2b0 2024-04-20T21:23:20.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.927+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf015ddd0 con 0x7febf015f2b0 2024-04-20T21:23:20.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.929+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf0074340 con 0x7febf015f2b0 2024-04-20T21:23:20.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.930+0000 7febe7400640 1 -- 172.21.15.102:0/3337200418 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febe803e850 con 0x7febf015f2b0 2024-04-20T21:23:20.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.930+0000 7febe7400640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7febcc06c8f0 0x7febcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.931+0000 7febe7400640 1 -- 172.21.15.102:0/3337200418 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 7421+0+0 (secure 0 0 0) 0x7febe80aede0 con 0x7febf015f2b0 2024-04-20T21:23:20.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.931+0000 7febeea00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7febcc06c8f0 0x7febcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.931+0000 7febeea00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7febcc06c8f0 0x7febcc06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febe0007fd0 tx=0x7febe0007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:20.936+0000 7febe7400640 1 -- 172.21.15.102:0/3337200418 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7febe807c620 con 0x7febf015f2b0 2024-04-20T21:23:21.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.181+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febf00725a0 con 0x7febf015f2b0 2024-04-20T21:23:21.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.182+0000 7febe7400640 1 -- 172.21.15.102:0/3337200418 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6859 (secure 0 0 0) 0x7febe8080920 con 0x7febf015f2b0 2024-04-20T21:23:21.182 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:21.182 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:23:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-04-20T21:23:21.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7febcc06c8f0 msgr2=0x7febcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7febcc06c8f0 0x7febcc06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febe0007fd0 tx=0x7febe0007e80 comp rx=0 tx=0).stop 2024-04-20T21:23:21.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 msgr2=0x7febf01f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf01f4d00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7febe8009eb0 tx=0x7febe8004a40 comp rx=0 tx=0).stop 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 shutdown_connections 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febf015f2b0 0x7febf01f4d00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7febf015ee00 0x7febf01f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7febcc06c8f0 0x7febcc06edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 --2- 172.21.15.102:0/3337200418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf015ea20 0x7febf0071060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 >> 172.21.15.102:0/3337200418 conn(0x7febf00c9390 msgr2=0x7febf01513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 shutdown_connections 2024-04-20T21:23:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:21.185+0000 7febf5e00640 1 -- 172.21.15.102:0/3337200418 wait complete. 2024-04-20T21:23:21.204 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:23:22.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:22.705 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:23.205 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.402+0000 7f71eaa00640 1 Processor -- start 2024-04-20T21:23:23.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.402+0000 7f71eaa00640 1 -- start start 2024-04-20T21:23:23.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.403+0000 7f71eaa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e4157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.403+0000 7f71eaa00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e40cd6f0 con 0x7f71e41516d0 2024-04-20T21:23:23.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.403+0000 7f71eaa00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e406ef60 con 0x7f71e4151af0 2024-04-20T21:23:23.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.403+0000 7f71eaa00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f71e406ed40 con 0x7f71e40cd310 2024-04-20T21:23:23.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e8c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f71e41516d0 0x7f71e406e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52756/0 (socket says 172.21.15.102:52756) 2024-04-20T21:23:23.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e8c00640 1 -- 172.21.15.102:0/656439260 learned_addr learned my addr 172.21.15.102:0/656439260 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:23.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e9600640 1 --2- 172.21.15.102:0/656439260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e4157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e3e00640 1 -- 172.21.15.102:0/656439260 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3266017831 0 0) 0x7f71e406ef60 con 0x7f71e4151af0 2024-04-20T21:23:23.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e3e00640 1 -- 172.21.15.102:0/656439260 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71cc003520 con 0x7f71e4151af0 2024-04-20T21:23:23.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e3e00640 1 -- 172.21.15.102:0/656439260 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 458970006 0 0) 0x7f71e40cd6f0 con 0x7f71e41516d0 2024-04-20T21:23:23.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e9600640 1 -- 172.21.15.102:0/656439260 >> v1:172.21.15.193:6789/0 conn(0x7f71e4151af0 legacy=0x7f71e40c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e9600640 1 -- 172.21.15.102:0/656439260 >> v1:172.21.15.102:6789/0 conn(0x7f71e41516d0 legacy=0x7f71e406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.404+0000 7f71e9600640 1 -- 172.21.15.102:0/656439260 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e406f240 con 0x7f71e40cd310 2024-04-20T21:23:23.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.406+0000 7f71e9600640 1 --2- 172.21.15.102:0/656439260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e4157ff0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71d80097d0 tx=0x7f71d802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e88d92e5ab4011a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.406+0000 7f71e3e00640 1 -- 172.21.15.102:0/656439260 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8034070 con 0x7f71e40cd310 2024-04-20T21:23:23.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.406+0000 7f71e3e00640 1 -- 172.21.15.102:0/656439260 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d8030880 con 0x7f71e40cd310 2024-04-20T21:23:23.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.406+0000 7f71e3e00640 1 -- 172.21.15.102:0/656439260 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8030c50 con 0x7f71e40cd310 2024-04-20T21:23:23.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.406+0000 7f71eaa00640 1 -- 172.21.15.102:0/656439260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 msgr2=0x7f71e4157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.406+0000 7f71eaa00640 1 --2- 172.21.15.102:0/656439260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e4157ff0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71d80097d0 tx=0x7f71d802b9c0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.407+0000 7f71eaa00640 1 -- 172.21.15.102:0/656439260 shutdown_connections 2024-04-20T21:23:23.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.407+0000 7f71eaa00640 1 --2- 172.21.15.102:0/656439260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e4157ff0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.407+0000 7f71eaa00640 1 -- 172.21.15.102:0/656439260 >> 172.21.15.102:0/656439260 conn(0x7f71e40cc820 msgr2=0x7f71e40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:23.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.407+0000 7f71eaa00640 1 -- 172.21.15.102:0/656439260 shutdown_connections 2024-04-20T21:23:23.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.407+0000 7f71eaa00640 1 -- 172.21.15.102:0/656439260 wait complete. 2024-04-20T21:23:23.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.408+0000 7f71eaa00640 1 Processor -- start 2024-04-20T21:23:23.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.409+0000 7f71eaa00640 1 -- start start 2024-04-20T21:23:23.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.409+0000 7f71eaa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e40730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.409+0000 7f71eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e41516d0 0x7f71e40c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.409+0000 7f71eaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e41497e0 0x7f71e4149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.409+0000 7f71eaa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71e40c5020 con 0x7f71e41516d0 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.409+0000 7f71eaa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71e40c4dd0 con 0x7f71e41497e0 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71eaa00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f71e40c5300 con 0x7f71e40cd310 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e41516d0 0x7f71e40c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e9600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e40730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71ea000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e41497e0 0x7f71e4149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e9600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e40730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:42924/0 (socket says 172.21.15.102:42924) 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e9600640 1 -- 172.21.15.102:0/471779306 learned_addr learned my addr 172.21.15.102:0/471779306 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e9600640 1 -- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e41497e0 msgr2=0x7f71e4149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e9600640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e41497e0 0x7f71e4149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e9600640 1 -- 172.21.15.102:0/471779306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e41516d0 msgr2=0x7f71e40c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e9600640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e41516d0 0x7f71e40c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.410+0000 7f71e9600640 1 -- 172.21.15.102:0/471779306 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e41f0bc0 con 0x7f71e40cd310 2024-04-20T21:23:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.411+0000 7f71e9600640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e40730d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71d80097d0 tx=0x7f71d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.411+0000 7f71e1600640 1 -- 172.21.15.102:0/471779306 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8034040 con 0x7f71e40cd310 2024-04-20T21:23:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.411+0000 7f71e1600640 1 -- 172.21.15.102:0/471779306 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d8030680 con 0x7f71e40cd310 2024-04-20T21:23:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.411+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e4073710 con 0x7f71e40cd310 2024-04-20T21:23:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.411+0000 7f71e1600640 1 -- 172.21.15.102:0/471779306 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8039c40 con 0x7f71e40cd310 2024-04-20T21:23:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.411+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e41f1160 con 0x7f71e40cd310 2024-04-20T21:23:23.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.412+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e404fd60 con 0x7f71e40cd310 2024-04-20T21:23:23.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.413+0000 7f71e1600640 1 -- 172.21.15.102:0/471779306 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71d804b060 con 0x7f71e40cd310 2024-04-20T21:23:23.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.414+0000 7f71e1600640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f71c406c8f0 0x7f71c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.414+0000 7f71e1600640 1 -- 172.21.15.102:0/471779306 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(43..43 src has 1..43) ==== 7421+0+0 (secure 0 0 0) 0x7f71d80adfc0 con 0x7f71e40cd310 2024-04-20T21:23:23.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.414+0000 7f71e8c00640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f71c406c8f0 0x7f71c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.415+0000 7f71e8c00640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f71c406c8f0 0x7f71c406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f71d0007f70 tx=0x7f71d0007f00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.419+0000 7f71e1600640 1 -- 172.21.15.102:0/471779306 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f71d807b650 con 0x7f71e40cd310 2024-04-20T21:23:23.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.664+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71e406dcb0 con 0x7f71e40cd310 2024-04-20T21:23:23.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.665+0000 7f71e1600640 1 -- 172.21.15.102:0/471779306 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6859 (secure 0 0 0) 0x7f71d807f950 con 0x7f71e40cd310 2024-04-20T21:23:23.664 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:23.665 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:23:23.665 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f71c406c8f0 msgr2=0x7f71c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f71c406c8f0 0x7f71c406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f71d0007f70 tx=0x7f71d0007f00 comp rx=0 tx=0).stop 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 msgr2=0x7f71e40730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e40730d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71d80097d0 tx=0x7f71d80046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 shutdown_connections 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f71c406c8f0 0x7f71c406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e41497e0 0x7f71e4149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e41516d0 0x7f71e40c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 --2- 172.21.15.102:0/471779306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71e40cd310 0x7f71e40730d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 >> 172.21.15.102:0/471779306 conn(0x7f71e40cc820 msgr2=0x7f71e414f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 shutdown_connections 2024-04-20T21:23:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:23.668+0000 7f71eaa00640 1 -- 172.21.15.102:0/471779306 wait complete. 2024-04-20T21:23:23.687 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:23:25.688 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:25.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.889+0000 7eff66600640 1 Processor -- start 2024-04-20T21:23:25.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.889+0000 7eff66600640 1 -- start start 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff66600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff60153180 0x7eff6006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff66600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff6015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff66600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff6006def0 con 0x7eff600caec0 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff66600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7eff6015cae0 con 0x7eff60153180 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff66600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7eff6006e1d0 con 0x7eff60073b20 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff65200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff60153180 0x7eff6006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff65c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7eff600caec0 0x7eff600c6fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52768/0 (socket says 172.21.15.102:52768) 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff65c00640 1 -- 172.21.15.102:0/788060376 learned_addr learned my addr 172.21.15.102:0/788060376 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.891+0000 7eff5fe00640 1 --2- 172.21.15.102:0/788060376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff6015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.892+0000 7eff65200640 1 -- 172.21.15.102:0/788060376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 msgr2=0x7eff6015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.892+0000 7eff65200640 1 --2- 172.21.15.102:0/788060376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff6015cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.892+0000 7eff65200640 1 -- 172.21.15.102:0/788060376 >> v1:172.21.15.102:6789/0 conn(0x7eff600caec0 legacy=0x7eff600c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.892+0000 7eff65200640 1 -- 172.21.15.102:0/788060376 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6006e4b0 con 0x7eff60153180 2024-04-20T21:23:25.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.892+0000 7eff5f400640 1 -- 172.21.15.102:0/788060376 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209843150 0 0) 0x7eff6006def0 con 0x7eff600caec0 2024-04-20T21:23:25.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.893+0000 7eff65200640 1 --2- 172.21.15.102:0/788060376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff60153180 0x7eff6006e8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff540097a0 tx=0x7eff5402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bfa702bbdddc5d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:25.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.893+0000 7eff5f400640 1 -- 172.21.15.102:0/788060376 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54035020 con 0x7eff60153180 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.893+0000 7eff5f400640 1 -- 172.21.15.102:0/788060376 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff54031810 con 0x7eff60153180 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.893+0000 7eff5f400640 1 -- 172.21.15.102:0/788060376 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54031be0 con 0x7eff60153180 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.894+0000 7eff66600640 1 -- 172.21.15.102:0/788060376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff60153180 msgr2=0x7eff6006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.894+0000 7eff66600640 1 --2- 172.21.15.102:0/788060376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff60153180 0x7eff6006e8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff540097a0 tx=0x7eff5402b8b0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.894+0000 7eff66600640 1 -- 172.21.15.102:0/788060376 shutdown_connections 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.894+0000 7eff66600640 1 --2- 172.21.15.102:0/788060376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff6015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.894+0000 7eff66600640 1 --2- 172.21.15.102:0/788060376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff60153180 0x7eff6006e8c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.894+0000 7eff66600640 1 -- 172.21.15.102:0/788060376 >> 172.21.15.102:0/788060376 conn(0x7eff600ca3a0 msgr2=0x7eff600ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.894+0000 7eff66600640 1 -- 172.21.15.102:0/788060376 shutdown_connections 2024-04-20T21:23:25.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.894+0000 7eff66600640 1 -- 172.21.15.102:0/788060376 wait complete. 2024-04-20T21:23:25.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.896+0000 7eff66600640 1 Processor -- start 2024-04-20T21:23:25.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.896+0000 7eff66600640 1 -- start start 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff66600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff601f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff66600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff600caec0 0x7eff6015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff66600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff60153180 0x7eff6015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff66600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff60153560 con 0x7eff600caec0 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff66600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7eff6015c770 con 0x7eff60153180 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff66600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7eff601536e0 con 0x7eff60073b20 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff600caec0 0x7eff6015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff65200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff601f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff600caec0 0x7eff6015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55708/0 (socket says 172.21.15.102:55708) 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff5fe00640 1 -- 172.21.15.102:0/3663917817 learned_addr learned my addr 172.21.15.102:0/3663917817 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.897+0000 7eff65c00640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff60153180 0x7eff6015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5fe00640 1 -- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 msgr2=0x7eff601f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5fe00640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff601f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5fe00640 1 -- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff60153180 msgr2=0x7eff6015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5fe00640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff60153180 0x7eff6015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5fe00640 1 -- 172.21.15.102:0/3663917817 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6015ada0 con 0x7eff600caec0 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff65200640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff601f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff65c00640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff60153180 0x7eff6015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5fe00640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff600caec0 0x7eff6015a2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00e9e0 tx=0x7eff4c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5cc00640 1 -- 172.21.15.102:0/3663917817 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c017070 con 0x7eff600caec0 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5cc00640 1 -- 172.21.15.102:0/3663917817 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4c00c9d0 con 0x7eff600caec0 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff5cc00640 1 -- 172.21.15.102:0/3663917817 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00cdc0 con 0x7eff600caec0 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6015b020 con 0x7eff600caec0 2024-04-20T21:23:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.898+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6015b5e0 con 0x7eff600caec0 2024-04-20T21:23:25.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.900+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6006def0 con 0x7eff600caec0 2024-04-20T21:23:25.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.901+0000 7eff5cc00640 1 -- 172.21.15.102:0/3663917817 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff4c01a070 con 0x7eff600caec0 2024-04-20T21:23:25.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.901+0000 7eff5cc00640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff4006c8f0 0x7eff4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:25.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.902+0000 7eff5cc00640 1 -- 172.21.15.102:0/3663917817 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 7421+0+0 (secure 0 0 0) 0x7eff4c08c5d0 con 0x7eff600caec0 2024-04-20T21:23:25.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.902+0000 7eff65200640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff4006c8f0 0x7eff4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:25.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.903+0000 7eff65200640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff4006c8f0 0x7eff4006edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff54036000 tx=0x7eff54008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:25.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:25.909+0000 7eff5cc00640 1 -- 172.21.15.102:0/3663917817 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7eff6006def0 con 0x7eff600caec0 2024-04-20T21:23:26.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.149+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff600c85f0 con 0x7eff600caec0 2024-04-20T21:23:26.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.151+0000 7eff5cc00640 1 -- 172.21.15.102:0/3663917817 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6859 (secure 0 0 0) 0x7eff6015ada0 con 0x7eff600caec0 2024-04-20T21:23:26.150 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:26.150 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:23:26.151 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-04-20T21:23:26.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff4006c8f0 msgr2=0x7eff4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:26.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff4006c8f0 0x7eff4006edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff54036000 tx=0x7eff54008030 comp rx=0 tx=0).stop 2024-04-20T21:23:26.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff600caec0 msgr2=0x7eff6015a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:26.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff600caec0 0x7eff6015a2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00e9e0 tx=0x7eff4c00eea0 comp rx=0 tx=0).stop 2024-04-20T21:23:26.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 shutdown_connections 2024-04-20T21:23:26.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff4006c8f0 0x7eff4006edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:26.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff60153180 0x7eff6015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:26.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff600caec0 0x7eff6015a2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:26.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 --2- 172.21.15.102:0/3663917817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff60073b20 0x7eff601f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:26.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.153+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 >> 172.21.15.102:0/3663917817 conn(0x7eff600ca3a0 msgr2=0x7eff600c8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:26.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.154+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 shutdown_connections 2024-04-20T21:23:26.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:26.154+0000 7eff66600640 1 -- 172.21.15.102:0/3663917817 wait complete. 2024-04-20T21:23:26.172 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:23:28.173 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:28.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.384+0000 7f3913e00640 1 Processor -- start 2024-04-20T21:23:28.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.384+0000 7f3913e00640 1 -- start start 2024-04-20T21:23:28.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.386+0000 7f3913e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3914155530 0x7f391415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:28.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.386+0000 7f3913e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391415ef60 con 0x7f39140cb6a0 2024-04-20T21:23:28.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.386+0000 7f3913e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391415f240 con 0x7f39140cc470 2024-04-20T21:23:28.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.386+0000 7f3913e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f39140c5ca0 con 0x7f3914155530 2024-04-20T21:23:28.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.386+0000 7f3913400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f39140cb6a0 0x7f39141659d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38366/0 (socket says 172.21.15.102:38366) 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.386+0000 7f3912000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f39140cc470 0x7f39140cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:39444/0 (socket says 172.21.15.102:39444) 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.386+0000 7f3913400640 1 -- 172.21.15.102:0/52007641 learned_addr learned my addr 172.21.15.102:0/52007641 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.386+0000 7f3912a00640 1 --2- 172.21.15.102:0/52007641 >> v2:172.21.15.102:3300/0 conn(0x7f3914155530 0x7f391415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.387+0000 7f3911600640 1 -- 172.21.15.102:0/52007641 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 41446282 0 0) 0x7f391415f240 con 0x7f39140cc470 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.387+0000 7f3911600640 1 -- 172.21.15.102:0/52007641 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38f8003520 con 0x7f39140cc470 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.387+0000 7f3911600640 1 -- 172.21.15.102:0/52007641 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3785629252 0 0) 0x7f391415ef60 con 0x7f39140cb6a0 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.387+0000 7f3911600640 1 -- 172.21.15.102:0/52007641 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f391415f240 con 0x7f39140cb6a0 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.387+0000 7f3912a00640 1 -- 172.21.15.102:0/52007641 >> v1:172.21.15.193:6789/0 conn(0x7f39140cc470 legacy=0x7f39140cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.387+0000 7f3912a00640 1 -- 172.21.15.102:0/52007641 >> v1:172.21.15.102:6789/0 conn(0x7f39140cb6a0 legacy=0x7f39141659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:28.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.387+0000 7f3912a00640 1 -- 172.21.15.102:0/52007641 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391415f520 con 0x7f3914155530 2024-04-20T21:23:28.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.389+0000 7f3912a00640 1 --2- 172.21.15.102:0/52007641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3914155530 0x7f391415e640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39040097a0 tx=0x7f390402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96da0786ace2a84e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:28.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.389+0000 7f3911600640 1 -- 172.21.15.102:0/52007641 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904034070 con 0x7f3914155530 2024-04-20T21:23:28.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.389+0000 7f3911600640 1 -- 172.21.15.102:0/52007641 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3904030610 con 0x7f3914155530 2024-04-20T21:23:28.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.389+0000 7f3911600640 1 -- 172.21.15.102:0/52007641 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39040309e0 con 0x7f3914155530 2024-04-20T21:23:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.390+0000 7f3913e00640 1 -- 172.21.15.102:0/52007641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3914155530 msgr2=0x7f391415e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.390+0000 7f3913e00640 1 --2- 172.21.15.102:0/52007641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3914155530 0x7f391415e640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39040097a0 tx=0x7f390402b960 comp rx=0 tx=0).stop 2024-04-20T21:23:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.390+0000 7f3913e00640 1 -- 172.21.15.102:0/52007641 shutdown_connections 2024-04-20T21:23:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.390+0000 7f3913e00640 1 --2- 172.21.15.102:0/52007641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3914155530 0x7f391415e640 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.390+0000 7f3913e00640 1 -- 172.21.15.102:0/52007641 >> 172.21.15.102:0/52007641 conn(0x7f3914075080 msgr2=0x7f3914075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.390+0000 7f3913e00640 1 -- 172.21.15.102:0/52007641 shutdown_connections 2024-04-20T21:23:28.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.390+0000 7f3913e00640 1 -- 172.21.15.102:0/52007641 wait complete. 2024-04-20T21:23:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.391+0000 7f3913e00640 1 Processor -- start 2024-04-20T21:23:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.392+0000 7f3913e00640 1 -- start start 2024-04-20T21:23:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3913e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39140cb6a0 0x7f39140b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3913e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39140cc470 0x7f39141f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3913e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3914155530 0x7f391415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3913e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f39140cd280 con 0x7f39140cc470 2024-04-20T21:23:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3913e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39140cd5f0 con 0x7f3914155530 2024-04-20T21:23:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3913e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f39140ccbe0 con 0x7f39140cb6a0 2024-04-20T21:23:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3912000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39140cc470 0x7f39141f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3912000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39140cc470 0x7f39141f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55686/0 (socket says 172.21.15.102:55686) 2024-04-20T21:23:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3912000640 1 -- 172.21.15.102:0/3060139996 learned_addr learned my addr 172.21.15.102:0/3060139996 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3913400640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3914155530 0x7f391415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3912a00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39140cb6a0 0x7f39140b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:28.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.393+0000 7f3912000640 1 -- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39140cb6a0 msgr2=0x7f39140b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:28.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3912000640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39140cb6a0 0x7f39140b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:28.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3912000640 1 -- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3914155530 msgr2=0x7f391415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3912000640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3914155530 0x7f391415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3912000640 1 -- 172.21.15.102:0/3060139996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39140b4ab0 con 0x7f39140cc470 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3912a00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39140cb6a0 0x7f39140b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3913400640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3914155530 0x7f391415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3912000640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39140cc470 0x7f39141f1910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3900001eb0 tx=0x7f390000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f390aa00640 1 -- 172.21.15.102:0/3060139996 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900018070 con 0x7f39140cc470 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f390aa00640 1 -- 172.21.15.102:0/3060139996 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3900007480 con 0x7f39140cc470 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f390aa00640 1 -- 172.21.15.102:0/3060139996 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900004a20 con 0x7f39140cc470 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39140b4d90 con 0x7f39140cc470 2024-04-20T21:23:28.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.394+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39140b5350 con 0x7f39140cc470 2024-04-20T21:23:28.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.395+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391404fd60 con 0x7f39140cc470 2024-04-20T21:23:28.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.396+0000 7f390aa00640 1 -- 172.21.15.102:0/3060139996 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3900004bc0 con 0x7f39140cc470 2024-04-20T21:23:28.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.397+0000 7f390aa00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38f406c920 0x7f38f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:28.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.397+0000 7f390aa00640 1 -- 172.21.15.102:0/3060139996 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 7421+0+0 (secure 0 0 0) 0x7f390008b3b0 con 0x7f39140cc470 2024-04-20T21:23:28.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.398+0000 7f3912a00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38f406c920 0x7f38f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:28.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.398+0000 7f3912a00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38f406c920 0x7f38f406ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3904036000 tx=0x7f390403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:28.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.402+0000 7f390aa00640 1 -- 172.21.15.102:0/3060139996 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3900058a40 con 0x7f39140cc470 2024-04-20T21:23:28.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.641+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39140d0070 con 0x7f39140cc470 2024-04-20T21:23:28.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.642+0000 7f390aa00640 1 -- 172.21.15.102:0/3060139996 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6859 (secure 0 0 0) 0x7f390005cd40 con 0x7f39140cc470 2024-04-20T21:23:28.642 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:28.642 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:22:38.112689+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5266":{"gid":5266,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6833/713712414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":713712414},{"type":"v1","addr":"172.21.15.102:6833","nonce":713712414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5266,"qdb_cluster":[5266]},"id":1}]} 2024-04-20T21:23:28.642 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38f406c920 msgr2=0x7f38f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38f406c920 0x7f38f406ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3904036000 tx=0x7f390403b040 comp rx=0 tx=0).stop 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39140cc470 msgr2=0x7f39141f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39140cc470 0x7f39141f1910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3900001eb0 tx=0x7f390000c960 comp rx=0 tx=0).stop 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 shutdown_connections 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38f406c920 0x7f38f406ede0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3914155530 0x7f391415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39140cc470 0x7f39141f1910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 --2- 172.21.15.102:0/3060139996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39140cb6a0 0x7f39140b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 >> 172.21.15.102:0/3060139996 conn(0x7f3914075080 msgr2=0x7f39140ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 shutdown_connections 2024-04-20T21:23:28.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:28.645+0000 7f3913e00640 1 -- 172.21.15.102:0/3060139996 wait complete. 2024-04-20T21:23:28.662 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:23:30.663 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:30.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.871+0000 7f2a13200640 1 Processor -- start 2024-04-20T21:23:30.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.871+0000 7f2a13200640 1 -- start start 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.872+0000 7f2a13200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14153180 0x7f2a14158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.872+0000 7f2a13200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a14159380 0x7f2a140c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.872+0000 7f2a13200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a14153560 con 0x7f2a140c75a0 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.872+0000 7f2a13200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f2a140c7a70 con 0x7f2a14159380 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.872+0000 7f2a13200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2a140c7d30 con 0x7f2a14153180 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.872+0000 7f2a12800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2a140c75a0 0x7f2a14070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38374/0 (socket says 172.21.15.102:38374) 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.872+0000 7f2a12800640 1 -- 172.21.15.102:0/2300461763 learned_addr learned my addr 172.21.15.102:0/2300461763 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.872+0000 7f2a11400640 1 --2- 172.21.15.102:0/2300461763 >> v2:172.21.15.102:3300/0 conn(0x7f2a14159380 0x7f2a140c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:30.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.873+0000 7f2a11e00640 1 --2- 172.21.15.102:0/2300461763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14153180 0x7f2a14158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.873+0000 7f2a11400640 1 -- 172.21.15.102:0/2300461763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14153180 msgr2=0x7f2a14158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.873+0000 7f2a11400640 1 --2- 172.21.15.102:0/2300461763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14153180 0x7f2a14158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.873+0000 7f2a11400640 1 -- 172.21.15.102:0/2300461763 >> v1:172.21.15.102:6789/0 conn(0x7f2a140c75a0 legacy=0x7f2a14070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:30.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.873+0000 7f2a11400640 1 -- 172.21.15.102:0/2300461763 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1415c4d0 con 0x7f2a14159380 2024-04-20T21:23:30.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.874+0000 7f2a11400640 1 --2- 172.21.15.102:0/2300461763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14159380 0x7f2a140c5f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a000097a0 tx=0x7f2a0002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e879c0f580fbe02 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:30.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.874+0000 7f2a10a00640 1 -- 172.21.15.102:0/2300461763 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00035020 con 0x7f2a14159380 2024-04-20T21:23:30.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.874+0000 7f2a10a00640 1 -- 172.21.15.102:0/2300461763 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a00031810 con 0x7f2a14159380 2024-04-20T21:23:30.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.874+0000 7f2a10a00640 1 -- 172.21.15.102:0/2300461763 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00031be0 con 0x7f2a14159380 2024-04-20T21:23:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.875+0000 7f2a13200640 1 -- 172.21.15.102:0/2300461763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14159380 msgr2=0x7f2a140c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.875+0000 7f2a13200640 1 --2- 172.21.15.102:0/2300461763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14159380 0x7f2a140c5f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a000097a0 tx=0x7f2a0002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.875+0000 7f2a13200640 1 -- 172.21.15.102:0/2300461763 shutdown_connections 2024-04-20T21:23:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.875+0000 7f2a13200640 1 --2- 172.21.15.102:0/2300461763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14159380 0x7f2a140c5f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.875+0000 7f2a13200640 1 --2- 172.21.15.102:0/2300461763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14153180 0x7f2a14158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.875+0000 7f2a13200640 1 -- 172.21.15.102:0/2300461763 >> 172.21.15.102:0/2300461763 conn(0x7f2a140cc820 msgr2=0x7f2a140ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.875+0000 7f2a13200640 1 -- 172.21.15.102:0/2300461763 shutdown_connections 2024-04-20T21:23:30.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.875+0000 7f2a13200640 1 -- 172.21.15.102:0/2300461763 wait complete. 2024-04-20T21:23:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.877+0000 7f2a13200640 1 Processor -- start 2024-04-20T21:23:30.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.877+0000 7f2a13200640 1 -- start start 2024-04-20T21:23:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a13200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a140c75a0 0x7f2a14147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a13200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14153180 0x7f2a14157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a13200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14159380 0x7f2a14157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a13200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1415d290 con 0x7f2a14153180 2024-04-20T21:23:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a13200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1415ca90 con 0x7f2a140c75a0 2024-04-20T21:23:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a13200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2a140c6be0 con 0x7f2a14159380 2024-04-20T21:23:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a11400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14153180 0x7f2a14157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:30.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a11400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14153180 0x7f2a14157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55700/0 (socket says 172.21.15.102:55700) 2024-04-20T21:23:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a11400640 1 -- 172.21.15.102:0/2828903033 learned_addr learned my addr 172.21.15.102:0/2828903033 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a12800640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14159380 0x7f2a14157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.878+0000 7f2a11e00640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a140c75a0 0x7f2a14147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a11400640 1 -- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14159380 msgr2=0x7f2a14157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a11400640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14159380 0x7f2a14157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a11400640 1 -- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a140c75a0 msgr2=0x7f2a14147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a11400640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a140c75a0 0x7f2a14147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a11400640 1 -- 172.21.15.102:0/2828903033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1415ade0 con 0x7f2a14153180 2024-04-20T21:23:30.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a11400640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14153180 0x7f2a14157830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a0002bec0 tx=0x7f2a000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:30.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a0a000640 1 -- 172.21.15.102:0/2828903033 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00041070 con 0x7f2a14153180 2024-04-20T21:23:30.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a0a000640 1 -- 172.21.15.102:0/2828903033 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0003ad00 con 0x7f2a14153180 2024-04-20T21:23:30.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1415b0a0 con 0x7f2a14153180 2024-04-20T21:23:30.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.879+0000 7f2a0a000640 1 -- 172.21.15.102:0/2828903033 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00035040 con 0x7f2a14153180 2024-04-20T21:23:30.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.880+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a14156d50 con 0x7f2a14153180 2024-04-20T21:23:30.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.881+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a14074340 con 0x7f2a14153180 2024-04-20T21:23:30.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.882+0000 7f2a0a000640 1 -- 172.21.15.102:0/2828903033 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a00038020 con 0x7f2a14153180 2024-04-20T21:23:30.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.884+0000 7f2a0a000640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f29f406c8f0 0x7f29f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:30.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.884+0000 7f2a0a000640 1 -- 172.21.15.102:0/2828903033 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7f2a000add00 con 0x7f2a14153180 2024-04-20T21:23:30.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.884+0000 7f2a11e00640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f29f406c8f0 0x7f29f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:30.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.885+0000 7f2a11e00640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f29f406c8f0 0x7f29f406edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009aa0 tx=0x7f2a04009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:30.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:30.888+0000 7f2a0a000640 1 -- 172.21.15.102:0/2828903033 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a14074340 con 0x7f2a14153180 2024-04-20T21:23:31.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.131+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a1404fd60 con 0x7f2a14153180 2024-04-20T21:23:31.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.132+0000 7f2a0a000640 1 -- 172.21.15.102:0/2828903033 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6026 (secure 0 0 0) 0x7f2a0007b450 con 0x7f2a14153180 2024-04-20T21:23:31.131 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:31.131 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:30.537438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:31.132 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 27 2024-04-20T21:23:31.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f29f406c8f0 msgr2=0x7f29f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f29f406c8f0 0x7f29f406edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009aa0 tx=0x7f2a04009190 comp rx=0 tx=0).stop 2024-04-20T21:23:31.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14153180 msgr2=0x7f2a14157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14153180 0x7f2a14157830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a0002bec0 tx=0x7f2a000046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 shutdown_connections 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a14159380 0x7f2a14157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a14153180 0x7f2a14157830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f29f406c8f0 0x7f29f406edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 --2- 172.21.15.102:0/2828903033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a140c75a0 0x7f2a14147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 >> 172.21.15.102:0/2828903033 conn(0x7f2a140cc820 msgr2=0x7f2a1406f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 shutdown_connections 2024-04-20T21:23:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.134+0000 7f2a13200640 1 -- 172.21.15.102:0/2828903033 wait complete. 2024-04-20T21:23:31.153 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-04-20T21:23:31.153 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:23:31.153 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:31.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.350+0000 7fcffbc00640 1 Processor -- start 2024-04-20T21:23:31.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.350+0000 7fcffbc00640 1 -- start start 2024-04-20T21:23:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.351+0000 7fcffbc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcff414fa50 0x7fcff40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.351+0000 7fcffbc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff41536d0 con 0x7fcff4073a10 2024-04-20T21:23:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcffbc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff40c76c0 con 0x7fcff40c9eb0 2024-04-20T21:23:31.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcffbc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fcff40c7960 con 0x7fcff414fa50 2024-04-20T21:23:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcff9c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcff4073a10 0x7fcff4163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38382/0 (socket says 172.21.15.102:38382) 2024-04-20T21:23:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcff9c00640 1 -- 172.21.15.102:0/710245105 learned_addr learned my addr 172.21.15.102:0/710245105 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcff9200640 1 --2- 172.21.15.102:0/710245105 >> v2:172.21.15.193:3300/0 conn(0x7fcff414fa50 0x7fcff40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcff9200640 1 -- 172.21.15.102:0/710245105 >> v1:172.21.15.193:6789/0 conn(0x7fcff40c9eb0 legacy=0x7fcff40c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcff3e00640 1 -- 172.21.15.102:0/710245105 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2998552790 0 0) 0x7fcfec001490 con 0x7fcff4073a10 2024-04-20T21:23:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcff9200640 1 -- 172.21.15.102:0/710245105 >> v1:172.21.15.102:6789/0 conn(0x7fcff4073a10 legacy=0x7fcff4163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.352+0000 7fcff9200640 1 -- 172.21.15.102:0/710245105 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff40c3d90 con 0x7fcff414fa50 2024-04-20T21:23:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.353+0000 7fcff9200640 1 --2- 172.21.15.102:0/710245105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff414fa50 0x7fcff40c4830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcff40c7890 tx=0x7fcfe402b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f2c64041a1ba30 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.354+0000 7fcff3e00640 1 -- 172.21.15.102:0/710245105 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4034070 con 0x7fcff414fa50 2024-04-20T21:23:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.354+0000 7fcff3e00640 1 -- 172.21.15.102:0/710245105 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe4030890 con 0x7fcff414fa50 2024-04-20T21:23:31.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.354+0000 7fcff3e00640 1 -- 172.21.15.102:0/710245105 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4030c60 con 0x7fcff414fa50 2024-04-20T21:23:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.354+0000 7fcffbc00640 1 -- 172.21.15.102:0/710245105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff414fa50 msgr2=0x7fcff40c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.354+0000 7fcffbc00640 1 --2- 172.21.15.102:0/710245105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff414fa50 0x7fcff40c4830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcff40c7890 tx=0x7fcfe402b9d0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.355+0000 7fcffbc00640 1 -- 172.21.15.102:0/710245105 shutdown_connections 2024-04-20T21:23:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.355+0000 7fcffbc00640 1 --2- 172.21.15.102:0/710245105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff414fa50 0x7fcff40c4830 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.355+0000 7fcffbc00640 1 -- 172.21.15.102:0/710245105 >> 172.21.15.102:0/710245105 conn(0x7fcff40c9390 msgr2=0x7fcff40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.355+0000 7fcffbc00640 1 -- 172.21.15.102:0/710245105 shutdown_connections 2024-04-20T21:23:31.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.355+0000 7fcffbc00640 1 -- 172.21.15.102:0/710245105 wait complete. 2024-04-20T21:23:31.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.357+0000 7fcffbc00640 1 Processor -- start 2024-04-20T21:23:31.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.357+0000 7fcffbc00640 1 -- start start 2024-04-20T21:23:31.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcffbc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff4073a10 0x7fcff4142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcffbc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcff40c9eb0 0x7fcff41e2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcffbc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcff414fa50 0x7fcff41ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcffbc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcff40c4030 con 0x7fcff414fa50 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcffbc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcff40c7c00 con 0x7fcff4073a10 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcffbc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fcff40c4310 con 0x7fcff40c9eb0 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcff9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcff414fa50 0x7fcff41ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcff9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcff414fa50 0x7fcff41ef6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55704/0 (socket says 172.21.15.102:55704) 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcff9c00640 1 -- 172.21.15.102:0/2185332486 learned_addr learned my addr 172.21.15.102:0/2185332486 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcff9200640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff4073a10 0x7fcff4142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.358+0000 7fcfebe00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcff40c9eb0 0x7fcff41e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.359+0000 7fcff9200640 1 -- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcff40c9eb0 msgr2=0x7fcff41e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.359+0000 7fcff9200640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcff40c9eb0 0x7fcff41e2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.359+0000 7fcff9200640 1 -- 172.21.15.102:0/2185332486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcff414fa50 msgr2=0x7fcff41ef6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.359+0000 7fcff9200640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcff414fa50 0x7fcff41ef6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.359+0000 7fcff9200640 1 -- 172.21.15.102:0/2185332486 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff41e4130 con 0x7fcff4073a10 2024-04-20T21:23:31.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.359+0000 7fcff9c00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcff414fa50 0x7fcff41ef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:31.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.359+0000 7fcff9200640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff4073a10 0x7fcff4142a10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4033000 tx=0x7fcfe40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.359+0000 7fcff1600640 1 -- 172.21.15.102:0/2185332486 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4047070 con 0x7fcff4073a10 2024-04-20T21:23:31.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.360+0000 7fcff1600640 1 -- 172.21.15.102:0/2185332486 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe4039c10 con 0x7fcff4073a10 2024-04-20T21:23:31.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.360+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff41e4410 con 0x7fcff4073a10 2024-04-20T21:23:31.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.360+0000 7fcff1600640 1 -- 172.21.15.102:0/2185332486 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4034040 con 0x7fcff4073a10 2024-04-20T21:23:31.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.360+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff41e49d0 con 0x7fcff4073a10 2024-04-20T21:23:31.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.361+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff414f600 con 0x7fcff4073a10 2024-04-20T21:23:31.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.362+0000 7fcff1600640 1 -- 172.21.15.102:0/2185332486 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfe404a070 con 0x7fcff4073a10 2024-04-20T21:23:31.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.363+0000 7fcff1600640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcfd006c8f0 0x7fcfd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.363+0000 7fcff1600640 1 -- 172.21.15.102:0/2185332486 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7fcfe40add20 con 0x7fcff4073a10 2024-04-20T21:23:31.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.363+0000 7fcfebe00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcfd006c8f0 0x7fcfd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.364+0000 7fcfebe00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcfd006c8f0 0x7fcfd006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcff40cbde0 tx=0x7fcfdc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.368+0000 7fcff1600640 1 -- 172.21.15.102:0/2185332486 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcff414f600 con 0x7fcff4073a10 2024-04-20T21:23:31.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.609+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcff4074340 con 0x7fcff4073a10 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.610+0000 7fcff1600640 1 -- 172.21.15.102:0/2185332486 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6026 (secure 0 0 0) 0x7fcfe407b590 con 0x7fcff4073a10 2024-04-20T21:23:31.610 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:31.610 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:30.537438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:31.610 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 27 2024-04-20T21:23:31.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcfd006c8f0 msgr2=0x7fcfd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcfd006c8f0 0x7fcfd006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcff40cbde0 tx=0x7fcfdc009040 comp rx=0 tx=0).stop 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff4073a10 msgr2=0x7fcff4142a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff4073a10 0x7fcff4142a10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4033000 tx=0x7fcfe40046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 shutdown_connections 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcff414fa50 0x7fcff41ef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcff40c9eb0 0x7fcff41e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcfd006c8f0 0x7fcfd006edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 --2- 172.21.15.102:0/2185332486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcff4073a10 0x7fcff4142a10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 >> 172.21.15.102:0/2185332486 conn(0x7fcff40c9390 msgr2=0x7fcff40c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 shutdown_connections 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:31.613+0000 7fcffbc00640 1 -- 172.21.15.102:0/2185332486 wait complete. 2024-04-20T21:23:31.632 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 27, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5039": { "addr": "172.21.15.102:6835/1097388930", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 1097388930, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 1097388930, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5039, "incarnation": 27, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 44, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:23:30.537438+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5039 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:23:33.633 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:33.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.843+0000 7f8f8d600640 1 Processor -- start 2024-04-20T21:23:33.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.843+0000 7f8f8d600640 1 -- start start 2024-04-20T21:23:33.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.843+0000 7f8f8d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ee00 0x7f8f8806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.844+0000 7f8f8d600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8f8815f2b0 0x7f8f880710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.844+0000 7f8f8d600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f880c8f20 con 0x7f8f8815ea20 2024-04-20T21:23:33.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.844+0000 7f8f8d600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8806cb80 con 0x7f8f8815f2b0 2024-04-20T21:23:33.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.844+0000 7f8f8d600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8f8806ce60 con 0x7f8f8815ee00 2024-04-20T21:23:33.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.844+0000 7f8f87400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8f8815f2b0 0x7f8f880710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.844+0000 7f8f7fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ee00 0x7f8f8806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.844+0000 7f8f86a00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f8f8815ea20 0x7f8f880c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:39454/0 (socket says 172.21.15.102:39454) 2024-04-20T21:23:33.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.844+0000 7f8f86a00640 1 -- 172.21.15.102:0/605018819 learned_addr learned my addr 172.21.15.102:0/605018819 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:33.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.845+0000 7f8f86000640 1 -- 172.21.15.102:0/605018819 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596749535 0 0) 0x7f8f880c8f20 con 0x7f8f8815ea20 2024-04-20T21:23:33.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.845+0000 7f8f86000640 1 -- 172.21.15.102:0/605018819 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f6c003520 con 0x7f8f8815ea20 2024-04-20T21:23:33.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.845+0000 7f8f7fe00640 1 -- 172.21.15.102:0/605018819 >> v2:172.21.15.102:3300/0 conn(0x7f8f8815f2b0 msgr2=0x7f8f880710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:33.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.845+0000 7f8f7fe00640 1 --2- 172.21.15.102:0/605018819 >> v2:172.21.15.102:3300/0 conn(0x7f8f8815f2b0 0x7f8f880710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.845+0000 7f8f7fe00640 1 -- 172.21.15.102:0/605018819 >> v1:172.21.15.193:6789/0 conn(0x7f8f8815ea20 legacy=0x7f8f880c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:33.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.845+0000 7f8f7fe00640 1 -- 172.21.15.102:0/605018819 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8806d140 con 0x7f8f8815ee00 2024-04-20T21:23:33.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.845+0000 7f8f87400640 1 --2- 172.21.15.102:0/605018819 >> v2:172.21.15.102:3300/0 conn(0x7f8f8815f2b0 0x7f8f880710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:33.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.846+0000 7f8f7fe00640 1 --2- 172.21.15.102:0/605018819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ee00 0x7f8f8806d940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f700097a0 tx=0x7f8f7002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=982b7bce3ceedcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.846+0000 7f8f86000640 1 -- 172.21.15.102:0/605018819 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70034070 con 0x7f8f8815ee00 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.846+0000 7f8f86000640 1 -- 172.21.15.102:0/605018819 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f70030800 con 0x7f8f8815ee00 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.846+0000 7f8f86000640 1 -- 172.21.15.102:0/605018819 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70030bb0 con 0x7f8f8815ee00 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.847+0000 7f8f8d600640 1 -- 172.21.15.102:0/605018819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ee00 msgr2=0x7f8f8806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.847+0000 7f8f8d600640 1 --2- 172.21.15.102:0/605018819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ee00 0x7f8f8806d940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f700097a0 tx=0x7f8f7002b940 comp rx=0 tx=0).stop 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.847+0000 7f8f8d600640 1 -- 172.21.15.102:0/605018819 shutdown_connections 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.847+0000 7f8f8d600640 1 --2- 172.21.15.102:0/605018819 >> v2:172.21.15.102:3300/0 conn(0x7f8f8815f2b0 0x7f8f880710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.847+0000 7f8f8d600640 1 --2- 172.21.15.102:0/605018819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ee00 0x7f8f8806d940 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.847+0000 7f8f8d600640 1 -- 172.21.15.102:0/605018819 >> 172.21.15.102:0/605018819 conn(0x7f8f880c9390 msgr2=0x7f8f880c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.847+0000 7f8f8d600640 1 -- 172.21.15.102:0/605018819 shutdown_connections 2024-04-20T21:23:33.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.847+0000 7f8f8d600640 1 -- 172.21.15.102:0/605018819 wait complete. 2024-04-20T21:23:33.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.849+0000 7f8f8d600640 1 Processor -- start 2024-04-20T21:23:33.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.849+0000 7f8f8d600640 1 -- start start 2024-04-20T21:23:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f8d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ea20 0x7f8f881e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f8d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8815ee00 0x7f8f881473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f8d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f8815f2b0 0x7f8f881e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f8d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8f880c8c80 con 0x7f8f8815f2b0 2024-04-20T21:23:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f8d600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f880c8b00 con 0x7f8f8815ee00 2024-04-20T21:23:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f8d600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8f88154e80 con 0x7f8f8815ea20 2024-04-20T21:23:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f8815f2b0 0x7f8f881e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f8815f2b0 0x7f8f881e5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55720/0 (socket says 172.21.15.102:55720) 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 -- 172.21.15.102:0/2380486211 learned_addr learned my addr 172.21.15.102:0/2380486211 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f7fe00640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8815ee00 0x7f8f881473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f86a00640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ea20 0x7f8f881e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 -- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ea20 msgr2=0x7f8f881e4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ea20 0x7f8f881e4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 -- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8815ee00 msgr2=0x7f8f881473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8815ee00 0x7f8f881473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 -- 172.21.15.102:0/2380486211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f881e5010 con 0x7f8f8815f2b0 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f87400640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f8815f2b0 0x7f8f881e5a60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000efb0 tx=0x7f8f8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f7ea00640 1 -- 172.21.15.102:0/2380486211 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80018020 con 0x7f8f8815f2b0 2024-04-20T21:23:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f7ea00640 1 -- 172.21.15.102:0/2380486211 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f80014480 con 0x7f8f8815f2b0 2024-04-20T21:23:33.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f881e52a0 con 0x7f8f8815f2b0 2024-04-20T21:23:33.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.851+0000 7f8f7ea00640 1 -- 172.21.15.102:0/2380486211 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80014850 con 0x7f8f8815f2b0 2024-04-20T21:23:33.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.852+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f881f1610 con 0x7f8f8815f2b0 2024-04-20T21:23:33.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.853+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f880c6f80 con 0x7f8f8815f2b0 2024-04-20T21:23:33.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.854+0000 7f8f7ea00640 1 -- 172.21.15.102:0/2380486211 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f80005d10 con 0x7f8f8815f2b0 2024-04-20T21:23:33.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.855+0000 7f8f7ea00640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8f6406c8f0 0x7f8f6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.855+0000 7f8f7ea00640 1 -- 172.21.15.102:0/2380486211 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7f8f8008cd10 con 0x7f8f8815f2b0 2024-04-20T21:23:33.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.855+0000 7f8f86a00640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8f6406c8f0 0x7f8f6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.856+0000 7f8f86a00640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8f6406c8f0 0x7f8f6406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005880 tx=0x7f8f78009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:33.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:33.860+0000 7f8f7ea00640 1 -- 172.21.15.102:0/2380486211 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8f880c6f80 con 0x7f8f8815f2b0 2024-04-20T21:23:34.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.099+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f880c7420 con 0x7f8f8815f2b0 2024-04-20T21:23:34.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.100+0000 7f8f7ea00640 1 -- 172.21.15.102:0/2380486211 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6026 (secure 0 0 0) 0x7f8f80092020 con 0x7f8f8815f2b0 2024-04-20T21:23:34.098 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:34.098 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:30.537438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:34.099 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 27 2024-04-20T21:23:34.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.102+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8f6406c8f0 msgr2=0x7f8f6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:34.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.102+0000 7f8f8d600640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8f6406c8f0 0x7f8f6406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005880 tx=0x7f8f78009040 comp rx=0 tx=0).stop 2024-04-20T21:23:34.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.102+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f8815f2b0 msgr2=0x7f8f881e5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:34.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.102+0000 7f8f8d600640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f8815f2b0 0x7f8f881e5a60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000efb0 tx=0x7f8f8000c490 comp rx=0 tx=0).stop 2024-04-20T21:23:34.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.103+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 shutdown_connections 2024-04-20T21:23:34.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.103+0000 7f8f8d600640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f8815f2b0 0x7f8f881e5a60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:34.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.103+0000 7f8f8d600640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8f6406c8f0 0x7f8f6406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:34.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.103+0000 7f8f8d600640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8815ee00 0x7f8f881473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:34.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.103+0000 7f8f8d600640 1 --2- 172.21.15.102:0/2380486211 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f8815ea20 0x7f8f881e4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:34.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.103+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 >> 172.21.15.102:0/2380486211 conn(0x7f8f880c9390 msgr2=0x7f8f8806e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:34.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.103+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 shutdown_connections 2024-04-20T21:23:34.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:34.103+0000 7f8f8d600640 1 -- 172.21.15.102:0/2380486211 wait complete. 2024-04-20T21:23:34.119 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:23:36.121 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:36.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.336+0000 7f1dbf000640 1 Processor -- start 2024-04-20T21:23:36.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.336+0000 7f1dbf000640 1 -- start start 2024-04-20T21:23:36.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.338+0000 7f1dbf000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1db80c80d0 0x7f1db806b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:36.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.338+0000 7f1dbf000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db8159420 con 0x7f1db806be80 2024-04-20T21:23:36.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.338+0000 7f1dbf000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db8159700 con 0x7f1db813efd0 2024-04-20T21:23:36.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.338+0000 7f1dbf000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1db815c7e0 con 0x7f1db80c80d0 2024-04-20T21:23:36.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.339+0000 7f1db7400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1db80c80d0 0x7f1db806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.339+0000 7f1db7e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f1db813efd0 0x7f1db8067fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:39458/0 (socket says 172.21.15.102:39458) 2024-04-20T21:23:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.339+0000 7f1db7e00640 1 -- 172.21.15.102:0/297676693 learned_addr learned my addr 172.21.15.102:0/297676693 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.339+0000 7f1db7400640 1 -- 172.21.15.102:0/297676693 >> v1:172.21.15.193:6789/0 conn(0x7f1db813efd0 legacy=0x7f1db8067fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.339+0000 7f1db7400640 1 -- 172.21.15.102:0/297676693 >> v1:172.21.15.102:6789/0 conn(0x7f1db806be80 legacy=0x7f1db815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.339+0000 7f1db7400640 1 -- 172.21.15.102:0/297676693 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db81599e0 con 0x7f1db80c80d0 2024-04-20T21:23:36.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.340+0000 7f1db7400640 1 --2- 172.21.15.102:0/297676693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 0x7f1db806b940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1db815c960 tx=0x7f1da402b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae74dab839e96b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:36.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.341+0000 7f1db6a00640 1 -- 172.21.15.102:0/297676693 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4034070 con 0x7f1db80c80d0 2024-04-20T21:23:36.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.341+0000 7f1db6a00640 1 -- 172.21.15.102:0/297676693 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da4030890 con 0x7f1db80c80d0 2024-04-20T21:23:36.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.341+0000 7f1db6a00640 1 -- 172.21.15.102:0/297676693 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4030c60 con 0x7f1db80c80d0 2024-04-20T21:23:36.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.342+0000 7f1dbf000640 1 -- 172.21.15.102:0/297676693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 msgr2=0x7f1db806b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:36.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.342+0000 7f1dbf000640 1 --2- 172.21.15.102:0/297676693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 0x7f1db806b940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1db815c960 tx=0x7f1da402b9d0 comp rx=0 tx=0).stop 2024-04-20T21:23:36.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.342+0000 7f1dbf000640 1 -- 172.21.15.102:0/297676693 shutdown_connections 2024-04-20T21:23:36.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.342+0000 7f1dbf000640 1 --2- 172.21.15.102:0/297676693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 0x7f1db806b940 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:36.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.342+0000 7f1dbf000640 1 -- 172.21.15.102:0/297676693 >> 172.21.15.102:0/297676693 conn(0x7f1db80ca050 msgr2=0x7f1db80ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:36.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.342+0000 7f1dbf000640 1 -- 172.21.15.102:0/297676693 shutdown_connections 2024-04-20T21:23:36.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.342+0000 7f1dbf000640 1 -- 172.21.15.102:0/297676693 wait complete. 2024-04-20T21:23:36.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.344+0000 7f1dbf000640 1 Processor -- start 2024-04-20T21:23:36.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.344+0000 7f1dbf000640 1 -- start start 2024-04-20T21:23:36.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbf000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1db806be80 0x7f1db813c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:36.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbf000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 0x7f1db80728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:36.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbf000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db813efd0 0x7f1db81f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:36.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbf000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81590c0 con 0x7f1db813efd0 2024-04-20T21:23:36.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbf000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1db8159fa0 con 0x7f1db80c80d0 2024-04-20T21:23:36.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbf000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1db813f3b0 con 0x7f1db806be80 2024-04-20T21:23:36.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db813efd0 0x7f1db81f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:36.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db813efd0 0x7f1db81f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55728/0 (socket says 172.21.15.102:55728) 2024-04-20T21:23:36.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.345+0000 7f1dbd000640 1 -- 172.21.15.102:0/4226599526 learned_addr learned my addr 172.21.15.102:0/4226599526 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:36.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1db7400640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 0x7f1db80728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:36.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1db7e00640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1db806be80 0x7f1db813c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:36.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dbd000640 1 -- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1db806be80 msgr2=0x7f1db813c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:36.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dbd000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1db806be80 0x7f1db813c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dbd000640 1 -- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 msgr2=0x7f1db80728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dbd000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 0x7f1db80728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dbd000640 1 -- 172.21.15.102:0/4226599526 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db8136c60 con 0x7f1db813efd0 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dbd000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db813efd0 0x7f1db81f9010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c760 tx=0x7f1db000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dafe00640 1 -- 172.21.15.102:0/4226599526 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0003b70 con 0x7f1db813efd0 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dafe00640 1 -- 172.21.15.102:0/4226599526 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0003d30 con 0x7f1db813efd0 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dafe00640 1 -- 172.21.15.102:0/4226599526 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db000aa30 con 0x7f1db813efd0 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.346+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db80cb4a0 con 0x7f1db813efd0 2024-04-20T21:23:36.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.347+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db8071e10 con 0x7f1db813efd0 2024-04-20T21:23:36.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.348+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db8068c30 con 0x7f1db813efd0 2024-04-20T21:23:36.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.349+0000 7f1dafe00640 1 -- 172.21.15.102:0/4226599526 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db000abd0 con 0x7f1db813efd0 2024-04-20T21:23:36.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.350+0000 7f1dafe00640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d9806c8f0 0x7f1d9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:36.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.350+0000 7f1dafe00640 1 -- 172.21.15.102:0/4226599526 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7f1db008bf80 con 0x7f1db813efd0 2024-04-20T21:23:36.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.351+0000 7f1db7e00640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d9806c8f0 0x7f1d9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:36.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.351+0000 7f1db7e00640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d9806c8f0 0x7f1d9806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005880 tx=0x7f1da8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:36.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.355+0000 7f1dafe00640 1 -- 172.21.15.102:0/4226599526 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1db0059dc0 con 0x7f1db813efd0 2024-04-20T21:23:36.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.603+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1db81589d0 con 0x7f1db813efd0 2024-04-20T21:23:36.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.604+0000 7f1dafe00640 1 -- 172.21.15.102:0/4226599526 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6026 (secure 0 0 0) 0x7f1db00596c0 con 0x7f1db813efd0 2024-04-20T21:23:36.604 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:36.604 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:30.537438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:36.604 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 27 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d9806c8f0 msgr2=0x7f1d9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d9806c8f0 0x7f1d9806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005880 tx=0x7f1da8009040 comp rx=0 tx=0).stop 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db813efd0 msgr2=0x7f1db81f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db813efd0 0x7f1db81f9010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c760 tx=0x7f1db000cc20 comp rx=0 tx=0).stop 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 shutdown_connections 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1db813efd0 0x7f1db81f9010 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1d9806c8f0 0x7f1d9806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1db80c80d0 0x7f1db80728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 --2- 172.21.15.102:0/4226599526 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1db806be80 0x7f1db813c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 >> 172.21.15.102:0/4226599526 conn(0x7f1db80ca050 msgr2=0x7f1db80c9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 shutdown_connections 2024-04-20T21:23:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:36.607+0000 7f1dbf000640 1 -- 172.21.15.102:0/4226599526 wait complete. 2024-04-20T21:23:36.627 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:23:38.628 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:38.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.829+0000 7f827cc00640 1 Processor -- start 2024-04-20T21:23:38.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.829+0000 7f827cc00640 1 -- start start 2024-04-20T21:23:38.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.830+0000 7f827cc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:38.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.830+0000 7f827cc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8278159380 0x7f82780c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.830+0000 7f827cc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f82780c75a0 0x7f82780c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.830+0000 7f827cc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8278153560 con 0x7f8278159380 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.830+0000 7f827cc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f82780c7a70 con 0x7f82780c75a0 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.830+0000 7f827cc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f82780c7d30 con 0x7f8278153180 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f82780c75a0 0x7f82780c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:49754/0 (socket says 172.21.15.102:49754) 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277400640 1 -- 172.21.15.102:0/3125593845 learned_addr learned my addr 172.21.15.102:0/3125593845 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8276a00640 1 --2- 172.21.15.102:0/3125593845 >> v2:172.21.15.102:3300/0 conn(0x7f8278159380 0x7f82780c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277400640 1 -- 172.21.15.102:0/3125593845 >> v2:172.21.15.193:3300/0 conn(0x7f82780c75a0 msgr2=0x7f82780c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277400640 1 --2- 172.21.15.102:0/3125593845 >> v2:172.21.15.193:3300/0 conn(0x7f82780c75a0 0x7f82780c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277400640 1 -- 172.21.15.102:0/3125593845 >> v2:172.21.15.102:3300/0 conn(0x7f8278159380 msgr2=0x7f82780c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277400640 1 --2- 172.21.15.102:0/3125593845 >> v2:172.21.15.102:3300/0 conn(0x7f8278159380 0x7f82780c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:38.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.831+0000 7f8277400640 1 -- 172.21.15.102:0/3125593845 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827815c4d0 con 0x7f8278153180 2024-04-20T21:23:38.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.832+0000 7f8277400640 1 --2- 172.21.15.102:0/3125593845 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278158e40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82780c72b0 tx=0x7f826002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a042974c66ae518b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:38.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.833+0000 7f8276000640 1 -- 172.21.15.102:0/3125593845 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260035020 con 0x7f8278153180 2024-04-20T21:23:38.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.833+0000 7f8276000640 1 -- 172.21.15.102:0/3125593845 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8260031870 con 0x7f8278153180 2024-04-20T21:23:38.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.833+0000 7f8276000640 1 -- 172.21.15.102:0/3125593845 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260031c40 con 0x7f8278153180 2024-04-20T21:23:38.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 -- 172.21.15.102:0/3125593845 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 msgr2=0x7f8278158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:38.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 --2- 172.21.15.102:0/3125593845 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278158e40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82780c72b0 tx=0x7f826002b910 comp rx=0 tx=0).stop 2024-04-20T21:23:38.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 -- 172.21.15.102:0/3125593845 shutdown_connections 2024-04-20T21:23:38.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 --2- 172.21.15.102:0/3125593845 >> v2:172.21.15.193:3300/0 conn(0x7f82780c75a0 0x7f82780c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:38.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 --2- 172.21.15.102:0/3125593845 >> v2:172.21.15.102:3300/0 conn(0x7f8278159380 0x7f82780c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:38.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 --2- 172.21.15.102:0/3125593845 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278158e40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:38.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 -- 172.21.15.102:0/3125593845 >> 172.21.15.102:0/3125593845 conn(0x7f82780cc820 msgr2=0x7f82780ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:38.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 -- 172.21.15.102:0/3125593845 shutdown_connections 2024-04-20T21:23:38.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.834+0000 7f827cc00640 1 -- 172.21.15.102:0/3125593845 wait complete. 2024-04-20T21:23:38.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.835+0000 7f827cc00640 1 Processor -- start 2024-04-20T21:23:38.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.836+0000 7f827cc00640 1 -- start start 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.836+0000 7f827cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82780c75a0 0x7f82780cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f827cc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f827cc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8278159380 0x7f82781464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f827cc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f827815d290 con 0x7f82780c75a0 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f827cc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f827815ca90 con 0x7f8278159380 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f827cc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f82780c6be0 con 0x7f8278153180 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82780c75a0 0x7f82780cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82780c75a0 0x7f82780cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47248/0 (socket says 172.21.15.102:47248) 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277400640 1 -- 172.21.15.102:0/4226157436 learned_addr learned my addr 172.21.15.102:0/4226157436 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8276a00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277e00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8278159380 0x7f82781464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277400640 1 -- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 msgr2=0x7f8278145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:38.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277400640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:38.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277400640 1 -- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8278159380 msgr2=0x7f82781464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:38.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277400640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8278159380 0x7f82781464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:38.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.837+0000 7f8277400640 1 -- 172.21.15.102:0/4226157436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82781584a0 con 0x7f82780c75a0 2024-04-20T21:23:38.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.838+0000 7f8277400640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82780c75a0 0x7f82780cba10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8260000c00 tx=0x7f82600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:38.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.838+0000 7f826f400640 1 -- 172.21.15.102:0/4226157436 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260040070 con 0x7f82780c75a0 2024-04-20T21:23:38.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.838+0000 7f826f400640 1 -- 172.21.15.102:0/4226157436 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8260031e10 con 0x7f82780c75a0 2024-04-20T21:23:38.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.838+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8278158780 con 0x7f82780c75a0 2024-04-20T21:23:38.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.838+0000 7f826f400640 1 -- 172.21.15.102:0/4226157436 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260035040 con 0x7f82780c75a0 2024-04-20T21:23:38.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.838+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8278158d40 con 0x7f82780c75a0 2024-04-20T21:23:38.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.839+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827804fd60 con 0x7f82780c75a0 2024-04-20T21:23:38.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.840+0000 7f826f400640 1 -- 172.21.15.102:0/4226157436 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f826004b060 con 0x7f82780c75a0 2024-04-20T21:23:38.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.842+0000 7f826f400640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f825406c920 0x7f825406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:38.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.842+0000 7f826f400640 1 -- 172.21.15.102:0/4226157436 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7f82600adb50 con 0x7f82780c75a0 2024-04-20T21:23:38.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.842+0000 7f8276a00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f825406c920 0x7f825406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:38.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.843+0000 7f8276a00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f825406c920 0x7f825406ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82780c7f00 tx=0x7f8268009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:38.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:38.846+0000 7f826f400640 1 -- 172.21.15.102:0/4226157436 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f826007b460 con 0x7f82780c75a0 2024-04-20T21:23:39.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.070+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82780c3690 con 0x7f82780c75a0 2024-04-20T21:23:39.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.072+0000 7f826f400640 1 -- 172.21.15.102:0/4226157436 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6030 (secure 0 0 0) 0x7f826007f760 con 0x7f82780c75a0 2024-04-20T21:23:39.071 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:39.072 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:37.910969+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:39.072 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 28 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f825406c920 msgr2=0x7f825406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f825406c920 0x7f825406ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82780c7f00 tx=0x7f8268009190 comp rx=0 tx=0).stop 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82780c75a0 msgr2=0x7f82780cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82780c75a0 0x7f82780cba10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8260000c00 tx=0x7f82600046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 shutdown_connections 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f825406c920 0x7f825406ede0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8278159380 0x7f82781464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8278153180 0x7f8278145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:39.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 --2- 172.21.15.102:0/4226157436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82780c75a0 0x7f82780cba10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:39.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.074+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 >> 172.21.15.102:0/4226157436 conn(0x7f82780cc820 msgr2=0x7f82780ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:39.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.075+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 shutdown_connections 2024-04-20T21:23:39.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:39.075+0000 7f827cc00640 1 -- 172.21.15.102:0/4226157436 wait complete. 2024-04-20T21:23:39.092 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:23:41.092 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:41.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.300+0000 7fea71200640 1 Processor -- start 2024-04-20T21:23:41.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.301+0000 7fea71200640 1 -- start start 2024-04-20T21:23:41.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.302+0000 7fea71200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.302+0000 7fea71200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fea6c073c30 0x7fea6c0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.302+0000 7fea71200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea6c0c6ec0 con 0x7fea6c13bc80 2024-04-20T21:23:41.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.302+0000 7fea71200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fea6c0c6a30 con 0x7fea6c073c30 2024-04-20T21:23:41.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.302+0000 7fea71200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c0c8d00 con 0x7fea6c0737f0 2024-04-20T21:23:41.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.302+0000 7fea6a800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fea6c073c30 0x7fea6c0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.302+0000 7fea69400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.303+0000 7fea69e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fea6c13bc80 0x7fea6c0c84e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:42180/0 (socket says 172.21.15.102:42180) 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.303+0000 7fea69e00640 1 -- 172.21.15.102:0/2542891035 learned_addr learned my addr 172.21.15.102:0/2542891035 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.303+0000 7fea69400640 1 -- 172.21.15.102:0/2542891035 >> v2:172.21.15.193:3300/0 conn(0x7fea6c073c30 msgr2=0x7fea6c0c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.303+0000 7fea69400640 1 --2- 172.21.15.102:0/2542891035 >> v2:172.21.15.193:3300/0 conn(0x7fea6c073c30 0x7fea6c0c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.303+0000 7fea69400640 1 -- 172.21.15.102:0/2542891035 >> v1:172.21.15.193:6789/0 conn(0x7fea6c13bc80 legacy=0x7fea6c0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.303+0000 7fea68a00640 1 -- 172.21.15.102:0/2542891035 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2583713540 0 0) 0x7fea6c0c6ec0 con 0x7fea6c13bc80 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.303+0000 7fea69400640 1 -- 172.21.15.102:0/2542891035 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c0c8fc0 con 0x7fea6c0737f0 2024-04-20T21:23:41.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.303+0000 7fea6a800640 1 --2- 172.21.15.102:0/2542891035 >> v2:172.21.15.193:3300/0 conn(0x7fea6c073c30 0x7fea6c0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:41.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.304+0000 7fea69400640 1 --2- 172.21.15.102:0/2542891035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c159020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea540097a0 tx=0x7fea5402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bd72a0fe6b9194e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:41.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.304+0000 7fea68a00640 1 -- 172.21.15.102:0/2542891035 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54035020 con 0x7fea6c0737f0 2024-04-20T21:23:41.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.304+0000 7fea68a00640 1 -- 172.21.15.102:0/2542891035 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea54031810 con 0x7fea6c0737f0 2024-04-20T21:23:41.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.304+0000 7fea68a00640 1 -- 172.21.15.102:0/2542891035 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54031be0 con 0x7fea6c0737f0 2024-04-20T21:23:41.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.305+0000 7fea71200640 1 -- 172.21.15.102:0/2542891035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 msgr2=0x7fea6c159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.305+0000 7fea71200640 1 --2- 172.21.15.102:0/2542891035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c159020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea540097a0 tx=0x7fea5402b8b0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.305+0000 7fea71200640 1 -- 172.21.15.102:0/2542891035 shutdown_connections 2024-04-20T21:23:41.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.305+0000 7fea71200640 1 --2- 172.21.15.102:0/2542891035 >> v2:172.21.15.193:3300/0 conn(0x7fea6c073c30 0x7fea6c0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.305+0000 7fea71200640 1 --2- 172.21.15.102:0/2542891035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c159020 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.305+0000 7fea71200640 1 -- 172.21.15.102:0/2542891035 >> 172.21.15.102:0/2542891035 conn(0x7fea6c06ebf0 msgr2=0x7fea6c06efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:41.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.305+0000 7fea71200640 1 -- 172.21.15.102:0/2542891035 shutdown_connections 2024-04-20T21:23:41.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.305+0000 7fea71200640 1 -- 172.21.15.102:0/2542891035 wait complete. 2024-04-20T21:23:41.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.307+0000 7fea71200640 1 Processor -- start 2024-04-20T21:23:41.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.307+0000 7fea71200640 1 -- start start 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea71200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea71200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea6c073c30 0x7fea6c0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea71200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6c13bc80 0x7fea6c1f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea71200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c0c66c0 con 0x7fea6c073c30 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea71200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c0c63e0 con 0x7fea6c13bc80 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea71200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c159560 con 0x7fea6c0737f0 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea6c073c30 0x7fea6c0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:49782/0 (socket says 172.21.15.102:49782) 2024-04-20T21:23:41.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69e00640 1 -- 172.21.15.102:0/2736763014 learned_addr learned my addr 172.21.15.102:0/2736763014 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea6a800640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6c13bc80 0x7fea6c1f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea6c073c30 0x7fea6c0ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47252/0 (socket says 172.21.15.102:47252) 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69e00640 1 -- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6c13bc80 msgr2=0x7fea6c1f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69e00640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6c13bc80 0x7fea6c1f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69e00640 1 -- 172.21.15.102:0/2736763014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea6c073c30 msgr2=0x7fea6c0ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69e00640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea6c073c30 0x7fea6c0ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea69e00640 1 -- 172.21.15.102:0/2736763014 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c14f0a0 con 0x7fea6c0737f0 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.309+0000 7fea6a800640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6c13bc80 0x7fea6c1f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:41.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.310+0000 7fea69e00640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c0ce7f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00c430 tx=0x7fea5c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:41.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.310+0000 7fea62000640 1 -- 172.21.15.102:0/2736763014 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0074e0 con 0x7fea6c0737f0 2024-04-20T21:23:41.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.310+0000 7fea62000640 1 -- 172.21.15.102:0/2736763014 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5c0047e0 con 0x7fea6c0737f0 2024-04-20T21:23:41.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.310+0000 7fea62000640 1 -- 172.21.15.102:0/2736763014 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c004bd0 con 0x7fea6c0737f0 2024-04-20T21:23:41.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.310+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6c0cd690 con 0x7fea6c0737f0 2024-04-20T21:23:41.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.310+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6c0cdc50 con 0x7fea6c0737f0 2024-04-20T21:23:41.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.311+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6c074340 con 0x7fea6c0737f0 2024-04-20T21:23:41.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.312+0000 7fea62000640 1 -- 172.21.15.102:0/2736763014 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea5c011070 con 0x7fea6c0737f0 2024-04-20T21:23:41.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.313+0000 7fea62000640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea4806c920 0x7fea4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.313+0000 7fea62000640 1 -- 172.21.15.102:0/2736763014 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7fea5c08bbf0 con 0x7fea6c0737f0 2024-04-20T21:23:41.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.314+0000 7fea69400640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea4806c920 0x7fea4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.314+0000 7fea69400640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea4806c920 0x7fea4806ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea54036000 tx=0x7fea54008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:41.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.318+0000 7fea62000640 1 -- 172.21.15.102:0/2736763014 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fea5c059a20 con 0x7fea6c0737f0 2024-04-20T21:23:41.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.561+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea6c136550 con 0x7fea6c0737f0 2024-04-20T21:23:41.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.562+0000 7fea62000640 1 -- 172.21.15.102:0/2736763014 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6030 (secure 0 0 0) 0x7fea5c059320 con 0x7fea6c0737f0 2024-04-20T21:23:41.562 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:41.562 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:37.910969+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:41.562 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 28 2024-04-20T21:23:41.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea4806c920 msgr2=0x7fea4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea4806c920 0x7fea4806ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea54036000 tx=0x7fea54008030 comp rx=0 tx=0).stop 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 msgr2=0x7fea6c0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c0ce7f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00c430 tx=0x7fea5c00c830 comp rx=0 tx=0).stop 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 shutdown_connections 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea4806c920 0x7fea4806ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6c13bc80 0x7fea6c1f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea6c073c30 0x7fea6c0ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 --2- 172.21.15.102:0/2736763014 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea6c0737f0 0x7fea6c0ce7f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 >> 172.21.15.102:0/2736763014 conn(0x7fea6c06ebf0 msgr2=0x7fea6c157000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 shutdown_connections 2024-04-20T21:23:41.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:41.565+0000 7fea71200640 1 -- 172.21.15.102:0/2736763014 wait complete. 2024-04-20T21:23:41.584 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:23:43.585 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:43.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.795+0000 7f1ab9200640 1 Processor -- start 2024-04-20T21:23:43.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.795+0000 7f1ab9200640 1 -- start start 2024-04-20T21:23:43.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.796+0000 7f1ab9200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 0x7f1ab40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:43.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab9200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1ab40c9eb0 0x7f1ab4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:43.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab9200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1ab4073a10 0x7f1ab40c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:43.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab9200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab4159500 con 0x7f1ab4073a10 2024-04-20T21:23:43.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab9200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab41536d0 con 0x7f1ab40c9eb0 2024-04-20T21:23:43.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab9200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab40c76c0 con 0x7f1ab414fa50 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab2800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1ab4073a10 0x7f1ab40c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab2800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1ab4073a10 0x7f1ab40c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47262/0 (socket says 172.21.15.102:47262) 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab2800640 1 -- 172.21.15.102:0/155464177 learned_addr learned my addr 172.21.15.102:0/155464177 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab1e00640 1 --2- 172.21.15.102:0/155464177 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 0x7f1ab40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab1400640 1 --2- 172.21.15.102:0/155464177 >> v2:172.21.15.193:3300/0 conn(0x7f1ab40c9eb0 0x7f1ab4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab2800640 1 -- 172.21.15.102:0/155464177 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 msgr2=0x7f1ab40c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab2800640 1 --2- 172.21.15.102:0/155464177 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 0x7f1ab40c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab2800640 1 -- 172.21.15.102:0/155464177 >> v2:172.21.15.193:3300/0 conn(0x7f1ab40c9eb0 msgr2=0x7f1ab4158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:43.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab2800640 1 --2- 172.21.15.102:0/155464177 >> v2:172.21.15.193:3300/0 conn(0x7f1ab40c9eb0 0x7f1ab4158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:43.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.797+0000 7f1ab2800640 1 -- 172.21.15.102:0/155464177 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab40c7960 con 0x7f1ab4073a10 2024-04-20T21:23:43.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.799+0000 7f1ab2800640 1 --2- 172.21.15.102:0/155464177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab40c6170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4069c80 tx=0x7f1aac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9f6c92daa60119 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:43.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.799+0000 7f1ab0a00640 1 -- 172.21.15.102:0/155464177 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac02cb00 con 0x7f1ab4073a10 2024-04-20T21:23:43.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.799+0000 7f1ab0a00640 1 -- 172.21.15.102:0/155464177 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac02cca0 con 0x7f1ab4073a10 2024-04-20T21:23:43.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.799+0000 7f1ab0a00640 1 -- 172.21.15.102:0/155464177 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac034610 con 0x7f1ab4073a10 2024-04-20T21:23:43.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 -- 172.21.15.102:0/155464177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 msgr2=0x7f1ab40c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:43.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 --2- 172.21.15.102:0/155464177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab40c6170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4069c80 tx=0x7f1aac02bf60 comp rx=0 tx=0).stop 2024-04-20T21:23:43.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 -- 172.21.15.102:0/155464177 shutdown_connections 2024-04-20T21:23:43.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 --2- 172.21.15.102:0/155464177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab40c6170 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:43.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 --2- 172.21.15.102:0/155464177 >> v2:172.21.15.193:3300/0 conn(0x7f1ab40c9eb0 0x7f1ab4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:43.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 --2- 172.21.15.102:0/155464177 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 0x7f1ab40c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:43.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 -- 172.21.15.102:0/155464177 >> 172.21.15.102:0/155464177 conn(0x7f1ab40c9390 msgr2=0x7f1ab40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:43.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 -- 172.21.15.102:0/155464177 shutdown_connections 2024-04-20T21:23:43.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.800+0000 7f1ab9200640 1 -- 172.21.15.102:0/155464177 wait complete. 2024-04-20T21:23:43.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.802+0000 7f1ab9200640 1 Processor -- start 2024-04-20T21:23:43.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.802+0000 7f1ab9200640 1 -- start start 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab4070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab9200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ab40c9eb0 0x7f1ab40726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab9200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 0x7f1ab414caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab9200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40c7c00 con 0x7f1ab4073a10 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab9200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40c3d90 con 0x7f1ab40c9eb0 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab9200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab40c4030 con 0x7f1ab414fa50 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab4070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab4070df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47272/0 (socket says 172.21.15.102:47272) 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1e00640 1 -- 172.21.15.102:0/4237835993 learned_addr learned my addr 172.21.15.102:0/4237835993 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab2800640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 0x7f1ab414caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1400640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ab40c9eb0 0x7f1ab40726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1e00640 1 -- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 msgr2=0x7f1ab414caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1e00640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 0x7f1ab414caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1e00640 1 -- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ab40c9eb0 msgr2=0x7f1ab40726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:43.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1e00640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ab40c9eb0 0x7f1ab40726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:43.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.803+0000 7f1ab1e00640 1 -- 172.21.15.102:0/4237835993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab41f4640 con 0x7f1ab4073a10 2024-04-20T21:23:43.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.804+0000 7f1ab1e00640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab4070df0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400e9b0 tx=0x7f1aa400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:43.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.804+0000 7f1aaa000640 1 -- 172.21.15.102:0/4237835993 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4017070 con 0x7f1ab4073a10 2024-04-20T21:23:43.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.804+0000 7f1aaa000640 1 -- 172.21.15.102:0/4237835993 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa400c9a0 con 0x7f1ab4073a10 2024-04-20T21:23:43.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.804+0000 7f1aaa000640 1 -- 172.21.15.102:0/4237835993 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400cd90 con 0x7f1ab4073a10 2024-04-20T21:23:43.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.804+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab41f4920 con 0x7f1ab4073a10 2024-04-20T21:23:43.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.804+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab41f4ee0 con 0x7f1ab4073a10 2024-04-20T21:23:43.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.805+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab4074340 con 0x7f1ab4073a10 2024-04-20T21:23:43.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.806+0000 7f1aaa000640 1 -- 172.21.15.102:0/4237835993 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1aa401a070 con 0x7f1ab4073a10 2024-04-20T21:23:43.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.807+0000 7f1aaa000640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:43.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.807+0000 7f1aaa000640 1 -- 172.21.15.102:0/4237835993 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7f1aa408bfe0 con 0x7f1ab4073a10 2024-04-20T21:23:43.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.807+0000 7f1ab1400640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:43.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.808+0000 7f1ab1400640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a9406c8f0 0x7f1a9406edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ab41eec60 tx=0x7f1aa0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:43.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:43.812+0000 7f1aaa000640 1 -- 172.21.15.102:0/4237835993 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1aa4059890 con 0x7f1ab4073a10 2024-04-20T21:23:44.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.046+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab406ef20 con 0x7f1ab4073a10 2024-04-20T21:23:44.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.048+0000 7f1aaa000640 1 -- 172.21.15.102:0/4237835993 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6027 (secure 0 0 0) 0x7f1aa405db90 con 0x7f1ab4073a10 2024-04-20T21:23:44.047 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:44.048 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:43.625627+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:44.048 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 29 2024-04-20T21:23:44.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a9406c8f0 msgr2=0x7f1a9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a9406c8f0 0x7f1a9406edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ab41eec60 tx=0x7f1aa0009190 comp rx=0 tx=0).stop 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 msgr2=0x7f1ab4070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab4070df0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400e9b0 tx=0x7f1aa400ee70 comp rx=0 tx=0).stop 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 shutdown_connections 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ab414fa50 0x7f1ab414caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ab40c9eb0 0x7f1ab40726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 --2- 172.21.15.102:0/4237835993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab4073a10 0x7f1ab4070df0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 >> 172.21.15.102:0/4237835993 conn(0x7f1ab40c9390 msgr2=0x7f1ab406f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 shutdown_connections 2024-04-20T21:23:44.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:44.050+0000 7f1ab9200640 1 -- 172.21.15.102:0/4237835993 wait complete. 2024-04-20T21:23:44.067 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 29, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5039": { "addr": "172.21.15.102:6835/1097388930", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 1097388930, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 1097388930, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5039, "incarnation": 27, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:rejoin", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 44, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:23:43.625627+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5039 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:23:46.068 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:46.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.272+0000 7fc86c800640 1 Processor -- start 2024-04-20T21:23:46.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.272+0000 7fc86c800640 1 -- start start 2024-04-20T21:23:46.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.274+0000 7fc86c800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc864152170 0x7fc8640c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.274+0000 7fc86c800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc864071f70 con 0x7fc864158b80 2024-04-20T21:23:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.274+0000 7fc86c800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc864072210 con 0x7fc864158770 2024-04-20T21:23:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.274+0000 7fc86c800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc8640c6dc0 con 0x7fc864152170 2024-04-20T21:23:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.274+0000 7fc86a800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc864158b80 0x7fc8641628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:44942/0 (socket says 172.21.15.102:44942) 2024-04-20T21:23:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.274+0000 7fc869e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc864152170 0x7fc8640c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.274+0000 7fc86a800640 1 -- 172.21.15.102:0/1243953450 learned_addr learned my addr 172.21.15.102:0/1243953450 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.274+0000 7fc869400640 1 --1- 172.21.15.102:0/1243953450 >> v1:172.21.15.193:6789/0 conn(0x7fc864158770 0x7fc864070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:42184/0 (socket says 172.21.15.102:42184) 2024-04-20T21:23:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.275+0000 7fc869e00640 1 -- 172.21.15.102:0/1243953450 >> v1:172.21.15.193:6789/0 conn(0x7fc864158770 legacy=0x7fc864070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.275+0000 7fc869e00640 1 -- 172.21.15.102:0/1243953450 >> v1:172.21.15.102:6789/0 conn(0x7fc864158b80 legacy=0x7fc8641628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.275+0000 7fc869e00640 1 -- 172.21.15.102:0/1243953450 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8640724b0 con 0x7fc864152170 2024-04-20T21:23:46.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.276+0000 7fc869e00640 1 --2- 172.21.15.102:0/1243953450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864152170 0x7fc8640c8db0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8580097a0 tx=0x7fc85802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82e3c4276338fa13 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:46.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.276+0000 7fc868a00640 1 -- 172.21.15.102:0/1243953450 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85802f810 con 0x7fc864152170 2024-04-20T21:23:46.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.276+0000 7fc868a00640 1 -- 172.21.15.102:0/1243953450 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85802f9b0 con 0x7fc864152170 2024-04-20T21:23:46.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.276+0000 7fc868a00640 1 -- 172.21.15.102:0/1243953450 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85802fd80 con 0x7fc864152170 2024-04-20T21:23:46.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.276+0000 7fc86c800640 1 -- 172.21.15.102:0/1243953450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864152170 msgr2=0x7fc8640c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:46.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.276+0000 7fc86c800640 1 --2- 172.21.15.102:0/1243953450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864152170 0x7fc8640c8db0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8580097a0 tx=0x7fc85802b9f0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.277+0000 7fc86c800640 1 -- 172.21.15.102:0/1243953450 shutdown_connections 2024-04-20T21:23:46.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.277+0000 7fc86c800640 1 --2- 172.21.15.102:0/1243953450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864152170 0x7fc8640c8db0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.277+0000 7fc86c800640 1 -- 172.21.15.102:0/1243953450 >> 172.21.15.102:0/1243953450 conn(0x7fc8640c9390 msgr2=0x7fc8640c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:46.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.277+0000 7fc86c800640 1 -- 172.21.15.102:0/1243953450 shutdown_connections 2024-04-20T21:23:46.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.277+0000 7fc86c800640 1 -- 172.21.15.102:0/1243953450 wait complete. 2024-04-20T21:23:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.279+0000 7fc86c800640 1 Processor -- start 2024-04-20T21:23:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.279+0000 7fc86c800640 1 -- start start 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc86c800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864152170 0x7fc8641efe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc86c800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc864158770 0x7fc864150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc86c800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864158b80 0x7fc8641423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc86c800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc8640c9f20 con 0x7fc864152170 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc86c800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc864158250 con 0x7fc864158b80 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc86c800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc864157f20 con 0x7fc864158770 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc869e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864152170 0x7fc8641efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc869e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864152170 0x7fc8641efe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47276/0 (socket says 172.21.15.102:47276) 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc869e00640 1 -- 172.21.15.102:0/2062233927 learned_addr learned my addr 172.21.15.102:0/2062233927 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc86a800640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864158b80 0x7fc8641423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:46.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.280+0000 7fc869400640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc864158770 0x7fc864150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:46.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc869e00640 1 -- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc864158770 msgr2=0x7fc864150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:46.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc869e00640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc864158770 0x7fc864150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc869e00640 1 -- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864158b80 msgr2=0x7fc8641423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:46.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc869e00640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864158b80 0x7fc8641423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc869e00640 1 -- 172.21.15.102:0/2062233927 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8641f0380 con 0x7fc864152170 2024-04-20T21:23:46.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc869e00640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864152170 0x7fc8641efe40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc85802c000 tx=0x7fc858004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:46.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc85e000640 1 -- 172.21.15.102:0/2062233927 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858047070 con 0x7fc864152170 2024-04-20T21:23:46.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc85e000640 1 -- 172.21.15.102:0/2062233927 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc858037c60 con 0x7fc864152170 2024-04-20T21:23:46.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc864155e30 con 0x7fc864152170 2024-04-20T21:23:46.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc85e000640 1 -- 172.21.15.102:0/2062233927 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85802fe60 con 0x7fc864152170 2024-04-20T21:23:46.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.281+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8641eee50 con 0x7fc864152170 2024-04-20T21:23:46.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.283+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc864074340 con 0x7fc864152170 2024-04-20T21:23:46.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.284+0000 7fc85e000640 1 -- 172.21.15.102:0/2062233927 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc85804a070 con 0x7fc864152170 2024-04-20T21:23:46.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.285+0000 7fc85e000640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc84806c920 0x7fc84806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:46.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.285+0000 7fc85e000640 1 -- 172.21.15.102:0/2062233927 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7fc8580adc40 con 0x7fc864152170 2024-04-20T21:23:46.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.285+0000 7fc869400640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc84806c920 0x7fc84806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:46.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.286+0000 7fc869400640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc84806c920 0x7fc84806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8640717d0 tx=0x7fc854000ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:46.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.290+0000 7fc85e000640 1 -- 172.21.15.102:0/2062233927 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc858033030 con 0x7fc864152170 2024-04-20T21:23:46.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.521+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8640c3460 con 0x7fc864152170 2024-04-20T21:23:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.523+0000 7fc85e000640 1 -- 172.21.15.102:0/2062233927 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6027 (secure 0 0 0) 0x7fc85807b530 con 0x7fc864152170 2024-04-20T21:23:46.522 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:46.522 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:43.625627+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 29 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc84806c920 msgr2=0x7fc84806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc84806c920 0x7fc84806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8640717d0 tx=0x7fc854000ed0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864152170 msgr2=0x7fc8641efe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864152170 0x7fc8641efe40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc85802c000 tx=0x7fc858004730 comp rx=0 tx=0).stop 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 shutdown_connections 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc84806c920 0x7fc84806ede0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc864158b80 0x7fc8641423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc864158770 0x7fc864150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 --2- 172.21.15.102:0/2062233927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864152170 0x7fc8641efe40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 >> 172.21.15.102:0/2062233927 conn(0x7fc8640c9390 msgr2=0x7fc86406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:46.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.525+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 shutdown_connections 2024-04-20T21:23:46.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:46.526+0000 7fc86c800640 1 -- 172.21.15.102:0/2062233927 wait complete. 2024-04-20T21:23:46.543 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:23:48.545 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:48.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.754+0000 7f7422c00640 1 Processor -- start 2024-04-20T21:23:48.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.755+0000 7f7422c00640 1 -- start start 2024-04-20T21:23:48.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.756+0000 7f7422c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f741c155560 0x7f741c15be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:48.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.756+0000 7f7422c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c073230 0x7f741c0736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:48.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.756+0000 7f7422c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f741c15e7b0 con 0x7f741c0c5cc0 2024-04-20T21:23:48.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.756+0000 7f7422c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f741c15f0c0 con 0x7f741c155560 2024-04-20T21:23:48.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.756+0000 7f7422c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f741c155940 con 0x7f741c073230 2024-04-20T21:23:48.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f7420c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c073230 0x7f741c0736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:48.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f741be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f741c155560 0x7f741c15be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:48.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f741b400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f741c0c5cc0 0x7f741c0ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:40100/0 (socket says 172.21.15.102:40100) 2024-04-20T21:23:48.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f741b400640 1 -- 172.21.15.102:0/2787613981 learned_addr learned my addr 172.21.15.102:0/2787613981 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:48.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f741be00640 1 -- 172.21.15.102:0/2787613981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c073230 msgr2=0x7f741c0736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:48.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f741be00640 1 --2- 172.21.15.102:0/2787613981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c073230 0x7f741c0736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f741be00640 1 -- 172.21.15.102:0/2787613981 >> v1:172.21.15.193:6789/0 conn(0x7f741c0c5cc0 legacy=0x7f741c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:48.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f741be00640 1 -- 172.21.15.102:0/2787613981 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c0cbea0 con 0x7f741c155560 2024-04-20T21:23:48.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f7420c00640 1 --2- 172.21.15.102:0/2787613981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c073230 0x7f741c0736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:48.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.757+0000 7f741aa00640 1 -- 172.21.15.102:0/2787613981 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321032198 0 0) 0x7f741c15e7b0 con 0x7f741c0c5cc0 2024-04-20T21:23:48.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.758+0000 7f741be00640 1 --2- 172.21.15.102:0/2787613981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c155560 0x7f741c15be30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097a0 tx=0x7f740c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5692f47cb9980289 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:48.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f741aa00640 1 -- 172.21.15.102:0/2787613981 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c034070 con 0x7f741c155560 2024-04-20T21:23:48.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f741aa00640 1 -- 172.21.15.102:0/2787613981 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740c030800 con 0x7f741c155560 2024-04-20T21:23:48.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f741aa00640 1 -- 172.21.15.102:0/2787613981 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c030bd0 con 0x7f741c155560 2024-04-20T21:23:48.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f7422c00640 1 -- 172.21.15.102:0/2787613981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c155560 msgr2=0x7f741c15be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:48.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f7422c00640 1 --2- 172.21.15.102:0/2787613981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c155560 0x7f741c15be30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097a0 tx=0x7f740c02b940 comp rx=0 tx=0).stop 2024-04-20T21:23:48.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f7422c00640 1 -- 172.21.15.102:0/2787613981 shutdown_connections 2024-04-20T21:23:48.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f7422c00640 1 --2- 172.21.15.102:0/2787613981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c073230 0x7f741c0736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f7422c00640 1 --2- 172.21.15.102:0/2787613981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c155560 0x7f741c15be30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.759+0000 7f7422c00640 1 -- 172.21.15.102:0/2787613981 >> 172.21.15.102:0/2787613981 conn(0x7f741c06cd40 msgr2=0x7f741c06d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:48.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.760+0000 7f7422c00640 1 -- 172.21.15.102:0/2787613981 shutdown_connections 2024-04-20T21:23:48.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.760+0000 7f7422c00640 1 -- 172.21.15.102:0/2787613981 wait complete. 2024-04-20T21:23:48.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.761+0000 7f7422c00640 1 Processor -- start 2024-04-20T21:23:48.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.762+0000 7f7422c00640 1 -- start start 2024-04-20T21:23:48.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f7422c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f741c073230 0x7f741c1e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f7422c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c0c5cc0 0x7f741c1e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f7422c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c155560 0x7f741c1e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f7422c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f741c15ed50 con 0x7f741c073230 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f7422c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f741c15ea70 con 0x7f741c0c5cc0 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f7422c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f741c0cbb60 con 0x7f741c155560 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f741b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c0c5cc0 0x7f741c1e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f741be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f741c073230 0x7f741c1e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f741b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c0c5cc0 0x7f741c1e45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:49486/0 (socket says 172.21.15.102:49486) 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f741b400640 1 -- 172.21.15.102:0/384827095 learned_addr learned my addr 172.21.15.102:0/384827095 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:48.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f7420c00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c155560 0x7f741c1e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:48.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.763+0000 7f741b400640 1 -- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c155560 msgr2=0x7f741c1e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:48.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f741b400640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c155560 0x7f741c1e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f741b400640 1 -- 172.21.15.102:0/384827095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f741c073230 msgr2=0x7f741c1e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:48.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f741b400640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f741c073230 0x7f741c1e63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f741b400640 1 -- 172.21.15.102:0/384827095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c0b6a40 con 0x7f741c0c5cc0 2024-04-20T21:23:48.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f741b400640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c0c5cc0 0x7f741c1e45f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f740800c4d0 tx=0x7f740800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:48.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f7413e00640 1 -- 172.21.15.102:0/384827095 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74080074e0 con 0x7f741c0c5cc0 2024-04-20T21:23:48.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f7413e00640 1 -- 172.21.15.102:0/384827095 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74080047e0 con 0x7f741c0c5cc0 2024-04-20T21:23:48.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741c1e53d0 con 0x7f741c0c5cc0 2024-04-20T21:23:48.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f7413e00640 1 -- 172.21.15.102:0/384827095 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408004bd0 con 0x7f741c0c5cc0 2024-04-20T21:23:48.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.764+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f741c0d0ae0 con 0x7f741c0c5cc0 2024-04-20T21:23:48.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.765+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f741c069340 con 0x7f741c0c5cc0 2024-04-20T21:23:48.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.767+0000 7f7413e00640 1 -- 172.21.15.102:0/384827095 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7408011070 con 0x7f741c0c5cc0 2024-04-20T21:23:48.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.768+0000 7f7413e00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f73fc06c8f0 0x7f73fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:48.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.768+0000 7f7413e00640 1 -- 172.21.15.102:0/384827095 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7f740808bde0 con 0x7f741c0c5cc0 2024-04-20T21:23:48.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.768+0000 7f741be00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f73fc06c8f0 0x7f73fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:48.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.769+0000 7f741be00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f73fc06c8f0 0x7f73fc06edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f740c036000 tx=0x7f740c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:48.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:48.771+0000 7f7413e00640 1 -- 172.21.15.102:0/384827095 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f741c069340 con 0x7f741c0c5cc0 2024-04-20T21:23:49.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.012+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f741c0d0fc0 con 0x7f741c0c5cc0 2024-04-20T21:23:49.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.013+0000 7f7413e00640 1 -- 172.21.15.102:0/384827095 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6033 (secure 0 0 0) 0x7f7408059d20 con 0x7f741c0c5cc0 2024-04-20T21:23:49.013 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:49.013 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:48.772276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:clientreplay","state_seq":19,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:49.013 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 30 2024-04-20T21:23:49.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f73fc06c8f0 msgr2=0x7f73fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:49.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f73fc06c8f0 0x7f73fc06edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f740c036000 tx=0x7f740c03b040 comp rx=0 tx=0).stop 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c0c5cc0 msgr2=0x7f741c1e45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c0c5cc0 0x7f741c1e45f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f740800c4d0 tx=0x7f740800c8d0 comp rx=0 tx=0).stop 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 shutdown_connections 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f741c155560 0x7f741c1e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f73fc06c8f0 0x7f73fc06edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f741c0c5cc0 0x7f741c1e45f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 --2- 172.21.15.102:0/384827095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f741c073230 0x7f741c1e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 >> 172.21.15.102:0/384827095 conn(0x7f741c06cd40 msgr2=0x7f741c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 shutdown_connections 2024-04-20T21:23:49.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:49.016+0000 7f7422c00640 1 -- 172.21.15.102:0/384827095 wait complete. 2024-04-20T21:23:49.034 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:23:51.034 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:51.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.238+0000 7fdbe4800640 1 Processor -- start 2024-04-20T21:23:51.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.238+0000 7fdbe4800640 1 -- start start 2024-04-20T21:23:51.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.240+0000 7fdbe4800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.240+0000 7fdbe4800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbdc0c8f20 con 0x7fdbdc15f2b0 2024-04-20T21:23:51.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.240+0000 7fdbe4800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbdc06cb80 con 0x7fdbdc15ee00 2024-04-20T21:23:51.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.240+0000 7fdbe4800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc06ce60 con 0x7fdbdc15ea20 2024-04-20T21:23:51.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.240+0000 7fdbe2800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdbdc15f2b0 0x7fdbdc154e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:44274/0 (socket says 172.21.15.102:44274) 2024-04-20T21:23:51.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.240+0000 7fdbe2800640 1 -- 172.21.15.102:0/1339479795 learned_addr learned my addr 172.21.15.102:0/1339479795 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:51.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.240+0000 7fdbe1e00640 1 --2- 172.21.15.102:0/1339479795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.240+0000 7fdbe0a00640 1 -- 172.21.15.102:0/1339479795 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487518326 0 0) 0x7fdbdc0c8f20 con 0x7fdbdc15f2b0 2024-04-20T21:23:51.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.241+0000 7fdbe0a00640 1 -- 172.21.15.102:0/1339479795 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbc8003520 con 0x7fdbdc15f2b0 2024-04-20T21:23:51.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.241+0000 7fdbe0a00640 1 -- 172.21.15.102:0/1339479795 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352184234 0 0) 0x7fdbdc06cb80 con 0x7fdbdc15ee00 2024-04-20T21:23:51.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.241+0000 7fdbe0a00640 1 -- 172.21.15.102:0/1339479795 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbdc0c8f20 con 0x7fdbdc15ee00 2024-04-20T21:23:51.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.241+0000 7fdbe1e00640 1 -- 172.21.15.102:0/1339479795 >> v1:172.21.15.193:6789/0 conn(0x7fdbdc15ee00 legacy=0x7fdbdc0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.241+0000 7fdbe1e00640 1 -- 172.21.15.102:0/1339479795 >> v1:172.21.15.102:6789/0 conn(0x7fdbdc15f2b0 legacy=0x7fdbdc154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.241+0000 7fdbe1e00640 1 -- 172.21.15.102:0/1339479795 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc06d140 con 0x7fdbdc15ea20 2024-04-20T21:23:51.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.242+0000 7fdbe1e00640 1 --2- 172.21.15.102:0/1339479795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc06d940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009dc0 tx=0x7fdbd802bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7b333fe6d94e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:51.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.242+0000 7fdbe0a00640 1 -- 172.21.15.102:0/1339479795 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8039070 con 0x7fdbdc15ea20 2024-04-20T21:23:51.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.242+0000 7fdbe0a00640 1 -- 172.21.15.102:0/1339479795 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd802cd50 con 0x7fdbdc15ea20 2024-04-20T21:23:51.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.242+0000 7fdbe0a00640 1 -- 172.21.15.102:0/1339479795 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8034660 con 0x7fdbdc15ea20 2024-04-20T21:23:51.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.243+0000 7fdbe4800640 1 -- 172.21.15.102:0/1339479795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 msgr2=0x7fdbdc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.243+0000 7fdbe4800640 1 --2- 172.21.15.102:0/1339479795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc06d940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009dc0 tx=0x7fdbd802bfa0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.243+0000 7fdbe4800640 1 -- 172.21.15.102:0/1339479795 shutdown_connections 2024-04-20T21:23:51.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.243+0000 7fdbe4800640 1 --2- 172.21.15.102:0/1339479795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc06d940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.243+0000 7fdbe4800640 1 -- 172.21.15.102:0/1339479795 >> 172.21.15.102:0/1339479795 conn(0x7fdbdc0c9390 msgr2=0x7fdbdc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:51.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.243+0000 7fdbe4800640 1 -- 172.21.15.102:0/1339479795 shutdown_connections 2024-04-20T21:23:51.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.243+0000 7fdbe4800640 1 -- 172.21.15.102:0/1339479795 wait complete. 2024-04-20T21:23:51.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.245+0000 7fdbe4800640 1 Processor -- start 2024-04-20T21:23:51.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.245+0000 7fdbe4800640 1 -- start start 2024-04-20T21:23:51.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe4800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe4800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbdc15ee00 0x7fdbdc158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe4800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbdc15f2b0 0x7fdbdc145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe4800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0c8c80 con 0x7fdbdc15f2b0 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe4800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0c8b00 con 0x7fdbdc15ee00 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe4800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc155590 con 0x7fdbdc15ea20 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbdc15f2b0 0x7fdbdc145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:49210/0 (socket says 172.21.15.102:49210) 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1e00640 1 -- 172.21.15.102:0/3786128596 learned_addr learned my addr 172.21.15.102:0/3786128596 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1400640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbdc15ee00 0x7fdbdc158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1e00640 1 -- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbdc15ee00 msgr2=0x7fdbdc158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1e00640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbdc15ee00 0x7fdbdc158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1e00640 1 -- 172.21.15.102:0/3786128596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbdc15f2b0 msgr2=0x7fdbdc145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1e00640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbdc15f2b0 0x7fdbdc145150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.246+0000 7fdbe1e00640 1 -- 172.21.15.102:0/3786128596 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc0c6cc0 con 0x7fdbdc15ea20 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.247+0000 7fdbe1e00640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc157470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8037040 tx=0x7fdbd8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.247+0000 7fdbd2000640 1 -- 172.21.15.102:0/3786128596 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8039050 con 0x7fdbdc15ea20 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.247+0000 7fdbd2000640 1 -- 172.21.15.102:0/3786128596 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd8045070 con 0x7fdbdc15ea20 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.247+0000 7fdbd2000640 1 -- 172.21.15.102:0/3786128596 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8009450 con 0x7fdbdc15ea20 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.247+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc1f0b20 con 0x7fdbdc15ea20 2024-04-20T21:23:51.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.247+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc0c4520 con 0x7fdbdc15ea20 2024-04-20T21:23:51.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.249+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc074340 con 0x7fdbdc15ea20 2024-04-20T21:23:51.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.249+0000 7fdbd2000640 1 -- 172.21.15.102:0/3786128596 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbd803e850 con 0x7fdbdc15ea20 2024-04-20T21:23:51.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.250+0000 7fdbd2000640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdbc006c8f0 0x7fdbc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.250+0000 7fdbd2000640 1 -- 172.21.15.102:0/3786128596 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7fdbd80aee40 con 0x7fdbdc15ea20 2024-04-20T21:23:51.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.250+0000 7fdbe1400640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdbc006c8f0 0x7fdbc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.251+0000 7fdbe1400640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdbc006c8f0 0x7fdbc006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0c39d0 tx=0x7fdbcc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:51.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.255+0000 7fdbd2000640 1 -- 172.21.15.102:0/3786128596 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdbdc0c6cc0 con 0x7fdbdc15ea20 2024-04-20T21:23:51.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.491+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbdc157fe0 con 0x7fdbdc15ea20 2024-04-20T21:23:51.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.492+0000 7fdbd2000640 1 -- 172.21.15.102:0/3786128596 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6033 (secure 0 0 0) 0x7fdbd807c740 con 0x7fdbdc15ea20 2024-04-20T21:23:51.492 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:48.772276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:clientreplay","state_seq":19,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:51.492 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 30 2024-04-20T21:23:51.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.494+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdbc006c8f0 msgr2=0x7fdbc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.494+0000 7fdbe4800640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdbc006c8f0 0x7fdbc006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0c39d0 tx=0x7fdbcc009040 comp rx=0 tx=0).stop 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.494+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 msgr2=0x7fdbdc157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.494+0000 7fdbe4800640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc157470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8037040 tx=0x7fdbd8004a40 comp rx=0 tx=0).stop 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.495+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 shutdown_connections 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.495+0000 7fdbe4800640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbdc15f2b0 0x7fdbdc145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.495+0000 7fdbe4800640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdbc006c8f0 0x7fdbc006edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.495+0000 7fdbe4800640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbdc15ee00 0x7fdbdc158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.495+0000 7fdbe4800640 1 --2- 172.21.15.102:0/3786128596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbdc15ea20 0x7fdbdc157470 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.495+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 >> 172.21.15.102:0/3786128596 conn(0x7fdbdc0c9390 msgr2=0x7fdbdc153470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.495+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 shutdown_connections 2024-04-20T21:23:51.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:51.495+0000 7fdbe4800640 1 -- 172.21.15.102:0/3786128596 wait complete. 2024-04-20T21:23:51.513 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:23:53.514 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:53.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.723+0000 7fafc4600640 1 Processor -- start 2024-04-20T21:23:53.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.723+0000 7fafc4600640 1 -- start start 2024-04-20T21:23:53.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.724+0000 7fafc4600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fafbc151c90 0x7fafbc0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:53.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.724+0000 7fafc4600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc0c7290 0x7fafbc158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:53.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.724+0000 7fafc4600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc158780 con 0x7fafbc071a70 2024-04-20T21:23:53.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.724+0000 7fafc4600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fafbc0c9200 con 0x7fafbc151c90 2024-04-20T21:23:53.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.724+0000 7fafc4600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc070df0 con 0x7fafbc0c7290 2024-04-20T21:23:53.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.725+0000 7fafc1200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc0c7290 0x7fafbc158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:53.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.725+0000 7fafc2600640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fafbc071a70 0x7fafbc06d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:44276/0 (socket says 172.21.15.102:44276) 2024-04-20T21:23:53.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.725+0000 7fafc2600640 1 -- 172.21.15.102:0/2785656051 learned_addr learned my addr 172.21.15.102:0/2785656051 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:53.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.725+0000 7fafc1c00640 1 --2- 172.21.15.102:0/2785656051 >> v2:172.21.15.102:3300/0 conn(0x7fafbc151c90 0x7fafbc0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:53.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.725+0000 7fafbbe00640 1 -- 172.21.15.102:0/2785656051 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1044246769 0 0) 0x7fafbc158780 con 0x7fafbc071a70 2024-04-20T21:23:53.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.725+0000 7fafbbe00640 1 -- 172.21.15.102:0/2785656051 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafa4003520 con 0x7fafbc071a70 2024-04-20T21:23:53.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.725+0000 7fafc1c00640 1 -- 172.21.15.102:0/2785656051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc0c7290 msgr2=0x7fafbc158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:53.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.725+0000 7fafc1c00640 1 --2- 172.21.15.102:0/2785656051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc0c7290 0x7fafbc158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:53.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.726+0000 7fafc1c00640 1 -- 172.21.15.102:0/2785656051 >> v1:172.21.15.102:6789/0 conn(0x7fafbc071a70 legacy=0x7fafbc06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:53.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.726+0000 7fafc1c00640 1 -- 172.21.15.102:0/2785656051 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc071040 con 0x7fafbc151c90 2024-04-20T21:23:53.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.726+0000 7fafc1200640 1 --2- 172.21.15.102:0/2785656051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc0c7290 0x7fafbc158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:53.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.727+0000 7fafc1c00640 1 --2- 172.21.15.102:0/2785656051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc151c90 0x7fafbc0c8cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fafb000a8c0 tx=0x7fafb002ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1859f4e9127eb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:53.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.727+0000 7fafbbe00640 1 -- 172.21.15.102:0/2785656051 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb002e820 con 0x7fafbc151c90 2024-04-20T21:23:53.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.727+0000 7fafbbe00640 1 -- 172.21.15.102:0/2785656051 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb002e9c0 con 0x7fafbc151c90 2024-04-20T21:23:53.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.727+0000 7fafbbe00640 1 -- 172.21.15.102:0/2785656051 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb002ed70 con 0x7fafbc151c90 2024-04-20T21:23:53.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.727+0000 7fafc4600640 1 -- 172.21.15.102:0/2785656051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc151c90 msgr2=0x7fafbc0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:53.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.727+0000 7fafc4600640 1 --2- 172.21.15.102:0/2785656051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc151c90 0x7fafbc0c8cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fafb000a8c0 tx=0x7fafb002ca60 comp rx=0 tx=0).stop 2024-04-20T21:23:53.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.728+0000 7fafc4600640 1 -- 172.21.15.102:0/2785656051 shutdown_connections 2024-04-20T21:23:53.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.728+0000 7fafc4600640 1 --2- 172.21.15.102:0/2785656051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc0c7290 0x7fafbc158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:53.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.728+0000 7fafc4600640 1 --2- 172.21.15.102:0/2785656051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc151c90 0x7fafbc0c8cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:53.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.728+0000 7fafc4600640 1 -- 172.21.15.102:0/2785656051 >> 172.21.15.102:0/2785656051 conn(0x7fafbc0c9390 msgr2=0x7fafbc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:53.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.728+0000 7fafc4600640 1 -- 172.21.15.102:0/2785656051 shutdown_connections 2024-04-20T21:23:53.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.728+0000 7fafc4600640 1 -- 172.21.15.102:0/2785656051 wait complete. 2024-04-20T21:23:53.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.729+0000 7fafc4600640 1 Processor -- start 2024-04-20T21:23:53.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.730+0000 7fafc4600640 1 -- start start 2024-04-20T21:23:53.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc4600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc071a70 0x7fafbc1f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:53.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc4600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc0c7290 0x7fafbc148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc4600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc151c90 0x7fafbc0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc4600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc071600 con 0x7fafbc071a70 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc4600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc157db0 con 0x7fafbc0c7290 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc4600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc157a40 con 0x7fafbc151c90 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc071a70 0x7fafbc1f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc071a70 0x7fafbc1f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59366/0 (socket says 172.21.15.102:59366) 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1c00640 1 -- 172.21.15.102:0/2972859740 learned_addr learned my addr 172.21.15.102:0/2972859740 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc2600640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc151c90 0x7fafbc0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1200640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc0c7290 0x7fafbc148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1c00640 1 -- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc151c90 msgr2=0x7fafbc0cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1c00640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc151c90 0x7fafbc0cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1c00640 1 -- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc0c7290 msgr2=0x7fafbc148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1c00640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc0c7290 0x7fafbc148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:53.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.731+0000 7fafc1c00640 1 -- 172.21.15.102:0/2972859740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc1470a0 con 0x7fafbc071a70 2024-04-20T21:23:53.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.732+0000 7fafc1c00640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc071a70 0x7fafbc1f3160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fafb000a130 tx=0x7fafb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:53.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.732+0000 7fafb9600640 1 -- 172.21.15.102:0/2972859740 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0038810 con 0x7fafbc071a70 2024-04-20T21:23:53.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.732+0000 7fafb9600640 1 -- 172.21.15.102:0/2972859740 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb00389b0 con 0x7fafbc071a70 2024-04-20T21:23:53.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.732+0000 7fafb9600640 1 -- 172.21.15.102:0/2972859740 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0038d60 con 0x7fafbc071a70 2024-04-20T21:23:53.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.732+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc147360 con 0x7fafbc071a70 2024-04-20T21:23:53.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.732+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1f2680 con 0x7fafbc071a70 2024-04-20T21:23:53.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.733+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc074340 con 0x7fafbc071a70 2024-04-20T21:23:53.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.734+0000 7fafb9600640 1 -- 172.21.15.102:0/2972859740 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafb0041410 con 0x7fafbc071a70 2024-04-20T21:23:53.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.735+0000 7fafb9600640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faf9c06c8f0 0x7faf9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:53.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.735+0000 7fafb9600640 1 -- 172.21.15.102:0/2972859740 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7fafb00adbe0 con 0x7fafbc071a70 2024-04-20T21:23:53.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.735+0000 7fafc1200640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faf9c06c8f0 0x7faf9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:53.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.736+0000 7fafc1200640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faf9c06c8f0 0x7faf9c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafbc149f40 tx=0x7fafa8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:53.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:53.740+0000 7fafb9600640 1 -- 172.21.15.102:0/2972859740 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fafb004f060 con 0x7fafbc071a70 2024-04-20T21:23:54.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.112+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafbc072520 con 0x7fafbc071a70 2024-04-20T21:23:54.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.114+0000 7fafb9600640 1 -- 172.21.15.102:0/2972859740 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6033 (secure 0 0 0) 0x7fafb003e050 con 0x7fafbc071a70 2024-04-20T21:23:54.114 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:54.114 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:48.772276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:clientreplay","state_seq":19,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:23:54.114 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 30 2024-04-20T21:23:54.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.119+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faf9c06c8f0 msgr2=0x7faf9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:54.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.119+0000 7fafc4600640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faf9c06c8f0 0x7faf9c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafbc149f40 tx=0x7fafa8009190 comp rx=0 tx=0).stop 2024-04-20T21:23:54.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.119+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc071a70 msgr2=0x7fafbc1f3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:54.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.119+0000 7fafc4600640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc071a70 0x7fafbc1f3160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fafb000a130 tx=0x7fafb00046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:54.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.120+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 shutdown_connections 2024-04-20T21:23:54.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.120+0000 7fafc4600640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafbc151c90 0x7fafbc0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:54.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.120+0000 7fafc4600640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faf9c06c8f0 0x7faf9c06edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:54.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.120+0000 7fafc4600640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafbc0c7290 0x7fafbc148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:54.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.120+0000 7fafc4600640 1 --2- 172.21.15.102:0/2972859740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafbc071a70 0x7fafbc1f3160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:54.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.120+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 >> 172.21.15.102:0/2972859740 conn(0x7fafbc0c9390 msgr2=0x7fafbc0c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:54.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.121+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 shutdown_connections 2024-04-20T21:23:54.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:54.122+0000 7fafc4600640 1 -- 172.21.15.102:0/2972859740 wait complete. 2024-04-20T21:23:54.146 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:23:55.548 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:55.556 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:56.147 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:56.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.354+0000 7fd268200640 1 Processor -- start 2024-04-20T21:23:56.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.354+0000 7fd268200640 1 -- start start 2024-04-20T21:23:56.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.356+0000 7fd268200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd2600c9eb0 0x7fd2600c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:56.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.356+0000 7fd268200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2601536d0 con 0x7fd26014fa50 2024-04-20T21:23:56.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.356+0000 7fd268200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2600c76c0 con 0x7fd260073a10 2024-04-20T21:23:56.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.356+0000 7fd268200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fd2600c7960 con 0x7fd2600c9eb0 2024-04-20T21:23:56.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.356+0000 7fd264e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd2600c9eb0 0x7fd2600c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:56.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.356+0000 7fd266200640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fd260073a10 0x7fd260163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:40120/0 (socket says 172.21.15.102:40120) 2024-04-20T21:23:56.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.356+0000 7fd266200640 1 -- 172.21.15.102:0/856796763 learned_addr learned my addr 172.21.15.102:0/856796763 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:56.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.356+0000 7fd265800640 1 --1- 172.21.15.102:0/856796763 >> v1:172.21.15.102:6789/0 conn(0x7fd26014fa50 0x7fd2600c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:44292/0 (socket says 172.21.15.102:44292) 2024-04-20T21:23:56.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.357+0000 7fd25fe00640 1 -- 172.21.15.102:0/856796763 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3130788797 0 0) 0x7fd2600c76c0 con 0x7fd260073a10 2024-04-20T21:23:56.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.357+0000 7fd25fe00640 1 -- 172.21.15.102:0/856796763 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd248003520 con 0x7fd260073a10 2024-04-20T21:23:56.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.357+0000 7fd25fe00640 1 -- 172.21.15.102:0/856796763 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197944516 0 0) 0x7fd254001110 con 0x7fd26014fa50 2024-04-20T21:23:56.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.357+0000 7fd264e00640 1 -- 172.21.15.102:0/856796763 >> v1:172.21.15.193:6789/0 conn(0x7fd260073a10 legacy=0x7fd260163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.357+0000 7fd264e00640 1 -- 172.21.15.102:0/856796763 >> v1:172.21.15.102:6789/0 conn(0x7fd26014fa50 legacy=0x7fd2600c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.357+0000 7fd264e00640 1 -- 172.21.15.102:0/856796763 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2600c3d90 con 0x7fd2600c9eb0 2024-04-20T21:23:56.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.358+0000 7fd264e00640 1 --2- 172.21.15.102:0/856796763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2600c9eb0 0x7fd2600c4830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd2600c7890 tx=0x7fd25002b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d44f733607087b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:56.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.358+0000 7fd25fe00640 1 -- 172.21.15.102:0/856796763 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250034070 con 0x7fd2600c9eb0 2024-04-20T21:23:56.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.358+0000 7fd25fe00640 1 -- 172.21.15.102:0/856796763 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250030890 con 0x7fd2600c9eb0 2024-04-20T21:23:56.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.358+0000 7fd25fe00640 1 -- 172.21.15.102:0/856796763 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250030c60 con 0x7fd2600c9eb0 2024-04-20T21:23:56.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.359+0000 7fd268200640 1 -- 172.21.15.102:0/856796763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2600c9eb0 msgr2=0x7fd2600c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.359+0000 7fd268200640 1 --2- 172.21.15.102:0/856796763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2600c9eb0 0x7fd2600c4830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd2600c7890 tx=0x7fd25002b9d0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.359+0000 7fd268200640 1 -- 172.21.15.102:0/856796763 shutdown_connections 2024-04-20T21:23:56.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.359+0000 7fd268200640 1 --2- 172.21.15.102:0/856796763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2600c9eb0 0x7fd2600c4830 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.359+0000 7fd268200640 1 -- 172.21.15.102:0/856796763 >> 172.21.15.102:0/856796763 conn(0x7fd2600c9390 msgr2=0x7fd2600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:56.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.359+0000 7fd268200640 1 -- 172.21.15.102:0/856796763 shutdown_connections 2024-04-20T21:23:56.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.359+0000 7fd268200640 1 -- 172.21.15.102:0/856796763 wait complete. 2024-04-20T21:23:56.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.361+0000 7fd268200640 1 Processor -- start 2024-04-20T21:23:56.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.361+0000 7fd268200640 1 -- start start 2024-04-20T21:23:56.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd268200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd260073a10 0x7fd2601560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:56.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd268200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2600c9eb0 0x7fd2601e5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:56.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd268200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd26014fa50 0x7fd26014b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:56.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd268200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd2600c4030 con 0x7fd2600c9eb0 2024-04-20T21:23:56.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd268200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2600c7c00 con 0x7fd260073a10 2024-04-20T21:23:56.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd268200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd2600c4310 con 0x7fd26014fa50 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd264e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2600c9eb0 0x7fd2601e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd264e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2600c9eb0 0x7fd2601e5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59374/0 (socket says 172.21.15.102:59374) 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd264e00640 1 -- 172.21.15.102:0/3926022500 learned_addr learned my addr 172.21.15.102:0/3926022500 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd265800640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd260073a10 0x7fd2601560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.362+0000 7fd266200640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd26014fa50 0x7fd26014b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd264e00640 1 -- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd26014fa50 msgr2=0x7fd26014b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd264e00640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd26014fa50 0x7fd26014b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd264e00640 1 -- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd260073a10 msgr2=0x7fd2601560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd264e00640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd260073a10 0x7fd2601560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd264e00640 1 -- 172.21.15.102:0/3926022500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd260155010 con 0x7fd2600c9eb0 2024-04-20T21:23:56.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd264e00640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2600c9eb0 0x7fd2601e5650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd250009890 tx=0x7fd2500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:56.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd25d600640 1 -- 172.21.15.102:0/3926022500 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250049070 con 0x7fd2600c9eb0 2024-04-20T21:23:56.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd25d600640 1 -- 172.21.15.102:0/3926022500 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250030690 con 0x7fd2600c9eb0 2024-04-20T21:23:56.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd25d600640 1 -- 172.21.15.102:0/3926022500 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250034040 con 0x7fd2600c9eb0 2024-04-20T21:23:56.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2601552f0 con 0x7fd2600c9eb0 2024-04-20T21:23:56.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.363+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2601558b0 con 0x7fd2600c9eb0 2024-04-20T21:23:56.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.364+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26014bd00 con 0x7fd2600c9eb0 2024-04-20T21:23:56.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.366+0000 7fd25d600640 1 -- 172.21.15.102:0/3926022500 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd25004c070 con 0x7fd2600c9eb0 2024-04-20T21:23:56.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.366+0000 7fd25d600640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd24406c8f0 0x7fd24406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:56.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.367+0000 7fd25d600640 1 -- 172.21.15.102:0/3926022500 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7fd2500ae150 con 0x7fd2600c9eb0 2024-04-20T21:23:56.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.367+0000 7fd265800640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd24406c8f0 0x7fd24406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:56.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.367+0000 7fd265800640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd24406c8f0 0x7fd24406edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd254007fd0 tx=0x7fd254007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.371+0000 7fd25d600640 1 -- 172.21.15.102:0/3926022500 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd26014bd00 con 0x7fd2600c9eb0 2024-04-20T21:23:56.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.613+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd260074340 con 0x7fd2600c9eb0 2024-04-20T21:23:56.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.614+0000 7fd25d600640 1 -- 172.21.15.102:0/3926022500 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+6034 (secure 0 0 0) 0x7fd25007b930 con 0x7fd2600c9eb0 2024-04-20T21:23:56.614 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:56.614 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:23:56.614 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 31 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd24406c8f0 msgr2=0x7fd24406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd24406c8f0 0x7fd24406edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd254007fd0 tx=0x7fd254007de0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2600c9eb0 msgr2=0x7fd2601e5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2600c9eb0 0x7fd2601e5650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd250009890 tx=0x7fd2500046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 shutdown_connections 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd26014fa50 0x7fd26014b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2600c9eb0 0x7fd2601e5650 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd24406c8f0 0x7fd24406edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 --2- 172.21.15.102:0/3926022500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd260073a10 0x7fd2601560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 >> 172.21.15.102:0/3926022500 conn(0x7fd2600c9390 msgr2=0x7fd2600c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:56.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 shutdown_connections 2024-04-20T21:23:56.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:56.617+0000 7fd268200640 1 -- 172.21.15.102:0/3926022500 wait complete. 2024-04-20T21:23:56.636 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=5266 2024-04-20T21:23:56.636 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.a 2024-04-20T21:23:58.637 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-04-20T21:23:58.637 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:23:58.637 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:23:58.645 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:23:58.646 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:58.801 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-20T21:23:58.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.826+0000 7f0d3b000640 1 Processor -- start 2024-04-20T21:23:58.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.826+0000 7f0d3b000640 1 -- start start 2024-04-20T21:23:58.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d3b000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0d34155560 0x7f0d3415be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d3b000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d3b000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3415e7b0 con 0x7f0d340c60c0 2024-04-20T21:23:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d3b000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3415f0c0 con 0x7f0d34155560 2024-04-20T21:23:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d3b000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0d34155940 con 0x7f0d340c5cc0 2024-04-20T21:23:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d39000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0d340c60c0 0x7f0d34073230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56024/0 (socket says 172.21.15.102:56024) 2024-04-20T21:23:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d39000640 1 -- 172.21.15.102:0/2172037562 learned_addr learned my addr 172.21.15.102:0/2172037562 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:23:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d2b400640 1 --2- 172.21.15.102:0/2172037562 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.827+0000 7f0d33e00640 1 --2- 172.21.15.102:0/2172037562 >> v2:172.21.15.193:3300/0 conn(0x7f0d34155560 0x7f0d3415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:58.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.828+0000 7f0d33400640 1 -- 172.21.15.102:0/2172037562 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025176002 0 0) 0x7f0d3415e7b0 con 0x7f0d340c60c0 2024-04-20T21:23:58.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.828+0000 7f0d33400640 1 -- 172.21.15.102:0/2172037562 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d18003520 con 0x7f0d340c60c0 2024-04-20T21:23:58.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.828+0000 7f0d2b400640 1 -- 172.21.15.102:0/2172037562 >> v2:172.21.15.193:3300/0 conn(0x7f0d34155560 msgr2=0x7f0d3415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:58.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.828+0000 7f0d2b400640 1 --2- 172.21.15.102:0/2172037562 >> v2:172.21.15.193:3300/0 conn(0x7f0d34155560 0x7f0d3415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:58.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.828+0000 7f0d2b400640 1 -- 172.21.15.102:0/2172037562 >> v1:172.21.15.102:6789/0 conn(0x7f0d340c60c0 legacy=0x7f0d34073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:58.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.828+0000 7f0d2b400640 1 -- 172.21.15.102:0/2172037562 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d340cbea0 con 0x7f0d340c5cc0 2024-04-20T21:23:58.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.828+0000 7f0d33e00640 1 --2- 172.21.15.102:0/2172037562 >> v2:172.21.15.193:3300/0 conn(0x7f0d34155560 0x7f0d3415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:58.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.829+0000 7f0d2b400640 1 --2- 172.21.15.102:0/2172037562 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340ccd20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0097a0 tx=0x7f0d1c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7c95e286b803ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:58.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.829+0000 7f0d33400640 1 -- 172.21.15.102:0/2172037562 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c034070 con 0x7f0d340c5cc0 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.829+0000 7f0d33400640 1 -- 172.21.15.102:0/2172037562 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1c030800 con 0x7f0d340c5cc0 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.829+0000 7f0d33400640 1 -- 172.21.15.102:0/2172037562 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c030bd0 con 0x7f0d340c5cc0 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.830+0000 7f0d3b000640 1 -- 172.21.15.102:0/2172037562 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 msgr2=0x7f0d340ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.830+0000 7f0d3b000640 1 --2- 172.21.15.102:0/2172037562 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340ccd20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0097a0 tx=0x7f0d1c02b940 comp rx=0 tx=0).stop 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.830+0000 7f0d3b000640 1 -- 172.21.15.102:0/2172037562 shutdown_connections 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.830+0000 7f0d3b000640 1 --2- 172.21.15.102:0/2172037562 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340ccd20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.830+0000 7f0d3b000640 1 --2- 172.21.15.102:0/2172037562 >> v2:172.21.15.193:3300/0 conn(0x7f0d34155560 0x7f0d3415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.830+0000 7f0d3b000640 1 -- 172.21.15.102:0/2172037562 >> 172.21.15.102:0/2172037562 conn(0x7f0d3406cd40 msgr2=0x7f0d3406d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.830+0000 7f0d3b000640 1 -- 172.21.15.102:0/2172037562 shutdown_connections 2024-04-20T21:23:58.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.830+0000 7f0d3b000640 1 -- 172.21.15.102:0/2172037562 wait complete. 2024-04-20T21:23:58.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.832+0000 7f0d3b000640 1 Processor -- start 2024-04-20T21:23:58.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.832+0000 7f0d3b000640 1 -- start start 2024-04-20T21:23:58.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d3b000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340d1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:58.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d3b000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d34155560 0x7f0d3415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:58.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d3b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d3415bd20 0x7f0d34157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:58.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d3b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3415ed50 con 0x7f0d3415bd20 2024-04-20T21:23:58.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d3b000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3415ea70 con 0x7f0d34155560 2024-04-20T21:23:58.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d3b000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0d340cbb60 con 0x7f0d340c5cc0 2024-04-20T21:23:58.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d39000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d3415bd20 0x7f0d34157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:58.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d33e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d39000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d3415bd20 0x7f0d34157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38716/0 (socket says 172.21.15.102:38716) 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d2b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d34155560 0x7f0d3415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.833+0000 7f0d39000640 1 -- 172.21.15.102:0/593242248 learned_addr learned my addr 172.21.15.102:0/593242248 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d39000640 1 -- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 msgr2=0x7f0d340d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d39000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d39000640 1 -- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d34155560 msgr2=0x7f0d3415b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d39000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d34155560 0x7f0d3415b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d39000640 1 -- 172.21.15.102:0/593242248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d340d1750 con 0x7f0d3415bd20 2024-04-20T21:23:58.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d39000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d3415bd20 0x7f0d34157e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c400 tx=0x7f0d2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:58.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d30c00640 1 -- 172.21.15.102:0/593242248 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017070 con 0x7f0d3415bd20 2024-04-20T21:23:58.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d30c00640 1 -- 172.21.15.102:0/593242248 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2c005ce0 con 0x7f0d3415bd20 2024-04-20T21:23:58.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d30c00640 1 -- 172.21.15.102:0/593242248 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0072d0 con 0x7f0d3415bd20 2024-04-20T21:23:58.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d34156d60 con 0x7f0d3415bd20 2024-04-20T21:23:58.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.834+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d34157320 con 0x7f0d3415bd20 2024-04-20T21:23:58.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.836+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d34074340 con 0x7f0d3415bd20 2024-04-20T21:23:58.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.837+0000 7f0d30c00640 1 -- 172.21.15.102:0/593242248 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d2c01a070 con 0x7f0d3415bd20 2024-04-20T21:23:58.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.838+0000 7f0d30c00640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0d1006c920 0x7f0d1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:58.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.838+0000 7f0d30c00640 1 -- 172.21.15.102:0/593242248 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7079+0+0 (secure 0 0 0) 0x7f0d2c08b360 con 0x7f0d3415bd20 2024-04-20T21:23:58.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.838+0000 7f0d33e00640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0d1006c920 0x7f0d1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:58.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.839+0000 7f0d33e00640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0d1006c920 0x7f0d1006ede0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009bd0 tx=0x7f0d2400b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:58.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:58.842+0000 7f0d30c00640 1 -- 172.21.15.102:0/593242248 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0d2c058be0 con 0x7f0d3415bd20 2024-04-20T21:23:59.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.086+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d34153b30 con 0x7f0d3415bd20 2024-04-20T21:23:59.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.087+0000 7f0d30c00640 1 -- 172.21.15.102:0/593242248 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6862 (secure 0 0 0) 0x7f0d2c05cee0 con 0x7f0d3415bd20 2024-04-20T21:23:59.086 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:23:59.087 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5533,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:23:59.087 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-04-20T21:23:59.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.089+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0d1006c920 msgr2=0x7f0d1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:59.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.089+0000 7f0d3b000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0d1006c920 0x7f0d1006ede0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009bd0 tx=0x7f0d2400b040 comp rx=0 tx=0).stop 2024-04-20T21:23:59.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.089+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d3415bd20 msgr2=0x7f0d34157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:59.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.089+0000 7f0d3b000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d3415bd20 0x7f0d34157e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c400 tx=0x7f0d2c00c8c0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.090+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 shutdown_connections 2024-04-20T21:23:59.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.090+0000 7f0d3b000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d3415bd20 0x7f0d34157e00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.090+0000 7f0d3b000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0d1006c920 0x7f0d1006ede0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.090+0000 7f0d3b000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d34155560 0x7f0d3415b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.090+0000 7f0d3b000640 1 --2- 172.21.15.102:0/593242248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d340c5cc0 0x7f0d340d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.090+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 >> 172.21.15.102:0/593242248 conn(0x7f0d3406cd40 msgr2=0x7f0d34154030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:59.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.090+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 shutdown_connections 2024-04-20T21:23:59.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:23:59.090+0000 7f0d3b000640 1 -- 172.21.15.102:0/593242248 wait complete. 2024-04-20T21:23:59.108 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-04-20T21:23:59.108 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 117 secs before thrashing 2024-04-20T21:23:59.892 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/11' 2024-04-20T21:23:59.892 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/16' 2024-04-20T21:23:59.892 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/32' 2024-04-20T21:23:59.892 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/30' 2024-04-20T21:23:59.892 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/14' 2024-04-20T21:23:59.892 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/18' 2024-04-20T21:23:59.892 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/13' 2024-04-20T21:23:59.892 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/33' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/17' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/35' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './167/37' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './167' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/19' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/15' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/31' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/36' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/20' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/12' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/39' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/10' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/34' 2024-04-20T21:23:59.893 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/38' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/11' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/16' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/32' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/30' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/14' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/18' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/13' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/33' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/17' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/35' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './168/37' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './168' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/19' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/15' 2024-04-20T21:23:59.894 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/31' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/36' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/20' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/12' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/39' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/10' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/34' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/38' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/11' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/16' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/32' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/30' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/14' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/18' 2024-04-20T21:23:59.895 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/13' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/33' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/17' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/35' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './169/37' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './169' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/19' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/15' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/31' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/36' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/20' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/12' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/39' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/10' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/34' 2024-04-20T21:23:59.896 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/38' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/11' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/16' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/32' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/30' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/14' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/18' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/13' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/33' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/17' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/35' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './170/37' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './170' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/19' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/15' 2024-04-20T21:23:59.897 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/31' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/36' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/20' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/12' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/39' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/10' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/34' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/38' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/11' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/16' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/32' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/30' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/14' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/18' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/13' 2024-04-20T21:23:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/33' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/17' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/35' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './171/37' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './171' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/19' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/15' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/31' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/36' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/20' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/12' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/39' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/10' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/34' 2024-04-20T21:23:59.899 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/38' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/11' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/16' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/32' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/30' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/14' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/18' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/13' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/33' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/17' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/35' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './172/37' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './172' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/19' 2024-04-20T21:23:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/15' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/31' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/36' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/20' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/12' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/39' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/10' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/34' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/38' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/11' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/16' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/32' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/30' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/14' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/18' 2024-04-20T21:23:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/13' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/33' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/17' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/35' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './173/37' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './173' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/19' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/15' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/31' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/36' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/20' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/12' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/39' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/10' 2024-04-20T21:23:59.902 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/34' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/38' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/11' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/16' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/32' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/30' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/14' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/18' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/13' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/33' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/17' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/35' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './174/37' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './174' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/19' 2024-04-20T21:23:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/15' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/31' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/36' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/20' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/12' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/39' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/10' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/34' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/38' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/11' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/16' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/32' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/30' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/14' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/18' 2024-04-20T21:23:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/13' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/33' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/17' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/35' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './175/37' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './175' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/19' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/15' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/31' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/36' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/20' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/12' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/39' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/10' 2024-04-20T21:23:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/34' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/38' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/11' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/16' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/32' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/30' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/14' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/18' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/13' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/33' 2024-04-20T21:23:59.906 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/17' 2024-04-20T21:23:59.907 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/35' 2024-04-20T21:23:59.907 INFO:tasks.workunit.client.0.smithi102.stdout:removed './176/37' 2024-04-20T21:23:59.907 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './176' 2024-04-20T21:23:59.907 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/19' 2024-04-20T21:23:59.907 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/15' 2024-04-20T21:24:00.153 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:24:00.154+0000 7fe2e5600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:24:03.382 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/31' 2024-04-20T21:24:03.382 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/36' 2024-04-20T21:24:03.382 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/20' 2024-04-20T21:24:03.382 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/12' 2024-04-20T21:24:03.382 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/39' 2024-04-20T21:24:03.382 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/10' 2024-04-20T21:24:03.382 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/34' 2024-04-20T21:24:03.382 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/38' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/11' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/16' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/32' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/30' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/14' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/18' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/13' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/33' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/17' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/35' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './177/37' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './177' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/19' 2024-04-20T21:24:03.383 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/15' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/31' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/36' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/20' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/12' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/39' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/10' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/34' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/38' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/11' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/16' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/32' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/30' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/14' 2024-04-20T21:24:03.384 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/18' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/13' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/33' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/17' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/35' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './178/37' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './178' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/19' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/15' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/31' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/36' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/20' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/12' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/39' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/10' 2024-04-20T21:24:03.385 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/34' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/38' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/11' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/16' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/32' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/30' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/14' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/18' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/13' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/33' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/17' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/35' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed './179/37' 2024-04-20T21:24:03.386 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './179' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/19' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/15' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/31' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/36' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/20' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/12' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/39' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/10' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/34' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/38' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/11' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/16' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/32' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/30' 2024-04-20T21:24:03.387 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/14' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/18' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/13' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/33' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/17' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/35' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './180/37' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './180' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/19' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/15' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/31' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/36' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/20' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/12' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/39' 2024-04-20T21:24:03.388 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/10' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/34' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/38' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/11' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/16' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/32' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/30' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/14' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/18' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/13' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/33' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/17' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/35' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed './181/37' 2024-04-20T21:24:03.389 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './181' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/19' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/15' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/31' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/36' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/20' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/12' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/39' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/10' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/34' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/38' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/11' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/16' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/32' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/30' 2024-04-20T21:24:03.390 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/14' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/18' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/13' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/33' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/17' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/35' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './182/37' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './182' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/19' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/15' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/31' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/36' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/20' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/12' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/39' 2024-04-20T21:24:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/10' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/34' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/38' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/11' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/16' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/32' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/30' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/14' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/18' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/13' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/33' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/17' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/35' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed './183/37' 2024-04-20T21:24:03.392 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './183' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/19' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/15' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/31' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/36' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/20' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/12' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/39' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/10' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/34' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/38' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/11' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/16' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/32' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/30' 2024-04-20T21:24:03.393 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/14' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/18' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/13' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/33' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/17' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/35' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './184/37' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './184' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/19' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/15' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/31' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/36' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/20' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/12' 2024-04-20T21:24:03.394 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/39' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/10' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/34' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/38' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/11' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/16' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/32' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/30' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/14' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/18' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/13' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/33' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/17' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/35' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed './185/37' 2024-04-20T21:24:03.395 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './185' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/19' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/15' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/31' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/36' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/20' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/12' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/39' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/10' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/34' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/38' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/11' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/16' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/32' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/30' 2024-04-20T21:24:03.396 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/14' 2024-04-20T21:24:03.397 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/18' 2024-04-20T21:24:06.091 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/13' 2024-04-20T21:24:06.091 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/33' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/17' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/35' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './186/37' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './186' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/19' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/15' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/31' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/36' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/20' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/12' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/39' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/10' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/34' 2024-04-20T21:24:06.092 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/38' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/11' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/16' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/32' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/30' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/14' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/18' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/13' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/33' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/17' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/35' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './187/37' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './187' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/19' 2024-04-20T21:24:06.093 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/15' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/31' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/36' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/20' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/12' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/39' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/10' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/34' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/38' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/11' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/16' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/32' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/30' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/14' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/18' 2024-04-20T21:24:06.094 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/13' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/33' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/17' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/35' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './188/37' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './188' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/19' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/15' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/31' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/36' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/20' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/12' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/39' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/10' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/34' 2024-04-20T21:24:06.095 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/38' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/11' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/16' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/32' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/30' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/14' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/18' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/13' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/33' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/17' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/35' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './189/37' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './189' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/19' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/15' 2024-04-20T21:24:06.096 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/31' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/36' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/20' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/12' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/39' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/10' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/34' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/38' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/11' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/16' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/32' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/30' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/14' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/18' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/13' 2024-04-20T21:24:06.097 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/33' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/17' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/35' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './190/37' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './190' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/19' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/15' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/31' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/36' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/20' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/12' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/39' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/10' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/34' 2024-04-20T21:24:06.098 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/38' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/11' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/16' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/32' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/30' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/14' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/18' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/13' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/33' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/17' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/35' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './191/37' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './191' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/19' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/15' 2024-04-20T21:24:06.099 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/31' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/36' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/20' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/12' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/39' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/10' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/34' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/38' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/11' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/16' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/32' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/30' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/14' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/18' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/13' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/33' 2024-04-20T21:24:06.100 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/17' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/35' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './192/37' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './192' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/19' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/15' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/31' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/36' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/20' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/12' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/39' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/10' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/34' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/38' 2024-04-20T21:24:06.101 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/11' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/16' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/32' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/30' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/14' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/18' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/13' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/33' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/17' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/35' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './193/37' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './193' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/19' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/15' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/31' 2024-04-20T21:24:06.102 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/36' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/20' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/12' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/39' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/10' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/34' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/38' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/11' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/16' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/32' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/30' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/14' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/18' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/13' 2024-04-20T21:24:06.103 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/33' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/17' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/35' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './194/37' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './194' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/19' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/15' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/31' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/36' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/20' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/12' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/39' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/10' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/34' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/38' 2024-04-20T21:24:06.104 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/11' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/16' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/32' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/30' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/14' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/18' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/13' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/33' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/17' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/35' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './195/37' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './195' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/19' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/15' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/31' 2024-04-20T21:24:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/36' 2024-04-20T21:24:06.106 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/20' 2024-04-20T21:24:06.106 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/12' 2024-04-20T21:24:06.106 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/39' 2024-04-20T21:24:06.106 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/10' 2024-04-20T21:24:08.743 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/34' 2024-04-20T21:24:08.743 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/38' 2024-04-20T21:24:08.743 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/11' 2024-04-20T21:24:08.743 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/16' 2024-04-20T21:24:08.743 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/32' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/30' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/14' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/18' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/13' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/33' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/17' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/35' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './196/37' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './196' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/19' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/15' 2024-04-20T21:24:08.744 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/31' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/36' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/20' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/12' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/39' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/10' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/34' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/38' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/11' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/16' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/32' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/30' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/14' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/18' 2024-04-20T21:24:08.745 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/13' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/33' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/17' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/35' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './197/37' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './197' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/19' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/15' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/31' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/36' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/20' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/12' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/39' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/10' 2024-04-20T21:24:08.746 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/34' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/38' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/11' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/16' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/32' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/30' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/14' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/18' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/13' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/33' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/17' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/35' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './198/37' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './198' 2024-04-20T21:24:08.747 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/19' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/15' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/31' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/36' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/20' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/12' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/39' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/10' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/34' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/38' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/11' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/16' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/32' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/30' 2024-04-20T21:24:08.748 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/14' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/18' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/13' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/33' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/17' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/35' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './199/37' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './199' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/21' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/27' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/31' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/36' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/20' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/39' 2024-04-20T21:24:08.749 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/34' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/38' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/28' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/24' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/29' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/32' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/22' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/25' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/30' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/33' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/23' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/35' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/26' 2024-04-20T21:24:08.750 INFO:tasks.workunit.client.0.smithi102.stdout:removed './200/37' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './200' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/21' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/27' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/31' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/36' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/20' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/39' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/34' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/38' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/28' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/24' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/29' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/32' 2024-04-20T21:24:08.751 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/22' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/25' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/30' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/33' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/23' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/35' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/26' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './201/37' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './201' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/21' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/27' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/31' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/36' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/20' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/39' 2024-04-20T21:24:08.752 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/34' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/38' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/28' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/24' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/29' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/32' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/22' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/25' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/30' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/33' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/23' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/35' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/26' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './202/37' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './202' 2024-04-20T21:24:08.753 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/21' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/27' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/31' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/36' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/20' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/39' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/34' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/38' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/28' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/24' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/29' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/32' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/22' 2024-04-20T21:24:08.754 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/25' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/30' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/33' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/23' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/35' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/26' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './203/37' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './203' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/21' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/27' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/31' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/36' 2024-04-20T21:24:08.755 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/20' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/39' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/34' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/38' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/28' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/24' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/29' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/32' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/22' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/25' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/30' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/33' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/23' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/35' 2024-04-20T21:24:08.756 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/26' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './204/37' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './204' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/21' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/27' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/31' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/36' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/20' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/39' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/34' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/38' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/28' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/24' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/29' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/32' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/22' 2024-04-20T21:24:08.757 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/25' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/30' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/33' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/23' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/35' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/26' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed './205/37' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './205' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/21' 2024-04-20T21:24:08.758 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/27' 2024-04-20T21:24:08.759 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/31' 2024-04-20T21:24:08.759 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/36' 2024-04-20T21:24:08.759 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/20' 2024-04-20T21:24:08.759 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/39' 2024-04-20T21:24:10.684 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/34' 2024-04-20T21:24:10.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/38' 2024-04-20T21:24:10.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/28' 2024-04-20T21:24:10.704 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/24' 2024-04-20T21:24:10.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/29' 2024-04-20T21:24:10.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/32' 2024-04-20T21:24:10.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/22' 2024-04-20T21:24:10.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/25' 2024-04-20T21:24:10.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/30' 2024-04-20T21:24:10.705 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/33' 2024-04-20T21:24:10.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/23' 2024-04-20T21:24:10.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/35' 2024-04-20T21:24:10.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/26' 2024-04-20T21:24:10.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './206/37' 2024-04-20T21:24:10.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './206' 2024-04-20T21:24:10.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/21' 2024-04-20T21:24:10.706 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/27' 2024-04-20T21:24:10.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/31' 2024-04-20T21:24:10.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/36' 2024-04-20T21:24:10.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/20' 2024-04-20T21:24:10.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/39' 2024-04-20T21:24:10.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/34' 2024-04-20T21:24:10.707 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/38' 2024-04-20T21:24:10.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/28' 2024-04-20T21:24:10.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/24' 2024-04-20T21:24:10.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/29' 2024-04-20T21:24:10.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/32' 2024-04-20T21:24:10.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/22' 2024-04-20T21:24:10.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/25' 2024-04-20T21:24:10.708 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/30' 2024-04-20T21:24:10.709 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/33' 2024-04-20T21:24:10.709 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/23' 2024-04-20T21:24:10.709 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/35' 2024-04-20T21:24:10.709 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/26' 2024-04-20T21:24:10.709 INFO:tasks.workunit.client.0.smithi102.stdout:removed './207/37' 2024-04-20T21:24:10.709 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './207' 2024-04-20T21:24:10.710 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/21' 2024-04-20T21:24:10.710 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/27' 2024-04-20T21:24:10.710 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/31' 2024-04-20T21:24:10.710 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/36' 2024-04-20T21:24:10.710 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/20' 2024-04-20T21:24:10.710 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/39' 2024-04-20T21:24:10.711 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/34' 2024-04-20T21:24:10.711 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/38' 2024-04-20T21:24:10.711 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/28' 2024-04-20T21:24:10.711 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/24' 2024-04-20T21:24:10.711 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/29' 2024-04-20T21:24:10.711 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/32' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/22' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/25' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/30' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/33' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/23' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/35' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/26' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed './208/37' 2024-04-20T21:24:10.712 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './208' 2024-04-20T21:24:10.713 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/21' 2024-04-20T21:24:10.713 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/27' 2024-04-20T21:24:10.713 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/31' 2024-04-20T21:24:10.713 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/36' 2024-04-20T21:24:10.713 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/20' 2024-04-20T21:24:10.713 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/39' 2024-04-20T21:24:10.713 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/34' 2024-04-20T21:24:10.714 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/38' 2024-04-20T21:24:10.714 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/28' 2024-04-20T21:24:10.714 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/24' 2024-04-20T21:24:10.714 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/29' 2024-04-20T21:24:10.714 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/32' 2024-04-20T21:24:10.714 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/22' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/25' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/30' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/33' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/23' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/35' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/26' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './209/37' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './209' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/21' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/27' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/31' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/36' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/20' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/39' 2024-04-20T21:24:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/34' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/38' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/28' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/24' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/29' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/32' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/22' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/25' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/30' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/33' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/23' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/35' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/26' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './210/37' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './210' 2024-04-20T21:24:10.716 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/21' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/27' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/31' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/36' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/20' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/39' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/34' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/38' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/28' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/24' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/29' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/32' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/22' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/25' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/30' 2024-04-20T21:24:10.717 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/33' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/23' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/35' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/26' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './211/37' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './211' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/21' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/27' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/31' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/36' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/20' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/39' 2024-04-20T21:24:10.718 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/34' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/38' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/28' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/24' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/29' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/32' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/22' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/25' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/30' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/33' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/23' 2024-04-20T21:24:10.719 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/35' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/26' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './212/37' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './212' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/21' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/27' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/31' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/36' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/20' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/39' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/34' 2024-04-20T21:24:10.720 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/38' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/28' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/24' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/29' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/32' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/22' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/25' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/30' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/33' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/23' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/35' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/26' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed './213/37' 2024-04-20T21:24:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './213' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/21' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/27' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/31' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/36' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/20' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/39' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/34' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/38' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/28' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/24' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/29' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/32' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/22' 2024-04-20T21:24:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/25' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/30' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/33' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/23' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/35' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/26' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './214/37' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './214' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/21' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/27' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/31' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/36' 2024-04-20T21:24:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/20' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/39' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/34' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/38' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/28' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/24' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/29' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/32' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/22' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/25' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/30' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/33' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/23' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/35' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/26' 2024-04-20T21:24:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:removed './215/37' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './215' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/21' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/27' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/31' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/36' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/20' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/39' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/34' 2024-04-20T21:24:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/38' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/28' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/24' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/29' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/32' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/22' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/25' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/30' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/33' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/23' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/35' 2024-04-20T21:24:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/26' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './216/37' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './216' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/21' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/27' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/31' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/36' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/20' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/39' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/34' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/38' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/28' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/24' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/29' 2024-04-20T21:24:13.294 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/32' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/22' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/25' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/30' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/33' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/23' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/35' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/26' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './217/37' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './217' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/21' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/27' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/31' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/36' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/20' 2024-04-20T21:24:13.295 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/39' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/34' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/38' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/28' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/24' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/29' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/32' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/22' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/25' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/30' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/33' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/23' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/35' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/26' 2024-04-20T21:24:13.296 INFO:tasks.workunit.client.0.smithi102.stdout:removed './218/37' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './218' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/21' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/27' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/31' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/36' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/20' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/39' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/34' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/38' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/28' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/24' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/29' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/32' 2024-04-20T21:24:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/22' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/25' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/30' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/33' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/23' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/35' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/26' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './219/37' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './219' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/21' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/27' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/31' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/36' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/20' 2024-04-20T21:24:13.298 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/39' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/34' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/38' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/28' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/24' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/29' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/32' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/22' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/25' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/30' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/33' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/23' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/35' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/26' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed './220/37' 2024-04-20T21:24:13.299 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './220' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/21' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/27' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/31' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/36' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/20' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/39' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/34' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/38' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/28' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/24' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/29' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/32' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/22' 2024-04-20T21:24:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/25' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/30' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/33' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/23' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/35' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/26' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './221/37' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './221' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/21' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/27' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/31' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/36' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/20' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/39' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/34' 2024-04-20T21:24:13.301 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/38' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/28' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/24' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/29' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/32' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/22' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/25' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/30' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/33' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/23' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/35' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/26' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './222/37' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './222' 2024-04-20T21:24:13.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/21' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/27' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/31' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/36' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/20' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/39' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/34' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/38' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/28' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/24' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/29' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/32' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/22' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/25' 2024-04-20T21:24:13.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/30' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/33' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/23' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/35' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/26' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './223/37' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './223' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/21' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/27' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/31' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/36' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/20' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/39' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/34' 2024-04-20T21:24:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/38' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/28' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/24' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/29' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/32' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/22' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/25' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/30' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/33' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/23' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/35' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/26' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './224/37' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './224' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/21' 2024-04-20T21:24:13.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/27' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/31' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/36' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/20' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/39' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/34' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/38' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/28' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/24' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/29' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/32' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/22' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/25' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/30' 2024-04-20T21:24:13.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/33' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/23' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/35' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/26' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './225/37' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './225' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/21' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/27' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/31' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/36' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/20' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/39' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/34' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/38' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/28' 2024-04-20T21:24:13.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/24' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/29' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/32' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/22' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/25' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/30' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/33' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/23' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/35' 2024-04-20T21:24:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/26' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './226/37' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './226' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/21' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/27' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/31' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/36' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/20' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/39' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/34' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/38' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/28' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/24' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/29' 2024-04-20T21:24:17.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/32' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/22' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/25' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/30' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/33' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/23' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/35' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/26' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './227/37' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './227' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/21' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/27' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/31' 2024-04-20T21:24:17.192 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/36' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/20' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/39' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/34' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/38' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/28' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/24' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/29' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/32' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/22' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/25' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/30' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/33' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/23' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/35' 2024-04-20T21:24:17.193 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/26' 2024-04-20T21:24:17.194 INFO:tasks.workunit.client.0.smithi102.stdout:removed './228/37' 2024-04-20T21:24:17.194 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './228' 2024-04-20T21:24:17.194 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/21' 2024-04-20T21:24:17.194 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/27' 2024-04-20T21:24:17.194 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/31' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/36' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/20' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/39' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/34' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/38' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/28' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/24' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/29' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/32' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/22' 2024-04-20T21:24:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/25' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/30' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/33' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/23' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/35' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/26' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './229/37' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './229' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/21' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/27' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/31' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/36' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/20' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/39' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/34' 2024-04-20T21:24:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/38' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/28' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/24' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/29' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/32' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/22' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/25' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/30' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/33' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/23' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/35' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/26' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './230/37' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './230' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/21' 2024-04-20T21:24:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/27' 2024-04-20T21:24:17.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/31' 2024-04-20T21:24:17.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/36' 2024-04-20T21:24:17.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/20' 2024-04-20T21:24:17.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/39' 2024-04-20T21:24:17.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/34' 2024-04-20T21:24:17.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/38' 2024-04-20T21:24:17.198 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/28' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/24' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/29' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/32' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/22' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/25' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/30' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/33' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/23' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/35' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/26' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './231/37' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './231' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/21' 2024-04-20T21:24:17.199 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/27' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/31' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/36' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/20' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/39' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/34' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/38' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/28' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/24' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/29' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/32' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/22' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/25' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/30' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/33' 2024-04-20T21:24:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/23' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/35' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/26' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './232/37' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './232' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/21' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/27' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/31' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/36' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/20' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/39' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/34' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/38' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/28' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/24' 2024-04-20T21:24:17.201 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/29' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/32' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/22' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/25' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/30' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/33' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/23' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/35' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/26' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed './233/37' 2024-04-20T21:24:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './233' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/21' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/27' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/31' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/36' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/20' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/39' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/34' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/38' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/28' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/24' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/29' 2024-04-20T21:24:17.203 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/32' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/22' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/25' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/30' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/33' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/23' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/35' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/26' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './234/37' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './234' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/21' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/27' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/31' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/36' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/20' 2024-04-20T21:24:17.204 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/39' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/34' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/38' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/28' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/24' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/29' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/32' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/22' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/25' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/30' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/33' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/23' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/35' 2024-04-20T21:24:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/26' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './235/37' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './235' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/21' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/27' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/31' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/36' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/20' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/39' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/34' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/38' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/28' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/24' 2024-04-20T21:24:17.206 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/29' 2024-04-20T21:24:17.207 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/32' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/22' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/25' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/30' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/33' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/23' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/35' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/26' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './236/37' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './236' 2024-04-20T21:24:20.302 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/21' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/27' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/31' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/36' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/20' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/39' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/34' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/38' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/28' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/24' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/29' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/32' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/22' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/25' 2024-04-20T21:24:20.303 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/30' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/33' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/23' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/35' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/26' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './237/37' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './237' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/21' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/27' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/31' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/36' 2024-04-20T21:24:20.304 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/20' 2024-04-20T21:24:20.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/39' 2024-04-20T21:24:20.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/34' 2024-04-20T21:24:20.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/38' 2024-04-20T21:24:20.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/28' 2024-04-20T21:24:20.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/24' 2024-04-20T21:24:20.305 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/29' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/32' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/22' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/25' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/30' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/33' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/23' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/35' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/26' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './238/37' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './238' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/21' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/27' 2024-04-20T21:24:20.306 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/31' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/36' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/20' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/39' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/34' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/38' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/28' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/24' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/29' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/32' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/22' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/25' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/30' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/33' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/23' 2024-04-20T21:24:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/35' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/26' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './239/37' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './239' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/21' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/27' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/31' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/36' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/20' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/39' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/34' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/38' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/28' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/24' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/29' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/32' 2024-04-20T21:24:20.308 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/22' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/25' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/30' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/33' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/23' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/35' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/26' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './240/37' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './240' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/21' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/27' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/31' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/36' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/20' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/39' 2024-04-20T21:24:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/34' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/38' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/28' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/24' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/29' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/32' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/22' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/25' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/30' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/33' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/23' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/35' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/26' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './241/37' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './241' 2024-04-20T21:24:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/21' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/27' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/31' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/36' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/20' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/39' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/34' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/38' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/28' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/24' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/29' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/32' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/22' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/25' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/30' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/33' 2024-04-20T21:24:20.311 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/23' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/35' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/26' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './242/37' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './242' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/21' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/27' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/31' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/36' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/20' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/39' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/34' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/38' 2024-04-20T21:24:20.312 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/28' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/24' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/29' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/32' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/22' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/25' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/30' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/33' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/23' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/35' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/26' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './243/37' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './243' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/21' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/27' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/31' 2024-04-20T21:24:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/36' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/20' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/39' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/34' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/38' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/28' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/24' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/29' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/32' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/22' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/25' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/30' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/33' 2024-04-20T21:24:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/23' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/35' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/26' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './244/37' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './244' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/21' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/27' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/31' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/36' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/20' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/39' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/34' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/38' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/28' 2024-04-20T21:24:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/24' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/29' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/32' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/22' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/25' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/30' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/33' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/23' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/35' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/26' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './245/37' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './245' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/21' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/27' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/31' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/36' 2024-04-20T21:24:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/20' 2024-04-20T21:24:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/39' 2024-04-20T21:24:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/34' 2024-04-20T21:24:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/38' 2024-04-20T21:24:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/28' 2024-04-20T21:24:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/24' 2024-04-20T21:24:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/29' 2024-04-20T21:24:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/32' 2024-04-20T21:24:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/22' 2024-04-20T21:24:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/25' 2024-04-20T21:24:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/30' 2024-04-20T21:24:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/33' 2024-04-20T21:24:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/23' 2024-04-20T21:24:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/35' 2024-04-20T21:24:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/26' 2024-04-20T21:24:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:removed './246/37' 2024-04-20T21:24:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './246' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/21' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/27' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/31' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/36' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/20' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/39' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/34' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/38' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/28' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/24' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/29' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/32' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/22' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/25' 2024-04-20T21:24:22.177 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/30' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/33' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/23' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/35' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/26' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './247/37' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './247' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/21' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/27' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/31' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/36' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/20' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/39' 2024-04-20T21:24:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/34' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/38' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/28' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/24' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/29' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/32' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/22' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/25' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/30' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/33' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/23' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/35' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/26' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './248/37' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './248' 2024-04-20T21:24:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/21' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/27' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/31' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/36' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/20' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/39' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/34' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/38' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/28' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/24' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/29' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/32' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/22' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/25' 2024-04-20T21:24:22.180 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/30' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/33' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/23' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/35' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/26' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './249/37' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './249' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/21' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/27' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/31' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/36' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/20' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/39' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/34' 2024-04-20T21:24:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/38' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/28' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/24' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/29' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/32' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/22' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/25' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/30' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/33' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/23' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/35' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/26' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './250/37' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './250' 2024-04-20T21:24:22.182 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/21' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/27' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/31' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/36' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/20' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/39' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/34' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/38' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/28' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/24' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/29' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/32' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/22' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/25' 2024-04-20T21:24:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/30' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/33' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/23' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/35' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/26' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './251/37' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './251' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/21' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/27' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/31' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/36' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/20' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/39' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/34' 2024-04-20T21:24:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/38' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/28' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/24' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/29' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/32' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/22' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/25' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/30' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/33' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/23' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/35' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/26' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './252/37' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './252' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/21' 2024-04-20T21:24:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/27' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/31' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/36' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/20' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/39' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/34' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/38' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/28' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/24' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/29' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/32' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/22' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/25' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/30' 2024-04-20T21:24:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/33' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/23' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/35' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/26' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './253/37' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './253' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/21' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/27' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/31' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/36' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/20' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/39' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/34' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/38' 2024-04-20T21:24:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/28' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/24' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/29' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/32' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/22' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/25' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/30' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/33' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/23' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/35' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/26' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './254/37' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './254' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/21' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/27' 2024-04-20T21:24:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/31' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/36' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/20' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/39' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/34' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/38' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/28' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/24' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/29' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/32' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/22' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/25' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/30' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/33' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/23' 2024-04-20T21:24:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/35' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/26' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './255/37' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './255' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/21' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/27' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/31' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/36' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/20' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/39' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/34' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/38' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/28' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/24' 2024-04-20T21:24:22.190 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/29' 2024-04-20T21:24:22.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/32' 2024-04-20T21:24:22.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/22' 2024-04-20T21:24:22.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/25' 2024-04-20T21:24:22.191 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/30' 2024-04-20T21:24:25.672 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:25.680 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:27.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/33' 2024-04-20T21:24:27.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/23' 2024-04-20T21:24:27.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/35' 2024-04-20T21:24:27.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/26' 2024-04-20T21:24:27.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './256/37' 2024-04-20T21:24:27.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './256' 2024-04-20T21:24:27.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/21' 2024-04-20T21:24:27.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/27' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/31' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/36' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/20' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/39' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/34' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/38' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/28' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/24' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/29' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/32' 2024-04-20T21:24:27.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/22' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/25' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/30' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/33' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/23' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/35' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/26' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './257/37' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './257' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/21' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/27' 2024-04-20T21:24:27.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/31' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/36' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/20' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/39' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/34' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/38' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/28' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/24' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/29' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/32' 2024-04-20T21:24:27.578 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/22' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/25' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/30' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/33' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/23' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/35' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/26' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './258/37' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './258' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/21' 2024-04-20T21:24:27.579 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/27' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/31' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/36' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/20' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/39' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/34' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/38' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/28' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/24' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/29' 2024-04-20T21:24:27.580 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/32' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/22' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/25' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/30' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/33' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/23' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/35' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/26' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './259/37' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './259' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/21' 2024-04-20T21:24:27.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/27' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/31' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/36' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/20' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/39' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/34' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/38' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/28' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/24' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/29' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/32' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/22' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/25' 2024-04-20T21:24:27.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/30' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/33' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/23' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/35' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/26' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './260/37' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './260' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/21' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/27' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/31' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/36' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/20' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/39' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/34' 2024-04-20T21:24:27.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/38' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/28' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/24' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/29' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/32' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/22' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/25' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/30' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/33' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/23' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/35' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/26' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './261/37' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './261' 2024-04-20T21:24:27.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/21' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/27' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/31' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/36' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/20' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/39' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/34' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/38' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/28' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/24' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/29' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/32' 2024-04-20T21:24:27.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/22' 2024-04-20T21:24:27.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/25' 2024-04-20T21:24:27.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/30' 2024-04-20T21:24:27.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/33' 2024-04-20T21:24:27.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/23' 2024-04-20T21:24:27.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/35' 2024-04-20T21:24:27.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/26' 2024-04-20T21:24:27.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './262/37' 2024-04-20T21:24:27.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './262' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/21' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/27' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/31' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/36' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/20' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/39' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/34' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/38' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/28' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/24' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/29' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/32' 2024-04-20T21:24:27.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/22' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/25' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/30' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/33' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/23' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/35' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/26' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './263/37' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './263' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/21' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/27' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/31' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/36' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/20' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/39' 2024-04-20T21:24:27.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/34' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/38' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/28' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/24' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/29' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/32' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/22' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/25' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/30' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/33' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/23' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/35' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/26' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed './264/37' 2024-04-20T21:24:27.589 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './264' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/21' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/27' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/31' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/36' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/20' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/39' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/34' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/38' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/28' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/24' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/29' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/32' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/22' 2024-04-20T21:24:27.590 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/25' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/30' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/33' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/23' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/35' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/26' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './265/37' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './265' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/21' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/27' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/31' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/36' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/20' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/39' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/34' 2024-04-20T21:24:27.591 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/38' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/28' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/24' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/29' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/32' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/22' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/25' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/30' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/33' 2024-04-20T21:24:27.592 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/23' 2024-04-20T21:24:30.405 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/35' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/26' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './266/37' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './266' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/21' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/27' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/31' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/36' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/20' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/39' 2024-04-20T21:24:30.406 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/34' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/38' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/28' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/24' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/29' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/32' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/22' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/25' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/30' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/33' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/23' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/35' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/26' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed './267/37' 2024-04-20T21:24:30.407 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './267' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/21' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/27' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/31' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/36' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/20' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/39' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/34' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/38' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/28' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/24' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/29' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/32' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/22' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/25' 2024-04-20T21:24:30.408 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/30' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/33' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/23' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/35' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/26' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './268/37' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './268' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/21' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/27' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/31' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/36' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/20' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/39' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/34' 2024-04-20T21:24:30.409 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/38' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/28' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/24' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/29' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/32' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/22' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/25' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/30' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/33' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/23' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/35' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/26' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './269/37' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './269' 2024-04-20T21:24:30.410 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/21' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/27' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/31' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/36' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/20' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/39' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/34' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/38' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/28' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/24' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/29' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/32' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/22' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/25' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/30' 2024-04-20T21:24:30.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/33' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/23' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/35' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/26' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './270/37' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './270' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/21' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/27' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/31' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/36' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/20' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/39' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/34' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/38' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/28' 2024-04-20T21:24:30.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/24' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/29' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/32' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/22' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/25' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/30' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/33' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/23' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/35' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/26' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './271/37' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './271' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/21' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/27' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/31' 2024-04-20T21:24:30.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/36' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/20' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/39' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/34' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/38' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/28' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/24' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/29' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/32' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/22' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/25' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/30' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/33' 2024-04-20T21:24:30.414 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/23' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/35' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/26' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './272/37' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './272' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/21' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/27' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/31' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/36' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/20' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/39' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/34' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/38' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/28' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/24' 2024-04-20T21:24:30.415 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/29' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/32' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/22' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/25' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/30' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/33' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/23' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/35' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/26' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './273/37' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './273' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/21' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/27' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/31' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/36' 2024-04-20T21:24:30.416 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/20' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/39' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/34' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/38' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/28' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/24' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/29' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/32' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/22' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/25' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/30' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/33' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/23' 2024-04-20T21:24:30.417 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/35' 2024-04-20T21:24:30.418 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/26' 2024-04-20T21:24:30.418 INFO:tasks.workunit.client.0.smithi102.stdout:removed './274/37' 2024-04-20T21:24:30.418 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './274' 2024-04-20T21:24:30.418 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/21' 2024-04-20T21:24:30.418 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/27' 2024-04-20T21:24:30.418 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/31' 2024-04-20T21:24:30.418 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/36' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/20' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/39' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/34' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/38' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/28' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/24' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/29' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/32' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/22' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/25' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/30' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/33' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/23' 2024-04-20T21:24:30.419 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/35' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/26' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './275/37' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './275' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/21' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/27' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/31' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/36' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/20' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/39' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/34' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/38' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/28' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/24' 2024-04-20T21:24:30.420 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/29' 2024-04-20T21:24:30.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/32' 2024-04-20T21:24:30.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/22' 2024-04-20T21:24:30.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/25' 2024-04-20T21:24:30.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/30' 2024-04-20T21:24:30.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/33' 2024-04-20T21:24:30.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/23' 2024-04-20T21:24:30.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/35' 2024-04-20T21:24:30.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/26' 2024-04-20T21:24:33.619 INFO:tasks.workunit.client.0.smithi102.stdout:removed './276/37' 2024-04-20T21:24:33.619 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './276' 2024-04-20T21:24:33.620 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/21' 2024-04-20T21:24:33.620 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/27' 2024-04-20T21:24:33.620 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/31' 2024-04-20T21:24:33.620 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/36' 2024-04-20T21:24:33.620 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/20' 2024-04-20T21:24:33.621 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/39' 2024-04-20T21:24:33.621 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/34' 2024-04-20T21:24:33.621 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/38' 2024-04-20T21:24:33.621 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/28' 2024-04-20T21:24:33.621 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/24' 2024-04-20T21:24:33.622 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/29' 2024-04-20T21:24:33.622 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/32' 2024-04-20T21:24:33.622 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/22' 2024-04-20T21:24:33.622 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/25' 2024-04-20T21:24:33.623 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/30' 2024-04-20T21:24:33.623 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/33' 2024-04-20T21:24:33.623 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/23' 2024-04-20T21:24:33.623 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/35' 2024-04-20T21:24:33.623 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/26' 2024-04-20T21:24:33.624 INFO:tasks.workunit.client.0.smithi102.stdout:removed './277/37' 2024-04-20T21:24:33.624 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './277' 2024-04-20T21:24:33.624 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/21' 2024-04-20T21:24:33.624 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/27' 2024-04-20T21:24:33.624 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/31' 2024-04-20T21:24:33.625 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/36' 2024-04-20T21:24:33.625 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/20' 2024-04-20T21:24:33.625 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/39' 2024-04-20T21:24:33.625 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/34' 2024-04-20T21:24:33.626 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/38' 2024-04-20T21:24:33.626 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/28' 2024-04-20T21:24:33.626 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/24' 2024-04-20T21:24:33.626 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/29' 2024-04-20T21:24:33.626 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/32' 2024-04-20T21:24:33.627 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/22' 2024-04-20T21:24:33.627 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/25' 2024-04-20T21:24:33.627 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/30' 2024-04-20T21:24:33.627 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/33' 2024-04-20T21:24:33.627 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/23' 2024-04-20T21:24:33.627 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/35' 2024-04-20T21:24:33.627 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/26' 2024-04-20T21:24:33.628 INFO:tasks.workunit.client.0.smithi102.stdout:removed './278/37' 2024-04-20T21:24:33.628 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './278' 2024-04-20T21:24:33.628 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/21' 2024-04-20T21:24:33.628 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/27' 2024-04-20T21:24:33.628 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/31' 2024-04-20T21:24:33.628 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/36' 2024-04-20T21:24:33.628 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/20' 2024-04-20T21:24:33.628 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/39' 2024-04-20T21:24:33.629 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/34' 2024-04-20T21:24:33.629 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/38' 2024-04-20T21:24:33.629 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/28' 2024-04-20T21:24:33.629 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/24' 2024-04-20T21:24:33.629 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/29' 2024-04-20T21:24:33.629 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/32' 2024-04-20T21:24:33.629 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/22' 2024-04-20T21:24:33.630 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/25' 2024-04-20T21:24:33.630 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/30' 2024-04-20T21:24:33.630 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/33' 2024-04-20T21:24:33.630 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/23' 2024-04-20T21:24:33.630 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/35' 2024-04-20T21:24:33.631 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/26' 2024-04-20T21:24:33.631 INFO:tasks.workunit.client.0.smithi102.stdout:removed './279/37' 2024-04-20T21:24:33.631 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './279' 2024-04-20T21:24:33.631 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/21' 2024-04-20T21:24:33.631 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/27' 2024-04-20T21:24:33.632 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/31' 2024-04-20T21:24:33.632 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/36' 2024-04-20T21:24:33.632 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/20' 2024-04-20T21:24:33.632 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/39' 2024-04-20T21:24:33.632 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/34' 2024-04-20T21:24:33.633 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/38' 2024-04-20T21:24:33.633 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/28' 2024-04-20T21:24:33.633 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/24' 2024-04-20T21:24:33.633 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/29' 2024-04-20T21:24:33.633 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/32' 2024-04-20T21:24:33.634 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/22' 2024-04-20T21:24:33.634 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/25' 2024-04-20T21:24:33.634 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/30' 2024-04-20T21:24:33.634 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/33' 2024-04-20T21:24:33.634 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/23' 2024-04-20T21:24:33.635 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/35' 2024-04-20T21:24:33.635 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/26' 2024-04-20T21:24:33.635 INFO:tasks.workunit.client.0.smithi102.stdout:removed './280/37' 2024-04-20T21:24:33.635 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './280' 2024-04-20T21:24:33.635 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/21' 2024-04-20T21:24:33.636 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/27' 2024-04-20T21:24:33.636 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/31' 2024-04-20T21:24:33.636 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/36' 2024-04-20T21:24:33.636 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/20' 2024-04-20T21:24:33.636 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/39' 2024-04-20T21:24:33.636 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/34' 2024-04-20T21:24:33.637 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/38' 2024-04-20T21:24:33.637 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/28' 2024-04-20T21:24:33.637 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/24' 2024-04-20T21:24:33.637 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/29' 2024-04-20T21:24:33.637 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/32' 2024-04-20T21:24:33.637 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/22' 2024-04-20T21:24:33.637 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/25' 2024-04-20T21:24:33.638 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/30' 2024-04-20T21:24:33.638 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/33' 2024-04-20T21:24:33.638 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/23' 2024-04-20T21:24:33.638 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/35' 2024-04-20T21:24:33.638 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/26' 2024-04-20T21:24:33.639 INFO:tasks.workunit.client.0.smithi102.stdout:removed './281/37' 2024-04-20T21:24:33.639 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './281' 2024-04-20T21:24:33.639 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/21' 2024-04-20T21:24:33.639 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/27' 2024-04-20T21:24:33.639 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/31' 2024-04-20T21:24:33.639 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/36' 2024-04-20T21:24:33.640 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/20' 2024-04-20T21:24:33.640 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/39' 2024-04-20T21:24:33.640 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/34' 2024-04-20T21:24:33.640 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/38' 2024-04-20T21:24:33.640 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/28' 2024-04-20T21:24:33.640 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/24' 2024-04-20T21:24:33.640 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/29' 2024-04-20T21:24:33.641 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/32' 2024-04-20T21:24:33.641 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/22' 2024-04-20T21:24:33.641 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/25' 2024-04-20T21:24:33.641 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/30' 2024-04-20T21:24:33.641 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/33' 2024-04-20T21:24:33.641 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/23' 2024-04-20T21:24:33.642 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/35' 2024-04-20T21:24:33.642 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/26' 2024-04-20T21:24:33.642 INFO:tasks.workunit.client.0.smithi102.stdout:removed './282/37' 2024-04-20T21:24:33.642 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './282' 2024-04-20T21:24:33.642 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/21' 2024-04-20T21:24:33.642 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/27' 2024-04-20T21:24:33.643 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/31' 2024-04-20T21:24:33.643 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/36' 2024-04-20T21:24:33.643 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/20' 2024-04-20T21:24:33.643 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/39' 2024-04-20T21:24:33.643 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/34' 2024-04-20T21:24:33.643 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/38' 2024-04-20T21:24:33.644 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/28' 2024-04-20T21:24:33.644 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/24' 2024-04-20T21:24:33.644 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/29' 2024-04-20T21:24:33.644 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/32' 2024-04-20T21:24:33.644 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/22' 2024-04-20T21:24:33.644 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/25' 2024-04-20T21:24:33.645 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/30' 2024-04-20T21:24:33.645 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/33' 2024-04-20T21:24:33.645 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/23' 2024-04-20T21:24:33.645 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/35' 2024-04-20T21:24:33.645 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/26' 2024-04-20T21:24:33.645 INFO:tasks.workunit.client.0.smithi102.stdout:removed './283/37' 2024-04-20T21:24:33.646 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './283' 2024-04-20T21:24:33.646 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/21' 2024-04-20T21:24:33.646 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/27' 2024-04-20T21:24:33.646 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/31' 2024-04-20T21:24:33.646 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/36' 2024-04-20T21:24:33.646 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/20' 2024-04-20T21:24:33.647 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/39' 2024-04-20T21:24:33.647 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/34' 2024-04-20T21:24:33.647 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/38' 2024-04-20T21:24:33.647 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/28' 2024-04-20T21:24:33.647 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/24' 2024-04-20T21:24:33.647 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/29' 2024-04-20T21:24:33.648 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/32' 2024-04-20T21:24:33.648 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/22' 2024-04-20T21:24:33.648 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/25' 2024-04-20T21:24:33.648 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/30' 2024-04-20T21:24:33.648 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/33' 2024-04-20T21:24:33.648 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/23' 2024-04-20T21:24:33.649 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/35' 2024-04-20T21:24:33.649 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/26' 2024-04-20T21:24:33.649 INFO:tasks.workunit.client.0.smithi102.stdout:removed './284/37' 2024-04-20T21:24:33.649 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './284' 2024-04-20T21:24:33.649 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/21' 2024-04-20T21:24:33.649 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/27' 2024-04-20T21:24:33.649 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/31' 2024-04-20T21:24:33.650 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/36' 2024-04-20T21:24:33.650 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/20' 2024-04-20T21:24:33.650 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/39' 2024-04-20T21:24:33.650 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/34' 2024-04-20T21:24:33.650 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/38' 2024-04-20T21:24:33.651 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/28' 2024-04-20T21:24:33.651 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/24' 2024-04-20T21:24:33.651 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/29' 2024-04-20T21:24:33.651 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/32' 2024-04-20T21:24:33.651 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/22' 2024-04-20T21:24:33.651 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/25' 2024-04-20T21:24:33.651 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/30' 2024-04-20T21:24:33.652 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/33' 2024-04-20T21:24:33.652 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/23' 2024-04-20T21:24:33.652 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/35' 2024-04-20T21:24:33.652 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/26' 2024-04-20T21:24:33.652 INFO:tasks.workunit.client.0.smithi102.stdout:removed './285/37' 2024-04-20T21:24:33.652 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './285' 2024-04-20T21:24:33.653 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/21' 2024-04-20T21:24:33.653 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/27' 2024-04-20T21:24:33.653 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/31' 2024-04-20T21:24:33.653 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/36' 2024-04-20T21:24:33.653 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/20' 2024-04-20T21:24:33.653 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/39' 2024-04-20T21:24:33.654 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/34' 2024-04-20T21:24:33.654 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/38' 2024-04-20T21:24:33.654 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/28' 2024-04-20T21:24:33.654 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/24' 2024-04-20T21:24:33.654 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/29' 2024-04-20T21:24:33.654 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/32' 2024-04-20T21:24:33.655 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/22' 2024-04-20T21:24:33.655 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/25' 2024-04-20T21:24:33.655 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/30' 2024-04-20T21:24:33.655 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/33' 2024-04-20T21:24:33.655 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/23' 2024-04-20T21:24:33.655 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/35' 2024-04-20T21:24:33.656 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/26' 2024-04-20T21:24:33.656 INFO:tasks.workunit.client.0.smithi102.stdout:removed './286/37' 2024-04-20T21:24:33.656 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './286' 2024-04-20T21:24:38.241 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/21' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/27' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/31' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/36' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/20' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/39' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/34' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/38' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/28' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/24' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/29' 2024-04-20T21:24:38.242 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/32' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/22' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/25' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/30' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/33' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/23' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/35' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/26' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './287/37' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './287' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/21' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/27' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/31' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/36' 2024-04-20T21:24:38.243 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/20' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/39' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/34' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/38' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/28' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/24' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/29' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/32' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/22' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/25' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/30' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/33' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/23' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/35' 2024-04-20T21:24:38.244 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/26' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './288/37' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './288' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/21' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/27' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/31' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/36' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/20' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/39' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/34' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/38' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/28' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/24' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/29' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/32' 2024-04-20T21:24:38.245 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/22' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/25' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/30' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/33' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/23' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/35' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/26' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './289/37' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './289' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/21' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/27' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/31' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/36' 2024-04-20T21:24:38.246 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/20' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/39' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/34' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/38' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/28' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/24' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/29' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/32' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/22' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/25' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/30' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/33' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/23' 2024-04-20T21:24:38.247 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/35' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/26' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './290/37' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './290' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/21' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/27' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/31' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/36' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/20' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/39' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/34' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/38' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/28' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/24' 2024-04-20T21:24:38.248 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/29' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/32' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/22' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/25' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/30' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/33' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/23' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/35' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/26' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './291/37' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './291' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/21' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/27' 2024-04-20T21:24:38.249 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/31' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/36' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/20' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/39' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/34' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/38' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/28' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/24' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/29' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/32' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/22' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/25' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/30' 2024-04-20T21:24:38.250 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/33' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/23' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/35' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/26' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './292/37' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './292' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/21' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/27' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/31' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/36' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/20' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/39' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/34' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/38' 2024-04-20T21:24:38.251 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/28' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/24' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/29' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/32' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/22' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/25' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/30' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/33' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/23' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/35' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/26' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './293/37' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './293' 2024-04-20T21:24:38.252 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/21' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/27' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/31' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/36' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/20' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/39' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/34' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/38' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/28' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/24' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/29' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/32' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/22' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/25' 2024-04-20T21:24:38.253 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/30' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/33' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/23' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/35' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/26' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './294/37' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './294' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/21' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/27' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/31' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/36' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/20' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/39' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/34' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/38' 2024-04-20T21:24:38.254 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/28' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/24' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/29' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/32' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/22' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/25' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/30' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/33' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/23' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/35' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/26' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './295/37' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './295' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/21' 2024-04-20T21:24:38.255 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/27' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/31' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/36' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/20' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/39' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/34' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/38' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/28' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/24' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/29' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/32' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/22' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/25' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/30' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/33' 2024-04-20T21:24:38.256 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/23' 2024-04-20T21:24:38.257 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/35' 2024-04-20T21:24:38.257 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/26' 2024-04-20T21:24:38.257 INFO:tasks.workunit.client.0.smithi102.stdout:removed './296/37' 2024-04-20T21:24:38.257 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './296' 2024-04-20T21:24:38.257 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/21' 2024-04-20T21:24:38.257 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/27' 2024-04-20T21:24:41.532 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/31' 2024-04-20T21:24:41.532 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/36' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/20' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/39' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/34' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/38' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/28' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/24' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/29' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/32' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/22' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/25' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/30' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/33' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/23' 2024-04-20T21:24:41.533 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/35' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/26' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './297/37' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './297' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/21' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/27' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/31' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/36' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/20' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/39' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/34' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/38' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/28' 2024-04-20T21:24:41.534 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/24' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/29' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/32' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/22' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/25' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/30' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/33' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/23' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/35' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/26' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './298/37' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './298' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/21' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/27' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/31' 2024-04-20T21:24:41.535 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/36' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/20' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/39' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/34' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/38' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/28' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/24' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/29' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/32' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/22' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/25' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/30' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/33' 2024-04-20T21:24:41.536 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/23' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/35' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/26' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './299/37' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './299' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/19' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/15' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/20' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/12' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/10' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/11' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/16' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/14' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/18' 2024-04-20T21:24:41.537 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/13' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/166/17' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/166' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/19' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/15' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/20' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/12' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/10' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/11' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/16' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/14' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/18' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/13' 2024-04-20T21:24:41.538 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/147/17' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/147' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/19' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/15' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/20' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/12' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/10' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/11' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/16' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/14' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/18' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/13' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/151/17' 2024-04-20T21:24:41.539 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/151' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/19' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/15' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/20' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/12' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/10' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/11' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/16' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/14' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/18' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/13' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/170/17' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/170' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/19' 2024-04-20T21:24:41.540 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/15' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/20' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/12' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/10' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/11' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/16' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/14' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/18' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/13' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/167/17' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/167' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/19' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/15' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/20' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/12' 2024-04-20T21:24:41.541 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/10' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/11' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/16' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/14' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/18' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/13' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/140/17' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/140' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/19' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/15' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/20' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/12' 2024-04-20T21:24:41.542 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/10' 2024-04-20T21:24:43.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/11' 2024-04-20T21:24:43.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/16' 2024-04-20T21:24:43.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/14' 2024-04-20T21:24:43.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/18' 2024-04-20T21:24:43.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/13' 2024-04-20T21:24:43.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/142/17' 2024-04-20T21:24:43.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/142' 2024-04-20T21:24:43.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/19' 2024-04-20T21:24:43.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/15' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/20' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/12' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/10' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/11' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/16' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/14' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/18' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/13' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/150/17' 2024-04-20T21:24:43.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/150' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/19' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/15' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/20' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/12' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/10' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/11' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/16' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/14' 2024-04-20T21:24:43.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/18' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/13' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/105/17' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/105' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/19' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/15' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/20' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/12' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/10' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/11' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/16' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/14' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/18' 2024-04-20T21:24:43.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/13' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/149/17' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/149' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/19' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/15' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/20' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/12' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/10' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/11' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/16' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/14' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/18' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/13' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/109/17' 2024-04-20T21:24:43.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/109' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/19' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/15' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/20' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/12' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/10' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/11' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/16' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/14' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/18' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/13' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/158/17' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/158' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/19' 2024-04-20T21:24:43.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/15' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/20' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/12' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/10' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/11' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/16' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/14' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/18' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/13' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/106/17' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/106' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/19' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/15' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/20' 2024-04-20T21:24:43.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/12' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/10' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/11' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/16' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/14' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/18' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/13' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/189/17' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/189' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/19' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/15' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/20' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/12' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/10' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/11' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/16' 2024-04-20T21:24:43.161 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/14' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/18' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/13' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/133/17' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/133' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/19' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/15' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/20' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/12' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/10' 2024-04-20T21:24:43.162 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/11' 2024-04-20T21:24:44.421 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/16' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/14' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/18' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/13' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/173/17' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/173' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/19' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/15' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/20' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/12' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/10' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/11' 2024-04-20T21:24:44.422 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/16' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/14' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/18' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/13' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/174/17' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/174' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/19' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/15' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/20' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/12' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/10' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/11' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/16' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/14' 2024-04-20T21:24:44.423 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/18' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/13' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/155/17' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/155' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/19' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/15' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/20' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/12' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/10' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/11' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/16' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/14' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/18' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/13' 2024-04-20T21:24:44.424 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/114/17' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/114' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/19' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/15' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/20' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/12' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/10' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/11' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/16' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/14' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/18' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/13' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/159/17' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/159' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/19' 2024-04-20T21:24:44.425 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/15' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/20' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/12' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/10' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/11' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/16' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/14' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/18' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/13' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/187/17' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/187' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/19' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/15' 2024-04-20T21:24:44.426 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/20' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/12' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/10' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/11' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/16' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/14' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/18' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/13' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/185/17' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/185' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/19' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/15' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/20' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/12' 2024-04-20T21:24:44.427 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/10' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/11' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/16' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/14' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/18' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/13' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/120/17' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/120' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/19' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/15' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/20' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/12' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/10' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/11' 2024-04-20T21:24:44.428 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/16' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/14' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/18' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/13' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/110/17' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/110' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/19' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/15' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/20' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/12' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/10' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/11' 2024-04-20T21:24:44.429 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/16' 2024-04-20T21:24:46.371 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/14' 2024-04-20T21:24:46.371 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/18' 2024-04-20T21:24:46.371 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/13' 2024-04-20T21:24:46.371 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/137/17' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/137' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/19' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/15' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/20' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/12' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/10' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/11' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/16' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/14' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/18' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/13' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/198/17' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/198' 2024-04-20T21:24:46.372 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/19' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/15' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/20' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/12' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/10' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/11' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/16' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/14' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/18' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/13' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/136/17' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/136' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/19' 2024-04-20T21:24:46.373 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/15' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/20' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/12' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/10' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/11' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/16' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/14' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/18' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/13' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/125/17' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/125' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/19' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/15' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/20' 2024-04-20T21:24:46.374 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/12' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/10' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/11' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/16' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/14' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/18' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/13' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/194/17' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/194' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/19' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/15' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/20' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/12' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/10' 2024-04-20T21:24:46.375 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/11' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/16' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/14' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/18' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/13' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/119/17' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/119' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/19' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/15' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/20' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/12' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/10' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/11' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/16' 2024-04-20T21:24:46.376 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/14' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/18' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/13' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/157/17' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/157' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/19' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/15' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/20' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/12' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/10' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/11' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/16' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/14' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/18' 2024-04-20T21:24:46.377 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/13' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/172/17' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/172' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/19' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/15' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/20' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/12' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/10' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/11' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/16' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/14' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/18' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/13' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/186/17' 2024-04-20T21:24:46.378 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/186' 2024-04-20T21:24:46.379 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/19' 2024-04-20T21:24:46.379 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/15' 2024-04-20T21:24:46.379 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/20' 2024-04-20T21:24:46.379 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/12' 2024-04-20T21:24:46.379 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/10' 2024-04-20T21:24:46.379 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/11' 2024-04-20T21:24:46.379 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/16' 2024-04-20T21:24:46.379 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/14' 2024-04-20T21:24:47.151 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/18' 2024-04-20T21:24:47.151 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/13' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/190/17' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/190' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/19' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/15' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/20' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/12' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/10' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/11' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/16' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/14' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/18' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/13' 2024-04-20T21:24:47.152 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/104/17' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/104' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/19' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/15' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/20' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/12' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/10' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/11' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/16' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/14' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/18' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/13' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/164/17' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/164' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/19' 2024-04-20T21:24:47.153 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/15' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/20' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/12' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/10' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/11' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/16' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/14' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/18' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/13' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/112/17' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/112' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/19' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/15' 2024-04-20T21:24:47.154 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/20' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/12' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/10' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/11' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/16' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/14' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/18' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/13' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/100/17' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/100' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/19' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/15' 2024-04-20T21:24:47.155 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/20' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/12' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/10' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/11' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/16' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/14' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/18' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/13' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/134/17' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/134' 2024-04-20T21:24:47.156 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/19' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/15' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/20' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/12' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/10' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/11' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/16' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/14' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/18' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/13' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/115/17' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/115' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/19' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/15' 2024-04-20T21:24:47.157 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/20' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/12' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/10' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/11' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/16' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/14' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/18' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/13' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/116/17' 2024-04-20T21:24:47.158 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/116' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/19' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/15' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/20' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/12' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/10' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/11' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/16' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/14' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/18' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/13' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/195/17' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/195' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/19' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/15' 2024-04-20T21:24:47.159 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/20' 2024-04-20T21:24:47.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/12' 2024-04-20T21:24:47.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/10' 2024-04-20T21:24:47.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/11' 2024-04-20T21:24:47.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/16' 2024-04-20T21:24:47.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/14' 2024-04-20T21:24:47.160 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/18' 2024-04-20T21:24:49.003 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/13' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/138/17' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/138' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/19' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/15' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/20' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/12' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/10' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/11' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/16' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/14' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/18' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/13' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/135/17' 2024-04-20T21:24:49.004 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/135' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/19' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/15' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/20' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/12' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/10' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/11' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/16' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/14' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/18' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/13' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/117/17' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/117' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/19' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/15' 2024-04-20T21:24:49.005 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/20' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/12' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/10' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/11' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/16' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/14' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/18' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/13' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/143/17' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/143' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/19' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/15' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/20' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/12' 2024-04-20T21:24:49.006 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/10' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/11' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/16' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/14' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/18' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/13' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/180/17' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/180' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/19' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/15' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/20' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/12' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/10' 2024-04-20T21:24:49.007 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/11' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/16' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/14' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/18' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/13' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/126/17' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/126' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/19' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/15' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/20' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/12' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/10' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/11' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/16' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/14' 2024-04-20T21:24:49.008 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/18' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/13' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/161/17' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/161' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/19' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/15' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/20' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/12' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/10' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/11' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/16' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/14' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/18' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/13' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/176/17' 2024-04-20T21:24:49.009 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/176' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/19' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/15' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/20' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/12' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/10' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/11' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/16' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/14' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/18' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/13' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/111/17' 2024-04-20T21:24:49.010 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/111' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/19' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/15' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/20' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/12' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/10' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/11' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/16' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/14' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/18' 2024-04-20T21:24:49.011 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/13' 2024-04-20T21:24:50.481 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/191/17' 2024-04-20T21:24:50.481 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/191' 2024-04-20T21:24:50.481 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/19' 2024-04-20T21:24:50.481 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/15' 2024-04-20T21:24:50.481 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/20' 2024-04-20T21:24:50.481 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/12' 2024-04-20T21:24:50.481 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/10' 2024-04-20T21:24:50.481 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/11' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/16' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/14' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/18' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/13' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/184/17' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/184' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/19' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/15' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/20' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/12' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/10' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/11' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/16' 2024-04-20T21:24:50.482 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/14' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/18' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/13' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/192/17' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/192' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/19' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/15' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/20' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/12' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/10' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/11' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/16' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/14' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/18' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/13' 2024-04-20T21:24:50.483 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/103/17' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/103' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/19' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/15' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/20' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/12' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/10' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/11' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/16' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/14' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/18' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/13' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/128/17' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/128' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/19' 2024-04-20T21:24:50.484 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/15' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/20' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/12' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/10' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/11' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/16' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/14' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/18' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/13' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/144/17' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/144' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/19' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/15' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/20' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/12' 2024-04-20T21:24:50.485 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/10' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/11' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/16' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/14' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/18' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/13' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/146/17' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/146' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/19' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/15' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/20' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/12' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/10' 2024-04-20T21:24:50.486 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/11' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/16' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/14' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/18' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/13' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/141/17' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/141' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/19' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/15' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/20' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/12' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/10' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/11' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/16' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/14' 2024-04-20T21:24:50.487 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/18' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/13' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/182/17' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/182' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/19' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/15' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/20' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/12' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/10' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/11' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/16' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/14' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/18' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/13' 2024-04-20T21:24:50.488 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/107/17' 2024-04-20T21:24:52.862 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/107' 2024-04-20T21:24:52.862 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/19' 2024-04-20T21:24:52.862 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/15' 2024-04-20T21:24:52.862 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/20' 2024-04-20T21:24:52.862 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/12' 2024-04-20T21:24:52.862 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/10' 2024-04-20T21:24:52.862 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/11' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/16' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/14' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/18' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/13' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/178/17' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/178' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/19' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/15' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/20' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/12' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/10' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/11' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/16' 2024-04-20T21:24:52.863 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/14' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/18' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/13' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/156/17' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/156' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/19' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/15' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/20' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/12' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/10' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/11' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/16' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/14' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/18' 2024-04-20T21:24:52.864 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/13' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/131/17' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/131' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/19' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/15' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/20' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/12' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/10' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/11' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/16' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/14' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/18' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/13' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/123/17' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/123' 2024-04-20T21:24:52.865 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/19' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/15' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/20' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/12' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/10' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/11' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/16' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/14' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/18' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/13' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/160/17' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/160' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/19' 2024-04-20T21:24:52.866 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/15' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/20' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/12' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/10' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/11' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/16' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/14' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/18' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/13' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/165/17' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/165' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/19' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/15' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/20' 2024-04-20T21:24:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/12' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/10' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/11' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/16' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/14' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/18' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/13' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/101/17' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/101' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/19' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/15' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/20' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/12' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/10' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/11' 2024-04-20T21:24:52.868 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/16' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/14' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/18' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/13' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/148/17' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/148' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/19' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/15' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/20' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/12' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/10' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/11' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/16' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/14' 2024-04-20T21:24:52.869 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/18' 2024-04-20T21:24:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/13' 2024-04-20T21:24:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/168/17' 2024-04-20T21:24:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/168' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/19' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/15' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/20' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/12' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/10' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/11' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/16' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/14' 2024-04-20T21:24:55.570 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/18' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/13' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/179/17' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/179' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/19' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/15' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/20' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/12' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/10' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/11' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/16' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/14' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/18' 2024-04-20T21:24:55.571 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/13' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/132/17' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/132' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/19' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/15' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/20' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/12' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/10' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/11' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/16' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/14' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/18' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/13' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/199/17' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/199' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/19' 2024-04-20T21:24:55.572 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/15' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/20' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/12' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/10' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/11' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/16' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/14' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/18' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/13' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/163/17' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/163' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/19' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/15' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/20' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/12' 2024-04-20T21:24:55.573 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/10' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/11' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/16' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/14' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/18' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/13' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/153/17' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/153' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/19' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/15' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/20' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/12' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/10' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/11' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/16' 2024-04-20T21:24:55.574 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/14' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/18' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/13' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/122/17' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/122' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/19' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/15' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/20' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/12' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/10' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/11' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/16' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/14' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/18' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/13' 2024-04-20T21:24:55.575 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/162/17' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/162' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/19' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/15' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/20' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/12' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/10' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/11' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/16' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/14' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/18' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/13' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/197/17' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/197' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/19' 2024-04-20T21:24:55.576 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/15' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/20' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/12' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/10' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/11' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/16' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/14' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/18' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/13' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/113/17' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/113' 2024-04-20T21:24:55.577 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/19' 2024-04-20T21:24:55.796 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:55.804 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/15' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/20' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/12' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/10' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/11' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/16' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/14' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/18' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/13' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/152/17' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/152' 2024-04-20T21:24:57.581 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/19' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/15' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/20' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/12' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/10' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/11' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/16' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/14' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/18' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/13' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/121/17' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/121' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/19' 2024-04-20T21:24:57.582 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/15' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/20' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/12' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/10' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/11' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/16' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/14' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/18' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/13' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/118/17' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/118' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/19' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/15' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/20' 2024-04-20T21:24:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/12' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/10' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/11' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/16' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/14' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/18' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/13' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/196/17' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/196' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/19' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/15' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/20' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/12' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/10' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/11' 2024-04-20T21:24:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/16' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/14' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/18' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/13' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/124/17' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/124' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/19' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/15' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/20' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/12' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/10' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/11' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/16' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/14' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/18' 2024-04-20T21:24:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/13' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/188/17' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/188' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/19' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/15' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/20' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/12' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/10' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/11' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/16' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/14' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/18' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/13' 2024-04-20T21:24:57.586 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/145/17' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/145' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/19' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/15' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/20' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/12' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/10' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/11' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/16' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/14' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/18' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/13' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/130/17' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/130' 2024-04-20T21:24:57.587 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/19' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/15' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/20' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/12' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/10' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/11' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/16' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/14' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/18' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/13' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/193/17' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/193' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/19' 2024-04-20T21:24:57.588 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/15' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/20' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/12' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/10' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/11' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/16' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/14' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/18' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/13' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/175/17' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/175' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/19' 2024-04-20T21:24:58.607 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/15' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/20' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/12' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/10' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/11' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/16' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/14' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/18' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/13' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/181/17' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/181' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/19' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/15' 2024-04-20T21:24:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/20' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/12' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/10' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/11' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/16' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/14' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/18' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/13' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/127/17' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/127' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/19' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/15' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/20' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/12' 2024-04-20T21:24:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/10' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/11' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/16' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/14' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/18' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/13' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/108/17' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/108' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/19' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/15' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/20' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/12' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/10' 2024-04-20T21:24:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/11' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/16' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/14' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/18' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/13' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/183/17' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/183' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/19' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/15' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/20' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/12' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/10' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/11' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/16' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/14' 2024-04-20T21:24:58.611 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/18' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/13' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/177/17' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/177' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/19' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/15' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/20' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/12' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/10' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/11' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/16' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/14' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/18' 2024-04-20T21:24:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/13' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/169/17' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/169' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/19' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/15' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/20' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/12' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/10' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/11' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/16' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/14' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/18' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/13' 2024-04-20T21:24:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/154/17' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/154' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/19' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/15' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/20' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/12' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/10' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/11' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/16' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/14' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/18' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/13' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/171/17' 2024-04-20T21:24:58.614 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/171' 2024-04-20T21:24:58.615 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/19' 2024-04-20T21:24:58.615 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/15' 2024-04-20T21:24:58.615 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/20' 2024-04-20T21:24:59.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/12' 2024-04-20T21:24:59.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/10' 2024-04-20T21:24:59.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/11' 2024-04-20T21:24:59.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/16' 2024-04-20T21:24:59.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/14' 2024-04-20T21:24:59.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/18' 2024-04-20T21:24:59.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/13' 2024-04-20T21:24:59.411 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/139/17' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/139' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/19' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/15' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/20' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/12' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/10' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/11' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/16' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/14' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/18' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/13' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/102/17' 2024-04-20T21:24:59.412 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/102' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/19' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/15' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/20' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/12' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/10' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/11' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/16' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/14' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/18' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/13' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed './copyofsnap1/test1/129/17' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1/129' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1/test1' 2024-04-20T21:24:59.413 INFO:tasks.workunit.client.0.smithi102.stdout:removed directory './copyofsnap1' 2024-04-20T21:24:59.414 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-04-20T21:24:59.415 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T21:24:59.415 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T21:24:59.509 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2024-04-20T21:24:59.510 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2024-04-20T21:24:59.578 INFO:tasks.workunit.client.0.smithi102.stderr:+ set -e 2024-04-20T21:24:59.578 INFO:tasks.workunit.client.0.smithi102.stderr:+ touch foo 2024-04-20T21:24:59.582 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod +x foo 2024-04-20T21:24:59.586 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/s 2024-04-20T21:24:59.632 INFO:tasks.workunit.client.0.smithi102.stderr:+ find .snap/s/foo -executable 2024-04-20T21:24:59.633 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep foo 2024-04-20T21:24:59.639 INFO:tasks.workunit.client.0.smithi102.stdout:.snap/s/foo 2024-04-20T21:24:59.639 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir .snap/s 2024-04-20T21:24:59.652 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm foo 2024-04-20T21:24:59.658 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo OK 2024-04-20T21:24:59.659 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-04-20T21:24:59.659 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T21:24:59.660 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T21:24:59.741 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2024-04-20T21:24:59.741 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2024-04-20T21:24:59.805 INFO:tasks.workunit.client.0.smithi102.stderr:+ set -e 2024-04-20T21:24:59.806 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir foo 2024-04-20T21:24:59.810 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo x 2024-04-20T21:24:59.812 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:01.462 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir foo/.snap/ss 2024-04-20T21:25:01.502 INFO:tasks.workunit.client.0.smithi102.stderr:+ ln foo/x foo/xx 2024-04-20T21:25:01.507 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat foo/.snap/ss/x 2024-04-20T21:25:01.522 INFO:tasks.workunit.client.0.smithi102.stdout:x 2024-04-20T21:25:01.522 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir foo/.snap/ss 2024-04-20T21:25:01.533 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo a 2024-04-20T21:25:01.535 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo b 2024-04-20T21:25:01.538 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir foo/.snap/s 2024-04-20T21:25:01.581 INFO:tasks.workunit.client.0.smithi102.stderr:++ cat foo/.snap/s/a 2024-04-20T21:25:01.601 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=a 2024-04-20T21:25:01.601 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -z a ']' 2024-04-20T21:25:01.602 INFO:tasks.workunit.client.0.smithi102.stderr:+ ln foo/b foo/b2 2024-04-20T21:25:01.610 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat foo/.snap/s/b 2024-04-20T21:25:01.630 INFO:tasks.workunit.client.0.smithi102.stdout:b 2024-04-20T21:25:01.630 INFO:tasks.workunit.client.0.smithi102.stdout:this used to hang: 2024-04-20T21:25:01.630 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'this used to hang:' 2024-04-20T21:25:01.631 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo more 2024-04-20T21:25:01.631 INFO:tasks.workunit.client.0.smithi102.stdout:oh, it didn't hang! good job. 2024-04-20T21:25:01.631 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'oh, it didn'\''t hang! good job.' 2024-04-20T21:25:01.631 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat foo/b 2024-04-20T21:25:01.633 INFO:tasks.workunit.client.0.smithi102.stdout:b 2024-04-20T21:25:01.633 INFO:tasks.workunit.client.0.smithi102.stdout:more 2024-04-20T21:25:01.633 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir foo/.snap/s 2024-04-20T21:25:01.642 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm -r foo 2024-04-20T21:25:01.678 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-04-20T21:25:01.678 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo OK 2024-04-20T21:25:01.679 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T21:25:01.680 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T21:25:01.770 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2024-04-20T21:25:01.771 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2024-04-20T21:25:01.842 INFO:tasks.workunit.client.0.smithi102.stderr:+ set -e 2024-04-20T21:25:01.842 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:01.847 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/bar 2024-04-20T21:25:01.889 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/foo 2024-04-20T21:25:01.923 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/foo 2024-04-20T21:25:01.937 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:01.943 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/bar/a 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout: File: .snap/bar/a 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout:Device: f9h/249d Inode: 1099511633797 Links: 2 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1316/ ubuntu) 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout:Access: 2024-04-20 21:25:01.846259265 +0000 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout:Modify: 2024-04-20 21:25:01.846259265 +0000 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout:Change: 2024-04-20 21:25:01.846259265 +0000 2024-04-20T21:25:01.952 INFO:tasks.workunit.client.0.smithi102.stdout: Birth: 2024-04-20 21:25:01.846259265 +0000 2024-04-20T21:25:01.953 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir .snap/bar 2024-04-20T21:25:01.971 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo OK 2024-04-20T21:25:01.972 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-04-20T21:25:01.973 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T21:25:01.973 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T21:25:02.057 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2024-04-20T21:25:02.057 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2024-04-20T21:25:02.124 INFO:tasks.workunit.client.0.smithi102.stderr:+ set -e 2024-04-20T21:25:02.125 INFO:tasks.workunit.client.0.smithi102.stderr:++ seq 1 20 2024-04-20T21:25:02.127 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:02.128 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:02.133 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:02.134 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:02.174 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:02.206 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:02.209 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:02.353 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:02.381 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:02.381 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:02.381 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:02.381 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:02.381 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:02.381 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:02.381 INFO:tasks.workunit.client.0.smithi102.stdout: 39522 ? 00:00:00 ps 2024-04-20T21:25:02.383 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:02.403 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:02.409 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:02.423 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:02.423 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:02.431 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:02.431 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:02.474 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:02.509 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:02.512 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:02.594 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:02.613 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:02.613 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:02.613 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:02.613 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:02.613 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:02.613 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:02.613 INFO:tasks.workunit.client.0.smithi102.stdout: 39532 ? 00:00:00 ps 2024-04-20T21:25:02.615 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:02.639 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:02.645 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:02.658 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:02.658 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:02.665 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:02.666 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:02.705 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:02.727 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:02.731 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:02.835 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:02.855 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:02.855 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:02.855 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:02.855 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:02.855 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:02.855 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:02.856 INFO:tasks.workunit.client.0.smithi102.stdout: 39542 ? 00:00:00 ps 2024-04-20T21:25:02.857 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:02.881 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:02.889 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:02.903 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:02.903 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:02.911 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:02.911 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:02.936 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:02.958 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:02.960 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:03.026 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:03.048 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:03.048 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:03.048 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:03.048 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:03.048 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:03.048 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:03.048 INFO:tasks.workunit.client.0.smithi102.stdout: 39552 ? 00:00:00 ps 2024-04-20T21:25:03.050 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:03.075 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:03.083 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:03.098 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:03.098 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:03.106 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:03.107 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:03.146 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:03.168 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:03.171 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:03.234 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:03.254 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:03.254 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:03.254 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:03.254 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:03.254 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:03.254 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:03.254 INFO:tasks.workunit.client.0.smithi102.stdout: 39562 ? 00:00:00 ps 2024-04-20T21:25:03.256 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:03.281 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:03.288 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:03.302 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:03.302 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:03.309 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:03.309 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:03.338 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:03.361 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:03.364 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:03.418 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:03.437 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:03.437 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:03.437 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:03.437 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:03.437 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:03.437 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:03.437 INFO:tasks.workunit.client.0.smithi102.stdout: 39572 ? 00:00:00 ps 2024-04-20T21:25:03.439 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:03.465 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:03.473 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:03.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:03.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:03.498 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:03.498 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:03.524 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:03.545 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:03.550 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:03.618 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:03.638 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:03.639 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:03.639 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:03.639 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:03.639 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:03.639 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:03.639 INFO:tasks.workunit.client.0.smithi102.stdout: 39582 ? 00:00:00 ps 2024-04-20T21:25:03.640 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:03.664 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:03.673 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:03.685 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:03.685 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:03.691 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:03.692 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:03.926 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:04.417 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:04.421 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:04.546 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:04.561 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:04.561 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:04.561 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:04.561 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:04.562 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:04.562 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:04.562 INFO:tasks.workunit.client.0.smithi102.stdout: 39592 ? 00:00:00 ps 2024-04-20T21:25:04.562 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:04.576 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:04.580 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:04.588 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:04.589 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:04.593 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:04.593 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:04.613 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:04.646 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:04.648 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:04.820 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:04.840 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:04.840 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:04.840 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:04.840 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:04.840 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:04.840 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:04.840 INFO:tasks.workunit.client.0.smithi102.stdout: 39602 ? 00:00:00 ps 2024-04-20T21:25:04.842 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:04.867 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:04.872 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:04.889 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:04.889 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:04.900 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:04.900 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:04.940 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:04.973 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:04.976 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:05.062 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:05.082 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:05.082 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:05.082 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:05.082 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:05.082 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:05.082 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:05.082 INFO:tasks.workunit.client.0.smithi102.stdout: 39612 ? 00:00:00 ps 2024-04-20T21:25:05.084 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:05.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:05.111 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:05.125 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:05.125 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:05.133 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:05.133 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:05.174 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:05.208 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:05.212 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:05.281 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:05.301 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:05.301 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:05.301 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:05.301 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:05.301 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:05.301 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:05.302 INFO:tasks.workunit.client.0.smithi102.stdout: 39622 ? 00:00:00 ps 2024-04-20T21:25:05.303 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:05.324 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:05.332 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:05.347 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:05.347 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:05.354 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:05.355 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:05.399 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:05.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:05.437 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:05.513 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:05.533 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:05.533 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:05.533 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:05.533 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:05.533 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:05.533 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:05.533 INFO:tasks.workunit.client.0.smithi102.stdout: 39632 ? 00:00:00 ps 2024-04-20T21:25:05.535 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:05.559 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:05.575 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:05.603 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:05.603 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:05.612 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:05.613 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:05.688 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:05.747 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:05.751 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:06.098 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:06.112 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:06.112 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:06.112 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:06.112 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:06.113 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:06.113 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:06.113 INFO:tasks.workunit.client.0.smithi102.stdout: 39642 ? 00:00:00 ps 2024-04-20T21:25:06.113 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:06.124 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:06.128 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:06.134 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:06.134 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:06.138 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:06.138 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:06.168 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:07.191 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:07.195 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:07.617 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:07.644 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:07.644 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:07.644 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:07.645 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:07.645 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:07.645 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:07.645 INFO:tasks.workunit.client.0.smithi102.stdout: 39652 ? 00:00:00 ps 2024-04-20T21:25:07.645 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:07.673 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:07.678 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:07.686 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:07.686 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:07.690 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:07.690 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:07.711 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:07.735 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:07.737 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:07.917 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:07.937 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:07.937 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:07.937 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:07.937 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:07.938 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:07.938 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:07.938 INFO:tasks.workunit.client.0.smithi102.stdout: 39663 ? 00:00:00 ps 2024-04-20T21:25:07.939 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:07.958 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:07.964 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:07.977 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:07.977 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:07.983 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:07.983 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:08.008 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:08.029 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:08.032 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:08.082 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:08.102 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:08.103 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:08.103 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:08.103 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:08.103 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:08.103 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:08.103 INFO:tasks.workunit.client.0.smithi102.stdout: 39673 ? 00:00:00 ps 2024-04-20T21:25:08.104 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:08.122 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:08.149 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:08.149 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:08.268 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:08.268 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:08.268 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:08.268 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:08.268 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:08.268 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:08.268 INFO:tasks.workunit.client.0.smithi102.stdout: 39683 ? 00:00:00 ps 2024-04-20T21:25:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:08.322 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:08.322 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:08.367 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:08.450 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:08.467 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:08.467 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:08.467 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:08.467 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:08.467 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:08.467 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:08.467 INFO:tasks.workunit.client.0.smithi102.stdout: 39693 ? 00:00:00 ps 2024-04-20T21:25:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:08.493 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:08.506 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:08.506 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:08.514 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:08.514 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:08.541 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:08.561 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:08.565 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:08.634 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:08.655 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:08.655 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:08.656 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:08.656 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:08.656 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:08.656 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:08.656 INFO:tasks.workunit.client.0.smithi102.stdout: 39703 ? 00:00:00 ps 2024-04-20T21:25:08.657 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:08.677 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:08.684 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:08.697 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 20` 2024-04-20T21:25:08.697 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a 2024-04-20T21:25:08.704 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/one 2024-04-20T21:25:08.704 INFO:tasks.workunit.client.0.smithi102.stderr:+ cat 2024-04-20T21:25:09.547 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir a/.snap/two 2024-04-20T21:25:09.571 INFO:tasks.workunit.client.0.smithi102.stderr:+ chmod 777 a/foo 2024-04-20T21:25:09.574 INFO:tasks.workunit.client.0.smithi102.stderr:+ sync 2024-04-20T21:25:09.685 INFO:tasks.workunit.client.0.smithi102.stderr:+ ps 2024-04-20T21:25:09.705 INFO:tasks.workunit.client.0.smithi102.stdout: PID TTY TIME CMD 2024-04-20T21:25:09.705 INFO:tasks.workunit.client.0.smithi102.stdout: 874 ? 00:00:00 systemd 2024-04-20T21:25:09.705 INFO:tasks.workunit.client.0.smithi102.stdout: 880 ? 00:00:00 (sd-pam) 2024-04-20T21:25:09.705 INFO:tasks.workunit.client.0.smithi102.stdout: 9395 ? 00:00:02 sshd 2024-04-20T21:25:09.705 INFO:tasks.workunit.client.0.smithi102.stdout: 39496 ? 00:00:00 timeout 2024-04-20T21:25:09.705 INFO:tasks.workunit.client.0.smithi102.stdout: 39514 ? 00:00:00 snaptest-double 2024-04-20T21:25:09.705 INFO:tasks.workunit.client.0.smithi102.stdout: 39713 ? 00:00:00 ps 2024-04-20T21:25:09.707 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a/.snap/one a/.snap/two 2024-04-20T21:25:09.726 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm a/foo 2024-04-20T21:25:09.733 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir a 2024-04-20T21:25:09.746 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo OK 2024-04-20T21:25:09.746 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-04-20T21:25:09.748 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T21:25:09.749 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T21:25:09.850 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2024-04-20T21:25:09.851 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2024-04-20T21:25:09.917 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/foo 2024-04-20T21:25:09.955 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'We want ENOENT, not ESTALE, here.' 2024-04-20T21:25:09.956 INFO:tasks.workunit.client.0.smithi102.stdout:We want ENOENT, not ESTALE, here. 2024-04-20T21:25:09.957 INFO:tasks.workunit.client.0.smithi102.stderr:++ seq 1 100 2024-04-20T21:25:09.959 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.961 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/1 2024-04-20T21:25:09.961 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:09.966 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2024-04-20T21:25:09.967 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.967 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/2 2024-04-20T21:25:09.968 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:09.970 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2024-04-20T21:25:09.971 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.971 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/3 2024-04-20T21:25:09.972 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:09.974 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2024-04-20T21:25:09.975 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.976 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/4 2024-04-20T21:25:09.976 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:09.979 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2024-04-20T21:25:09.979 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.980 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/5 2024-04-20T21:25:09.980 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:09.983 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2024-04-20T21:25:09.983 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.984 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/6 2024-04-20T21:25:09.984 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:09.987 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2024-04-20T21:25:09.988 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.988 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/7 2024-04-20T21:25:09.989 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:09.991 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2024-04-20T21:25:09.992 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.992 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/8 2024-04-20T21:25:09.993 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:09.996 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2024-04-20T21:25:09.996 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:09.997 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/9 2024-04-20T21:25:09.997 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.000 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2024-04-20T21:25:10.000 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.001 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/10 2024-04-20T21:25:10.002 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.004 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2024-04-20T21:25:10.005 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.005 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/11 2024-04-20T21:25:10.006 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.009 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2024-04-20T21:25:10.009 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.009 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/12 2024-04-20T21:25:10.010 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.277 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2024-04-20T21:25:10.277 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.278 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/13 2024-04-20T21:25:10.280 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.283 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2024-04-20T21:25:10.283 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.285 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/14 2024-04-20T21:25:10.286 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.288 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2024-04-20T21:25:10.289 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.290 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/15 2024-04-20T21:25:10.291 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.294 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2024-04-20T21:25:10.295 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.296 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/16 2024-04-20T21:25:10.297 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.300 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2024-04-20T21:25:10.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.302 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/17 2024-04-20T21:25:10.303 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.306 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2024-04-20T21:25:10.306 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.308 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/18 2024-04-20T21:25:10.309 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.311 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2024-04-20T21:25:10.312 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.313 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/19 2024-04-20T21:25:10.314 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.317 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2024-04-20T21:25:10.317 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.318 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/20 2024-04-20T21:25:10.319 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.322 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2024-04-20T21:25:10.322 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.324 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/21 2024-04-20T21:25:10.325 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.327 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2024-04-20T21:25:10.328 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/22 2024-04-20T21:25:10.330 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.333 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2024-04-20T21:25:10.333 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.334 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/23 2024-04-20T21:25:10.335 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.338 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2024-04-20T21:25:10.339 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.340 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/24 2024-04-20T21:25:10.341 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.345 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2024-04-20T21:25:10.345 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.347 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/25 2024-04-20T21:25:10.348 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.350 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2024-04-20T21:25:10.351 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.352 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/26 2024-04-20T21:25:10.353 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.356 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2024-04-20T21:25:10.356 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.358 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/27 2024-04-20T21:25:10.358 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.361 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2024-04-20T21:25:10.362 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.363 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/28 2024-04-20T21:25:10.364 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.366 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2024-04-20T21:25:10.367 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.368 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/29 2024-04-20T21:25:10.369 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.372 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2024-04-20T21:25:10.372 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.373 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/30 2024-04-20T21:25:10.374 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.377 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2024-04-20T21:25:10.378 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.379 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/31 2024-04-20T21:25:10.380 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.383 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2024-04-20T21:25:10.383 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.385 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/32 2024-04-20T21:25:10.385 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.388 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2024-04-20T21:25:10.389 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.390 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/33 2024-04-20T21:25:10.391 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.394 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2024-04-20T21:25:10.394 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.395 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/34 2024-04-20T21:25:10.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.399 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2024-04-20T21:25:10.400 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.401 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/35 2024-04-20T21:25:10.402 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.404 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2024-04-20T21:25:10.405 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.406 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/36 2024-04-20T21:25:10.407 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.410 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2024-04-20T21:25:10.410 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.411 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/37 2024-04-20T21:25:10.412 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.415 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2024-04-20T21:25:10.415 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.417 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/38 2024-04-20T21:25:10.418 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.420 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2024-04-20T21:25:10.421 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.422 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/39 2024-04-20T21:25:10.423 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.426 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2024-04-20T21:25:10.426 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.427 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/40 2024-04-20T21:25:10.428 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.431 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2024-04-20T21:25:10.432 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.433 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/41 2024-04-20T21:25:10.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.437 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2024-04-20T21:25:10.438 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.439 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/42 2024-04-20T21:25:10.440 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.442 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2024-04-20T21:25:10.443 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.444 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/43 2024-04-20T21:25:10.445 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.448 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2024-04-20T21:25:10.448 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.450 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/44 2024-04-20T21:25:10.451 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.454 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2024-04-20T21:25:10.454 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.456 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/45 2024-04-20T21:25:10.457 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.459 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2024-04-20T21:25:10.460 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.461 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/46 2024-04-20T21:25:10.462 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.512 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2024-04-20T21:25:10.512 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.514 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/47 2024-04-20T21:25:10.515 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.519 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2024-04-20T21:25:10.519 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.521 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/48 2024-04-20T21:25:10.521 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.524 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2024-04-20T21:25:10.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/49 2024-04-20T21:25:10.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.530 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2024-04-20T21:25:10.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.531 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/50 2024-04-20T21:25:10.532 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.535 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2024-04-20T21:25:10.535 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.537 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/51 2024-04-20T21:25:10.538 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.541 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2024-04-20T21:25:10.541 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.542 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/52 2024-04-20T21:25:10.543 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.546 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2024-04-20T21:25:10.547 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.548 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/53 2024-04-20T21:25:10.549 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.552 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2024-04-20T21:25:10.553 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.554 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/54 2024-04-20T21:25:10.555 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.558 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2024-04-20T21:25:10.558 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.560 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/55 2024-04-20T21:25:10.560 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.563 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2024-04-20T21:25:10.564 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.565 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/56 2024-04-20T21:25:10.566 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.572 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2024-04-20T21:25:10.573 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.575 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/57 2024-04-20T21:25:10.578 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.585 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2024-04-20T21:25:10.586 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.588 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/58 2024-04-20T21:25:10.588 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.595 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2024-04-20T21:25:10.595 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.598 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/59 2024-04-20T21:25:10.599 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.604 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2024-04-20T21:25:10.606 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.607 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/60 2024-04-20T21:25:10.608 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.614 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2024-04-20T21:25:10.614 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.616 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.616 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/61 2024-04-20T21:25:10.621 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2024-04-20T21:25:10.622 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.624 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/62 2024-04-20T21:25:10.624 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.629 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2024-04-20T21:25:10.630 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.631 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.636 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/63 2024-04-20T21:25:10.641 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2024-04-20T21:25:10.643 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.647 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/64 2024-04-20T21:25:10.647 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.653 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2024-04-20T21:25:10.654 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.656 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.656 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/65 2024-04-20T21:25:10.666 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2024-04-20T21:25:10.667 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.671 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/66 2024-04-20T21:25:10.671 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.677 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2024-04-20T21:25:10.677 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.679 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/67 2024-04-20T21:25:10.680 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.686 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2024-04-20T21:25:10.687 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.688 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.689 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/68 2024-04-20T21:25:10.694 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2024-04-20T21:25:10.695 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.700 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.700 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/69 2024-04-20T21:25:10.703 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2024-04-20T21:25:10.703 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.705 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/70 2024-04-20T21:25:10.706 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.713 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2024-04-20T21:25:10.714 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.716 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/71 2024-04-20T21:25:10.717 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.724 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2024-04-20T21:25:10.724 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.726 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/72 2024-04-20T21:25:10.728 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.733 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2024-04-20T21:25:10.733 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.739 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/73 2024-04-20T21:25:10.739 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.741 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2024-04-20T21:25:10.745 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.747 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/74 2024-04-20T21:25:10.748 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.753 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2024-04-20T21:25:10.753 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.757 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/75 2024-04-20T21:25:10.757 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.763 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2024-04-20T21:25:10.763 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.766 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/76 2024-04-20T21:25:10.766 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.772 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2024-04-20T21:25:10.772 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.774 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/77 2024-04-20T21:25:10.774 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.779 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2024-04-20T21:25:10.781 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.782 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/78 2024-04-20T21:25:10.787 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.787 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2024-04-20T21:25:10.789 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.790 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/79 2024-04-20T21:25:10.791 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.796 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2024-04-20T21:25:10.797 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.798 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/80 2024-04-20T21:25:10.800 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.805 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2024-04-20T21:25:10.807 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.808 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/81 2024-04-20T21:25:10.809 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.814 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2024-04-20T21:25:10.815 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.816 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/82 2024-04-20T21:25:10.817 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.823 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2024-04-20T21:25:10.823 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.825 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/83 2024-04-20T21:25:10.827 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.829 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2024-04-20T21:25:10.830 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.831 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/84 2024-04-20T21:25:10.832 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.836 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2024-04-20T21:25:10.837 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.839 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.839 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/85 2024-04-20T21:25:10.843 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2024-04-20T21:25:10.844 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.845 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/86 2024-04-20T21:25:10.846 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.851 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2024-04-20T21:25:10.852 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.853 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/87 2024-04-20T21:25:10.854 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.857 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2024-04-20T21:25:10.858 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.859 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/88 2024-04-20T21:25:10.860 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.865 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2024-04-20T21:25:10.865 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.868 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/89 2024-04-20T21:25:10.868 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.872 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2024-04-20T21:25:10.873 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/90 2024-04-20T21:25:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.880 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2024-04-20T21:25:10.881 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.882 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/91 2024-04-20T21:25:10.882 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.890 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2024-04-20T21:25:10.890 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.892 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/92 2024-04-20T21:25:10.892 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.896 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2024-04-20T21:25:10.897 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.898 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/93 2024-04-20T21:25:10.899 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.902 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2024-04-20T21:25:10.903 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.904 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/94 2024-04-20T21:25:10.904 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.908 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2024-04-20T21:25:10.908 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.911 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/95 2024-04-20T21:25:10.911 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.914 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2024-04-20T21:25:10.914 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.916 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/96 2024-04-20T21:25:10.916 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.920 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2024-04-20T21:25:10.920 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.921 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/97 2024-04-20T21:25:10.922 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.925 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2024-04-20T21:25:10.926 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.927 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/98 2024-04-20T21:25:10.928 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.932 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2024-04-20T21:25:10.932 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.933 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/99 2024-04-20T21:25:10.934 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.937 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2024-04-20T21:25:10.937 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 100` 2024-04-20T21:25:10.938 INFO:tasks.workunit.client.0.smithi102.stderr:+ stat .snap/foo/100 2024-04-20T21:25:10.939 INFO:tasks.workunit.client.0.smithi102.stderr:+ grep 'No such file' 2024-04-20T21:25:10.943 INFO:tasks.workunit.client.0.smithi102.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2024-04-20T21:25:10.943 INFO:tasks.workunit.client.0.smithi102.stderr:+ rmdir .snap/foo 2024-04-20T21:25:10.954 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-04-20T21:25:10.955 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo OK 2024-04-20T21:25:10.955 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T21:25:10.955 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T21:25:11.017 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2024-04-20T21:25:11.018 DEBUG:teuthology.orchestra.run.smithi102:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2024-04-20T21:25:11.070 INFO:tasks.workunit.client.0.smithi102.stderr:+ set -e 2024-04-20T21:25:11.070 INFO:tasks.workunit.client.0.smithi102.stderr:+ sudo git config --global http.sslVerify false 2024-04-20T21:25:11.101 INFO:tasks.workunit.client.0.smithi102.stderr:+ sudo git config --global http.postBuffer 1048576000 2024-04-20T21:25:11.129 INFO:tasks.workunit.client.0.smithi102.stderr:+ retried=false 2024-04-20T21:25:11.129 INFO:tasks.workunit.client.0.smithi102.stderr:+ trap -- retry EXIT 2024-04-20T21:25:11.129 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm -rf ceph 2024-04-20T21:25:11.131 INFO:tasks.workunit.client.0.smithi102.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2024-04-20T21:25:11.138 INFO:tasks.workunit.client.0.smithi102.stderr:Cloning into 'ceph'... 2024-04-20T21:25:25.928 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:25.937 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:56.070 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:56.079 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:56.108 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:56.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.293+0000 7f746fe00640 1 Processor -- start 2024-04-20T21:25:56.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.293+0000 7f746fe00640 1 -- start start 2024-04-20T21:25:56.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.294+0000 7f746fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f74700cc470 0x7f74700cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.294+0000 7f746fe00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747015ef60 con 0x7f74700cdfc0 2024-04-20T21:25:56.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.294+0000 7f746fe00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747015f240 con 0x7f7470155530 2024-04-20T21:25:56.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.294+0000 7f746fe00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f74700c5ca0 con 0x7f74700cc470 2024-04-20T21:25:56.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746f400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f74700cdfc0 0x7f74700cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41402/0 (socket says 172.21.15.102:41402) 2024-04-20T21:25:56.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746f400640 1 -- 172.21.15.102:0/1626477206 learned_addr learned my addr 172.21.15.102:0/1626477206 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:25:56.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746ea00640 1 --1- 172.21.15.102:0/1626477206 >> v1:172.21.15.193:6789/0 conn(0x7f7470155530 0x7f747015e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:44892/0 (socket says 172.21.15.102:44892) 2024-04-20T21:25:56.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746e000640 1 --2- 172.21.15.102:0/1626477206 >> v2:172.21.15.193:3300/0 conn(0x7f74700cc470 0x7f74700cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746d600640 1 -- 172.21.15.102:0/1626477206 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2112557655 0 0) 0x7f747015f240 con 0x7f7470155530 2024-04-20T21:25:56.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746d600640 1 -- 172.21.15.102:0/1626477206 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7454003520 con 0x7f7470155530 2024-04-20T21:25:56.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746d600640 1 -- 172.21.15.102:0/1626477206 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642658499 0 0) 0x7f747015ef60 con 0x7f74700cdfc0 2024-04-20T21:25:56.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746e000640 1 -- 172.21.15.102:0/1626477206 >> v1:172.21.15.193:6789/0 conn(0x7f7470155530 legacy=0x7f747015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746e000640 1 -- 172.21.15.102:0/1626477206 >> v1:172.21.15.102:6789/0 conn(0x7f74700cdfc0 legacy=0x7f74700cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.295+0000 7f746e000640 1 -- 172.21.15.102:0/1626477206 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747015f520 con 0x7f74700cc470 2024-04-20T21:25:56.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.297+0000 7f746e000640 1 --2- 172.21.15.102:0/1626477206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74700cc470 0x7f74700cda80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f745c0097a0 tx=0x7f745c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1722162d21664e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:56.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.297+0000 7f746d600640 1 -- 172.21.15.102:0/1626477206 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c034070 con 0x7f74700cc470 2024-04-20T21:25:56.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.297+0000 7f746d600640 1 -- 172.21.15.102:0/1626477206 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c030610 con 0x7f74700cc470 2024-04-20T21:25:56.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.297+0000 7f746d600640 1 -- 172.21.15.102:0/1626477206 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c0309e0 con 0x7f74700cc470 2024-04-20T21:25:56.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.298+0000 7f746fe00640 1 -- 172.21.15.102:0/1626477206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74700cc470 msgr2=0x7f74700cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.298+0000 7f746fe00640 1 --2- 172.21.15.102:0/1626477206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74700cc470 0x7f74700cda80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f745c0097a0 tx=0x7f745c02b960 comp rx=0 tx=0).stop 2024-04-20T21:25:56.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.298+0000 7f746fe00640 1 -- 172.21.15.102:0/1626477206 shutdown_connections 2024-04-20T21:25:56.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.298+0000 7f746fe00640 1 --2- 172.21.15.102:0/1626477206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74700cc470 0x7f74700cda80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.298+0000 7f746fe00640 1 -- 172.21.15.102:0/1626477206 >> 172.21.15.102:0/1626477206 conn(0x7f7470075080 msgr2=0x7f7470075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:56.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.298+0000 7f746fe00640 1 -- 172.21.15.102:0/1626477206 shutdown_connections 2024-04-20T21:25:56.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.298+0000 7f746fe00640 1 -- 172.21.15.102:0/1626477206 wait complete. 2024-04-20T21:25:56.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.300+0000 7f746fe00640 1 Processor -- start 2024-04-20T21:25:56.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.300+0000 7f746fe00640 1 -- start start 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74700cc470 0x7f74701446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700cdfc0 0x7f747014ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7470155530 0x7f747014b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f74700cd280 con 0x7f74700cc470 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74700cd5f0 con 0x7f7470155530 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746fe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f74700ccbe0 con 0x7f74700cdfc0 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74700cc470 0x7f74701446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74700cc470 0x7f74701446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42430/0 (socket says 172.21.15.102:42430) 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746ea00640 1 -- 172.21.15.102:0/335398198 learned_addr learned my addr 172.21.15.102:0/335398198 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746f400640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7470155530 0x7f747014b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.301+0000 7f746e000640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700cdfc0 0x7f747014ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f746ea00640 1 -- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700cdfc0 msgr2=0x7f747014ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f746ea00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700cdfc0 0x7f747014ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f746ea00640 1 -- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7470155530 msgr2=0x7f747014b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f746ea00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7470155530 0x7f747014b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f746ea00640 1 -- 172.21.15.102:0/335398198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7470152970 con 0x7f74700cc470 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f746f400640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7470155530 0x7f747014b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f746ea00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74700cc470 0x7f74701446e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f746000c440 tx=0x7f746000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f7466a00640 1 -- 172.21.15.102:0/335398198 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460018070 con 0x7f74700cc470 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f7466a00640 1 -- 172.21.15.102:0/335398198 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7460007480 con 0x7f74700cc470 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f7466a00640 1 -- 172.21.15.102:0/335398198 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74600047e0 con 0x7f74700cc470 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.302+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7470144c20 con 0x7f74700cc470 2024-04-20T21:25:56.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.303+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74700b5980 con 0x7f74700cc470 2024-04-20T21:25:56.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.304+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f747004fd60 con 0x7f74700cc470 2024-04-20T21:25:56.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.305+0000 7f7466a00640 1 -- 172.21.15.102:0/335398198 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7460004980 con 0x7f74700cc470 2024-04-20T21:25:56.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.306+0000 7f7466a00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f745006c8f0 0x7f745006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.306+0000 7f7466a00640 1 -- 172.21.15.102:0/335398198 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 7135+0+0 (secure 0 0 0) 0x7f746008b3c0 con 0x7f74700cc470 2024-04-20T21:25:56.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.306+0000 7f746e000640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f745006c8f0 0x7f745006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.307+0000 7f746e000640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f745006c8f0 0x7f745006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f745c036000 tx=0x7f745c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:56.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.310+0000 7f7466a00640 1 -- 172.21.15.102:0/335398198 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7470144c20 con 0x7f74700cc470 2024-04-20T21:25:56.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.568+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74700b3c10 con 0x7f74700cc470 2024-04-20T21:25:56.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.569+0000 7f7466a00640 1 -- 172.21.15.102:0/335398198 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6862 (secure 0 0 0) 0x7f7470152970 con 0x7f74700cc470 2024-04-20T21:25:56.568 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:25:56.569 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5533,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:25:56.569 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f745006c8f0 msgr2=0x7f745006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f745006c8f0 0x7f745006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f745c036000 tx=0x7f745c008030 comp rx=0 tx=0).stop 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74700cc470 msgr2=0x7f74701446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74700cc470 0x7f74701446e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f746000c440 tx=0x7f746000c900 comp rx=0 tx=0).stop 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 shutdown_connections 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f745006c8f0 0x7f745006edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7470155530 0x7f747014b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700cdfc0 0x7f747014ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 --2- 172.21.15.102:0/335398198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74700cc470 0x7f74701446e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.571+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 >> 172.21.15.102:0/335398198 conn(0x7f7470075080 msgr2=0x7f74700ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:56.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.572+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 shutdown_connections 2024-04-20T21:25:56.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.572+0000 7f746fe00640 1 -- 172.21.15.102:0/335398198 wait complete. 2024-04-20T21:25:56.589 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:25:56.590 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:25:56.590 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:25:56.612 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:25:56.613 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:56.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.831+0000 7f737a000640 1 Processor -- start 2024-04-20T21:25:56.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.831+0000 7f737a000640 1 -- start start 2024-04-20T21:25:56.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.832+0000 7f737a000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7374153180 0x7f7374158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.832+0000 7f737a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7374159380 0x7f73740c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.832+0000 7f737a000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73740c75a0 0x7f73740c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.832+0000 7f737a000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f7374153560 con 0x7f73740c75a0 2024-04-20T21:25:56.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.832+0000 7f737a000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f73740c7a70 con 0x7f7374153180 2024-04-20T21:25:56.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.832+0000 7f737a000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f73740c7d30 con 0x7f7374159380 2024-04-20T21:25:56.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7379600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73740c75a0 0x7f73740c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7378c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7374153180 0x7f7374158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7373e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7374159380 0x7f73740c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7379600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73740c75a0 0x7f73740c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42436/0 (socket says 172.21.15.102:42436) 2024-04-20T21:25:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7379600640 1 -- 172.21.15.102:0/4000398222 learned_addr learned my addr 172.21.15.102:0/4000398222 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:25:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7378c00640 1 -- 172.21.15.102:0/4000398222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7374159380 msgr2=0x7f73740c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7378c00640 1 --2- 172.21.15.102:0/4000398222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7374159380 0x7f73740c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7378c00640 1 -- 172.21.15.102:0/4000398222 >> v2:172.21.15.102:3300/0 conn(0x7f73740c75a0 msgr2=0x7f73740c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:56.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7378c00640 1 --2- 172.21.15.102:0/4000398222 >> v2:172.21.15.102:3300/0 conn(0x7f73740c75a0 0x7f73740c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.833+0000 7f7378c00640 1 -- 172.21.15.102:0/4000398222 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737415c4d0 con 0x7f7374153180 2024-04-20T21:25:56.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.835+0000 7f7378c00640 1 --2- 172.21.15.102:0/4000398222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374153180 0x7f7374158e40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73740cd860 tx=0x7f736802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7b944a749b30d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:56.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.835+0000 7f7373400640 1 -- 172.21.15.102:0/4000398222 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680315a0 con 0x7f7374153180 2024-04-20T21:25:56.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.835+0000 7f7373400640 1 -- 172.21.15.102:0/4000398222 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7368037070 con 0x7f7374153180 2024-04-20T21:25:56.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.835+0000 7f7373400640 1 -- 172.21.15.102:0/4000398222 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368031960 con 0x7f7374153180 2024-04-20T21:25:56.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 -- 172.21.15.102:0/4000398222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374153180 msgr2=0x7f7374158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 --2- 172.21.15.102:0/4000398222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374153180 0x7f7374158e40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73740cd860 tx=0x7f736802b840 comp rx=0 tx=0).stop 2024-04-20T21:25:56.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 -- 172.21.15.102:0/4000398222 shutdown_connections 2024-04-20T21:25:56.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 --2- 172.21.15.102:0/4000398222 >> v2:172.21.15.102:3300/0 conn(0x7f73740c75a0 0x7f73740c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 --2- 172.21.15.102:0/4000398222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7374159380 0x7f73740c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 --2- 172.21.15.102:0/4000398222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374153180 0x7f7374158e40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 -- 172.21.15.102:0/4000398222 >> 172.21.15.102:0/4000398222 conn(0x7f73740cc820 msgr2=0x7f73740ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:56.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 -- 172.21.15.102:0/4000398222 shutdown_connections 2024-04-20T21:25:56.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.836+0000 7f737a000640 1 -- 172.21.15.102:0/4000398222 wait complete. 2024-04-20T21:25:56.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.844+0000 7f737a000640 1 Processor -- start 2024-04-20T21:25:56.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.844+0000 7f737a000640 1 -- start start 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f737a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73740c75a0 0x7f73740c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f737a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7374153180 0x7f7374149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f737a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374159380 0x7f73741f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f737a000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f737415d290 con 0x7f7374153180 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f737a000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f737415ca90 con 0x7f7374159380 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f737a000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f73740c6be0 con 0x7f73740c75a0 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f7373e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7374153180 0x7f7374149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f7373e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7374153180 0x7f7374149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42442/0 (socket says 172.21.15.102:42442) 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f7373e00640 1 -- 172.21.15.102:0/854495516 learned_addr learned my addr 172.21.15.102:0/854495516 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f7378c00640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73740c75a0 0x7f73740c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.845+0000 7f7379600640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374159380 0x7f73741f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7373e00640 1 -- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73740c75a0 msgr2=0x7f73740c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7373e00640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73740c75a0 0x7f73740c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7373e00640 1 -- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374159380 msgr2=0x7f73741f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7373e00640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374159380 0x7f73741f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7373e00640 1 -- 172.21.15.102:0/854495516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7374150660 con 0x7f7374153180 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7379600640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374159380 0x7f73741f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7373e00640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7374153180 0x7f7374149ea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f736000e9b0 tx=0x7f736000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7370c00640 1 -- 172.21.15.102:0/854495516 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360017070 con 0x7f7374153180 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7370c00640 1 -- 172.21.15.102:0/854495516 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736000c9a0 con 0x7f7374153180 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.846+0000 7f7370c00640 1 -- 172.21.15.102:0/854495516 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736000cd70 con 0x7f7374153180 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.847+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73741590f0 con 0x7f7374153180 2024-04-20T21:25:56.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.847+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7374147180 con 0x7f7374153180 2024-04-20T21:25:56.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.848+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73741589e0 con 0x7f7374153180 2024-04-20T21:25:56.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.849+0000 7f7370c00640 1 -- 172.21.15.102:0/854495516 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f736001a070 con 0x7f7374153180 2024-04-20T21:25:56.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.850+0000 7f7370c00640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f735406c920 0x7f735406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:56.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.851+0000 7f7370c00640 1 -- 172.21.15.102:0/854495516 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 7135+0+0 (secure 0 0 0) 0x7f736008c090 con 0x7f7374153180 2024-04-20T21:25:56.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.851+0000 7f7378c00640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f735406c920 0x7f735406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:56.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.851+0000 7f7378c00640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f735406c920 0x7f735406ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7368036000 tx=0x7f736803d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:56.859+0000 7f7370c00640 1 -- 172.21.15.102:0/854495516 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f73600597f0 con 0x7f7374153180 2024-04-20T21:25:57.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.153+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f737404fd60 con 0x7f7374153180 2024-04-20T21:25:57.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.154+0000 7f7370c00640 1 -- 172.21.15.102:0/854495516 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6862 (secure 0 0 0) 0x7f736005daf0 con 0x7f7374153180 2024-04-20T21:25:57.153 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:25:57.154 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5533,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:25:57.154 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.156+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f735406c920 msgr2=0x7f735406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.156+0000 7f737a000640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f735406c920 0x7f735406ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7368036000 tx=0x7f736803d040 comp rx=0 tx=0).stop 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.156+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7374153180 msgr2=0x7f7374149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.156+0000 7f737a000640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7374153180 0x7f7374149ea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f736000e9b0 tx=0x7f736000ee70 comp rx=0 tx=0).stop 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.156+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 shutdown_connections 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.156+0000 7f737a000640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f735406c920 0x7f735406ede0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.157+0000 7f737a000640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7374159380 0x7f73741f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.157+0000 7f737a000640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7374153180 0x7f7374149ea0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:57.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.157+0000 7f737a000640 1 --2- 172.21.15.102:0/854495516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73740c75a0 0x7f73740c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.157+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 >> 172.21.15.102:0/854495516 conn(0x7f73740cc820 msgr2=0x7f73740c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.157+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 shutdown_connections 2024-04-20T21:25:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:57.157+0000 7f737a000640 1 -- 172.21.15.102:0/854495516 wait complete. 2024-04-20T21:25:57.176 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:25:59.178 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:59.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.357+0000 7fd0a7e00640 1 Processor -- start 2024-04-20T21:25:59.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.357+0000 7fd0a7e00640 1 -- start start 2024-04-20T21:25:59.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.358+0000 7fd0a7e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd0a8155530 0x7fd0a815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.358+0000 7fd0a7e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a815ef60 con 0x7fd0a80cc470 2024-04-20T21:25:59.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.358+0000 7fd0a7e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a815f240 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.358+0000 7fd0a7e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a80c5ca0 con 0x7fd0a8155530 2024-04-20T21:25:59.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.359+0000 7fd0a6000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd0a80cc470 0x7fd0a80cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36294/0 (socket says 172.21.15.102:36294) 2024-04-20T21:25:59.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.359+0000 7fd0a6000640 1 -- 172.21.15.102:0/2884363365 learned_addr learned my addr 172.21.15.102:0/2884363365 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:25:59.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.359+0000 7fd0a6a00640 1 --2- 172.21.15.102:0/2884363365 >> v2:172.21.15.193:3300/0 conn(0x7fd0a8155530 0x7fd0a815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.359+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2884363365 >> v1:172.21.15.193:6789/0 conn(0x7fd0a80cb6a0 legacy=0x7fd0a81659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.359+0000 7fd0a5600640 1 -- 172.21.15.102:0/2884363365 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643591722 0 0) 0x7fd0a815f240 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.359+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2884363365 >> v1:172.21.15.102:6789/0 conn(0x7fd0a80cc470 legacy=0x7fd0a80cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.359+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2884363365 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a815f520 con 0x7fd0a8155530 2024-04-20T21:25:59.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.361+0000 7fd0a6a00640 1 --2- 172.21.15.102:0/2884363365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 0x7fd0a815e640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd0980097a0 tx=0x7fd09802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f1f0fb6e441d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.361+0000 7fd0a5600640 1 -- 172.21.15.102:0/2884363365 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098037070 con 0x7fd0a8155530 2024-04-20T21:25:59.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.361+0000 7fd0a5600640 1 -- 172.21.15.102:0/2884363365 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd098030710 con 0x7fd0a8155530 2024-04-20T21:25:59.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.361+0000 7fd0a5600640 1 -- 172.21.15.102:0/2884363365 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098030ae0 con 0x7fd0a8155530 2024-04-20T21:25:59.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.361+0000 7fd0a7e00640 1 -- 172.21.15.102:0/2884363365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 msgr2=0x7fd0a815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.361+0000 7fd0a7e00640 1 --2- 172.21.15.102:0/2884363365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 0x7fd0a815e640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd0980097a0 tx=0x7fd09802b790 comp rx=0 tx=0).stop 2024-04-20T21:25:59.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.362+0000 7fd0a7e00640 1 -- 172.21.15.102:0/2884363365 shutdown_connections 2024-04-20T21:25:59.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.362+0000 7fd0a7e00640 1 --2- 172.21.15.102:0/2884363365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 0x7fd0a815e640 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.362+0000 7fd0a7e00640 1 -- 172.21.15.102:0/2884363365 >> 172.21.15.102:0/2884363365 conn(0x7fd0a8075080 msgr2=0x7fd0a8075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.362+0000 7fd0a7e00640 1 -- 172.21.15.102:0/2884363365 shutdown_connections 2024-04-20T21:25:59.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.362+0000 7fd0a7e00640 1 -- 172.21.15.102:0/2884363365 wait complete. 2024-04-20T21:25:59.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.364+0000 7fd0a7e00640 1 Processor -- start 2024-04-20T21:25:59.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.364+0000 7fd0a7e00640 1 -- start start 2024-04-20T21:25:59.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a80cb6a0 0x7fd0a80b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0a80cc470 0x7fd0a81e4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 0x7fd0a81e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80cd280 con 0x7fd0a80cc470 2024-04-20T21:25:59.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80cd5f0 con 0x7fd0a8155530 2024-04-20T21:25:59.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a7e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a80ccbe0 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a6a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a80cb6a0 0x7fd0a80b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a6a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a80cb6a0 0x7fd0a80b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:43128/0 (socket says 172.21.15.102:43128) 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a6a00640 1 -- 172.21.15.102:0/487832229 learned_addr learned my addr 172.21.15.102:0/487832229 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a6000640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0a80cc470 0x7fd0a81e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.366+0000 7fd0a7400640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 0x7fd0a81e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd0a6a00640 1 -- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 msgr2=0x7fd0a81e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd0a6a00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 0x7fd0a81e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd0a6a00640 1 -- 172.21.15.102:0/487832229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0a80cc470 msgr2=0x7fd0a81e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd0a6a00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0a80cc470 0x7fd0a81e4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd0a6a00640 1 -- 172.21.15.102:0/487832229 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a8147540 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd0a7400640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 0x7fd0a81e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:25:59.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd0a6a00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a80cb6a0 0x7fd0a80b5f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd098000c00 tx=0x7fd0980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd09ea00640 1 -- 172.21.15.102:0/487832229 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098042070 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd09ea00640 1 -- 172.21.15.102:0/487832229 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd098032a40 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81e61a0 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.367+0000 7fd09ea00640 1 -- 172.21.15.102:0/487832229 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098037040 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.368+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81e6760 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.369+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a804fd60 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.370+0000 7fd09ea00640 1 -- 172.21.15.102:0/487832229 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd098045070 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.371+0000 7fd09ea00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08806c8f0 0x7fd08806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.371+0000 7fd09ea00640 1 -- 172.21.15.102:0/487832229 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 7135+0+0 (secure 0 0 0) 0x7fd0980ae220 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.371+0000 7fd0a6000640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08806c8f0 0x7fd08806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.372+0000 7fd0a6000640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08806c8f0 0x7fd08806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80b6a30 tx=0x7fd094007e00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.375+0000 7fd09ea00640 1 -- 172.21.15.102:0/487832229 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd09807ba50 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.609+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a8148410 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.609+0000 7fd09ea00640 1 -- 172.21.15.102:0/487832229 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6862 (secure 0 0 0) 0x7fd09807fd50 con 0x7fd0a80cb6a0 2024-04-20T21:25:59.609 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:25:59.609 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5533,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:25:59.609 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-04-20T21:25:59.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.611+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08806c8f0 msgr2=0x7fd08806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.611+0000 7fd0a7e00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08806c8f0 0x7fd08806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80b6a30 tx=0x7fd094007e00 comp rx=0 tx=0).stop 2024-04-20T21:25:59.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.611+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a80cb6a0 msgr2=0x7fd0a80b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.611+0000 7fd0a7e00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a80cb6a0 0x7fd0a80b5f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd098000c00 tx=0x7fd0980046c0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.612+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 shutdown_connections 2024-04-20T21:25:59.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.612+0000 7fd0a7e00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08806c8f0 0x7fd08806edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.612+0000 7fd0a7e00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a8155530 0x7fd0a81e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.612+0000 7fd0a7e00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0a80cc470 0x7fd0a81e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.612+0000 7fd0a7e00640 1 --2- 172.21.15.102:0/487832229 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a80cb6a0 0x7fd0a80b5f70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.612+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 >> 172.21.15.102:0/487832229 conn(0x7fd0a8075080 msgr2=0x7fd0a80ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.612+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 shutdown_connections 2024-04-20T21:25:59.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:25:59.612+0000 7fd0a7e00640 1 -- 172.21.15.102:0/487832229 wait complete. 2024-04-20T21:25:59.628 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:26:01.629 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:01.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.801+0000 7f0861e00640 1 Processor -- start 2024-04-20T21:26:01.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.801+0000 7f0861e00640 1 -- start start 2024-04-20T21:26:01.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.803+0000 7f0861e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f085c0ceff0 0x7f085c06b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:01.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.803+0000 7f0861e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f085c15e790 0x7f085c15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:01.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.803+0000 7f0861e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c144140 0x7f085c1445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:01.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.803+0000 7f0861e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f085c0cf3d0 con 0x7f085c15e790 2024-04-20T21:26:01.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.803+0000 7f0861e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f085c06bf60 con 0x7f085c0ceff0 2024-04-20T21:26:01.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.803+0000 7f0861e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f085c06c240 con 0x7f085c144140 2024-04-20T21:26:01.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f085be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f085c15e790 0x7f085c15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f085be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f085c15e790 0x7f085c15bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59142/0 (socket says 172.21.15.102:59142) 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f0860a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f085c0ceff0 0x7f085c06b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f085be00640 1 -- 172.21.15.102:0/1767443434 learned_addr learned my addr 172.21.15.102:0/1767443434 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f0861400640 1 --2- 172.21.15.102:0/1767443434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c144140 0x7f085c1445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f085be00640 1 -- 172.21.15.102:0/1767443434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c144140 msgr2=0x7f085c1445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f085be00640 1 --2- 172.21.15.102:0/1767443434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c144140 0x7f085c1445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f085be00640 1 -- 172.21.15.102:0/1767443434 >> v2:172.21.15.193:3300/0 conn(0x7f085c0ceff0 msgr2=0x7f085c06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f085be00640 1 --2- 172.21.15.102:0/1767443434 >> v2:172.21.15.193:3300/0 conn(0x7f085c0ceff0 0x7f085c06b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:01.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.804+0000 7f085be00640 1 -- 172.21.15.102:0/1767443434 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c06c520 con 0x7f085c15e790 2024-04-20T21:26:01.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.806+0000 7f085be00640 1 --2- 172.21.15.102:0/1767443434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c15e790 0x7f085c15bd10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f085c06db20 tx=0x7f084802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57803b47e96b6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:01.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.806+0000 7f085b400640 1 -- 172.21.15.102:0/1767443434 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848034070 con 0x7f085c15e790 2024-04-20T21:26:01.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.806+0000 7f085b400640 1 -- 172.21.15.102:0/1767443434 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0848030860 con 0x7f085c15e790 2024-04-20T21:26:01.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.806+0000 7f085b400640 1 -- 172.21.15.102:0/1767443434 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848030c30 con 0x7f085c15e790 2024-04-20T21:26:01.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 -- 172.21.15.102:0/1767443434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c15e790 msgr2=0x7f085c15bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 --2- 172.21.15.102:0/1767443434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c15e790 0x7f085c15bd10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f085c06db20 tx=0x7f084802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:26:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 -- 172.21.15.102:0/1767443434 shutdown_connections 2024-04-20T21:26:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 --2- 172.21.15.102:0/1767443434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c144140 0x7f085c1445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 --2- 172.21.15.102:0/1767443434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c15e790 0x7f085c15bd10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 --2- 172.21.15.102:0/1767443434 >> v2:172.21.15.193:3300/0 conn(0x7f085c0ceff0 0x7f085c06b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 -- 172.21.15.102:0/1767443434 >> 172.21.15.102:0/1767443434 conn(0x7f085c075820 msgr2=0x7f085c070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 -- 172.21.15.102:0/1767443434 shutdown_connections 2024-04-20T21:26:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.807+0000 7f0861e00640 1 -- 172.21.15.102:0/1767443434 wait complete. 2024-04-20T21:26:01.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.810+0000 7f0861e00640 1 Processor -- start 2024-04-20T21:26:01.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.810+0000 7f0861e00640 1 -- start start 2024-04-20T21:26:01.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.811+0000 7f0861e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f085c0ceff0 0x7f085c142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:01.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.811+0000 7f0861e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c144140 0x7f085c1f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:01.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.811+0000 7f0861e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c15e790 0x7f085c13d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:01.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.811+0000 7f0861e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f085c15b1c0 con 0x7f085c144140 2024-04-20T21:26:01.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.811+0000 7f0861e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f085c15b7f0 con 0x7f085c0ceff0 2024-04-20T21:26:01.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.811+0000 7f0861e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f085c0734b0 con 0x7f085c15e790 2024-04-20T21:26:01.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f085c0ceff0 0x7f085c142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:01.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f085be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c144140 0x7f085c1f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:01.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f085c0ceff0 0x7f085c142af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:43516/0 (socket says 172.21.15.102:43516) 2024-04-20T21:26:01.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 -- 172.21.15.102:0/3931165180 learned_addr learned my addr 172.21.15.102:0/3931165180 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:01.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0861400640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c15e790 0x7f085c13d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:01.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 -- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c15e790 msgr2=0x7f085c13d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:01.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c15e790 0x7f085c13d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:01.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 -- 172.21.15.102:0/3931165180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c144140 msgr2=0x7f085c1f0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:01.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c144140 0x7f085c1f0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:01.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 -- 172.21.15.102:0/3931165180 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c15e180 con 0x7f085c0ceff0 2024-04-20T21:26:01.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.812+0000 7f0860a00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f085c0ceff0 0x7f085c142af0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f085000e8c0 tx=0x7f085000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:01.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.813+0000 7f0858c00640 1 -- 172.21.15.102:0/3931165180 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850017070 con 0x7f085c0ceff0 2024-04-20T21:26:01.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.813+0000 7f0858c00640 1 -- 172.21.15.102:0/3931165180 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085000c940 con 0x7f085c0ceff0 2024-04-20T21:26:01.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.813+0000 7f0858c00640 1 -- 172.21.15.102:0/3931165180 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085000cd10 con 0x7f085c0ceff0 2024-04-20T21:26:01.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.813+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c157e70 con 0x7f085c0ceff0 2024-04-20T21:26:01.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.813+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c1f4240 con 0x7f085c0ceff0 2024-04-20T21:26:01.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.814+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f085c06d140 con 0x7f085c0ceff0 2024-04-20T21:26:01.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.816+0000 7f0858c00640 1 -- 172.21.15.102:0/3931165180 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f085001a070 con 0x7f085c0ceff0 2024-04-20T21:26:01.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.817+0000 7f0858c00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f083c06c8f0 0x7f083c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:01.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.817+0000 7f0858c00640 1 -- 172.21.15.102:0/3931165180 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 7135+0+0 (secure 0 0 0) 0x7f085008bf40 con 0x7f085c0ceff0 2024-04-20T21:26:01.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.817+0000 7f085be00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f083c06c8f0 0x7f083c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:01.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.818+0000 7f085be00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f083c06c8f0 0x7f083c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0848036000 tx=0x7f084803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:01.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:01.821+0000 7f0858c00640 1 -- 172.21.15.102:0/3931165180 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f085c06d140 con 0x7f085c0ceff0 2024-04-20T21:26:02.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.066+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f085c04fd60 con 0x7f085c0ceff0 2024-04-20T21:26:02.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.068+0000 7f0858c00640 1 -- 172.21.15.102:0/3931165180 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6862 (secure 0 0 0) 0x7f0850059820 con 0x7f085c0ceff0 2024-04-20T21:26:02.066 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:02.066 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5533,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:26:02.066 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-04-20T21:26:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f083c06c8f0 msgr2=0x7f083c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f083c06c8f0 0x7f083c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0848036000 tx=0x7f084803b040 comp rx=0 tx=0).stop 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f085c0ceff0 msgr2=0x7f085c142af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f085c0ceff0 0x7f085c142af0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f085000e8c0 tx=0x7f085000ed80 comp rx=0 tx=0).stop 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 shutdown_connections 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f085c15e790 0x7f085c13d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f085c144140 0x7f085c1f0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f083c06c8f0 0x7f083c06edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 --2- 172.21.15.102:0/3931165180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f085c0ceff0 0x7f085c142af0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 >> 172.21.15.102:0/3931165180 conn(0x7f085c075820 msgr2=0x7f085c15d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 shutdown_connections 2024-04-20T21:26:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:02.070+0000 7f0861e00640 1 -- 172.21.15.102:0/3931165180 wait complete. 2024-04-20T21:26:02.091 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:26:04.092 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:04.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.259+0000 7f62d6400640 1 Processor -- start 2024-04-20T21:26:04.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.259+0000 7f62d6400640 1 -- start start 2024-04-20T21:26:04.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.260+0000 7f62d6400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f62d0153180 0x7f62d0158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.260+0000 7f62d6400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d0153560 con 0x7f62d0159380 2024-04-20T21:26:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.260+0000 7f62d6400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d00c7a70 con 0x7f62d00c75a0 2024-04-20T21:26:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.260+0000 7f62d6400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f62d00c7d30 con 0x7f62d0153180 2024-04-20T21:26:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.261+0000 7f62cfe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f62d0159380 0x7f62d00c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36304/0 (socket says 172.21.15.102:36304) 2024-04-20T21:26:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.261+0000 7f62cfe00640 1 -- 172.21.15.102:0/906927630 learned_addr learned my addr 172.21.15.102:0/906927630 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.261+0000 7f62d5000640 1 --2- 172.21.15.102:0/906927630 >> v2:172.21.15.193:3300/0 conn(0x7f62d0153180 0x7f62d0158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:04.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.261+0000 7f62d5000640 1 -- 172.21.15.102:0/906927630 >> v1:172.21.15.193:6789/0 conn(0x7f62d00c75a0 legacy=0x7f62d01637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:04.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.261+0000 7f62d5000640 1 -- 172.21.15.102:0/906927630 >> v1:172.21.15.102:6789/0 conn(0x7f62d0159380 legacy=0x7f62d00c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:04.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.261+0000 7f62d5000640 1 -- 172.21.15.102:0/906927630 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d015c4d0 con 0x7f62d0153180 2024-04-20T21:26:04.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.261+0000 7f62cf400640 1 -- 172.21.15.102:0/906927630 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1661719110 0 0) 0x7f62d0153560 con 0x7f62d0159380 2024-04-20T21:26:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.263+0000 7f62d5000640 1 --2- 172.21.15.102:0/906927630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d0153180 0x7f62d0158e40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62c40097a0 tx=0x7f62c402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e38afd11e96d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.263+0000 7f62cf400640 1 -- 172.21.15.102:0/906927630 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4034070 con 0x7f62d0153180 2024-04-20T21:26:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.263+0000 7f62cf400640 1 -- 172.21.15.102:0/906927630 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c4030610 con 0x7f62d0153180 2024-04-20T21:26:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.263+0000 7f62cf400640 1 -- 172.21.15.102:0/906927630 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c40309e0 con 0x7f62d0153180 2024-04-20T21:26:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.264+0000 7f62d6400640 1 -- 172.21.15.102:0/906927630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d0153180 msgr2=0x7f62d0158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.264+0000 7f62d6400640 1 --2- 172.21.15.102:0/906927630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d0153180 0x7f62d0158e40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62c40097a0 tx=0x7f62c402b960 comp rx=0 tx=0).stop 2024-04-20T21:26:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.264+0000 7f62d6400640 1 -- 172.21.15.102:0/906927630 shutdown_connections 2024-04-20T21:26:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.264+0000 7f62d6400640 1 --2- 172.21.15.102:0/906927630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d0153180 0x7f62d0158e40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.264+0000 7f62d6400640 1 -- 172.21.15.102:0/906927630 >> 172.21.15.102:0/906927630 conn(0x7f62d00cc820 msgr2=0x7f62d00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.264+0000 7f62d6400640 1 -- 172.21.15.102:0/906927630 shutdown_connections 2024-04-20T21:26:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.264+0000 7f62d6400640 1 -- 172.21.15.102:0/906927630 wait complete. 2024-04-20T21:26:04.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.267+0000 7f62d6400640 1 Processor -- start 2024-04-20T21:26:04.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.267+0000 7f62d6400640 1 -- start start 2024-04-20T21:26:04.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d6400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d00c75a0 0x7f62d00cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:04.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d0153180 0x7f62d0158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:04.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d6400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62d0159380 0x7f62d0158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d6400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62d015d290 con 0x7f62d0153180 2024-04-20T21:26:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d6400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62d015ca90 con 0x7f62d00c75a0 2024-04-20T21:26:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d6400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f62d00c6be0 con 0x7f62d0159380 2024-04-20T21:26:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d5000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d00c75a0 0x7f62d00cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d5a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62d0159380 0x7f62d0158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d5000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d00c75a0 0x7f62d00cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:43534/0 (socket says 172.21.15.102:43534) 2024-04-20T21:26:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62d5000640 1 -- 172.21.15.102:0/2782497299 learned_addr learned my addr 172.21.15.102:0/2782497299 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.268+0000 7f62cfe00640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d0153180 0x7f62d0158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62d5000640 1 -- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62d0159380 msgr2=0x7f62d0158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62d5000640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62d0159380 0x7f62d0158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62d5000640 1 -- 172.21.15.102:0/2782497299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d0153180 msgr2=0x7f62d0158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62d5000640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d0153180 0x7f62d0158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62d5000640 1 -- 172.21.15.102:0/2782497299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d00caaa0 con 0x7f62d00c75a0 2024-04-20T21:26:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62d5000640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d00c75a0 0x7f62d00cb6f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62c4000c00 tx=0x7f62c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62ccc00640 1 -- 172.21.15.102:0/2782497299 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4034040 con 0x7f62d00c75a0 2024-04-20T21:26:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62ccc00640 1 -- 172.21.15.102:0/2782497299 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c4004230 con 0x7f62d00c75a0 2024-04-20T21:26:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62ccc00640 1 -- 172.21.15.102:0/2782497299 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4039680 con 0x7f62d00c75a0 2024-04-20T21:26:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.269+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d01429f0 con 0x7f62d00c75a0 2024-04-20T21:26:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.270+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d00cbc30 con 0x7f62d00c75a0 2024-04-20T21:26:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.271+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d004fd60 con 0x7f62d00c75a0 2024-04-20T21:26:04.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.272+0000 7f62ccc00640 1 -- 172.21.15.102:0/2782497299 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62c4037070 con 0x7f62d00c75a0 2024-04-20T21:26:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.273+0000 7f62ccc00640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f62b006c8f0 0x7f62b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.273+0000 7f62ccc00640 1 -- 172.21.15.102:0/2782497299 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 7135+0+0 (secure 0 0 0) 0x7f62c40b2100 con 0x7f62d00c75a0 2024-04-20T21:26:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.274+0000 7f62cfe00640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f62b006c8f0 0x7f62b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.274+0000 7f62cfe00640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f62b006c8f0 0x7f62b006edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f62d006e820 tx=0x7f62bc007bb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:04.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.278+0000 7f62ccc00640 1 -- 172.21.15.102:0/2782497299 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f62c40305e0 con 0x7f62d00c75a0 2024-04-20T21:26:04.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.583+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62d00c3690 con 0x7f62d00c75a0 2024-04-20T21:26:04.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.584+0000 7f62ccc00640 1 -- 172.21.15.102:0/2782497299 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6862 (secure 0 0 0) 0x7f62c40800a0 con 0x7f62d00c75a0 2024-04-20T21:26:04.584 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:04.584 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5533,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:26:04.584 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.587+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f62b006c8f0 msgr2=0x7f62b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.587+0000 7f62d6400640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f62b006c8f0 0x7f62b006edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f62d006e820 tx=0x7f62bc007bb0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.587+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d00c75a0 msgr2=0x7f62d00cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.587+0000 7f62d6400640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d00c75a0 0x7f62d00cb6f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62c4000c00 tx=0x7f62c40046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.588+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 shutdown_connections 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.588+0000 7f62d6400640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62d0159380 0x7f62d0158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.588+0000 7f62d6400640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d0153180 0x7f62d0158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.588+0000 7f62d6400640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f62b006c8f0 0x7f62b006edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.588+0000 7f62d6400640 1 --2- 172.21.15.102:0/2782497299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62d00c75a0 0x7f62d00cb6f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:04.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.588+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 >> 172.21.15.102:0/2782497299 conn(0x7f62d00cc820 msgr2=0x7f62d0148e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:04.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.588+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 shutdown_connections 2024-04-20T21:26:04.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:04.588+0000 7f62d6400640 1 -- 172.21.15.102:0/2782497299 wait complete. 2024-04-20T21:26:04.608 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:26:06.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:06.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.775+0000 7f201e400640 1 Processor -- start 2024-04-20T21:26:06.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.776+0000 7f201e400640 1 -- start start 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.777+0000 7f201e400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f20180cc470 0x7f20180cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.777+0000 7f201e400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f20180cdfc0 0x7f20180cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.777+0000 7f201e400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f201815ef60 con 0x7f2018155530 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.777+0000 7f201e400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f20180c5ca0 con 0x7f20180cc470 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.777+0000 7f201e400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f20180cd5f0 con 0x7f20180cdfc0 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.777+0000 7f2017e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f20180cc470 0x7f20180cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.777+0000 7f201d000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2018155530 0x7f201815e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36308/0 (socket says 172.21.15.102:36308) 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.778+0000 7f201d000640 1 -- 172.21.15.102:0/1945966958 learned_addr learned my addr 172.21.15.102:0/1945966958 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:06.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.778+0000 7f201da00640 1 --2- 172.21.15.102:0/1945966958 >> v2:172.21.15.193:3300/0 conn(0x7f20180cdfc0 0x7f20180cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:06.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.778+0000 7f201da00640 1 -- 172.21.15.102:0/1945966958 >> v2:172.21.15.102:3300/0 conn(0x7f20180cc470 msgr2=0x7f20180cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:06.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.778+0000 7f201da00640 1 --2- 172.21.15.102:0/1945966958 >> v2:172.21.15.102:3300/0 conn(0x7f20180cc470 0x7f20180cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:06.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.778+0000 7f201da00640 1 -- 172.21.15.102:0/1945966958 >> v1:172.21.15.102:6789/0 conn(0x7f2018155530 legacy=0x7f201815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:06.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.778+0000 7f201da00640 1 -- 172.21.15.102:0/1945966958 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201815f240 con 0x7f20180cdfc0 2024-04-20T21:26:06.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.778+0000 7f2017e00640 1 --2- 172.21.15.102:0/1945966958 >> v2:172.21.15.102:3300/0 conn(0x7f20180cc470 0x7f20180cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:06.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.779+0000 7f201da00640 1 --2- 172.21.15.102:0/1945966958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20180cdfc0 0x7f20180cb6a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f201808e290 tx=0x7f201002bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=793f761685c76a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:06.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.780+0000 7f2017400640 1 -- 172.21.15.102:0/1945966958 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010039070 con 0x7f20180cdfc0 2024-04-20T21:26:06.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.780+0000 7f2017400640 1 -- 172.21.15.102:0/1945966958 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201002cc70 con 0x7f20180cdfc0 2024-04-20T21:26:06.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.780+0000 7f2017400640 1 -- 172.21.15.102:0/1945966958 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010034660 con 0x7f20180cdfc0 2024-04-20T21:26:06.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.780+0000 7f201e400640 1 -- 172.21.15.102:0/1945966958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20180cdfc0 msgr2=0x7f20180cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:06.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.780+0000 7f201e400640 1 --2- 172.21.15.102:0/1945966958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20180cdfc0 0x7f20180cb6a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f201808e290 tx=0x7f201002bec0 comp rx=0 tx=0).stop 2024-04-20T21:26:06.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.780+0000 7f201e400640 1 -- 172.21.15.102:0/1945966958 shutdown_connections 2024-04-20T21:26:06.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.781+0000 7f201e400640 1 --2- 172.21.15.102:0/1945966958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20180cdfc0 0x7f20180cb6a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:06.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.781+0000 7f201e400640 1 --2- 172.21.15.102:0/1945966958 >> v2:172.21.15.102:3300/0 conn(0x7f20180cc470 0x7f20180cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:06.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.781+0000 7f201e400640 1 -- 172.21.15.102:0/1945966958 >> 172.21.15.102:0/1945966958 conn(0x7f2018075080 msgr2=0x7f2018075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:06.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.781+0000 7f201e400640 1 -- 172.21.15.102:0/1945966958 shutdown_connections 2024-04-20T21:26:06.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.781+0000 7f201e400640 1 -- 172.21.15.102:0/1945966958 wait complete. 2024-04-20T21:26:06.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.782+0000 7f201e400640 1 Processor -- start 2024-04-20T21:26:06.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.783+0000 7f201e400640 1 -- start start 2024-04-20T21:26:06.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.783+0000 7f201e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20180cc470 0x7f20181f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.783+0000 7f201e400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20180cdfc0 0x7f201814ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.783+0000 7f201e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2018155530 0x7f201815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.783+0000 7f201e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f20180cd280 con 0x7f20180cc470 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.783+0000 7f201e400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20180ccfa0 con 0x7f2018155530 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.783+0000 7f201e400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f20180ccbe0 con 0x7f20180cdfc0 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2018155530 0x7f201815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2018155530 0x7f201815dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:43542/0 (socket says 172.21.15.102:43542) 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201da00640 1 -- 172.21.15.102:0/4179859728 learned_addr learned my addr 172.21.15.102:0/4179859728 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201d000640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20180cc470 0x7f20181f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:06.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f2017e00640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20180cdfc0 0x7f201814ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:06.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201da00640 1 -- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20180cdfc0 msgr2=0x7f201814ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:06.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201da00640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20180cdfc0 0x7f201814ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:06.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201da00640 1 -- 172.21.15.102:0/4179859728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20180cc470 msgr2=0x7f20181f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:06.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201da00640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20180cc470 0x7f20181f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:06.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201da00640 1 -- 172.21.15.102:0/4179859728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20181f1380 con 0x7f2018155530 2024-04-20T21:26:06.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.784+0000 7f201d000640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20180cc470 0x7f20181f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:26:06.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.785+0000 7f201da00640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2018155530 0x7f201815dd70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f201002c4d0 tx=0x7f2010004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:06.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.785+0000 7f2014c00640 1 -- 172.21.15.102:0/4179859728 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010039050 con 0x7f2018155530 2024-04-20T21:26:06.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.785+0000 7f2014c00640 1 -- 172.21.15.102:0/4179859728 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010045070 con 0x7f2018155530 2024-04-20T21:26:06.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.785+0000 7f2014c00640 1 -- 172.21.15.102:0/4179859728 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010009450 con 0x7f2018155530 2024-04-20T21:26:06.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.785+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20181f1660 con 0x7f2018155530 2024-04-20T21:26:06.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.786+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20181f1c20 con 0x7f2018155530 2024-04-20T21:26:06.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.787+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201804fd60 con 0x7f2018155530 2024-04-20T21:26:06.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.788+0000 7f2014c00640 1 -- 172.21.15.102:0/4179859728 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f201003e850 con 0x7f2018155530 2024-04-20T21:26:06.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.789+0000 7f2014c00640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff806c8f0 0x7f1ff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:06.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.789+0000 7f2014c00640 1 -- 172.21.15.102:0/4179859728 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 7135+0+0 (secure 0 0 0) 0x7f20100aee10 con 0x7f2018155530 2024-04-20T21:26:06.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.790+0000 7f201d000640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff806c8f0 0x7f1ff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:06.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.790+0000 7f201d000640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff806c8f0 0x7f1ff806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f200c000f60 tx=0x7f200c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:06.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:06.794+0000 7f2014c00640 1 -- 172.21.15.102:0/4179859728 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f201002ce20 con 0x7f2018155530 2024-04-20T21:26:07.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.065+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2018148410 con 0x7f2018155530 2024-04-20T21:26:07.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.066+0000 7f2014c00640 1 -- 172.21.15.102:0/4179859728 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6862 (secure 0 0 0) 0x7f201007cdd0 con 0x7f2018155530 2024-04-20T21:26:07.066 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:07.066 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5533,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:26:07.066 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-04-20T21:26:07.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff806c8f0 msgr2=0x7f1ff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:07.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff806c8f0 0x7f1ff806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f200c000f60 tx=0x7f200c009040 comp rx=0 tx=0).stop 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2018155530 msgr2=0x7f201815dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2018155530 0x7f201815dd70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f201002c4d0 tx=0x7f2010004a40 comp rx=0 tx=0).stop 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 shutdown_connections 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1ff806c8f0 0x7f1ff806edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2018155530 0x7f201815dd70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20180cdfc0 0x7f201814ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 --2- 172.21.15.102:0/4179859728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20180cc470 0x7f20181f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 >> 172.21.15.102:0/4179859728 conn(0x7f2018075080 msgr2=0x7f20181542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 shutdown_connections 2024-04-20T21:26:07.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:07.069+0000 7f201e400640 1 -- 172.21.15.102:0/4179859728 wait complete. 2024-04-20T21:26:07.084 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:26:09.085 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:09.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.303+0000 7fa882800640 1 Processor -- start 2024-04-20T21:26:09.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.304+0000 7fa882800640 1 -- start start 2024-04-20T21:26:09.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa882800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c155530 0x7fa87c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa882800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa87c0cb6a0 0x7fa87c0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa882800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c15ef60 con 0x7fa87c0cc470 2024-04-20T21:26:09.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa882800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa87c0c5ca0 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa882800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c0cd5f0 con 0x7fa87c155530 2024-04-20T21:26:09.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa880a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa87c0cc470 0x7fa87c0cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36512/0 (socket says 172.21.15.102:36512) 2024-04-20T21:26:09.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa880a00640 1 -- 172.21.15.102:0/3357505868 learned_addr learned my addr 172.21.15.102:0/3357505868 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:09.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa881400640 1 --2- 172.21.15.102:0/3357505868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c155530 0x7fa87c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.306+0000 7fa881e00640 1 --2- 172.21.15.102:0/3357505868 >> v2:172.21.15.102:3300/0 conn(0x7fa87c0cb6a0 0x7fa87c0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.307+0000 7fa881400640 1 -- 172.21.15.102:0/3357505868 >> v2:172.21.15.102:3300/0 conn(0x7fa87c0cb6a0 msgr2=0x7fa87c0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:09.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.307+0000 7fa881400640 1 --2- 172.21.15.102:0/3357505868 >> v2:172.21.15.102:3300/0 conn(0x7fa87c0cb6a0 0x7fa87c0cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.307+0000 7fa881400640 1 -- 172.21.15.102:0/3357505868 >> v1:172.21.15.102:6789/0 conn(0x7fa87c0cc470 legacy=0x7fa87c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.307+0000 7fa881400640 1 -- 172.21.15.102:0/3357505868 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c15f240 con 0x7fa87c155530 2024-04-20T21:26:09.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.307+0000 7fa87be00640 1 -- 172.21.15.102:0/3357505868 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104344681 0 0) 0x7fa87c15ef60 con 0x7fa87c0cc470 2024-04-20T21:26:09.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.307+0000 7fa881e00640 1 --2- 172.21.15.102:0/3357505868 >> v2:172.21.15.102:3300/0 conn(0x7fa87c0cb6a0 0x7fa87c0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:09.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.309+0000 7fa881400640 1 --2- 172.21.15.102:0/3357505868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c155530 0x7fa87c15e640 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa8700097a0 tx=0x7fa87002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae923aaf4a8310e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:09.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.309+0000 7fa87be00640 1 -- 172.21.15.102:0/3357505868 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870034070 con 0x7fa87c155530 2024-04-20T21:26:09.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.309+0000 7fa87be00640 1 -- 172.21.15.102:0/3357505868 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870030800 con 0x7fa87c155530 2024-04-20T21:26:09.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.309+0000 7fa87be00640 1 -- 172.21.15.102:0/3357505868 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870030bd0 con 0x7fa87c155530 2024-04-20T21:26:09.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.310+0000 7fa882800640 1 -- 172.21.15.102:0/3357505868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c155530 msgr2=0x7fa87c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.310+0000 7fa882800640 1 --2- 172.21.15.102:0/3357505868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c155530 0x7fa87c15e640 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa8700097a0 tx=0x7fa87002b940 comp rx=0 tx=0).stop 2024-04-20T21:26:09.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.310+0000 7fa882800640 1 -- 172.21.15.102:0/3357505868 shutdown_connections 2024-04-20T21:26:09.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.310+0000 7fa882800640 1 --2- 172.21.15.102:0/3357505868 >> v2:172.21.15.102:3300/0 conn(0x7fa87c0cb6a0 0x7fa87c0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.310+0000 7fa882800640 1 --2- 172.21.15.102:0/3357505868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c155530 0x7fa87c15e640 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.310+0000 7fa882800640 1 -- 172.21.15.102:0/3357505868 >> 172.21.15.102:0/3357505868 conn(0x7fa87c075080 msgr2=0x7fa87c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:09.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.310+0000 7fa882800640 1 -- 172.21.15.102:0/3357505868 shutdown_connections 2024-04-20T21:26:09.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.310+0000 7fa882800640 1 -- 172.21.15.102:0/3357505868 wait complete. 2024-04-20T21:26:09.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.313+0000 7fa882800640 1 Processor -- start 2024-04-20T21:26:09.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.313+0000 7fa882800640 1 -- start start 2024-04-20T21:26:09.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.314+0000 7fa882800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c0cb6a0 0x7fa87c0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.314+0000 7fa882800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa87c0cc470 0x7fa87c1f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa882800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa87c155530 0x7fa87c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa882800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c0cd280 con 0x7fa87c155530 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa882800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c0ccfa0 con 0x7fa87c0cc470 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa882800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c0ccbe0 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c0cb6a0 0x7fa87c0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c0cb6a0 0x7fa87c0ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:47488/0 (socket says 172.21.15.102:47488) 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881400640 1 -- 172.21.15.102:0/1825782353 learned_addr learned my addr 172.21.15.102:0/1825782353 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa880a00640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa87c0cc470 0x7fa87c1f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881e00640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa87c155530 0x7fa87c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881400640 1 -- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa87c0cc470 msgr2=0x7fa87c1f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881400640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa87c0cc470 0x7fa87c1f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881400640 1 -- 172.21.15.102:0/1825782353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa87c155530 msgr2=0x7fa87c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881400640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa87c155530 0x7fa87c0cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.315+0000 7fa881400640 1 -- 172.21.15.102:0/1825782353 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c0c84c0 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.316+0000 7fa881400640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c0cb6a0 0x7fa87c0ca5a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa870009890 tx=0x7fa8700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:09.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.316+0000 7fa879600640 1 -- 172.21.15.102:0/1825782353 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870034040 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.316+0000 7fa879600640 1 -- 172.21.15.102:0/1825782353 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa87003add0 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.316+0000 7fa879600640 1 -- 172.21.15.102:0/1825782353 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870006e10 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.316+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c15c3d0 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.316+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c15ccf0 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.318+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa87c04fd60 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.319+0000 7fa879600640 1 -- 172.21.15.102:0/1825782353 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa87004b060 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.321+0000 7fa879600640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa85c06c920 0x7fa85c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.321+0000 7fa879600640 1 -- 172.21.15.102:0/1825782353 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 7135+0+0 (secure 0 0 0) 0x7fa8700adff0 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.321+0000 7fa880a00640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa85c06c920 0x7fa85c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.322+0000 7fa880a00640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa85c06c920 0x7fa85c06ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0d09d0 tx=0x7fa868007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:09.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.325+0000 7fa879600640 1 -- 172.21.15.102:0/1825782353 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa87c0c84c0 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.788+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa87c151760 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.789+0000 7fa879600640 1 -- 172.21.15.102:0/1825782353 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6862 (secure 0 0 0) 0x7fa87003f070 con 0x7fa87c0cb6a0 2024-04-20T21:26:09.788 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5533,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:23:54.838998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5039":{"gid":5039,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":21,"addr":"172.21.15.102:6835/1097388930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1097388930},{"type":"v1","addr":"172.21.15.102:6835","nonce":1097388930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5039,"qdb_cluster":[5039]},"id":1}]} 2024-04-20T21:26:09.789 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-04-20T21:26:09.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa85c06c920 msgr2=0x7fa85c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa85c06c920 0x7fa85c06ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0d09d0 tx=0x7fa868007d50 comp rx=0 tx=0).stop 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c0cb6a0 msgr2=0x7fa87c0ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c0cb6a0 0x7fa87c0ca5a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa870009890 tx=0x7fa8700046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 shutdown_connections 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa87c155530 0x7fa87c0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa85c06c920 0x7fa85c06ede0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa87c0cc470 0x7fa87c1f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 --2- 172.21.15.102:0/1825782353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87c0cb6a0 0x7fa87c0ca5a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 >> 172.21.15.102:0/1825782353 conn(0x7fa87c075080 msgr2=0x7fa87c150b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 shutdown_connections 2024-04-20T21:26:09.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:09.792+0000 7fa882800640 1 -- 172.21.15.102:0/1825782353 wait complete. 2024-04-20T21:26:09.816 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:26:11.817 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:12.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.006+0000 7f4475800640 1 Processor -- start 2024-04-20T21:26:12.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.006+0000 7f4475800640 1 -- start start 2024-04-20T21:26:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.007+0000 7f4475800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.007+0000 7f4475800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f44700cdfc0 0x7f44700cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.007+0000 7f4475800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447015ef60 con 0x7f4470155530 2024-04-20T21:26:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.007+0000 7f4475800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f44700c5ca0 con 0x7f44700cdfc0 2024-04-20T21:26:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.007+0000 7f4475800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f44700cd5f0 con 0x7f44700cc470 2024-04-20T21:26:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f4474e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f44700cdfc0 0x7f44700cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f446f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f446f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:47502/0 (socket says 172.21.15.102:47502) 2024-04-20T21:26:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f4474e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f44700cdfc0 0x7f44700cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59318/0 (socket says 172.21.15.102:59318) 2024-04-20T21:26:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f446f400640 1 -- 172.21.15.102:0/273066544 learned_addr learned my addr 172.21.15.102:0/273066544 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f446f400640 1 -- 172.21.15.102:0/273066544 >> v2:172.21.15.102:3300/0 conn(0x7f44700cdfc0 msgr2=0x7f44700cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f446f400640 1 --2- 172.21.15.102:0/273066544 >> v2:172.21.15.102:3300/0 conn(0x7f44700cdfc0 0x7f44700cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f446f400640 1 -- 172.21.15.102:0/273066544 >> v1:172.21.15.193:6789/0 conn(0x7f4470155530 legacy=0x7f447015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.008+0000 7f446f400640 1 -- 172.21.15.102:0/273066544 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447015f240 con 0x7f44700cc470 2024-04-20T21:26:12.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.010+0000 7f446f400640 1 --2- 172.21.15.102:0/273066544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700cda80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f445c009870 tx=0x7f445c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18940e9ade45e4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:12.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.010+0000 7f446ea00640 1 -- 172.21.15.102:0/273066544 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c034070 con 0x7f44700cc470 2024-04-20T21:26:12.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.010+0000 7f446ea00640 1 -- 172.21.15.102:0/273066544 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445c030840 con 0x7f44700cc470 2024-04-20T21:26:12.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.010+0000 7f446ea00640 1 -- 172.21.15.102:0/273066544 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c030c10 con 0x7f44700cc470 2024-04-20T21:26:12.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.011+0000 7f4475800640 1 -- 172.21.15.102:0/273066544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 msgr2=0x7f44700cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.011+0000 7f4475800640 1 --2- 172.21.15.102:0/273066544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700cda80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f445c009870 tx=0x7f445c02b980 comp rx=0 tx=0).stop 2024-04-20T21:26:12.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.011+0000 7f4475800640 1 -- 172.21.15.102:0/273066544 shutdown_connections 2024-04-20T21:26:12.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.011+0000 7f4475800640 1 --2- 172.21.15.102:0/273066544 >> v2:172.21.15.102:3300/0 conn(0x7f44700cdfc0 0x7f44700cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.011+0000 7f4475800640 1 --2- 172.21.15.102:0/273066544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700cda80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.011+0000 7f4475800640 1 -- 172.21.15.102:0/273066544 >> 172.21.15.102:0/273066544 conn(0x7f4470075080 msgr2=0x7f4470075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:12.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.011+0000 7f4475800640 1 -- 172.21.15.102:0/273066544 shutdown_connections 2024-04-20T21:26:12.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.011+0000 7f4475800640 1 -- 172.21.15.102:0/273066544 wait complete. 2024-04-20T21:26:12.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.014+0000 7f4475800640 1 Processor -- start 2024-04-20T21:26:12.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.014+0000 7f4475800640 1 -- start start 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.015+0000 7f4475800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.015+0000 7f4475800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44700cdfc0 0x7f44701564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.015+0000 7f4475800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4470155530 0x7f4470156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.015+0000 7f4475800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f44700cd280 con 0x7f4470155530 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.015+0000 7f4475800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44700ccfa0 con 0x7f44700cdfc0 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.015+0000 7f4475800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f44700ccbe0 con 0x7f44700cc470 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f4474e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4470155530 0x7f4470156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:47518/0 (socket says 172.21.15.102:47518) 2024-04-20T21:26:12.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446fe00640 1 -- 172.21.15.102:0/342215087 learned_addr learned my addr 172.21.15.102:0/342215087 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:12.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f4474e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4470155530 0x7f4470156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59328/0 (socket says 172.21.15.102:59328) 2024-04-20T21:26:12.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446f400640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44700cdfc0 0x7f44701564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446fe00640 1 -- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44700cdfc0 msgr2=0x7f44701564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446fe00640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44700cdfc0 0x7f44701564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446fe00640 1 -- 172.21.15.102:0/342215087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4470155530 msgr2=0x7f4470156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446fe00640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4470155530 0x7f4470156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446fe00640 1 -- 172.21.15.102:0/342215087 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4470155a40 con 0x7f44700cc470 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.016+0000 7f446f400640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44700cdfc0 0x7f44701564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.017+0000 7f446fe00640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700c9b60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f446000c3f0 tx=0x7f446000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.017+0000 7f4467e00640 1 -- 172.21.15.102:0/342215087 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44600052d0 con 0x7f44700cc470 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.017+0000 7f4467e00640 1 -- 172.21.15.102:0/342215087 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4460005490 con 0x7f44700cc470 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.017+0000 7f4467e00640 1 -- 172.21.15.102:0/342215087 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460007260 con 0x7f44700cc470 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.017+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44700c8520 con 0x7f44700cc470 2024-04-20T21:26:12.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.017+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44700c8ae0 con 0x7f44700cc470 2024-04-20T21:26:12.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.018+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447004fd60 con 0x7f44700cc470 2024-04-20T21:26:12.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.020+0000 7f4467e00640 1 -- 172.21.15.102:0/342215087 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4460007400 con 0x7f44700cc470 2024-04-20T21:26:12.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.021+0000 7f4467e00640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.022+0000 7f4467e00640 1 -- 172.21.15.102:0/342215087 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f446008b630 con 0x7f44700cc470 2024-04-20T21:26:12.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.022+0000 7f446f400640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.023+0000 7f446f400640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f445c036000 tx=0x7f445c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:12.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.024+0000 7f4467e00640 1 -- 172.21.15.102:0/342215087 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4460058e40 con 0x7f44700cc470 2024-04-20T21:26:12.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.480+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44700cc9c0 con 0x7f44700cc470 2024-04-20T21:26:12.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.481+0000 7f4467e00640 1 -- 172.21.15.102:0/342215087 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6026 (secure 0 0 0) 0x7f446005d140 con 0x7f44700cc470 2024-04-20T21:26:12.480 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:12.480 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:10.588749+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:26:12.481 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 33 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 msgr2=0x7f445006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f445c036000 tx=0x7f445c008030 comp rx=0 tx=0).stop 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 msgr2=0x7f44700c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700c9b60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f446000c3f0 tx=0x7f446000c8b0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 shutdown_connections 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4470155530 0x7f4470156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44700cdfc0 0x7f44701564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 --2- 172.21.15.102:0/342215087 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44700cc470 0x7f44700c9b60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 >> 172.21.15.102:0/342215087 conn(0x7f4470075080 msgr2=0x7f44700d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:12.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 shutdown_connections 2024-04-20T21:26:12.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.484+0000 7f4475800640 1 -- 172.21.15.102:0/342215087 wait complete. 2024-04-20T21:26:12.508 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:26:12.508 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:26:12.508 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.694+0000 7f27de400640 1 Processor -- start 2024-04-20T21:26:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.694+0000 7f27de400640 1 -- start start 2024-04-20T21:26:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.695+0000 7f27de400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f27d815c6a0 0x7f27d815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.696+0000 7f27de400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f27d8073950 0x7f27d80ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.696+0000 7f27de400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d815cac0 con 0x7f27d8155610 2024-04-20T21:26:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.696+0000 7f27de400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f27d815b8d0 con 0x7f27d8073950 2024-04-20T21:26:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.696+0000 7f27de400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f27d815b2c0 con 0x7f27d815c6a0 2024-04-20T21:26:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.696+0000 7f27d7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f27d8073950 0x7f27d80ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.696+0000 7f27d7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f27d8073950 0x7f27d80ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59332/0 (socket says 172.21.15.102:59332) 2024-04-20T21:26:12.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.696+0000 7f27d7e00640 1 -- 172.21.15.102:0/3856906578 learned_addr learned my addr 172.21.15.102:0/3856906578 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:12.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.696+0000 7f27d6a00640 1 --2- 172.21.15.102:0/3856906578 >> v2:172.21.15.193:3300/0 conn(0x7f27d815c6a0 0x7f27d815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.697+0000 7f27d7e00640 1 -- 172.21.15.102:0/3856906578 >> v2:172.21.15.193:3300/0 conn(0x7f27d815c6a0 msgr2=0x7f27d815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:12.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.697+0000 7f27d7e00640 1 --2- 172.21.15.102:0/3856906578 >> v2:172.21.15.193:3300/0 conn(0x7f27d815c6a0 0x7f27d815bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.697+0000 7f27d7e00640 1 -- 172.21.15.102:0/3856906578 >> v1:172.21.15.102:6789/0 conn(0x7f27d8155610 legacy=0x7f27d815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.697+0000 7f27d7e00640 1 -- 172.21.15.102:0/3856906578 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d80c9a70 con 0x7f27d8073950 2024-04-20T21:26:12.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.698+0000 7f27d7e00640 1 --2- 172.21.15.102:0/3856906578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 0x7f27d80ceb10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f27d806fc20 tx=0x7f27d002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db17633a4cb139 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:12.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.698+0000 7f27d6000640 1 -- 172.21.15.102:0/3856906578 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d002cba0 con 0x7f27d8073950 2024-04-20T21:26:12.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.698+0000 7f27d6000640 1 -- 172.21.15.102:0/3856906578 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d002cd40 con 0x7f27d8073950 2024-04-20T21:26:12.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.698+0000 7f27d6000640 1 -- 172.21.15.102:0/3856906578 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0034610 con 0x7f27d8073950 2024-04-20T21:26:12.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.699+0000 7f27de400640 1 -- 172.21.15.102:0/3856906578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 msgr2=0x7f27d80ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.699+0000 7f27de400640 1 --2- 172.21.15.102:0/3856906578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 0x7f27d80ceb10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f27d806fc20 tx=0x7f27d002c000 comp rx=0 tx=0).stop 2024-04-20T21:26:12.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.699+0000 7f27de400640 1 -- 172.21.15.102:0/3856906578 shutdown_connections 2024-04-20T21:26:12.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.699+0000 7f27de400640 1 --2- 172.21.15.102:0/3856906578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 0x7f27d80ceb10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.699+0000 7f27de400640 1 --2- 172.21.15.102:0/3856906578 >> v2:172.21.15.193:3300/0 conn(0x7f27d815c6a0 0x7f27d815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.699+0000 7f27de400640 1 -- 172.21.15.102:0/3856906578 >> 172.21.15.102:0/3856906578 conn(0x7f27d806ebf0 msgr2=0x7f27d806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.699+0000 7f27de400640 1 -- 172.21.15.102:0/3856906578 shutdown_connections 2024-04-20T21:26:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.699+0000 7f27de400640 1 -- 172.21.15.102:0/3856906578 wait complete. 2024-04-20T21:26:12.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.701+0000 7f27de400640 1 Processor -- start 2024-04-20T21:26:12.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.701+0000 7f27de400640 1 -- start start 2024-04-20T21:26:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27de400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 0x7f27d814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27de400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27d8155610 0x7f27d814fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27de400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27d815c6a0 0x7f27d80c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27de400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f27d80ca030 con 0x7f27d8073950 2024-04-20T21:26:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27de400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27d815b560 con 0x7f27d815c6a0 2024-04-20T21:26:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27de400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f27d8073de0 con 0x7f27d8155610 2024-04-20T21:26:12.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 0x7f27d814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27d7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27d815c6a0 0x7f27d80c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 0x7f27d814f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59336/0 (socket says 172.21.15.102:59336) 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27d7400640 1 -- 172.21.15.102:0/2746668287 learned_addr learned my addr 172.21.15.102:0/2746668287 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.702+0000 7f27d6a00640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27d8155610 0x7f27d814fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.703+0000 7f27d7e00640 1 -- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27d8155610 msgr2=0x7f27d814fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.703+0000 7f27d7e00640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27d8155610 0x7f27d814fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.703+0000 7f27d7e00640 1 -- 172.21.15.102:0/2746668287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 msgr2=0x7f27d814f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.703+0000 7f27d7e00640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 0x7f27d814f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.703+0000 7f27d7e00640 1 -- 172.21.15.102:0/2746668287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d81f10c0 con 0x7f27d815c6a0 2024-04-20T21:26:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.703+0000 7f27d7e00640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27d815c6a0 0x7f27d80c7680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f27d0032000 tx=0x7f27d0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:12.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.704+0000 7f27cf400640 1 -- 172.21.15.102:0/2746668287 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0043020 con 0x7f27d815c6a0 2024-04-20T21:26:12.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.704+0000 7f27cf400640 1 -- 172.21.15.102:0/2746668287 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d003e450 con 0x7f27d815c6a0 2024-04-20T21:26:12.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.704+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d81f1380 con 0x7f27d815c6a0 2024-04-20T21:26:12.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.704+0000 7f27cf400640 1 -- 172.21.15.102:0/2746668287 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0009510 con 0x7f27d815c6a0 2024-04-20T21:26:12.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.704+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d81f1940 con 0x7f27d815c6a0 2024-04-20T21:26:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.706+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d81f2000 con 0x7f27d815c6a0 2024-04-20T21:26:12.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.708+0000 7f27cf400640 1 -- 172.21.15.102:0/2746668287 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27d0046020 con 0x7f27d815c6a0 2024-04-20T21:26:12.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.709+0000 7f27cf400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f27b806c8f0 0x7f27b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:12.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.709+0000 7f27cf400640 1 -- 172.21.15.102:0/2746668287 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f27d00ae8f0 con 0x7f27d815c6a0 2024-04-20T21:26:12.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.709+0000 7f27d7400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f27b806c8f0 0x7f27b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:12.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.710+0000 7f27d7400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f27b806c8f0 0x7f27b806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f27c8001840 tx=0x7f27c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:12.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:12.712+0000 7f27cf400640 1 -- 172.21.15.102:0/2746668287 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f27d81f2000 con 0x7f27d815c6a0 2024-04-20T21:26:13.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.128+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27d80c7480 con 0x7f27d815c6a0 2024-04-20T21:26:13.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.129+0000 7f27cf400640 1 -- 172.21.15.102:0/2746668287 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6026 (secure 0 0 0) 0x7f27d004d030 con 0x7f27d815c6a0 2024-04-20T21:26:13.128 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:10.588749+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:26:13.129 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 33 2024-04-20T21:26:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.131+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f27b806c8f0 msgr2=0x7f27b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.131+0000 7f27de400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f27b806c8f0 0x7f27b806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f27c8001840 tx=0x7f27c8009040 comp rx=0 tx=0).stop 2024-04-20T21:26:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27d815c6a0 msgr2=0x7f27d80c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27d815c6a0 0x7f27d80c7680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f27d0032000 tx=0x7f27d0004a40 comp rx=0 tx=0).stop 2024-04-20T21:26:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 shutdown_connections 2024-04-20T21:26:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f27b806c8f0 0x7f27b806edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27d815c6a0 0x7f27d80c7680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f27d8155610 0x7f27d814fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 --2- 172.21.15.102:0/2746668287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27d8073950 0x7f27d814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 >> 172.21.15.102:0/2746668287 conn(0x7f27d806ebf0 msgr2=0x7f27d8153bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 shutdown_connections 2024-04-20T21:26:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:13.132+0000 7f27de400640 1 -- 172.21.15.102:0/2746668287 wait complete. 2024-04-20T21:26:13.148 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 33, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 33, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5533": { "addr": "172.21.15.102:6833/2239658043", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 2239658043, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 2239658043, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5533, "incarnation": 33, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:26:10.588749+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5533 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:26:15.148 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:15.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.409+0000 7f3ba2e00640 1 Processor -- start 2024-04-20T21:26:15.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.410+0000 7f3ba2e00640 1 -- start start 2024-04-20T21:26:15.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba2e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3b9c0cc470 0x7f3b9c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:15.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba2e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cdfc0 0x7f3b9c0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:15.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba2e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9c15ef60 con 0x7f3b9c155530 2024-04-20T21:26:15.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba2e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9c0c5ca0 con 0x7f3b9c0cc470 2024-04-20T21:26:15.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba2e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c0cd5f0 con 0x7f3b9c0cdfc0 2024-04-20T21:26:15.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba1a00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f3b9c155530 0x7f3b9c15e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:52988/0 (socket says 172.21.15.102:52988) 2024-04-20T21:26:15.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba1a00640 1 -- 172.21.15.102:0/2300542364 learned_addr learned my addr 172.21.15.102:0/2300542364 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:15.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba2400640 1 --2- 172.21.15.102:0/2300542364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cdfc0 0x7f3b9c0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.412+0000 7f3ba1000640 1 --2- 172.21.15.102:0/2300542364 >> v2:172.21.15.102:3300/0 conn(0x7f3b9c0cc470 0x7f3b9c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.413+0000 7f3ba2400640 1 -- 172.21.15.102:0/2300542364 >> v2:172.21.15.102:3300/0 conn(0x7f3b9c0cc470 msgr2=0x7f3b9c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.413+0000 7f3ba2400640 1 --2- 172.21.15.102:0/2300542364 >> v2:172.21.15.102:3300/0 conn(0x7f3b9c0cc470 0x7f3b9c0cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.413+0000 7f3ba2400640 1 -- 172.21.15.102:0/2300542364 >> v1:172.21.15.193:6789/0 conn(0x7f3b9c155530 legacy=0x7f3b9c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.413+0000 7f3ba2400640 1 -- 172.21.15.102:0/2300542364 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c15f240 con 0x7f3b9c0cdfc0 2024-04-20T21:26:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.413+0000 7f3ba1000640 1 --2- 172.21.15.102:0/2300542364 >> v2:172.21.15.102:3300/0 conn(0x7f3b9c0cc470 0x7f3b9c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:26:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.414+0000 7f3ba2400640 1 --2- 172.21.15.102:0/2300542364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cdfc0 0x7f3b9c0cb6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009d90 tx=0x7f3b9402bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca563e509edc04a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.415+0000 7f3b9be00640 1 -- 172.21.15.102:0/2300542364 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94039070 con 0x7f3b9c0cdfc0 2024-04-20T21:26:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.415+0000 7f3b9be00640 1 -- 172.21.15.102:0/2300542364 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9402ca10 con 0x7f3b9c0cdfc0 2024-04-20T21:26:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.415+0000 7f3b9be00640 1 -- 172.21.15.102:0/2300542364 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9402cde0 con 0x7f3b9c0cdfc0 2024-04-20T21:26:15.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.416+0000 7f3ba2e00640 1 -- 172.21.15.102:0/2300542364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cdfc0 msgr2=0x7f3b9c0cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:15.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.416+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/2300542364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cdfc0 0x7f3b9c0cb6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009d90 tx=0x7f3b9402bce0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.416+0000 7f3ba2e00640 1 -- 172.21.15.102:0/2300542364 shutdown_connections 2024-04-20T21:26:15.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.416+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/2300542364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cdfc0 0x7f3b9c0cb6a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.416+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/2300542364 >> v2:172.21.15.102:3300/0 conn(0x7f3b9c0cc470 0x7f3b9c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.416+0000 7f3ba2e00640 1 -- 172.21.15.102:0/2300542364 >> 172.21.15.102:0/2300542364 conn(0x7f3b9c075080 msgr2=0x7f3b9c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:15.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.416+0000 7f3ba2e00640 1 -- 172.21.15.102:0/2300542364 shutdown_connections 2024-04-20T21:26:15.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.416+0000 7f3ba2e00640 1 -- 172.21.15.102:0/2300542364 wait complete. 2024-04-20T21:26:15.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.419+0000 7f3ba2e00640 1 Processor -- start 2024-04-20T21:26:15.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.419+0000 7f3ba2e00640 1 -- start start 2024-04-20T21:26:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.420+0000 7f3ba2e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cc470 0x7f3b9c1e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.420+0000 7f3ba2e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b9c0cdfc0 0x7f3b9c1e4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.420+0000 7f3ba2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9c155530 0x7f3b9c0cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.420+0000 7f3ba2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0cd280 con 0x7f3b9c155530 2024-04-20T21:26:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.420+0000 7f3ba2e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0ccfa0 con 0x7f3b9c0cdfc0 2024-04-20T21:26:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.420+0000 7f3ba2e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c0ccbe0 con 0x7f3b9c0cc470 2024-04-20T21:26:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.421+0000 7f3ba1a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cc470 0x7f3b9c1e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.421+0000 7f3ba1a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cc470 0x7f3b9c1e6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:47542/0 (socket says 172.21.15.102:47542) 2024-04-20T21:26:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.421+0000 7f3ba1a00640 1 -- 172.21.15.102:0/3473533734 learned_addr learned my addr 172.21.15.102:0/3473533734 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.421+0000 7f3ba1000640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b9c0cdfc0 0x7f3b9c1e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.421+0000 7f3ba2400640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9c155530 0x7f3b9c0cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.421+0000 7f3ba1a00640 1 -- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b9c0cdfc0 msgr2=0x7f3b9c1e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.421+0000 7f3ba1a00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b9c0cdfc0 0x7f3b9c1e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.421+0000 7f3ba1a00640 1 -- 172.21.15.102:0/3473533734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9c155530 msgr2=0x7f3b9c0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.422+0000 7f3ba1a00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9c155530 0x7f3b9c0cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.422+0000 7f3ba1a00640 1 -- 172.21.15.102:0/3473533734 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c15e740 con 0x7f3b9c0cc470 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.422+0000 7f3ba2400640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9c155530 0x7f3b9c0cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.422+0000 7f3ba1a00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cc470 0x7f3b9c1e6d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000c780 tx=0x7f3b9000cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.422+0000 7f3b99600640 1 -- 172.21.15.102:0/3473533734 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90018070 con 0x7f3b9c0cc470 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.422+0000 7f3b99600640 1 -- 172.21.15.102:0/3473533734 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b900072d0 con 0x7f3b9c0cc470 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.422+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c15ea00 con 0x7f3b9c0cc470 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.423+0000 7f3b99600640 1 -- 172.21.15.102:0/3473533734 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b900047e0 con 0x7f3b9c0cc470 2024-04-20T21:26:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.423+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c15ce50 con 0x7f3b9c0cc470 2024-04-20T21:26:15.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.424+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9c04fd60 con 0x7f3b9c0cc470 2024-04-20T21:26:15.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.426+0000 7f3b99600640 1 -- 172.21.15.102:0/3473533734 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b90004980 con 0x7f3b9c0cc470 2024-04-20T21:26:15.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.427+0000 7f3b99600640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b7c06c8f0 0x7f3b7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:15.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.427+0000 7f3b99600640 1 -- 172.21.15.102:0/3473533734 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f3b9008b560 con 0x7f3b9c0cc470 2024-04-20T21:26:15.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.428+0000 7f3ba1000640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b7c06c8f0 0x7f3b7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:15.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.428+0000 7f3ba1000640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b7c06c8f0 0x7f3b7c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c152a70 tx=0x7f3b88008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:15.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.432+0000 7f3b99600640 1 -- 172.21.15.102:0/3473533734 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3b90004c70 con 0x7f3b9c0cc470 2024-04-20T21:26:15.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.737+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b9c0cc9c0 con 0x7f3b9c0cc470 2024-04-20T21:26:15.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.738+0000 7f3b99600640 1 -- 172.21.15.102:0/3473533734 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6026 (secure 0 0 0) 0x7f3b90058db0 con 0x7f3b9c0cc470 2024-04-20T21:26:15.737 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:15.737 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:10.588749+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:26:15.738 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 33 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b7c06c8f0 msgr2=0x7f3b7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b7c06c8f0 0x7f3b7c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c152a70 tx=0x7f3b88008040 comp rx=0 tx=0).stop 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cc470 msgr2=0x7f3b9c1e6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cc470 0x7f3b9c1e6d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000c780 tx=0x7f3b9000cc40 comp rx=0 tx=0).stop 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 shutdown_connections 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9c155530 0x7f3b9c0cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b7c06c8f0 0x7f3b7c06edb0 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c152a70 tx=0x7f3b88008040 comp rx=0 tx=0).stop 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b9c0cdfc0 0x7f3b9c1e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 --2- 172.21.15.102:0/3473533734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b9c0cc470 0x7f3b9c1e6d60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:15.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 >> 172.21.15.102:0/3473533734 conn(0x7f3b9c075080 msgr2=0x7f3b9c153e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:15.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.740+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 shutdown_connections 2024-04-20T21:26:15.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:15.742+0000 7f3ba2e00640 1 -- 172.21.15.102:0/3473533734 wait complete. 2024-04-20T21:26:15.759 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:26:17.760 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:18.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.024+0000 7fc185800640 1 Processor -- start 2024-04-20T21:26:18.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.024+0000 7fc185800640 1 -- start start 2024-04-20T21:26:18.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.026+0000 7fc185800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1800ca9b0 0x7fc18015b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.026+0000 7fc185800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc18015bc00 con 0x7fc1800cbac0 2024-04-20T21:26:18.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.026+0000 7fc185800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc18015bee0 con 0x7fc1800cbf30 2024-04-20T21:26:18.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.026+0000 7fc185800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc18015c1c0 con 0x7fc1800ca9b0 2024-04-20T21:26:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.027+0000 7fc17fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1800ca9b0 0x7fc18015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.027+0000 7fc17fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1800ca9b0 0x7fc18015b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37234/0 (socket says 172.21.15.102:37234) 2024-04-20T21:26:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.027+0000 7fc184e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fc1800cbf30 0x7fc1800b4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:41950/0 (socket says 172.21.15.102:41950) 2024-04-20T21:26:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.027+0000 7fc17fe00640 1 -- 172.21.15.102:0/153751592 learned_addr learned my addr 172.21.15.102:0/153751592 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.028+0000 7fc17fe00640 1 -- 172.21.15.102:0/153751592 >> v1:172.21.15.193:6789/0 conn(0x7fc1800cbf30 legacy=0x7fc1800b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.028+0000 7fc17fe00640 1 -- 172.21.15.102:0/153751592 >> v1:172.21.15.102:6789/0 conn(0x7fc1800cbac0 legacy=0x7fc1800b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.028+0000 7fc17ea00640 1 -- 172.21.15.102:0/153751592 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329190568 0 0) 0x7fc18015bee0 con 0x7fc1800cbf30 2024-04-20T21:26:18.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.028+0000 7fc17fe00640 1 -- 172.21.15.102:0/153751592 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800b43d0 con 0x7fc1800ca9b0 2024-04-20T21:26:18.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.030+0000 7fc17fe00640 1 --2- 172.21.15.102:0/153751592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800ca9b0 0x7fc18015b370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc170009870 tx=0x7fc17002bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2c395d04258a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:18.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.030+0000 7fc17ea00640 1 -- 172.21.15.102:0/153751592 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17002d870 con 0x7fc1800ca9b0 2024-04-20T21:26:18.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.030+0000 7fc17ea00640 1 -- 172.21.15.102:0/153751592 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc17002da10 con 0x7fc1800ca9b0 2024-04-20T21:26:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.030+0000 7fc17ea00640 1 -- 172.21.15.102:0/153751592 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17002dde0 con 0x7fc1800ca9b0 2024-04-20T21:26:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.031+0000 7fc185800640 1 -- 172.21.15.102:0/153751592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800ca9b0 msgr2=0x7fc18015b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.031+0000 7fc185800640 1 --2- 172.21.15.102:0/153751592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800ca9b0 0x7fc18015b370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc170009870 tx=0x7fc17002bab0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.031+0000 7fc185800640 1 -- 172.21.15.102:0/153751592 shutdown_connections 2024-04-20T21:26:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.031+0000 7fc185800640 1 --2- 172.21.15.102:0/153751592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800ca9b0 0x7fc18015b370 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.031+0000 7fc185800640 1 -- 172.21.15.102:0/153751592 >> 172.21.15.102:0/153751592 conn(0x7fc180075080 msgr2=0x7fc180075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.032+0000 7fc185800640 1 -- 172.21.15.102:0/153751592 shutdown_connections 2024-04-20T21:26:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.032+0000 7fc185800640 1 -- 172.21.15.102:0/153751592 wait complete. 2024-04-20T21:26:18.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.034+0000 7fc185800640 1 Processor -- start 2024-04-20T21:26:18.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.034+0000 7fc185800640 1 -- start start 2024-04-20T21:26:18.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.036+0000 7fc185800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800cbac0 0x7fc1800c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.036+0000 7fc185800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1800cbf30 0x7fc1801f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.036+0000 7fc185800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1800c2650 0x7fc180153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.036+0000 7fc185800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800cad90 con 0x7fc1800cbac0 2024-04-20T21:26:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.036+0000 7fc185800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800b4990 con 0x7fc1800c2650 2024-04-20T21:26:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.036+0000 7fc185800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc18015b8b0 con 0x7fc1800cbf30 2024-04-20T21:26:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.037+0000 7fc17fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800cbac0 0x7fc1800c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.037+0000 7fc17f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1800cbf30 0x7fc1801f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.037+0000 7fc17f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1800cbf30 0x7fc1801f8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:55336/0 (socket says 172.21.15.102:55336) 2024-04-20T21:26:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.037+0000 7fc17f400640 1 -- 172.21.15.102:0/1454978936 learned_addr learned my addr 172.21.15.102:0/1454978936 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.037+0000 7fc184e00640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1800c2650 0x7fc180153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.038+0000 7fc17fe00640 1 -- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1800cbf30 msgr2=0x7fc1801f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.038+0000 7fc17fe00640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1800cbf30 0x7fc1801f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.038+0000 7fc17fe00640 1 -- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1800c2650 msgr2=0x7fc180153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.038+0000 7fc17fe00640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1800c2650 0x7fc180153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.038+0000 7fc17fe00640 1 -- 172.21.15.102:0/1454978936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800c1810 con 0x7fc1800cbac0 2024-04-20T21:26:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.039+0000 7fc17fe00640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800cbac0 0x7fc1800c0f60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc18008e130 tx=0x7fc1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.039+0000 7fc177e00640 1 -- 172.21.15.102:0/1454978936 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170046070 con 0x7fc1800cbac0 2024-04-20T21:26:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.039+0000 7fc177e00640 1 -- 172.21.15.102:0/1454978936 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc170004230 con 0x7fc1800cbac0 2024-04-20T21:26:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.039+0000 7fc177e00640 1 -- 172.21.15.102:0/1454978936 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170037830 con 0x7fc1800cbac0 2024-04-20T21:26:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.039+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18006e5f0 con 0x7fc1800cbac0 2024-04-20T21:26:18.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.039+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18006eb90 con 0x7fc1800cbac0 2024-04-20T21:26:18.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.042+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18004fd60 con 0x7fc1800cbac0 2024-04-20T21:26:18.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.043+0000 7fc177e00640 1 -- 172.21.15.102:0/1454978936 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1700379d0 con 0x7fc1800cbac0 2024-04-20T21:26:18.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.048+0000 7fc177e00640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc16006c920 0x7fc16006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.048+0000 7fc177e00640 1 -- 172.21.15.102:0/1454978936 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fc1700ad8e0 con 0x7fc1800cbac0 2024-04-20T21:26:18.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.049+0000 7fc17f400640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc16006c920 0x7fc16006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.050+0000 7fc17f400640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc16006c920 0x7fc16006ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1800c7770 tx=0x7fc16c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:18.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.052+0000 7fc177e00640 1 -- 172.21.15.102:0/1454978936 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc17003e070 con 0x7fc1800cbac0 2024-04-20T21:26:18.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.506+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1800cc3b0 con 0x7fc1800cbac0 2024-04-20T21:26:18.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.507+0000 7fc177e00640 1 -- 172.21.15.102:0/1454978936 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6026 (secure 0 0 0) 0x7fc17007afe0 con 0x7fc1800cbac0 2024-04-20T21:26:18.508 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:18.508 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:10.588749+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:26:18.508 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 33 2024-04-20T21:26:18.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.510+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc16006c920 msgr2=0x7fc16006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.510+0000 7fc185800640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc16006c920 0x7fc16006ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1800c7770 tx=0x7fc16c009040 comp rx=0 tx=0).stop 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800cbac0 msgr2=0x7fc1800c0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800cbac0 0x7fc1800c0f60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc18008e130 tx=0x7fc1700046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 shutdown_connections 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc16006c920 0x7fc16006ede0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1800c2650 0x7fc180153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1800cbf30 0x7fc1801f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 --2- 172.21.15.102:0/1454978936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1800cbac0 0x7fc1800c0f60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 >> 172.21.15.102:0/1454978936 conn(0x7fc180075080 msgr2=0x7fc1800c3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 shutdown_connections 2024-04-20T21:26:18.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:18.511+0000 7fc185800640 1 -- 172.21.15.102:0/1454978936 wait complete. 2024-04-20T21:26:18.539 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:26:20.540 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.714+0000 7f5c5ec00640 1 Processor -- start 2024-04-20T21:26:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.714+0000 7f5c5ec00640 1 -- start start 2024-04-20T21:26:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c5ec00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5c5814fa50 0x7f5c580c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c5ec00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c58073a10 0x7f5c58158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c5ec00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c581536d0 con 0x7f5c580c9eb0 2024-04-20T21:26:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c5ec00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5c58159500 con 0x7f5c58073a10 2024-04-20T21:26:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c5ec00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f5c580c76c0 con 0x7f5c5814fa50 2024-04-20T21:26:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c57e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5c5814fa50 0x7f5c580c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c5cc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c58073a10 0x7f5c58158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c57e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5c5814fa50 0x7f5c580c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58442/0 (socket says 172.21.15.102:58442) 2024-04-20T21:26:20.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c4f400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f5c580c9eb0 0x7f5c580c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:41952/0 (socket says 172.21.15.102:41952) 2024-04-20T21:26:20.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.715+0000 7f5c57e00640 1 -- 172.21.15.102:0/3843444039 learned_addr learned my addr 172.21.15.102:0/3843444039 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:20.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.716+0000 7f5c57e00640 1 -- 172.21.15.102:0/3843444039 >> v2:172.21.15.102:3300/0 conn(0x7f5c58073a10 msgr2=0x7f5c58158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:20.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.716+0000 7f5c57e00640 1 --2- 172.21.15.102:0/3843444039 >> v2:172.21.15.102:3300/0 conn(0x7f5c58073a10 0x7f5c58158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.716+0000 7f5c57e00640 1 -- 172.21.15.102:0/3843444039 >> v1:172.21.15.193:6789/0 conn(0x7f5c580c9eb0 legacy=0x7f5c580c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:20.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.716+0000 7f5c57e00640 1 -- 172.21.15.102:0/3843444039 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c580c7960 con 0x7f5c5814fa50 2024-04-20T21:26:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.717+0000 7f5c57e00640 1 --2- 172.21.15.102:0/3843444039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c580c4830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5c580706a0 tx=0x7f5c4802ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db62aabfe7e90d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.717+0000 7f5c57400640 1 -- 172.21.15.102:0/3843444039 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4802f860 con 0x7f5c5814fa50 2024-04-20T21:26:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.717+0000 7f5c57400640 1 -- 172.21.15.102:0/3843444039 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4802fa00 con 0x7f5c5814fa50 2024-04-20T21:26:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.717+0000 7f5c57400640 1 -- 172.21.15.102:0/3843444039 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4802fdd0 con 0x7f5c5814fa50 2024-04-20T21:26:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.717+0000 7f5c5ec00640 1 -- 172.21.15.102:0/3843444039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 msgr2=0x7f5c580c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:20.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.717+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/3843444039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c580c4830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5c580706a0 tx=0x7f5c4802ba40 comp rx=0 tx=0).stop 2024-04-20T21:26:20.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.718+0000 7f5c5ec00640 1 -- 172.21.15.102:0/3843444039 shutdown_connections 2024-04-20T21:26:20.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.718+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/3843444039 >> v2:172.21.15.102:3300/0 conn(0x7f5c58073a10 0x7f5c58158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.718+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/3843444039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c580c4830 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.718+0000 7f5c5ec00640 1 -- 172.21.15.102:0/3843444039 >> 172.21.15.102:0/3843444039 conn(0x7f5c580c9390 msgr2=0x7f5c580c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:20.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.718+0000 7f5c5ec00640 1 -- 172.21.15.102:0/3843444039 shutdown_connections 2024-04-20T21:26:20.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.718+0000 7f5c5ec00640 1 -- 172.21.15.102:0/3843444039 wait complete. 2024-04-20T21:26:20.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.719+0000 7f5c5ec00640 1 Processor -- start 2024-04-20T21:26:20.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.719+0000 7f5c5ec00640 1 -- start start 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c58073a10 0x7f5c5814ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5ec00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c580c9eb0 0x7f5c581497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5ec00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c58149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c580c7c00 con 0x7f5c58073a10 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5ec00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c580c3d90 con 0x7f5c5814fa50 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5ec00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5c580c4030 con 0x7f5c580c9eb0 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5cc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c58149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c58073a10 0x7f5c5814ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5cc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c58149cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58450/0 (socket says 172.21.15.102:58450) 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.720+0000 7f5c5cc00640 1 -- 172.21.15.102:0/821052311 learned_addr learned my addr 172.21.15.102:0/821052311 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c4f400640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c580c9eb0 0x7f5c581497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c5cc00640 1 -- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c580c9eb0 msgr2=0x7f5c581497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c5cc00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c580c9eb0 0x7f5c581497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c5cc00640 1 -- 172.21.15.102:0/821052311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c58073a10 msgr2=0x7f5c5814ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c5cc00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c58073a10 0x7f5c5814ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c5cc00640 1 -- 172.21.15.102:0/821052311 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c580721e0 con 0x7f5c5814fa50 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c57e00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c58073a10 0x7f5c5814ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c5cc00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c58149cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000efc0 tx=0x7f5c5000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c54c00640 1 -- 172.21.15.102:0/821052311 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50018020 con 0x7f5c5814fa50 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c54c00640 1 -- 172.21.15.102:0/821052311 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c50014480 con 0x7f5c5814fa50 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c58148a20 con 0x7f5c5814fa50 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.721+0000 7f5c54c00640 1 -- 172.21.15.102:0/821052311 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50014870 con 0x7f5c5814fa50 2024-04-20T21:26:20.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.722+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c58072830 con 0x7f5c5814fa50 2024-04-20T21:26:20.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.723+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c58074340 con 0x7f5c5814fa50 2024-04-20T21:26:20.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.723+0000 7f5c54c00640 1 -- 172.21.15.102:0/821052311 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c50005d10 con 0x7f5c5814fa50 2024-04-20T21:26:20.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.724+0000 7f5c54c00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3406c920 0x7f5c3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:20.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.724+0000 7f5c54c00640 1 -- 172.21.15.102:0/821052311 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f5c5008ce50 con 0x7f5c5814fa50 2024-04-20T21:26:20.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.725+0000 7f5c57e00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3406c920 0x7f5c3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:20.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.725+0000 7f5c57e00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3406c920 0x7f5c3406ede0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c48036000 tx=0x7f5c48008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:20.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.729+0000 7f5c54c00640 1 -- 172.21.15.102:0/821052311 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5c50022030 con 0x7f5c5814fa50 2024-04-20T21:26:20.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.951+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c580c7320 con 0x7f5c5814fa50 2024-04-20T21:26:20.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.952+0000 7f5c54c00640 1 -- 172.21.15.102:0/821052311 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+6030 (secure 0 0 0) 0x7f5c50010040 con 0x7f5c5814fa50 2024-04-20T21:26:20.952 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:20.952 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:19.975920+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":37,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:26:20.952 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 34 2024-04-20T21:26:20.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3406c920 msgr2=0x7f5c3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:20.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3406c920 0x7f5c3406ede0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c48036000 tx=0x7f5c48008030 comp rx=0 tx=0).stop 2024-04-20T21:26:20.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 msgr2=0x7f5c58149cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:20.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c58149cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000efc0 tx=0x7f5c5000c490 comp rx=0 tx=0).stop 2024-04-20T21:26:20.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 shutdown_connections 2024-04-20T21:26:20.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3406c920 0x7f5c3406ede0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5814fa50 0x7f5c58149cf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c580c9eb0 0x7f5c581497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 --2- 172.21.15.102:0/821052311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c58073a10 0x7f5c5814ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:20.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 >> 172.21.15.102:0/821052311 conn(0x7f5c580c9390 msgr2=0x7f5c5806ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:20.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 shutdown_connections 2024-04-20T21:26:20.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:20.955+0000 7f5c5ec00640 1 -- 172.21.15.102:0/821052311 wait complete. 2024-04-20T21:26:20.971 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:26:22.972 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:23.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.178+0000 7fdfc3800640 1 Processor -- start 2024-04-20T21:26:23.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.179+0000 7fdfc3800640 1 -- start start 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.180+0000 7fdfc3800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdfbc15ee00 0x7fdfbc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.180+0000 7fdfc3800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.180+0000 7fdfc3800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfbc0c8f20 con 0x7fdfbc15ea20 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.180+0000 7fdfc3800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fdfbc06cb80 con 0x7fdfbc15ee00 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.180+0000 7fdfc3800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc06ce60 con 0x7fdfbc15f2b0 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfb3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdfbc15ee00 0x7fdfbc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfc1800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfc0e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fdfbc15ea20 0x7fdfbc0c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:41966/0 (socket says 172.21.15.102:41966) 2024-04-20T21:26:23.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfc0e00640 1 -- 172.21.15.102:0/2042665458 learned_addr learned my addr 172.21.15.102:0/2042665458 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:23.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfb3e00640 1 -- 172.21.15.102:0/2042665458 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 msgr2=0x7fdfbc0710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:23.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfb3e00640 1 --2- 172.21.15.102:0/2042665458 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfb3e00640 1 -- 172.21.15.102:0/2042665458 >> v1:172.21.15.193:6789/0 conn(0x7fdfbc15ea20 legacy=0x7fdfbc0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:23.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfb3e00640 1 -- 172.21.15.102:0/2042665458 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc06d140 con 0x7fdfbc15ee00 2024-04-20T21:26:23.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfbbe00640 1 -- 172.21.15.102:0/2042665458 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154713161 0 0) 0x7fdfbc0c8f20 con 0x7fdfbc15ea20 2024-04-20T21:26:23.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.181+0000 7fdfc1800640 1 --2- 172.21.15.102:0/2042665458 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:23.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.182+0000 7fdfb3e00640 1 --2- 172.21.15.102:0/2042665458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ee00 0x7fdfbc06d940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40097a0 tx=0x7fdfa402b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7b8c9ac8adccfa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:23.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.182+0000 7fdfbbe00640 1 -- 172.21.15.102:0/2042665458 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4034070 con 0x7fdfbc15ee00 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.182+0000 7fdfbbe00640 1 -- 172.21.15.102:0/2042665458 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa4030800 con 0x7fdfbc15ee00 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.182+0000 7fdfbbe00640 1 -- 172.21.15.102:0/2042665458 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4030bd0 con 0x7fdfbc15ee00 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.183+0000 7fdfc3800640 1 -- 172.21.15.102:0/2042665458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ee00 msgr2=0x7fdfbc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.183+0000 7fdfc3800640 1 --2- 172.21.15.102:0/2042665458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ee00 0x7fdfbc06d940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40097a0 tx=0x7fdfa402b940 comp rx=0 tx=0).stop 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.183+0000 7fdfc3800640 1 -- 172.21.15.102:0/2042665458 shutdown_connections 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.183+0000 7fdfc3800640 1 --2- 172.21.15.102:0/2042665458 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.183+0000 7fdfc3800640 1 --2- 172.21.15.102:0/2042665458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ee00 0x7fdfbc06d940 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.183+0000 7fdfc3800640 1 -- 172.21.15.102:0/2042665458 >> 172.21.15.102:0/2042665458 conn(0x7fdfbc0c9390 msgr2=0x7fdfbc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.183+0000 7fdfc3800640 1 -- 172.21.15.102:0/2042665458 shutdown_connections 2024-04-20T21:26:23.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.183+0000 7fdfc3800640 1 -- 172.21.15.102:0/2042665458 wait complete. 2024-04-20T21:26:23.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.185+0000 7fdfc3800640 1 Processor -- start 2024-04-20T21:26:23.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.185+0000 7fdfc3800640 1 -- start start 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc3800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ea20 0x7fdfbc15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc3800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfbc15ee00 0x7fdfbc15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc3800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc3800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc0c8c80 con 0x7fdfbc15ee00 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc3800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc0c8b00 con 0x7fdfbc15ea20 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc3800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc154e80 con 0x7fdfbc15f2b0 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ea20 0x7fdfbc15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfb3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfbc15ee00 0x7fdfbc15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ea20 0x7fdfbc15d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58462/0 (socket says 172.21.15.102:58462) 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc0e00640 1 -- 172.21.15.102:0/37676464 learned_addr learned my addr 172.21.15.102:0/37676464 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfb3e00640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfbc15ee00 0x7fdfbc15ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37270/0 (socket says 172.21.15.102:37270) 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.186+0000 7fdfc1800640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfc0e00640 1 -- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 msgr2=0x7fdfbc0c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:23.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfc0e00640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfc0e00640 1 -- 172.21.15.102:0/37676464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfbc15ee00 msgr2=0x7fdfbc15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfc0e00640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfbc15ee00 0x7fdfbc15ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfc0e00640 1 -- 172.21.15.102:0/37676464 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc15d930 con 0x7fdfbc15ea20 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfc0e00640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ea20 0x7fdfbc15d3f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c4d0 tx=0x7fdfac00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfb9600640 1 -- 172.21.15.102:0/37676464 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac018070 con 0x7fdfbc15ea20 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfb9600640 1 -- 172.21.15.102:0/37676464 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfac007500 con 0x7fdfbc15ea20 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfb9600640 1 -- 172.21.15.102:0/37676464 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac004a20 con 0x7fdfbc15ea20 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.187+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc1f4100 con 0x7fdfbc15ea20 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.188+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1f46c0 con 0x7fdfbc15ea20 2024-04-20T21:26:23.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.189+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc074340 con 0x7fdfbc15ea20 2024-04-20T21:26:23.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.190+0000 7fdfb9600640 1 -- 172.21.15.102:0/37676464 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfac004bc0 con 0x7fdfbc15ea20 2024-04-20T21:26:23.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.191+0000 7fdfb9600640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf9806c8f0 0x7fdf9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:23.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.191+0000 7fdfb9600640 1 -- 172.21.15.102:0/37676464 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fdfac08b390 con 0x7fdfbc15ea20 2024-04-20T21:26:23.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.191+0000 7fdfb3e00640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf9806c8f0 0x7fdf9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:23.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.192+0000 7fdfb3e00640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf9806c8f0 0x7fdf9806edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4036000 tx=0x7fdfa403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:23.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.196+0000 7fdfb9600640 1 -- 172.21.15.102:0/37676464 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdfac058b20 con 0x7fdfbc15ea20 2024-04-20T21:26:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.435+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfbc072930 con 0x7fdfbc15ea20 2024-04-20T21:26:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.436+0000 7fdfb9600640 1 -- 172.21.15.102:0/37676464 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+6030 (secure 0 0 0) 0x7fdfac05ce20 con 0x7fdfbc15ea20 2024-04-20T21:26:23.435 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:23.436 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:19.975920+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":37,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:26:23.436 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 34 2024-04-20T21:26:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.438+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf9806c8f0 msgr2=0x7fdf9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.438+0000 7fdfc3800640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf9806c8f0 0x7fdf9806edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4036000 tx=0x7fdfa403b040 comp rx=0 tx=0).stop 2024-04-20T21:26:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.438+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ea20 msgr2=0x7fdfbc15d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.438+0000 7fdfc3800640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ea20 0x7fdfbc15d3f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c4d0 tx=0x7fdfac00c990 comp rx=0 tx=0).stop 2024-04-20T21:26:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.439+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 shutdown_connections 2024-04-20T21:26:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.439+0000 7fdfc3800640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdfbc15f2b0 0x7fdfbc0c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.439+0000 7fdfc3800640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfbc15ee00 0x7fdfbc15ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.439+0000 7fdfc3800640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf9806c8f0 0x7fdf9806edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.439+0000 7fdfc3800640 1 --2- 172.21.15.102:0/37676464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfbc15ea20 0x7fdfbc15d3f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:23.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.439+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 >> 172.21.15.102:0/37676464 conn(0x7fdfbc0c9390 msgr2=0x7fdfbc14a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:23.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.439+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 shutdown_connections 2024-04-20T21:26:23.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:23.439+0000 7fdfc3800640 1 -- 172.21.15.102:0/37676464 wait complete. 2024-04-20T21:26:23.456 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:26:25.456 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:25.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.663+0000 7faa0ac00640 1 Processor -- start 2024-04-20T21:26:25.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.663+0000 7faa0ac00640 1 -- start start 2024-04-20T21:26:25.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.665+0000 7faa0ac00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faa04153180 0x7faa04158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:25.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.665+0000 7faa0ac00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa04153560 con 0x7faa04159380 2024-04-20T21:26:25.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.665+0000 7faa0ac00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa040c7a70 con 0x7faa040c75a0 2024-04-20T21:26:25.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.665+0000 7faa0ac00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7faa040c7d30 con 0x7faa04153180 2024-04-20T21:26:25.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.665+0000 7faa0a200640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7faa040c75a0 0x7faa041637c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:41978/0 (socket says 172.21.15.102:41978) 2024-04-20T21:26:25.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.665+0000 7faa09800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faa04153180 0x7faa04158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:25.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.665+0000 7faa0a200640 1 -- 172.21.15.102:0/4070356120 learned_addr learned my addr 172.21.15.102:0/4070356120 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:25.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.665+0000 7faa08e00640 1 --1- 172.21.15.102:0/4070356120 >> v1:172.21.15.102:6789/0 conn(0x7faa04159380 0x7faa040c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37052/0 (socket says 172.21.15.102:37052) 2024-04-20T21:26:25.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.666+0000 7faa09800640 1 -- 172.21.15.102:0/4070356120 >> v1:172.21.15.193:6789/0 conn(0x7faa040c75a0 legacy=0x7faa041637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:25.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.666+0000 7faa09800640 1 -- 172.21.15.102:0/4070356120 >> v1:172.21.15.102:6789/0 conn(0x7faa04159380 legacy=0x7faa040c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:25.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.666+0000 7faa09800640 1 -- 172.21.15.102:0/4070356120 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0415c4d0 con 0x7faa04153180 2024-04-20T21:26:25.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.667+0000 7faa09800640 1 --2- 172.21.15.102:0/4070356120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa04153180 0x7faa04158e40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80097a0 tx=0x7fa9f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4f812d7c1663e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:25.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.667+0000 7faa03e00640 1 -- 172.21.15.102:0/4070356120 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8034070 con 0x7faa04153180 2024-04-20T21:26:25.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.667+0000 7faa03e00640 1 -- 172.21.15.102:0/4070356120 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f8030610 con 0x7faa04153180 2024-04-20T21:26:25.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.667+0000 7faa03e00640 1 -- 172.21.15.102:0/4070356120 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80309e0 con 0x7faa04153180 2024-04-20T21:26:25.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.668+0000 7faa0ac00640 1 -- 172.21.15.102:0/4070356120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa04153180 msgr2=0x7faa04158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:25.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.668+0000 7faa0ac00640 1 --2- 172.21.15.102:0/4070356120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa04153180 0x7faa04158e40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80097a0 tx=0x7fa9f802b960 comp rx=0 tx=0).stop 2024-04-20T21:26:25.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.668+0000 7faa0ac00640 1 -- 172.21.15.102:0/4070356120 shutdown_connections 2024-04-20T21:26:25.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.668+0000 7faa0ac00640 1 --2- 172.21.15.102:0/4070356120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa04153180 0x7faa04158e40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:25.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.668+0000 7faa0ac00640 1 -- 172.21.15.102:0/4070356120 >> 172.21.15.102:0/4070356120 conn(0x7faa040cc820 msgr2=0x7faa040ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:25.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.668+0000 7faa0ac00640 1 -- 172.21.15.102:0/4070356120 shutdown_connections 2024-04-20T21:26:25.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.668+0000 7faa0ac00640 1 -- 172.21.15.102:0/4070356120 wait complete. 2024-04-20T21:26:25.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.670+0000 7faa0ac00640 1 Processor -- start 2024-04-20T21:26:25.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.670+0000 7faa0ac00640 1 -- start start 2024-04-20T21:26:25.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa0ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa040c75a0 0x7faa0414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:25.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa0ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa04153180 0x7faa0415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:25.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa0ac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa04159380 0x7faa0415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:25.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa0ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa0415d290 con 0x7faa040c75a0 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa0ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa0415ca90 con 0x7faa04153180 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa0ac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faa040c6be0 con 0x7faa04159380 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa09800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa040c75a0 0x7faa0414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa09800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa040c75a0 0x7faa0414fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37274/0 (socket says 172.21.15.102:37274) 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa09800640 1 -- 172.21.15.102:0/2887522393 learned_addr learned my addr 172.21.15.102:0/2887522393 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.671+0000 7faa08e00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa04153180 0x7faa0415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.672+0000 7faa0a200640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa04159380 0x7faa0415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.672+0000 7faa09800640 1 -- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa04159380 msgr2=0x7faa0415b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.672+0000 7faa09800640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa04159380 0x7faa0415b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.672+0000 7faa09800640 1 -- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa04153180 msgr2=0x7faa0415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.672+0000 7faa09800640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa04153180 0x7faa0415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.672+0000 7faa09800640 1 -- 172.21.15.102:0/2887522393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa041474c0 con 0x7faa040c75a0 2024-04-20T21:26:25.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.672+0000 7faa09800640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa040c75a0 0x7faa0414fe30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa9f802bf70 tx=0x7fa9f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:25.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.672+0000 7faa01600640 1 -- 172.21.15.102:0/2887522393 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8034040 con 0x7faa040c75a0 2024-04-20T21:26:25.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.673+0000 7faa01600640 1 -- 172.21.15.102:0/2887522393 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f803acc0 con 0x7faa040c75a0 2024-04-20T21:26:25.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.673+0000 7faa01600640 1 -- 172.21.15.102:0/2887522393 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8044630 con 0x7faa040c75a0 2024-04-20T21:26:25.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.673+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa040c5820 con 0x7faa040c75a0 2024-04-20T21:26:25.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.673+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0414e2b0 con 0x7faa040c75a0 2024-04-20T21:26:25.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.674+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa040c5610 con 0x7faa040c75a0 2024-04-20T21:26:25.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.675+0000 7faa01600640 1 -- 172.21.15.102:0/2887522393 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9f8037070 con 0x7faa040c75a0 2024-04-20T21:26:25.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.676+0000 7faa01600640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa9e406c920 0x7fa9e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:25.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.676+0000 7faa01600640 1 -- 172.21.15.102:0/2887522393 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fa9f80ae0a0 con 0x7faa040c75a0 2024-04-20T21:26:25.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.676+0000 7faa08e00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa9e406c920 0x7fa9e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:25.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.677+0000 7faa08e00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa9e406c920 0x7fa9e406ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa041e4820 tx=0x7fa9f0007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:25.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.681+0000 7faa01600640 1 -- 172.21.15.102:0/2887522393 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa9f803f070 con 0x7faa040c75a0 2024-04-20T21:26:25.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.924+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa0404fd60 con 0x7faa040c75a0 2024-04-20T21:26:25.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.925+0000 7faa01600640 1 -- 172.21.15.102:0/2887522393 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6027 (secure 0 0 0) 0x7fa9f807b8c0 con 0x7faa040c75a0 2024-04-20T21:26:25.925 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:25.925 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:25.638360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":39,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:26:25.925 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 35 2024-04-20T21:26:25.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa9e406c920 msgr2=0x7fa9e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:25.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa9e406c920 0x7fa9e406ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa041e4820 tx=0x7fa9f0007e80 comp rx=0 tx=0).stop 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa040c75a0 msgr2=0x7faa0414fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa040c75a0 0x7faa0414fe30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa9f802bf70 tx=0x7fa9f80046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 shutdown_connections 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faa04159380 0x7faa0415b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa9e406c920 0x7fa9e406ede0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa04153180 0x7faa0415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 --2- 172.21.15.102:0/2887522393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa040c75a0 0x7faa0414fe30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 >> 172.21.15.102:0/2887522393 conn(0x7faa040cc820 msgr2=0x7faa0406ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 shutdown_connections 2024-04-20T21:26:25.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:25.928+0000 7faa0ac00640 1 -- 172.21.15.102:0/2887522393 wait complete. 2024-04-20T21:26:25.945 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 35, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 35, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5533": { "addr": "172.21.15.102:6833/2239658043", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 2239658043, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 2239658043, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5533, "incarnation": 33, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 39 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:26:25.638360+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5533 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:26:26.231 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:26.239 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:27.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:28.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.154+0000 7f6c82a00640 1 Processor -- start 2024-04-20T21:26:28.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.154+0000 7f6c82a00640 1 -- start start 2024-04-20T21:26:28.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.155+0000 7f6c82a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c7c153180 0x7f6c7c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:28.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.155+0000 7f6c82a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c7c0c75a0 0x7f6c7c1637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:28.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.155+0000 7f6c82a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c7c153560 con 0x7f6c7c159380 2024-04-20T21:26:28.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.155+0000 7f6c82a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6c7c0c7a70 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.155+0000 7f6c82a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6c7c0c7d30 con 0x7f6c7c153180 2024-04-20T21:26:28.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c81600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c7c153180 0x7f6c7c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:28.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c81600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c7c153180 0x7f6c7c158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:39256/0 (socket says 172.21.15.102:39256) 2024-04-20T21:26:28.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c80c00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6c7c159380 0x7f6c7c0c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:54300/0 (socket says 172.21.15.102:54300) 2024-04-20T21:26:28.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c81600640 1 -- 172.21.15.102:0/774023540 learned_addr learned my addr 172.21.15.102:0/774023540 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:28.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c82000640 1 --2- 172.21.15.102:0/774023540 >> v2:172.21.15.102:3300/0 conn(0x7f6c7c0c75a0 0x7f6c7c1637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:28.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c81600640 1 -- 172.21.15.102:0/774023540 >> v2:172.21.15.102:3300/0 conn(0x7f6c7c0c75a0 msgr2=0x7f6c7c1637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:28.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c81600640 1 --2- 172.21.15.102:0/774023540 >> v2:172.21.15.102:3300/0 conn(0x7f6c7c0c75a0 0x7f6c7c1637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c81600640 1 -- 172.21.15.102:0/774023540 >> v1:172.21.15.193:6789/0 conn(0x7f6c7c159380 legacy=0x7f6c7c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:28.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c81600640 1 -- 172.21.15.102:0/774023540 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c15c4d0 con 0x7f6c7c153180 2024-04-20T21:26:28.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.156+0000 7f6c7be00640 1 -- 172.21.15.102:0/774023540 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389066398 0 0) 0x7f6c7c153560 con 0x7f6c7c159380 2024-04-20T21:26:28.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.157+0000 7f6c81600640 1 --2- 172.21.15.102:0/774023540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 0x7f6c7c158e40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0cd860 tx=0x7f6c7002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9290e7e1848dcd91 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c7be00640 1 -- 172.21.15.102:0/774023540 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70034070 con 0x7f6c7c153180 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c7be00640 1 -- 172.21.15.102:0/774023540 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c70030860 con 0x7f6c7c153180 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c7be00640 1 -- 172.21.15.102:0/774023540 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70030c30 con 0x7f6c7c153180 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c82a00640 1 -- 172.21.15.102:0/774023540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 msgr2=0x7f6c7c158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c82a00640 1 --2- 172.21.15.102:0/774023540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 0x7f6c7c158e40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0cd860 tx=0x7f6c7002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c82a00640 1 -- 172.21.15.102:0/774023540 shutdown_connections 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c82a00640 1 --2- 172.21.15.102:0/774023540 >> v2:172.21.15.102:3300/0 conn(0x7f6c7c0c75a0 0x7f6c7c1637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c82a00640 1 --2- 172.21.15.102:0/774023540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 0x7f6c7c158e40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c82a00640 1 -- 172.21.15.102:0/774023540 >> 172.21.15.102:0/774023540 conn(0x7f6c7c0cc820 msgr2=0x7f6c7c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c82a00640 1 -- 172.21.15.102:0/774023540 shutdown_connections 2024-04-20T21:26:28.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.158+0000 7f6c82a00640 1 -- 172.21.15.102:0/774023540 wait complete. 2024-04-20T21:26:28.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.160+0000 7f6c82a00640 1 Processor -- start 2024-04-20T21:26:28.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.160+0000 7f6c82a00640 1 -- start start 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.161+0000 7f6c82a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c7c0c75a0 0x7f6c7c070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.161+0000 7f6c82a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 0x7f6c7c146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.161+0000 7f6c82a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c7c159380 0x7f6c7c1466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.161+0000 7f6c82a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c15d290 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.161+0000 7f6c82a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c15ca90 con 0x7f6c7c153180 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.161+0000 7f6c82a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c0c6be0 con 0x7f6c7c159380 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.161+0000 7f6c81600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c7c0c75a0 0x7f6c7c070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.161+0000 7f6c81600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c7c0c75a0 0x7f6c7c070f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51874/0 (socket says 172.21.15.102:51874) 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c81600640 1 -- 172.21.15.102:0/1133425312 learned_addr learned my addr 172.21.15.102:0/1133425312 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c82000640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c7c159380 0x7f6c7c1466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c80c00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 0x7f6c7c146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c81600640 1 -- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c7c159380 msgr2=0x7f6c7c1466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c81600640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c7c159380 0x7f6c7c1466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c81600640 1 -- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 msgr2=0x7f6c7c146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c81600640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 0x7f6c7c146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c81600640 1 -- 172.21.15.102:0/1133425312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c0cc120 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.162+0000 7f6c81600640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c7c0c75a0 0x7f6c7c070f40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c7002b9e0 tx=0x7f6c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.163+0000 7f6c79600640 1 -- 172.21.15.102:0/1133425312 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70034040 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.163+0000 7f6c79600640 1 -- 172.21.15.102:0/1133425312 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c70039d90 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.163+0000 7f6c79600640 1 -- 172.21.15.102:0/1133425312 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70043610 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.163+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c0c4860 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.163+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c0c4e20 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.164+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c04fd60 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.165+0000 7f6c79600640 1 -- 172.21.15.102:0/1133425312 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c70037070 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.166+0000 7f6c79600640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c5c06c8f0 0x7f6c5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.166+0000 7f6c79600640 1 -- 172.21.15.102:0/1133425312 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f6c700ae370 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.166+0000 7f6c80c00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c5c06c8f0 0x7f6c5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.167+0000 7f6c80c00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c5c06c8f0 0x7f6c5c06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0718e0 tx=0x7f6c68007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:28.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.171+0000 7f6c79600640 1 -- 172.21.15.102:0/1133425312 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c7003e070 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.409+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c7c0c3690 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.410+0000 7f6c79600640 1 -- 172.21.15.102:0/1133425312 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6027 (secure 0 0 0) 0x7f6c7007bb90 con 0x7f6c7c0c75a0 2024-04-20T21:26:28.410 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:28.410 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:25.638360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":39,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:26:28.410 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 35 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c5c06c8f0 msgr2=0x7f6c5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c5c06c8f0 0x7f6c5c06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0718e0 tx=0x7f6c68007de0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c7c0c75a0 msgr2=0x7f6c7c070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c7c0c75a0 0x7f6c7c070f40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c7002b9e0 tx=0x7f6c700046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 shutdown_connections 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c7c159380 0x7f6c7c1466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c5c06c8f0 0x7f6c5c06edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c7c153180 0x7f6c7c146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 --2- 172.21.15.102:0/1133425312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c7c0c75a0 0x7f6c7c070f40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:28.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.413+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 >> 172.21.15.102:0/1133425312 conn(0x7f6c7c0cc820 msgr2=0x7f6c7c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:28.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.414+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 shutdown_connections 2024-04-20T21:26:28.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:28.414+0000 7f6c82a00640 1 -- 172.21.15.102:0/1133425312 wait complete. 2024-04-20T21:26:28.431 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:26:30.432 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:30.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.753+0000 7fcc15400640 1 Processor -- start 2024-04-20T21:26:30.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.753+0000 7fcc15400640 1 -- start start 2024-04-20T21:26:30.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc15400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcc1015ea20 0x7fcc1006d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:30.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc15400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:30.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc15400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc100c8f20 con 0x7fcc1015f2b0 2024-04-20T21:26:30.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc15400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fcc1006cb80 con 0x7fcc1015ea20 2024-04-20T21:26:30.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc15400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1006ce60 con 0x7fcc1015ee00 2024-04-20T21:26:30.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc05e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:30.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc0e800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcc1015ea20 0x7fcc1006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:30.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc0f200640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fcc1015f2b0 0x7fcc100c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:54316/0 (socket says 172.21.15.102:54316) 2024-04-20T21:26:30.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc0f200640 1 -- 172.21.15.102:0/4253341551 learned_addr learned my addr 172.21.15.102:0/4253341551 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:30.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.755+0000 7fcc05e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:57436/0 (socket says 172.21.15.102:57436) 2024-04-20T21:26:30.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.756+0000 7fcc05e00640 1 -- 172.21.15.102:0/4253341551 >> v2:172.21.15.193:3300/0 conn(0x7fcc1015ea20 msgr2=0x7fcc1006d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:30.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.756+0000 7fcc05e00640 1 --2- 172.21.15.102:0/4253341551 >> v2:172.21.15.193:3300/0 conn(0x7fcc1015ea20 0x7fcc1006d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.756+0000 7fcc05e00640 1 -- 172.21.15.102:0/4253341551 >> v1:172.21.15.193:6789/0 conn(0x7fcc1015f2b0 legacy=0x7fcc100c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:30.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.756+0000 7fcc05e00640 1 -- 172.21.15.102:0/4253341551 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1006d140 con 0x7fcc1015ee00 2024-04-20T21:26:30.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.757+0000 7fcc05e00640 1 --2- 172.21.15.102:0/4253341551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100710a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80097a0 tx=0x7fcbf802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968c68dd5e418b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:30.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.757+0000 7fcc0de00640 1 -- 172.21.15.102:0/4253341551 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8034070 con 0x7fcc1015ee00 2024-04-20T21:26:30.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.757+0000 7fcc0de00640 1 -- 172.21.15.102:0/4253341551 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf8030800 con 0x7fcc1015ee00 2024-04-20T21:26:30.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.757+0000 7fcc0de00640 1 -- 172.21.15.102:0/4253341551 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8030bd0 con 0x7fcc1015ee00 2024-04-20T21:26:30.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.758+0000 7fcc15400640 1 -- 172.21.15.102:0/4253341551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 msgr2=0x7fcc100710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:30.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.758+0000 7fcc15400640 1 --2- 172.21.15.102:0/4253341551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100710a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80097a0 tx=0x7fcbf802b940 comp rx=0 tx=0).stop 2024-04-20T21:26:30.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.758+0000 7fcc15400640 1 -- 172.21.15.102:0/4253341551 shutdown_connections 2024-04-20T21:26:30.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.758+0000 7fcc15400640 1 --2- 172.21.15.102:0/4253341551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100710a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.758+0000 7fcc15400640 1 --2- 172.21.15.102:0/4253341551 >> v2:172.21.15.193:3300/0 conn(0x7fcc1015ea20 0x7fcc1006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.758+0000 7fcc15400640 1 -- 172.21.15.102:0/4253341551 >> 172.21.15.102:0/4253341551 conn(0x7fcc100c9390 msgr2=0x7fcc100c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:30.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.758+0000 7fcc15400640 1 -- 172.21.15.102:0/4253341551 shutdown_connections 2024-04-20T21:26:30.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.758+0000 7fcc15400640 1 -- 172.21.15.102:0/4253341551 wait complete. 2024-04-20T21:26:30.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.760+0000 7fcc15400640 1 Processor -- start 2024-04-20T21:26:30.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.760+0000 7fcc15400640 1 -- start start 2024-04-20T21:26:30.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc15400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1015ea20 0x7fcc100c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:30.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc15400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:30.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc15400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1015f2b0 0x7fcc10149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:30.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc15400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcc100c8c80 con 0x7fcc1015ea20 2024-04-20T21:26:30.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc15400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc100c8b00 con 0x7fcc1015f2b0 2024-04-20T21:26:30.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc15400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fcc10154e80 con 0x7fcc1015ee00 2024-04-20T21:26:30.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc0e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1015ea20 0x7fcc100c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:30.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc05e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:30.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc0f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1015f2b0 0x7fcc10149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:30.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc0e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1015ea20 0x7fcc100c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51884/0 (socket says 172.21.15.102:51884) 2024-04-20T21:26:30.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.761+0000 7fcc0e800640 1 -- 172.21.15.102:0/1679464044 learned_addr learned my addr 172.21.15.102:0/1679464044 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:30.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc05e00640 1 -- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1015f2b0 msgr2=0x7fcc10149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:30.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc05e00640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1015f2b0 0x7fcc10149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc05e00640 1 -- 172.21.15.102:0/1679464044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1015ea20 msgr2=0x7fcc100c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:30.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc05e00640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1015ea20 0x7fcc100c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc05e00640 1 -- 172.21.15.102:0/1679464044 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1014a060 con 0x7fcc1015ee00 2024-04-20T21:26:30.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc05e00640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100c5a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8033000 tx=0x7fcbf80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:30.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc07400640 1 -- 172.21.15.102:0/1679464044 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8047070 con 0x7fcc1015ee00 2024-04-20T21:26:30.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc07400640 1 -- 172.21.15.102:0/1679464044 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf8030600 con 0x7fcc1015ee00 2024-04-20T21:26:30.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc07400640 1 -- 172.21.15.102:0/1679464044 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8034040 con 0x7fcc1015ee00 2024-04-20T21:26:30.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc100c6370 con 0x7fcc1015ee00 2024-04-20T21:26:30.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.762+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc101f1120 con 0x7fcc1015ee00 2024-04-20T21:26:30.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.764+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc10072f30 con 0x7fcc1015ee00 2024-04-20T21:26:30.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.765+0000 7fcc07400640 1 -- 172.21.15.102:0/1679464044 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbf8039d10 con 0x7fcc1015ee00 2024-04-20T21:26:30.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.765+0000 7fcc07400640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcbec06c920 0x7fcbec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:30.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.766+0000 7fcc07400640 1 -- 172.21.15.102:0/1679464044 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fcbf80adbc0 con 0x7fcc1015ee00 2024-04-20T21:26:30.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.766+0000 7fcc0e800640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcbec06c920 0x7fcbec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:30.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.766+0000 7fcc0e800640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcbec06c920 0x7fcbec06ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009c00 tx=0x7fcc00009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:30.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:30.770+0000 7fcc07400640 1 -- 172.21.15.102:0/1679464044 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcc10072f30 con 0x7fcc1015ee00 2024-04-20T21:26:31.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.176+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc10074340 con 0x7fcc1015ee00 2024-04-20T21:26:31.177 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:31.177 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:26:31.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.177+0000 7fcc07400640 1 -- 172.21.15.102:0/1679464044 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6034 (secure 0 0 0) 0x7fcbf807b2c0 con 0x7fcc1015ee00 2024-04-20T21:26:31.177 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 36 2024-04-20T21:26:31.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcbec06c920 msgr2=0x7fcbec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcbec06c920 0x7fcbec06ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009c00 tx=0x7fcc00009190 comp rx=0 tx=0).stop 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 msgr2=0x7fcc100c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100c5a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8033000 tx=0x7fcbf80046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 shutdown_connections 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcbec06c920 0x7fcbec06ede0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1015f2b0 0x7fcc10149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc1015ee00 0x7fcc100c5a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 --2- 172.21.15.102:0/1679464044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc1015ea20 0x7fcc100c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 >> 172.21.15.102:0/1679464044 conn(0x7fcc100c9390 msgr2=0x7fcc100c76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 shutdown_connections 2024-04-20T21:26:31.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:31.180+0000 7fcc15400640 1 -- 172.21.15.102:0/1679464044 wait complete. 2024-04-20T21:26:31.205 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5039 2024-04-20T21:26:31.205 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 9 secs before reviving mds.c 2024-04-20T21:26:38.870 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-04-20T21:26:38.871+0000 7fcd5ba00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:26:40.206 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:26:40.206 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:26:40.206 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:26:40.215 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:26:40.215 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:40.372 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:26:40.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.397+0000 7fc8ee000640 1 Processor -- start 2024-04-20T21:26:40.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.397+0000 7fc8ee000640 1 -- start start 2024-04-20T21:26:40.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.398+0000 7fc8ee000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8e8073c70 0x7fc8e81647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:40.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.398+0000 7fc8ee000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e8153560 con 0x7fc8e80c75a0 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.398+0000 7fc8ee000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e80c7a70 con 0x7fc8e8153180 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.398+0000 7fc8ee000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e80c7d30 con 0x7fc8e8073c70 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8e7e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc8e80c75a0 0x7fc8e80c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35258/0 (socket says 172.21.15.102:35258) 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8e7e00640 1 -- 172.21.15.102:0/474511624 learned_addr learned my addr 172.21.15.102:0/474511624 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8ecc00640 1 --1- 172.21.15.102:0/474511624 >> v1:172.21.15.193:6789/0 conn(0x7fc8e8153180 0x7fc8e8158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43498/0 (socket says 172.21.15.102:43498) 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8ed600640 1 --2- 172.21.15.102:0/474511624 >> v2:172.21.15.102:3300/0 conn(0x7fc8e8073c70 0x7fc8e81647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8e7400640 1 -- 172.21.15.102:0/474511624 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794912740 0 0) 0x7fc8e80c7a70 con 0x7fc8e8153180 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8e7400640 1 -- 172.21.15.102:0/474511624 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8d0003520 con 0x7fc8e8153180 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8e7400640 1 -- 172.21.15.102:0/474511624 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411983090 0 0) 0x7fc8e8153560 con 0x7fc8e80c75a0 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8e7400640 1 -- 172.21.15.102:0/474511624 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8e80c7a70 con 0x7fc8e80c75a0 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8ed600640 1 -- 172.21.15.102:0/474511624 >> v1:172.21.15.193:6789/0 conn(0x7fc8e8153180 legacy=0x7fc8e8158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8ed600640 1 -- 172.21.15.102:0/474511624 >> v1:172.21.15.102:6789/0 conn(0x7fc8e80c75a0 legacy=0x7fc8e80c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:40.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.399+0000 7fc8ed600640 1 -- 172.21.15.102:0/474511624 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e815c4d0 con 0x7fc8e8073c70 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.400+0000 7fc8ed600640 1 --2- 172.21.15.102:0/474511624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8073c70 0x7fc8e81647d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009d90 tx=0x7fc8e002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e38d92873b981f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.400+0000 7fc8e7400640 1 -- 172.21.15.102:0/474511624 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0039070 con 0x7fc8e8073c70 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8e7400640 1 -- 172.21.15.102:0/474511624 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e002ccf0 con 0x7fc8e8073c70 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8e7400640 1 -- 172.21.15.102:0/474511624 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0034660 con 0x7fc8e8073c70 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8ee000640 1 -- 172.21.15.102:0/474511624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8073c70 msgr2=0x7fc8e81647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8ee000640 1 --2- 172.21.15.102:0/474511624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8073c70 0x7fc8e81647d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009d90 tx=0x7fc8e002bf40 comp rx=0 tx=0).stop 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8ee000640 1 -- 172.21.15.102:0/474511624 shutdown_connections 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8ee000640 1 --2- 172.21.15.102:0/474511624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8073c70 0x7fc8e81647d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8ee000640 1 -- 172.21.15.102:0/474511624 >> 172.21.15.102:0/474511624 conn(0x7fc8e80cc820 msgr2=0x7fc8e80ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:40.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8ee000640 1 -- 172.21.15.102:0/474511624 shutdown_connections 2024-04-20T21:26:40.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.401+0000 7fc8ee000640 1 -- 172.21.15.102:0/474511624 wait complete. 2024-04-20T21:26:40.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.403+0000 7fc8ee000640 1 Processor -- start 2024-04-20T21:26:40.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.403+0000 7fc8ee000640 1 -- start start 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8e8073c70 0x7fc8e80c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8e80c75a0 0x7fc8e81475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8153180 0x7fc8e80cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ee000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8159550 con 0x7fc8e8153180 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ee000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e815ca90 con 0x7fc8e8073c70 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ee000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e815d290 con 0x7fc8e80c75a0 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ed600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8153180 0x7fc8e80cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ed600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8153180 0x7fc8e80cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39742/0 (socket says 172.21.15.102:39742) 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ed600640 1 -- 172.21.15.102:0/485653263 learned_addr learned my addr 172.21.15.102:0/485653263 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8e7e00640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8e80c75a0 0x7fc8e81475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.404+0000 7fc8ecc00640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8e8073c70 0x7fc8e80c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:40.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8ed600640 1 -- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8e80c75a0 msgr2=0x7fc8e81475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:40.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8ed600640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8e80c75a0 0x7fc8e81475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:40.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8ed600640 1 -- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8e8073c70 msgr2=0x7fc8e80c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:40.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8ed600640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8e8073c70 0x7fc8e80c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:40.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8ed600640 1 -- 172.21.15.102:0/485653263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e81f22b0 con 0x7fc8e8153180 2024-04-20T21:26:40.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8ed600640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8153180 0x7fc8e80cbc20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009eb0 tx=0x7fc8e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:40.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8e4c00640 1 -- 172.21.15.102:0/485653263 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0039050 con 0x7fc8e8153180 2024-04-20T21:26:40.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8e4c00640 1 -- 172.21.15.102:0/485653263 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e0045070 con 0x7fc8e8153180 2024-04-20T21:26:40.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8e4c00640 1 -- 172.21.15.102:0/485653263 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0009450 con 0x7fc8e8153180 2024-04-20T21:26:40.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.405+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e80c5730 con 0x7fc8e8153180 2024-04-20T21:26:40.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.406+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e80c5cb0 con 0x7fc8e8153180 2024-04-20T21:26:40.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.407+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e8152d30 con 0x7fc8e8153180 2024-04-20T21:26:40.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.408+0000 7fc8e4c00640 1 -- 172.21.15.102:0/485653263 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8e003e850 con 0x7fc8e8153180 2024-04-20T21:26:40.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.408+0000 7fc8e4c00640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8c806c8f0 0x7fc8c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:40.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.409+0000 7fc8e4c00640 1 -- 172.21.15.102:0/485653263 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fc8e00aed20 con 0x7fc8e8153180 2024-04-20T21:26:40.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.409+0000 7fc8ecc00640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8c806c8f0 0x7fc8c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:40.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.409+0000 7fc8ecc00640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8c806c8f0 0x7fc8c806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000f60 tx=0x7fc8dc007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:40.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.413+0000 7fc8e4c00640 1 -- 172.21.15.102:0/485653263 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc8e8152d30 con 0x7fc8e8153180 2024-04-20T21:26:40.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.650+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8e804fd60 con 0x7fc8e8153180 2024-04-20T21:26:40.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.651+0000 7fc8e4c00640 1 -- 172.21.15.102:0/485653263 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6034 (secure 0 0 0) 0x7fc8e007c5d0 con 0x7fc8e8153180 2024-04-20T21:26:40.651 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:40.651 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:26:40.651 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 36 2024-04-20T21:26:40.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8c806c8f0 msgr2=0x7fc8c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:40.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8c806c8f0 0x7fc8c806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000f60 tx=0x7fc8dc007d50 comp rx=0 tx=0).stop 2024-04-20T21:26:40.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8153180 msgr2=0x7fc8e80cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8153180 0x7fc8e80cbc20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009eb0 tx=0x7fc8e0004a40 comp rx=0 tx=0).stop 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 shutdown_connections 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8e8153180 0x7fc8e80cbc20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8e80c75a0 0x7fc8e81475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8c806c8f0 0x7fc8c806edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 --2- 172.21.15.102:0/485653263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8e8073c70 0x7fc8e80c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 >> 172.21.15.102:0/485653263 conn(0x7fc8e80cc820 msgr2=0x7fc8e80ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 shutdown_connections 2024-04-20T21:26:40.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:40.654+0000 7fc8ee000640 1 -- 172.21.15.102:0/485653263 wait complete. 2024-04-20T21:26:40.671 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-20T21:26:42.672 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:42.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.884+0000 7fd626c00640 1 Processor -- start 2024-04-20T21:26:42.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.884+0000 7fd626c00640 1 -- start start 2024-04-20T21:26:42.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.885+0000 7fd626c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd6200c9eb0 0x7fd6200c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.885+0000 7fd626c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd620073a10 0x7fd620158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.885+0000 7fd626c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6201536d0 con 0x7fd62014fa50 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.885+0000 7fd626c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fd620159500 con 0x7fd6200c9eb0 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.885+0000 7fd626c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fd6200c76c0 con 0x7fd620073a10 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd61f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd6200c9eb0 0x7fd6200c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd61fe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fd62014fa50 0x7fd6200c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43510/0 (socket says 172.21.15.102:43510) 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd61fe00640 1 -- 172.21.15.102:0/2303497431 learned_addr learned my addr 172.21.15.102:0/2303497431 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd624c00640 1 --2- 172.21.15.102:0/2303497431 >> v2:172.21.15.193:3300/0 conn(0x7fd620073a10 0x7fd620158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd61ea00640 1 -- 172.21.15.102:0/2303497431 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760670792 0 0) 0x7fd610001490 con 0x7fd62014fa50 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd61ea00640 1 -- 172.21.15.102:0/2303497431 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd604003520 con 0x7fd62014fa50 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd624c00640 1 -- 172.21.15.102:0/2303497431 >> v2:172.21.15.102:3300/0 conn(0x7fd6200c9eb0 msgr2=0x7fd6200c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd624c00640 1 --2- 172.21.15.102:0/2303497431 >> v2:172.21.15.102:3300/0 conn(0x7fd6200c9eb0 0x7fd6200c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:42.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd624c00640 1 -- 172.21.15.102:0/2303497431 >> v1:172.21.15.193:6789/0 conn(0x7fd62014fa50 legacy=0x7fd6200c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:42.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.886+0000 7fd624c00640 1 -- 172.21.15.102:0/2303497431 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6200c7960 con 0x7fd620073a10 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.887+0000 7fd624c00640 1 --2- 172.21.15.102:0/2303497431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 0x7fd620158fc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd620156b00 tx=0x7fd61802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207f5aac90371fff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.888+0000 7fd61ea00640 1 -- 172.21.15.102:0/2303497431 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618034070 con 0x7fd620073a10 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.888+0000 7fd61ea00640 1 -- 172.21.15.102:0/2303497431 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd618030610 con 0x7fd620073a10 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.888+0000 7fd61ea00640 1 -- 172.21.15.102:0/2303497431 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180309c0 con 0x7fd620073a10 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.888+0000 7fd626c00640 1 -- 172.21.15.102:0/2303497431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 msgr2=0x7fd620158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.888+0000 7fd626c00640 1 --2- 172.21.15.102:0/2303497431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 0x7fd620158fc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd620156b00 tx=0x7fd61802b960 comp rx=0 tx=0).stop 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.889+0000 7fd626c00640 1 -- 172.21.15.102:0/2303497431 shutdown_connections 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.889+0000 7fd626c00640 1 --2- 172.21.15.102:0/2303497431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 0x7fd620158fc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.889+0000 7fd626c00640 1 --2- 172.21.15.102:0/2303497431 >> v2:172.21.15.102:3300/0 conn(0x7fd6200c9eb0 0x7fd6200c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:42.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.889+0000 7fd626c00640 1 -- 172.21.15.102:0/2303497431 >> 172.21.15.102:0/2303497431 conn(0x7fd6200c9390 msgr2=0x7fd6200c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:42.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.889+0000 7fd626c00640 1 -- 172.21.15.102:0/2303497431 shutdown_connections 2024-04-20T21:26:42.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.889+0000 7fd626c00640 1 -- 172.21.15.102:0/2303497431 wait complete. 2024-04-20T21:26:42.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.890+0000 7fd626c00640 1 Processor -- start 2024-04-20T21:26:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.891+0000 7fd626c00640 1 -- start start 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd626c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 0x7fd6201e4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd626c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6200c9eb0 0x7fd6200727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd626c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd62014fa50 0x7fd6200cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd626c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200c7c00 con 0x7fd6200c9eb0 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd626c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200c3d90 con 0x7fd620073a10 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd626c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200c4030 con 0x7fd62014fa50 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd61f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6200c9eb0 0x7fd6200727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd61f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6200c9eb0 0x7fd6200727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39768/0 (socket says 172.21.15.102:39768) 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd61f400640 1 -- 172.21.15.102:0/4182679267 learned_addr learned my addr 172.21.15.102:0/4182679267 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd624c00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd62014fa50 0x7fd6200cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.892+0000 7fd61fe00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 0x7fd6201e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd61f400640 1 -- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd62014fa50 msgr2=0x7fd6200cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd61f400640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd62014fa50 0x7fd6200cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd61f400640 1 -- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 msgr2=0x7fd6201e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd61f400640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 0x7fd6201e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd61f400640 1 -- 172.21.15.102:0/4182679267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6201534d0 con 0x7fd6200c9eb0 2024-04-20T21:26:42.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd61f400640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6200c9eb0 0x7fd6200727f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00e9b0 tx=0x7fd60c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:42.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd617e00640 1 -- 172.21.15.102:0/4182679267 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c017070 con 0x7fd6200c9eb0 2024-04-20T21:26:42.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd617e00640 1 -- 172.21.15.102:0/4182679267 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd60c00c9a0 con 0x7fd6200c9eb0 2024-04-20T21:26:42.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd617e00640 1 -- 172.21.15.102:0/4182679267 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00cd90 con 0x7fd6200c9eb0 2024-04-20T21:26:42.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.893+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd620071910 con 0x7fd6200c9eb0 2024-04-20T21:26:42.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.894+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6201e3ab0 con 0x7fd6200c9eb0 2024-04-20T21:26:42.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.895+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd620074340 con 0x7fd6200c9eb0 2024-04-20T21:26:42.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.896+0000 7fd617e00640 1 -- 172.21.15.102:0/4182679267 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd60c01a070 con 0x7fd6200c9eb0 2024-04-20T21:26:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.897+0000 7fd617e00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd60006c8f0 0x7fd60006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.897+0000 7fd617e00640 1 -- 172.21.15.102:0/4182679267 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fd60c08c510 con 0x7fd6200c9eb0 2024-04-20T21:26:42.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.897+0000 7fd61fe00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd60006c8f0 0x7fd60006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.898+0000 7fd61fe00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd60006c8f0 0x7fd60006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd610005c00 tx=0x7fd610009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:42.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:42.902+0000 7fd617e00640 1 -- 172.21.15.102:0/4182679267 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd60c059c10 con 0x7fd6200c9eb0 2024-04-20T21:26:43.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.140+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6200c7320 con 0x7fd6200c9eb0 2024-04-20T21:26:43.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.141+0000 7fd617e00640 1 -- 172.21.15.102:0/4182679267 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6856 (secure 0 0 0) 0x7fd60c05df10 con 0x7fd6200c9eb0 2024-04-20T21:26:43.141 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:26:43.141 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5659,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:26:43.141 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd60006c8f0 msgr2=0x7fd60006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd60006c8f0 0x7fd60006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd610005c00 tx=0x7fd610009040 comp rx=0 tx=0).stop 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6200c9eb0 msgr2=0x7fd6200727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6200c9eb0 0x7fd6200727f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00e9b0 tx=0x7fd60c00ee70 comp rx=0 tx=0).stop 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 shutdown_connections 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd62014fa50 0x7fd6200cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6200c9eb0 0x7fd6200727f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd60006c8f0 0x7fd60006edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 --2- 172.21.15.102:0/4182679267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd620073a10 0x7fd6201e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 >> 172.21.15.102:0/4182679267 conn(0x7fd6200c9390 msgr2=0x7fd6201e75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 shutdown_connections 2024-04-20T21:26:43.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:26:43.144+0000 7fd626c00640 1 -- 172.21.15.102:0/4182679267 wait complete. 2024-04-20T21:26:43.162 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:26:43.163 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 97 secs before thrashing 2024-04-20T21:26:56.394 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:56.402 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:26.509 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:26.518 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:56.663 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:56.671 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:20.164 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:20.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.372+0000 7f6c3be00640 1 Processor -- start 2024-04-20T21:28:20.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.372+0000 7f6c3be00640 1 -- start start 2024-04-20T21:28:20.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.373+0000 7f6c3be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c3c155530 0x7f6c3c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.373+0000 7f6c3be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cc470 0x7f6c3c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.373+0000 7f6c3be00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c3c15ef60 con 0x7f6c3c0cdfc0 2024-04-20T21:28:20.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.373+0000 7f6c3be00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6c3c0c5ca0 con 0x7f6c3c155530 2024-04-20T21:28:20.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.373+0000 7f6c3be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6c3c0cd5f0 con 0x7f6c3c0cc470 2024-04-20T21:28:20.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.373+0000 7f6c3aa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c3c155530 0x7f6c3c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.373+0000 7f6c3aa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c3c155530 0x7f6c3c15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42344/0 (socket says 172.21.15.102:42344) 2024-04-20T21:28:20.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.373+0000 7f6c3aa00640 1 -- 172.21.15.102:0/2064396749 learned_addr learned my addr 172.21.15.102:0/2064396749 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:20.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.374+0000 7f6c3a000640 1 --2- 172.21.15.102:0/2064396749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cc470 0x7f6c3c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.374+0000 7f6c3aa00640 1 -- 172.21.15.102:0/2064396749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cc470 msgr2=0x7f6c3c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:20.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.374+0000 7f6c3aa00640 1 --2- 172.21.15.102:0/2064396749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cc470 0x7f6c3c0cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.374+0000 7f6c3aa00640 1 -- 172.21.15.102:0/2064396749 >> v1:172.21.15.102:6789/0 conn(0x7f6c3c0cdfc0 legacy=0x7f6c3c0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.374+0000 7f6c3aa00640 1 -- 172.21.15.102:0/2064396749 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c15f240 con 0x7f6c3c155530 2024-04-20T21:28:20.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.374+0000 7f6c3a000640 1 --2- 172.21.15.102:0/2064396749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cc470 0x7f6c3c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:20.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.375+0000 7f6c3aa00640 1 --2- 172.21.15.102:0/2064396749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 0x7f6c3c15e640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009870 tx=0x7f6c2c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5996d9a01734a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:20.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.375+0000 7f6c39600640 1 -- 172.21.15.102:0/2064396749 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c034070 con 0x7f6c3c155530 2024-04-20T21:28:20.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.376+0000 7f6c39600640 1 -- 172.21.15.102:0/2064396749 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c2c030840 con 0x7f6c3c155530 2024-04-20T21:28:20.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.376+0000 7f6c39600640 1 -- 172.21.15.102:0/2064396749 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c030c10 con 0x7f6c3c155530 2024-04-20T21:28:20.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.376+0000 7f6c3be00640 1 -- 172.21.15.102:0/2064396749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 msgr2=0x7f6c3c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.376+0000 7f6c3be00640 1 --2- 172.21.15.102:0/2064396749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 0x7f6c3c15e640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009870 tx=0x7f6c2c02b980 comp rx=0 tx=0).stop 2024-04-20T21:28:20.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.376+0000 7f6c3be00640 1 -- 172.21.15.102:0/2064396749 shutdown_connections 2024-04-20T21:28:20.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.376+0000 7f6c3be00640 1 --2- 172.21.15.102:0/2064396749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cc470 0x7f6c3c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.376+0000 7f6c3be00640 1 --2- 172.21.15.102:0/2064396749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 0x7f6c3c15e640 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.376+0000 7f6c3be00640 1 -- 172.21.15.102:0/2064396749 >> 172.21.15.102:0/2064396749 conn(0x7f6c3c075080 msgr2=0x7f6c3c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:20.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.377+0000 7f6c3be00640 1 -- 172.21.15.102:0/2064396749 shutdown_connections 2024-04-20T21:28:20.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.377+0000 7f6c3be00640 1 -- 172.21.15.102:0/2064396749 wait complete. 2024-04-20T21:28:20.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.378+0000 7f6c3be00640 1 Processor -- start 2024-04-20T21:28:20.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.378+0000 7f6c3be00640 1 -- start start 2024-04-20T21:28:20.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.379+0000 7f6c3be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c3c0cc470 0x7f6c3c1f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.379+0000 7f6c3be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cdfc0 0x7f6c3c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.379+0000 7f6c3be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 0x7f6c3c15d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.379+0000 7f6c3be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0cd280 con 0x7f6c3c155530 2024-04-20T21:28:20.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.379+0000 7f6c3be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0ccfa0 con 0x7f6c3c0cc470 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.379+0000 7f6c3be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6c3c0ccbe0 con 0x7f6c3c0cdfc0 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cdfc0 0x7f6c3c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c3c0cc470 0x7f6c3c1f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cdfc0 0x7f6c3c15cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:40146/0 (socket says 172.21.15.102:40146) 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3a000640 1 -- 172.21.15.102:0/858366775 learned_addr learned my addr 172.21.15.102:0/858366775 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3b400640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 0x7f6c3c15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3aa00640 1 -- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cdfc0 msgr2=0x7f6c3c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3aa00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cdfc0 0x7f6c3c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3aa00640 1 -- 172.21.15.102:0/858366775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 msgr2=0x7f6c3c15d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3aa00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 0x7f6c3c15d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3aa00640 1 -- 172.21.15.102:0/858366775 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c1500a0 con 0x7f6c3c0cc470 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3b400640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 0x7f6c3c15d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:28:20.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.380+0000 7f6c3a000640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cdfc0 0x7f6c3c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:20.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.381+0000 7f6c3aa00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c3c0cc470 0x7f6c3c1f12f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c033000 tx=0x7f6c2c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:20.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.381+0000 7f6c32a00640 1 -- 172.21.15.102:0/858366775 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c034040 con 0x7f6c3c0cc470 2024-04-20T21:28:20.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.381+0000 7f6c32a00640 1 -- 172.21.15.102:0/858366775 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c2c0433f0 con 0x7f6c3c0cc470 2024-04-20T21:28:20.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.381+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c1f1b10 con 0x7f6c3c0cc470 2024-04-20T21:28:20.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.381+0000 7f6c32a00640 1 -- 172.21.15.102:0/858366775 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0437e0 con 0x7f6c3c0cc470 2024-04-20T21:28:20.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.381+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c1f20d0 con 0x7f6c3c0cc470 2024-04-20T21:28:20.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.382+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c3c04fd60 con 0x7f6c3c0cc470 2024-04-20T21:28:20.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.383+0000 7f6c32a00640 1 -- 172.21.15.102:0/858366775 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c2c04a060 con 0x7f6c3c0cc470 2024-04-20T21:28:20.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.384+0000 7f6c32a00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c1c06c8f0 0x7f6c1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.384+0000 7f6c32a00640 1 -- 172.21.15.102:0/858366775 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f6c2c0ae250 con 0x7f6c3c0cc470 2024-04-20T21:28:20.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.384+0000 7f6c3a000640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c1c06c8f0 0x7f6c1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.385+0000 7f6c3a000640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c1c06c8f0 0x7f6c1c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c1f47c0 tx=0x7f6c28008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:20.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.390+0000 7f6c32a00640 1 -- 172.21.15.102:0/858366775 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c2c03e070 con 0x7f6c3c0cc470 2024-04-20T21:28:20.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.637+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c3c0cc9c0 con 0x7f6c3c0cc470 2024-04-20T21:28:20.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.638+0000 7f6c32a00640 1 -- 172.21.15.102:0/858366775 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6856 (secure 0 0 0) 0x7f6c2c07c030 con 0x7f6c3c0cc470 2024-04-20T21:28:20.637 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:20.637 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5659,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:28:20.637 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.640+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c1c06c8f0 msgr2=0x7f6c1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.640+0000 7f6c3be00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c1c06c8f0 0x7f6c1c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c1f47c0 tx=0x7f6c28008040 comp rx=0 tx=0).stop 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.640+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c3c0cc470 msgr2=0x7f6c3c1f12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.640+0000 7f6c3be00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c3c0cc470 0x7f6c3c1f12f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c033000 tx=0x7f6c2c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.641+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 shutdown_connections 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.641+0000 7f6c3be00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c3c155530 0x7f6c3c15d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.641+0000 7f6c3be00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c3c0cdfc0 0x7f6c3c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.641+0000 7f6c3be00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c1c06c8f0 0x7f6c1c06edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.641+0000 7f6c3be00640 1 --2- 172.21.15.102:0/858366775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c3c0cc470 0x7f6c3c1f12f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.641+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 >> 172.21.15.102:0/858366775 conn(0x7f6c3c075080 msgr2=0x7f6c3c0ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:20.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.641+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 shutdown_connections 2024-04-20T21:28:20.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.641+0000 7f6c3be00640 1 -- 172.21.15.102:0/858366775 wait complete. 2024-04-20T21:28:20.658 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-04-20T21:28:20.658 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:28:20.658 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:28:20.695 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:28:20.695 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:20.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.941+0000 7fe643200640 1 Processor -- start 2024-04-20T21:28:20.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.942+0000 7fe643200640 1 -- start start 2024-04-20T21:28:20.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe643200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe63c15ea20 0x7fe63c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe643200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe63c15ee00 0x7fe63c0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe643200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15f2b0 0x7fe63c0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe643200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe63c0c81e0 con 0x7fe63c15ea20 2024-04-20T21:28:20.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe643200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fe63c0c8f20 con 0x7fe63c15ee00 2024-04-20T21:28:20.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe643200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe63c06cb80 con 0x7fe63c15f2b0 2024-04-20T21:28:20.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe63be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe63c15ea20 0x7fe63c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe63be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe63c15ea20 0x7fe63c06d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42364/0 (socket says 172.21.15.102:42364) 2024-04-20T21:28:20.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe63be00640 1 -- 172.21.15.102:0/2491306745 learned_addr learned my addr 172.21.15.102:0/2491306745 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:20.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe641200640 1 --2- 172.21.15.102:0/2491306745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15f2b0 0x7fe63c0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.943+0000 7fe63b400640 1 --2- 172.21.15.102:0/2491306745 >> v2:172.21.15.193:3300/0 conn(0x7fe63c15ee00 0x7fe63c0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.944+0000 7fe63be00640 1 -- 172.21.15.102:0/2491306745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15f2b0 msgr2=0x7fe63c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:20.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.944+0000 7fe63be00640 1 --2- 172.21.15.102:0/2491306745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15f2b0 0x7fe63c0c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.944+0000 7fe63be00640 1 -- 172.21.15.102:0/2491306745 >> v2:172.21.15.193:3300/0 conn(0x7fe63c15ee00 msgr2=0x7fe63c0710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:20.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.944+0000 7fe63be00640 1 --2- 172.21.15.102:0/2491306745 >> v2:172.21.15.193:3300/0 conn(0x7fe63c15ee00 0x7fe63c0710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.944+0000 7fe63be00640 1 -- 172.21.15.102:0/2491306745 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c06ce60 con 0x7fe63c15ea20 2024-04-20T21:28:20.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.945+0000 7fe63be00640 1 --2- 172.21.15.102:0/2491306745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15ea20 0x7fe63c06d940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe63c06c2f0 tx=0x7fe62c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180e312cb7449856 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:20.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.945+0000 7fe63aa00640 1 -- 172.21.15.102:0/2491306745 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c034070 con 0x7fe63c15ea20 2024-04-20T21:28:20.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.945+0000 7fe63aa00640 1 -- 172.21.15.102:0/2491306745 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c030850 con 0x7fe63c15ea20 2024-04-20T21:28:20.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.945+0000 7fe63aa00640 1 -- 172.21.15.102:0/2491306745 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c030c20 con 0x7fe63c15ea20 2024-04-20T21:28:20.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 -- 172.21.15.102:0/2491306745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15ea20 msgr2=0x7fe63c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 --2- 172.21.15.102:0/2491306745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15ea20 0x7fe63c06d940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe63c06c2f0 tx=0x7fe62c02b990 comp rx=0 tx=0).stop 2024-04-20T21:28:20.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 -- 172.21.15.102:0/2491306745 shutdown_connections 2024-04-20T21:28:20.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 --2- 172.21.15.102:0/2491306745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15f2b0 0x7fe63c0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 --2- 172.21.15.102:0/2491306745 >> v2:172.21.15.193:3300/0 conn(0x7fe63c15ee00 0x7fe63c0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 --2- 172.21.15.102:0/2491306745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15ea20 0x7fe63c06d940 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 -- 172.21.15.102:0/2491306745 >> 172.21.15.102:0/2491306745 conn(0x7fe63c0c9390 msgr2=0x7fe63c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:20.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 -- 172.21.15.102:0/2491306745 shutdown_connections 2024-04-20T21:28:20.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.946+0000 7fe643200640 1 -- 172.21.15.102:0/2491306745 wait complete. 2024-04-20T21:28:20.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.947+0000 7fe643200640 1 Processor -- start 2024-04-20T21:28:20.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.948+0000 7fe643200640 1 -- start start 2024-04-20T21:28:20.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.948+0000 7fe643200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe63c15ea20 0x7fe63c147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.948+0000 7fe643200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15ee00 0x7fe63c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.948+0000 7fe643200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15f2b0 0x7fe63c1f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.948+0000 7fe643200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c0c8b00 con 0x7fe63c15f2b0 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe643200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c06d420 con 0x7fe63c15ea20 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe643200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe63c0c8c80 con 0x7fe63c15ee00 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe63c15ea20 0x7fe63c147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63b400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15ee00 0x7fe63c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe63c15ea20 0x7fe63c147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:55818/0 (socket says 172.21.15.102:55818) 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63b400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15ee00 0x7fe63c071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:40164/0 (socket says 172.21.15.102:40164) 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63be00640 1 -- 172.21.15.102:0/351993997 learned_addr learned my addr 172.21.15.102:0/351993997 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe641200640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15f2b0 0x7fe63c1f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63b400640 1 -- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe63c15ea20 msgr2=0x7fe63c147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63b400640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe63c15ea20 0x7fe63c147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63b400640 1 -- 172.21.15.102:0/351993997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15f2b0 msgr2=0x7fe63c1f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:20.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63b400640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15f2b0 0x7fe63c1f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:20.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63b400640 1 -- 172.21.15.102:0/351993997 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c1f16d0 con 0x7fe63c15ee00 2024-04-20T21:28:20.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.949+0000 7fe63be00640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe63c15ea20 0x7fe63c147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:20.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.950+0000 7fe63b400640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15ee00 0x7fe63c071fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe62400e870 tx=0x7fe62400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:20.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.950+0000 7fe633e00640 1 -- 172.21.15.102:0/351993997 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624017070 con 0x7fe63c15ee00 2024-04-20T21:28:20.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.950+0000 7fe633e00640 1 -- 172.21.15.102:0/351993997 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62400c940 con 0x7fe63c15ee00 2024-04-20T21:28:20.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.950+0000 7fe633e00640 1 -- 172.21.15.102:0/351993997 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62400ccf0 con 0x7fe63c15ee00 2024-04-20T21:28:20.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.950+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63c145fd0 con 0x7fe63c15ee00 2024-04-20T21:28:20.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.950+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63c0c65b0 con 0x7fe63c15ee00 2024-04-20T21:28:20.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.951+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63c074340 con 0x7fe63c15ee00 2024-04-20T21:28:20.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.952+0000 7fe633e00640 1 -- 172.21.15.102:0/351993997 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe62401a070 con 0x7fe63c15ee00 2024-04-20T21:28:20.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.953+0000 7fe633e00640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe61806c8f0 0x7fe61806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:20.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.953+0000 7fe633e00640 1 -- 172.21.15.102:0/351993997 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fe62408bfb0 con 0x7fe63c15ee00 2024-04-20T21:28:20.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.953+0000 7fe63be00640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe61806c8f0 0x7fe61806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:20.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.954+0000 7fe63be00640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe61806c8f0 0x7fe61806edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009630 tx=0x7fe62c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:20.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:20.958+0000 7fe633e00640 1 -- 172.21.15.102:0/351993997 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe624059740 con 0x7fe63c15ee00 2024-04-20T21:28:21.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.415+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe63c150280 con 0x7fe63c15ee00 2024-04-20T21:28:21.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.419+0000 7fe633e00640 1 -- 172.21.15.102:0/351993997 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6856 (secure 0 0 0) 0x7fe62405da40 con 0x7fe63c15ee00 2024-04-20T21:28:21.422 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:21.422 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5659,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:28:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-04-20T21:28:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe61806c8f0 msgr2=0x7fe61806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe61806c8f0 0x7fe61806edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009630 tx=0x7fe62c008030 comp rx=0 tx=0).stop 2024-04-20T21:28:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15ee00 msgr2=0x7fe63c071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:21.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15ee00 0x7fe63c071fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe62400e870 tx=0x7fe62400ed30 comp rx=0 tx=0).stop 2024-04-20T21:28:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 shutdown_connections 2024-04-20T21:28:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe63c15f2b0 0x7fe63c1f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe63c15ee00 0x7fe63c071fc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe61806c8f0 0x7fe61806edb0 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009630 tx=0x7fe62c008030 comp rx=0 tx=0).stop 2024-04-20T21:28:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 --2- 172.21.15.102:0/351993997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe63c15ea20 0x7fe63c147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.424+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 >> 172.21.15.102:0/351993997 conn(0x7fe63c0c9390 msgr2=0x7fe63c0c6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.427+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 shutdown_connections 2024-04-20T21:28:21.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:21.427+0000 7fe643200640 1 -- 172.21.15.102:0/351993997 wait complete. 2024-04-20T21:28:21.485 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:28:23.486 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:23.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.696+0000 7f17da000640 1 Processor -- start 2024-04-20T21:28:23.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.696+0000 7f17da000640 1 -- start start 2024-04-20T21:28:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.697+0000 7f17da000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f17d4155530 0x7f17d415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17da000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d40cb6a0 0x7f17d40cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17da000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d415ef60 con 0x7f17d40cc470 2024-04-20T21:28:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17da000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f17d40c5ca0 con 0x7f17d4155530 2024-04-20T21:28:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17da000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f17d40cd5f0 con 0x7f17d40cb6a0 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17d8c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f17d4155530 0x7f17d415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17d8c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f17d4155530 0x7f17d415e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42380/0 (socket says 172.21.15.102:42380) 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17d8c00640 1 -- 172.21.15.102:0/2326921711 learned_addr learned my addr 172.21.15.102:0/2326921711 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17d9600640 1 --2- 172.21.15.102:0/2326921711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d40cb6a0 0x7f17d40cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17d8c00640 1 -- 172.21.15.102:0/2326921711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d40cb6a0 msgr2=0x7f17d40cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17d8c00640 1 --2- 172.21.15.102:0/2326921711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d40cb6a0 0x7f17d40cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17d8c00640 1 -- 172.21.15.102:0/2326921711 >> v1:172.21.15.102:6789/0 conn(0x7f17d40cc470 legacy=0x7f17d40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.698+0000 7f17d8c00640 1 -- 172.21.15.102:0/2326921711 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d415f240 con 0x7f17d4155530 2024-04-20T21:28:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17d8c00640 1 --2- 172.21.15.102:0/2326921711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d4155530 0x7f17d415e640 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009870 tx=0x7f17c402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fefa21ebb4bf8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:23.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17d3e00640 1 -- 172.21.15.102:0/2326921711 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4034070 con 0x7f17d4155530 2024-04-20T21:28:23.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17d3e00640 1 -- 172.21.15.102:0/2326921711 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4030840 con 0x7f17d4155530 2024-04-20T21:28:23.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17d3e00640 1 -- 172.21.15.102:0/2326921711 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4030c10 con 0x7f17d4155530 2024-04-20T21:28:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17da000640 1 -- 172.21.15.102:0/2326921711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d4155530 msgr2=0x7f17d415e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17da000640 1 --2- 172.21.15.102:0/2326921711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d4155530 0x7f17d415e640 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009870 tx=0x7f17c402b980 comp rx=0 tx=0).stop 2024-04-20T21:28:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17da000640 1 -- 172.21.15.102:0/2326921711 shutdown_connections 2024-04-20T21:28:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17da000640 1 --2- 172.21.15.102:0/2326921711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d40cb6a0 0x7f17d40cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17da000640 1 --2- 172.21.15.102:0/2326921711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d4155530 0x7f17d415e640 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.700+0000 7f17da000640 1 -- 172.21.15.102:0/2326921711 >> 172.21.15.102:0/2326921711 conn(0x7f17d4075080 msgr2=0x7f17d4075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.701+0000 7f17da000640 1 -- 172.21.15.102:0/2326921711 shutdown_connections 2024-04-20T21:28:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.701+0000 7f17da000640 1 -- 172.21.15.102:0/2326921711 wait complete. 2024-04-20T21:28:23.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.702+0000 7f17da000640 1 Processor -- start 2024-04-20T21:28:23.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.702+0000 7f17da000640 1 -- start start 2024-04-20T21:28:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17da000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d40cb6a0 0x7f17d414b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17da000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d40cc470 0x7f17d415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17da000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d4155530 0x7f17d415d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17da000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f17d40cd280 con 0x7f17d40cb6a0 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17da000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17d40ccfa0 con 0x7f17d40cc470 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17da000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f17d40ccbe0 con 0x7f17d4155530 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17d8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d40cb6a0 0x7f17d414b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17d8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d40cb6a0 0x7f17d414b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42390/0 (socket says 172.21.15.102:42390) 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.703+0000 7f17d8c00640 1 -- 172.21.15.102:0/1302920800 learned_addr learned my addr 172.21.15.102:0/1302920800 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d9600640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d4155530 0x7f17d415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17cbe00640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d40cc470 0x7f17d415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d8c00640 1 -- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d4155530 msgr2=0x7f17d415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d8c00640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d4155530 0x7f17d415d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d8c00640 1 -- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d40cc470 msgr2=0x7f17d415cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d8c00640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d40cc470 0x7f17d415cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d8c00640 1 -- 172.21.15.102:0/1302920800 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d415c2a0 con 0x7f17d40cb6a0 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17cbe00640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d40cc470 0x7f17d415cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d8c00640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d40cb6a0 0x7f17d414b6e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f17c402b9c0 tx=0x7f17c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d1600640 1 -- 172.21.15.102:0/1302920800 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4034040 con 0x7f17d40cb6a0 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.704+0000 7f17d1600640 1 -- 172.21.15.102:0/1302920800 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4039d90 con 0x7f17d40cb6a0 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.705+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d415c780 con 0x7f17d40cb6a0 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.705+0000 7f17d1600640 1 -- 172.21.15.102:0/1302920800 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4043630 con 0x7f17d40cb6a0 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.705+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41f1a90 con 0x7f17d40cb6a0 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.706+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d404fd60 con 0x7f17d40cb6a0 2024-04-20T21:28:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.706+0000 7f17d1600640 1 -- 172.21.15.102:0/1302920800 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17c404a060 con 0x7f17d40cb6a0 2024-04-20T21:28:23.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.707+0000 7f17d1600640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17b006c8f0 0x7f17b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:23.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.708+0000 7f17d1600640 1 -- 172.21.15.102:0/1302920800 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f17c40ae190 con 0x7f17d40cb6a0 2024-04-20T21:28:23.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.708+0000 7f17cbe00640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17b006c8f0 0x7f17b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:23.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.708+0000 7f17cbe00640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17b006c8f0 0x7f17b006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17bc007fd0 tx=0x7f17bc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.712+0000 7f17d1600640 1 -- 172.21.15.102:0/1302920800 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f17c403e070 con 0x7f17d40cb6a0 2024-04-20T21:28:23.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.935+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17d40cc9c0 con 0x7f17d40cb6a0 2024-04-20T21:28:23.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.936+0000 7f17d1600640 1 -- 172.21.15.102:0/1302920800 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6856 (secure 0 0 0) 0x7f17c407bee0 con 0x7f17d40cb6a0 2024-04-20T21:28:23.936 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:23.936 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5659,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:28:23.936 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17b006c8f0 msgr2=0x7f17b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17b006c8f0 0x7f17b006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17bc007fd0 tx=0x7f17bc009040 comp rx=0 tx=0).stop 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d40cb6a0 msgr2=0x7f17d414b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d40cb6a0 0x7f17d414b6e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f17c402b9c0 tx=0x7f17c40046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 shutdown_connections 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f17d4155530 0x7f17d415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f17b006c8f0 0x7f17b006edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d40cc470 0x7f17d415cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 --2- 172.21.15.102:0/1302920800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d40cb6a0 0x7f17d414b6e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 >> 172.21.15.102:0/1302920800 conn(0x7f17d4075080 msgr2=0x7f17d4152d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 shutdown_connections 2024-04-20T21:28:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:23.939+0000 7f17da000640 1 -- 172.21.15.102:0/1302920800 wait complete. 2024-04-20T21:28:23.957 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:28:25.957 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:26.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.171+0000 7f584f200640 1 Processor -- start 2024-04-20T21:28:26.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.171+0000 7f584f200640 1 -- start start 2024-04-20T21:28:26.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f584f200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f58500cc470 0x7f58500cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:26.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f584f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58500cdfc0 0x7f58500cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:26.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f584f200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585015ef60 con 0x7f5850155530 2024-04-20T21:28:26.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f584f200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f58500c5ca0 con 0x7f58500cc470 2024-04-20T21:28:26.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f584f200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f58500cd5f0 con 0x7f58500cdfc0 2024-04-20T21:28:26.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f584de00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5850155530 0x7f585015e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47930/0 (socket says 172.21.15.102:47930) 2024-04-20T21:28:26.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f584de00640 1 -- 172.21.15.102:0/4165953402 learned_addr learned my addr 172.21.15.102:0/4165953402 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:26.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f584e800640 1 --2- 172.21.15.102:0/4165953402 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58500cdfc0 0x7f58500cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:26.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.173+0000 7f5845400640 1 --2- 172.21.15.102:0/4165953402 >> v2:172.21.15.193:3300/0 conn(0x7f58500cc470 0x7f58500cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:26.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.174+0000 7f584e800640 1 -- 172.21.15.102:0/4165953402 >> v2:172.21.15.193:3300/0 conn(0x7f58500cc470 msgr2=0x7f58500cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:26.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.174+0000 7f584e800640 1 --2- 172.21.15.102:0/4165953402 >> v2:172.21.15.193:3300/0 conn(0x7f58500cc470 0x7f58500cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.174+0000 7f584e800640 1 -- 172.21.15.102:0/4165953402 >> v1:172.21.15.102:6789/0 conn(0x7f5850155530 legacy=0x7f585015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:26.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.174+0000 7f584e800640 1 -- 172.21.15.102:0/4165953402 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585015f240 con 0x7f58500cdfc0 2024-04-20T21:28:26.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.174+0000 7f5845400640 1 --2- 172.21.15.102:0/4165953402 >> v2:172.21.15.193:3300/0 conn(0x7f58500cc470 0x7f58500cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:26.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.174+0000 7f584d400640 1 -- 172.21.15.102:0/4165953402 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444137943 0 0) 0x7f585015ef60 con 0x7f5850155530 2024-04-20T21:28:26.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.175+0000 7f584e800640 1 --2- 172.21.15.102:0/4165953402 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58500cdfc0 0x7f58500cb6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5848009d90 tx=0x7f584802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91f83b9e388345 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:26.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.175+0000 7f584d400640 1 -- 172.21.15.102:0/4165953402 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848039070 con 0x7f58500cdfc0 2024-04-20T21:28:26.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.175+0000 7f584d400640 1 -- 172.21.15.102:0/4165953402 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584802cc60 con 0x7f58500cdfc0 2024-04-20T21:28:26.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.175+0000 7f584d400640 1 -- 172.21.15.102:0/4165953402 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848034660 con 0x7f58500cdfc0 2024-04-20T21:28:26.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.176+0000 7f584f200640 1 -- 172.21.15.102:0/4165953402 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58500cdfc0 msgr2=0x7f58500cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:26.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.176+0000 7f584f200640 1 --2- 172.21.15.102:0/4165953402 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58500cdfc0 0x7f58500cb6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5848009d90 tx=0x7f584802beb0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.176+0000 7f584f200640 1 -- 172.21.15.102:0/4165953402 shutdown_connections 2024-04-20T21:28:26.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.176+0000 7f584f200640 1 --2- 172.21.15.102:0/4165953402 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58500cdfc0 0x7f58500cb6a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.176+0000 7f584f200640 1 --2- 172.21.15.102:0/4165953402 >> v2:172.21.15.193:3300/0 conn(0x7f58500cc470 0x7f58500cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.176+0000 7f584f200640 1 -- 172.21.15.102:0/4165953402 >> 172.21.15.102:0/4165953402 conn(0x7f5850075080 msgr2=0x7f5850075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:26.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.176+0000 7f584f200640 1 -- 172.21.15.102:0/4165953402 shutdown_connections 2024-04-20T21:28:26.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.176+0000 7f584f200640 1 -- 172.21.15.102:0/4165953402 wait complete. 2024-04-20T21:28:26.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.178+0000 7f584f200640 1 Processor -- start 2024-04-20T21:28:26.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.179+0000 7f584f200640 1 -- start start 2024-04-20T21:28:26.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58500cc470 0x7f585014acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:26.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58500cdfc0 0x7f5850150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5850155530 0x7f585015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58500cd280 con 0x7f58500cc470 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58500ccfa0 con 0x7f58500cdfc0 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584f200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f58500ccbe0 con 0x7f5850155530 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58500cc470 0x7f585014acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58500cc470 0x7f585014acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42406/0 (socket says 172.21.15.102:42406) 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584de00640 1 -- 172.21.15.102:0/1170987384 learned_addr learned my addr 172.21.15.102:0/1170987384 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f584e800640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5850155530 0x7f585015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:26.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.180+0000 7f5845400640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58500cdfc0 0x7f5850150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:26.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f584de00640 1 -- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5850155530 msgr2=0x7f585015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:26.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f584de00640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5850155530 0x7f585015de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f584de00640 1 -- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58500cdfc0 msgr2=0x7f5850150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f584de00640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58500cdfc0 0x7f5850150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f584de00640 1 -- 172.21.15.102:0/1170987384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585015e810 con 0x7f58500cc470 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f584e800640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5850155530 0x7f585015de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f5845400640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58500cdfc0 0x7f5850150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f584de00640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58500cc470 0x7f585014acf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f584000c430 tx=0x7f584000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f5846a00640 1 -- 172.21.15.102:0/1170987384 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58400074e0 con 0x7f58500cc470 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f5846a00640 1 -- 172.21.15.102:0/1170987384 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58400047e0 con 0x7f58500cc470 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58501f49a0 con 0x7f58500cc470 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.181+0000 7f5846a00640 1 -- 172.21.15.102:0/1170987384 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840004bd0 con 0x7f58500cc470 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.182+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58501f4f60 con 0x7f58500cc470 2024-04-20T21:28:26.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.183+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58501547f0 con 0x7f58500cc470 2024-04-20T21:28:26.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.184+0000 7f5846a00640 1 -- 172.21.15.102:0/1170987384 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5840011070 con 0x7f58500cc470 2024-04-20T21:28:26.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.185+0000 7f5846a00640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f582c06c8f0 0x7f582c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:26.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.185+0000 7f5846a00640 1 -- 172.21.15.102:0/1170987384 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f584008bd70 con 0x7f58500cc470 2024-04-20T21:28:26.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.185+0000 7f5845400640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f582c06c8f0 0x7f582c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:26.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.186+0000 7f5845400640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f582c06c8f0 0x7f582c06edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f585015bd10 tx=0x7f5838009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:26.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.191+0000 7f5846a00640 1 -- 172.21.15.102:0/1170987384 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f58501547f0 con 0x7f58500cc470 2024-04-20T21:28:26.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.437+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f585004fd60 con 0x7f58500cc470 2024-04-20T21:28:26.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.439+0000 7f5846a00640 1 -- 172.21.15.102:0/1170987384 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6856 (secure 0 0 0) 0x7f5840059470 con 0x7f58500cc470 2024-04-20T21:28:26.438 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:26.438 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5659,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:28:26.439 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-04-20T21:28:26.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.441+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f582c06c8f0 msgr2=0x7f582c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:26.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.441+0000 7f584f200640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f582c06c8f0 0x7f582c06edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f585015bd10 tx=0x7f5838009190 comp rx=0 tx=0).stop 2024-04-20T21:28:26.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.441+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58500cc470 msgr2=0x7f585014acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:26.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.441+0000 7f584f200640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58500cc470 0x7f585014acf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f584000c430 tx=0x7f584000c8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.442+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 shutdown_connections 2024-04-20T21:28:26.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.442+0000 7f584f200640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5850155530 0x7f585015de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.442+0000 7f584f200640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f582c06c8f0 0x7f582c06edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.442+0000 7f584f200640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58500cdfc0 0x7f5850150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.442+0000 7f584f200640 1 --2- 172.21.15.102:0/1170987384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58500cc470 0x7f585014acf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:26.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.442+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 >> 172.21.15.102:0/1170987384 conn(0x7f5850075080 msgr2=0x7f5850153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:26.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.442+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 shutdown_connections 2024-04-20T21:28:26.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:26.442+0000 7f584f200640 1 -- 172.21.15.102:0/1170987384 wait complete. 2024-04-20T21:28:26.458 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:28:26.835 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:26.843 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:28.458 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:28.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.665+0000 7fcea6200640 1 Processor -- start 2024-04-20T21:28:28.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.665+0000 7fcea6200640 1 -- start start 2024-04-20T21:28:28.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea6200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:28.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea6200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcea00c3700 0x7fcea006e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:28.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea6200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcea015c220 con 0x7fcea006eb70 2024-04-20T21:28:28.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea6200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fcea00c9180 con 0x7fcea00c3700 2024-04-20T21:28:28.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea6200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fcea00c4840 con 0x7fcea0152290 2024-04-20T21:28:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fce9fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcea00c3700 0x7fcea006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea5800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcea006eb70 0x7fcea00c8330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:39280/0 (socket says 172.21.15.102:39280) 2024-04-20T21:28:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea5800640 1 -- 172.21.15.102:0/3256771745 learned_addr learned my addr 172.21.15.102:0/3256771745 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea4e00640 1 --2- 172.21.15.102:0/3256771745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea4e00640 1 -- 172.21.15.102:0/3256771745 >> v2:172.21.15.102:3300/0 conn(0x7fcea00c3700 msgr2=0x7fcea006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea4e00640 1 --2- 172.21.15.102:0/3256771745 >> v2:172.21.15.102:3300/0 conn(0x7fcea00c3700 0x7fcea006e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fce9f400640 1 -- 172.21.15.102:0/3256771745 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188068426 0 0) 0x7fcea015c220 con 0x7fcea006eb70 2024-04-20T21:28:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea4e00640 1 -- 172.21.15.102:0/3256771745 >> v1:172.21.15.102:6789/0 conn(0x7fcea006eb70 legacy=0x7fcea00c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.667+0000 7fcea4e00640 1 -- 172.21.15.102:0/3256771745 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea00c4560 con 0x7fcea0152290 2024-04-20T21:28:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.668+0000 7fcea4e00640 1 --2- 172.21.15.102:0/3256771745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea00cd310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0097a0 tx=0x7fce8c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2a862466f2e660 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.668+0000 7fce9f400640 1 -- 172.21.15.102:0/3256771745 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c034070 con 0x7fcea0152290 2024-04-20T21:28:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.668+0000 7fce9f400640 1 -- 172.21.15.102:0/3256771745 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8c030800 con 0x7fcea0152290 2024-04-20T21:28:28.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.668+0000 7fce9f400640 1 -- 172.21.15.102:0/3256771745 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c030bd0 con 0x7fcea0152290 2024-04-20T21:28:28.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.669+0000 7fcea6200640 1 -- 172.21.15.102:0/3256771745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 msgr2=0x7fcea00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:28.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.669+0000 7fcea6200640 1 --2- 172.21.15.102:0/3256771745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea00cd310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0097a0 tx=0x7fce8c02b940 comp rx=0 tx=0).stop 2024-04-20T21:28:28.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.669+0000 7fcea6200640 1 -- 172.21.15.102:0/3256771745 shutdown_connections 2024-04-20T21:28:28.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.669+0000 7fcea6200640 1 --2- 172.21.15.102:0/3256771745 >> v2:172.21.15.102:3300/0 conn(0x7fcea00c3700 0x7fcea006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.669+0000 7fcea6200640 1 --2- 172.21.15.102:0/3256771745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea00cd310 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.669+0000 7fcea6200640 1 -- 172.21.15.102:0/3256771745 >> 172.21.15.102:0/3256771745 conn(0x7fcea00cc820 msgr2=0x7fcea00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:28.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.669+0000 7fcea6200640 1 -- 172.21.15.102:0/3256771745 shutdown_connections 2024-04-20T21:28:28.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.669+0000 7fcea6200640 1 -- 172.21.15.102:0/3256771745 wait complete. 2024-04-20T21:28:28.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.670+0000 7fcea6200640 1 Processor -- start 2024-04-20T21:28:28.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.670+0000 7fcea6200640 1 -- start start 2024-04-20T21:28:28.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.671+0000 7fcea6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcea006eb70 0x7fcea013db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:28.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.671+0000 7fcea6200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcea00c3700 0x7fcea00cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:28.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.671+0000 7fcea6200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea013c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:28.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.671+0000 7fcea6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcea015ba20 con 0x7fcea006eb70 2024-04-20T21:28:28.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.671+0000 7fcea6200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcea015b740 con 0x7fcea00c3700 2024-04-20T21:28:28.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.671+0000 7fcea6200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fcea015bd00 con 0x7fcea0152290 2024-04-20T21:28:28.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fcea4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcea006eb70 0x7fcea013db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fcea4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcea006eb70 0x7fcea013db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33102/0 (socket says 172.21.15.102:33102) 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fcea4e00640 1 -- 172.21.15.102:0/3070020349 learned_addr learned my addr 172.21.15.102:0/3070020349 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fce9fe00640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcea00c3700 0x7fcea00cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fcea5800640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea013c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fce9fe00640 1 -- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 msgr2=0x7fcea013c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fce9fe00640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea013c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fce9fe00640 1 -- 172.21.15.102:0/3070020349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcea006eb70 msgr2=0x7fcea013db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fce9fe00640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcea006eb70 0x7fcea013db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fce9fe00640 1 -- 172.21.15.102:0/3070020349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea01f50a0 con 0x7fcea00c3700 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fcea4e00640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcea006eb70 0x7fcea013db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.672+0000 7fce9fe00640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcea00c3700 0x7fcea00cbeb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce9000e880 tx=0x7fce9000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.673+0000 7fce9cc00640 1 -- 172.21.15.102:0/3070020349 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90017070 con 0x7fcea00c3700 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.673+0000 7fce9cc00640 1 -- 172.21.15.102:0/3070020349 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9000c940 con 0x7fcea00c3700 2024-04-20T21:28:28.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.673+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea01f5380 con 0x7fcea00c3700 2024-04-20T21:28:28.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.673+0000 7fce9cc00640 1 -- 172.21.15.102:0/3070020349 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9000cd30 con 0x7fcea00c3700 2024-04-20T21:28:28.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.673+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea01f5940 con 0x7fcea00c3700 2024-04-20T21:28:28.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.674+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea004fd60 con 0x7fcea00c3700 2024-04-20T21:28:28.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.675+0000 7fce9cc00640 1 -- 172.21.15.102:0/3070020349 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce9001a070 con 0x7fcea00c3700 2024-04-20T21:28:28.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.676+0000 7fce9cc00640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce8006c8f0 0x7fce8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:28.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.676+0000 7fce9cc00640 1 -- 172.21.15.102:0/3070020349 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fce9008c3f0 con 0x7fcea00c3700 2024-04-20T21:28:28.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.676+0000 7fcea4e00640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce8006c8f0 0x7fce8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:28.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.677+0000 7fcea4e00640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce8006c8f0 0x7fce8006edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009560 tx=0x7fce8c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:28.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.681+0000 7fce9cc00640 1 -- 172.21.15.102:0/3070020349 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fce90059b80 con 0x7fcea00c3700 2024-04-20T21:28:28.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.911+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcea01424e0 con 0x7fcea00c3700 2024-04-20T21:28:28.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.913+0000 7fce9cc00640 1 -- 172.21.15.102:0/3070020349 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6856 (secure 0 0 0) 0x7fce9005de80 con 0x7fcea00c3700 2024-04-20T21:28:28.911 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:28.911 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5659,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:28:28.912 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce8006c8f0 msgr2=0x7fce8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce8006c8f0 0x7fce8006edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009560 tx=0x7fce8c008030 comp rx=0 tx=0).stop 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcea00c3700 msgr2=0x7fcea00cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcea00c3700 0x7fcea00cbeb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce9000e880 tx=0x7fce9000ed40 comp rx=0 tx=0).stop 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 shutdown_connections 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcea0152290 0x7fcea013c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce8006c8f0 0x7fce8006edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcea00c3700 0x7fcea00cbeb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 --2- 172.21.15.102:0/3070020349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcea006eb70 0x7fcea013db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:28.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.915+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 >> 172.21.15.102:0/3070020349 conn(0x7fcea00cc820 msgr2=0x7fcea0137d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:28.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.916+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 shutdown_connections 2024-04-20T21:28:28.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:28.916+0000 7fcea6200640 1 -- 172.21.15.102:0/3070020349 wait complete. 2024-04-20T21:28:28.931 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:28:30.932 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:31.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.144+0000 7fb348e00640 1 Processor -- start 2024-04-20T21:28:31.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.144+0000 7fb348e00640 1 -- start start 2024-04-20T21:28:31.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.145+0000 7fb348e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb3440c75a0 0x7fb3440c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.145+0000 7fb348e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344073c70 0x7fb3440c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.145+0000 7fb348e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb344153560 con 0x7fb344153180 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.145+0000 7fb348e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb3440c7a70 con 0x7fb3440c75a0 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.145+0000 7fb348e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440c7d30 con 0x7fb344073c70 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb343400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb344153180 0x7fb344158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:39284/0 (socket says 172.21.15.102:39284) 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb343400640 1 -- 172.21.15.102:0/4023221169 learned_addr learned my addr 172.21.15.102:0/4023221169 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb33be00640 1 --2- 172.21.15.102:0/4023221169 >> v2:172.21.15.193:3300/0 conn(0x7fb3440c75a0 0x7fb3440c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb343e00640 1 --2- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344073c70 0x7fb3440c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb342a00640 1 -- 172.21.15.102:0/4023221169 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180831813 0 0) 0x7fb344153560 con 0x7fb344153180 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb342a00640 1 -- 172.21.15.102:0/4023221169 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb328003520 con 0x7fb344153180 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb33be00640 1 -- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344073c70 msgr2=0x7fb3440c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb33be00640 1 --2- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344073c70 0x7fb3440c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb33be00640 1 -- 172.21.15.102:0/4023221169 >> v1:172.21.15.102:6789/0 conn(0x7fb344153180 legacy=0x7fb344158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb33be00640 1 -- 172.21.15.102:0/4023221169 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34415c4d0 con 0x7fb3440c75a0 2024-04-20T21:28:31.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.146+0000 7fb343e00640 1 --2- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344073c70 0x7fb3440c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:31.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.147+0000 7fb33be00640 1 --2- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 0x7fb3440c5f30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0097a0 tx=0x7fb32c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ae6ceca628bd078 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb342a00640 1 -- 172.21.15.102:0/4023221169 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c035020 con 0x7fb3440c75a0 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb342a00640 1 -- 172.21.15.102:0/4023221169 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32c031810 con 0x7fb3440c75a0 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb342a00640 1 -- 172.21.15.102:0/4023221169 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c031bc0 con 0x7fb3440c75a0 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb348e00640 1 -- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 msgr2=0x7fb3440c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb348e00640 1 --2- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 0x7fb3440c5f30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0097a0 tx=0x7fb32c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb348e00640 1 -- 172.21.15.102:0/4023221169 shutdown_connections 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb348e00640 1 --2- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344073c70 0x7fb3440c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb348e00640 1 --2- 172.21.15.102:0/4023221169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 0x7fb3440c5f30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb348e00640 1 -- 172.21.15.102:0/4023221169 >> 172.21.15.102:0/4023221169 conn(0x7fb3440cc820 msgr2=0x7fb3440ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb348e00640 1 -- 172.21.15.102:0/4023221169 shutdown_connections 2024-04-20T21:28:31.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.148+0000 7fb348e00640 1 -- 172.21.15.102:0/4023221169 wait complete. 2024-04-20T21:28:31.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.150+0000 7fb348e00640 1 Processor -- start 2024-04-20T21:28:31.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.150+0000 7fb348e00640 1 -- start start 2024-04-20T21:28:31.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb348e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb344073c70 0x7fb3440cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb348e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 0x7fb3441e4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb348e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344153180 0x7fb344146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb348e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb344159550 con 0x7fb344073c70 2024-04-20T21:28:31.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb348e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb34415ca90 con 0x7fb3440c75a0 2024-04-20T21:28:31.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb348e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb34415d290 con 0x7fb344153180 2024-04-20T21:28:31.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb343400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb344073c70 0x7fb3440cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb33be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 0x7fb3441e4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb343400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb344073c70 0x7fb3440cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33104/0 (socket says 172.21.15.102:33104) 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb343400640 1 -- 172.21.15.102:0/853210853 learned_addr learned my addr 172.21.15.102:0/853210853 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.151+0000 7fb343e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344153180 0x7fb344146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33be00640 1 -- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344153180 msgr2=0x7fb344146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33be00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344153180 0x7fb344146960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33be00640 1 -- 172.21.15.102:0/853210853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb344073c70 msgr2=0x7fb3440cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33be00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb344073c70 0x7fb3440cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33be00640 1 -- 172.21.15.102:0/853210853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb344147aa0 con 0x7fb3440c75a0 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb343400640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb344073c70 0x7fb3440cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33be00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 0x7fb3441e4c20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb344070590 tx=0x7fb32c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:31.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33b400640 1 -- 172.21.15.102:0/853210853 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c035040 con 0x7fb3440c75a0 2024-04-20T21:28:31.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33b400640 1 -- 172.21.15.102:0/853210853 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32c039cf0 con 0x7fb3440c75a0 2024-04-20T21:28:31.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3440cbb20 con 0x7fb3440c75a0 2024-04-20T21:28:31.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.152+0000 7fb33b400640 1 -- 172.21.15.102:0/853210853 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c006e10 con 0x7fb3440c75a0 2024-04-20T21:28:31.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.153+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3441f13b0 con 0x7fb3440c75a0 2024-04-20T21:28:31.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.154+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34404fd60 con 0x7fb3440c75a0 2024-04-20T21:28:31.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.155+0000 7fb33b400640 1 -- 172.21.15.102:0/853210853 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb32c038020 con 0x7fb3440c75a0 2024-04-20T21:28:31.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.156+0000 7fb33b400640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c8f0 0x7fb32006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.156+0000 7fb33b400640 1 -- 172.21.15.102:0/853210853 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7fb32c0addd0 con 0x7fb3440c75a0 2024-04-20T21:28:31.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.156+0000 7fb343400640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c8f0 0x7fb32006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.157+0000 7fb343400640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c8f0 0x7fb32006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb334005880 tx=0x7fb334009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:31.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.160+0000 7fb33b400640 1 -- 172.21.15.102:0/853210853 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb32c07bc30 con 0x7fb3440c75a0 2024-04-20T21:28:31.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.401+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb34406ea90 con 0x7fb3440c75a0 2024-04-20T21:28:31.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.402+0000 7fb33b400640 1 -- 172.21.15.102:0/853210853 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6856 (secure 0 0 0) 0x7fb32c07b530 con 0x7fb3440c75a0 2024-04-20T21:28:31.401 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:31.401 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5659,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:28:31.402 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-04-20T21:28:31.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c8f0 msgr2=0x7fb32006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c8f0 0x7fb32006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb334005880 tx=0x7fb334009040 comp rx=0 tx=0).stop 2024-04-20T21:28:31.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 msgr2=0x7fb3441e4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 0x7fb3441e4c20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb344070590 tx=0x7fb32c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 shutdown_connections 2024-04-20T21:28:31.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb344153180 0x7fb344146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32006c8f0 0x7fb32006edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3440c75a0 0x7fb3441e4c20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 --2- 172.21.15.102:0/853210853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb344073c70 0x7fb3440cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 >> 172.21.15.102:0/853210853 conn(0x7fb3440cc820 msgr2=0x7fb344148210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:31.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 shutdown_connections 2024-04-20T21:28:31.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:31.405+0000 7fb348e00640 1 -- 172.21.15.102:0/853210853 wait complete. 2024-04-20T21:28:31.423 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:28:33.424 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:33.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.627+0000 7f10b0c00640 1 Processor -- start 2024-04-20T21:28:33.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.627+0000 7f10b0c00640 1 -- start start 2024-04-20T21:28:33.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.629+0000 7f10b0c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f10ac0cb980 0x7f10ac15e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:33.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.629+0000 7f10b0c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 0x7f10ac0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:33.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.629+0000 7f10b0c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10ac0ccdd0 con 0x7f10ac0cbd60 2024-04-20T21:28:33.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.629+0000 7f10b0c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f10ac15f7e0 con 0x7f10ac0cb980 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.629+0000 7f10b0c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f10ac15ef40 con 0x7f10ac0cdb30 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a9400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f10ac0cbd60 0x7f10ac0cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:39290/0 (socket says 172.21.15.102:39290) 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a9400640 1 -- 172.21.15.102:0/1453670598 learned_addr learned my addr 172.21.15.102:0/1453670598 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10aa800640 1 --2- 172.21.15.102:0/1453670598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 0x7f10ac0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a9e00640 1 --2- 172.21.15.102:0/1453670598 >> v2:172.21.15.193:3300/0 conn(0x7f10ac0cb980 0x7f10ac15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a8a00640 1 -- 172.21.15.102:0/1453670598 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1966537001 0 0) 0x7f10ac0ccdd0 con 0x7f10ac0cbd60 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a8a00640 1 -- 172.21.15.102:0/1453670598 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1090003520 con 0x7f10ac0cbd60 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a9e00640 1 -- 172.21.15.102:0/1453670598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 msgr2=0x7f10ac0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a9e00640 1 --2- 172.21.15.102:0/1453670598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 0x7f10ac0cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a9e00640 1 -- 172.21.15.102:0/1453670598 >> v1:172.21.15.102:6789/0 conn(0x7f10ac0cbd60 legacy=0x7f10ac0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.630+0000 7f10a9e00640 1 -- 172.21.15.102:0/1453670598 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac15f220 con 0x7f10ac0cb980 2024-04-20T21:28:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.631+0000 7f10a9e00640 1 --2- 172.21.15.102:0/1453670598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 0x7f10ac15e890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f109c0097a0 tx=0x7f109c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95a41688392d125 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.632+0000 7f10a8a00640 1 -- 172.21.15.102:0/1453670598 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c035020 con 0x7f10ac0cb980 2024-04-20T21:28:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.632+0000 7f10a8a00640 1 -- 172.21.15.102:0/1453670598 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109c031810 con 0x7f10ac0cb980 2024-04-20T21:28:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.632+0000 7f10a8a00640 1 -- 172.21.15.102:0/1453670598 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c031be0 con 0x7f10ac0cb980 2024-04-20T21:28:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.632+0000 7f10b0c00640 1 -- 172.21.15.102:0/1453670598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 msgr2=0x7f10ac15e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.632+0000 7f10b0c00640 1 --2- 172.21.15.102:0/1453670598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 0x7f10ac15e890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f109c0097a0 tx=0x7f109c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.633+0000 7f10b0c00640 1 -- 172.21.15.102:0/1453670598 shutdown_connections 2024-04-20T21:28:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.633+0000 7f10b0c00640 1 --2- 172.21.15.102:0/1453670598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 0x7f10ac0cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.633+0000 7f10b0c00640 1 --2- 172.21.15.102:0/1453670598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 0x7f10ac15e890 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.633+0000 7f10b0c00640 1 -- 172.21.15.102:0/1453670598 >> 172.21.15.102:0/1453670598 conn(0x7f10ac075080 msgr2=0x7f10ac075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.633+0000 7f10b0c00640 1 -- 172.21.15.102:0/1453670598 shutdown_connections 2024-04-20T21:28:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.633+0000 7f10b0c00640 1 -- 172.21.15.102:0/1453670598 wait complete. 2024-04-20T21:28:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.634+0000 7f10b0c00640 1 Processor -- start 2024-04-20T21:28:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.635+0000 7f10b0c00640 1 -- start start 2024-04-20T21:28:33.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.635+0000 7f10b0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 0x7f10ac15e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:33.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.635+0000 7f10b0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10ac0cbd60 0x7f10ac0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:33.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.635+0000 7f10b0c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 0x7f10ac1f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:33.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10b0c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0cd610 con 0x7f10ac0cbd60 2024-04-20T21:28:33.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10b0c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0cd330 con 0x7f10ac0cb980 2024-04-20T21:28:33.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10b0c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f10ac0cc1d0 con 0x7f10ac0cdb30 2024-04-20T21:28:33.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10ac0cbd60 0x7f10ac0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10ac0cbd60 0x7f10ac0b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33108/0 (socket says 172.21.15.102:33108) 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9400640 1 -- 172.21.15.102:0/596999099 learned_addr learned my addr 172.21.15.102:0/596999099 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9e00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 0x7f10ac15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10aa800640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 0x7f10ac1f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9400640 1 -- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 msgr2=0x7f10ac1f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9400640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 0x7f10ac1f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9400640 1 -- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 msgr2=0x7f10ac15e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9400640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 0x7f10ac15e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.636+0000 7f10a9400640 1 -- 172.21.15.102:0/596999099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac15eb70 con 0x7f10ac0cbd60 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.637+0000 7f10a9400640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10ac0cbd60 0x7f10ac0b4e70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f109800c430 tx=0x7f109800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.637+0000 7f10a2000640 1 -- 172.21.15.102:0/596999099 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098017070 con 0x7f10ac0cbd60 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.637+0000 7f10a2000640 1 -- 172.21.15.102:0/596999099 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1098005ce0 con 0x7f10ac0cbd60 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.637+0000 7f10a2000640 1 -- 172.21.15.102:0/596999099 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10980072f0 con 0x7f10ac0cbd60 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.637+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac157300 con 0x7f10ac0cbd60 2024-04-20T21:28:33.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.637+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac15db50 con 0x7f10ac0cbd60 2024-04-20T21:28:33.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.638+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ac0c7830 con 0x7f10ac0cbd60 2024-04-20T21:28:33.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.639+0000 7f10a2000640 1 -- 172.21.15.102:0/596999099 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f109801a070 con 0x7f10ac0cbd60 2024-04-20T21:28:33.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.640+0000 7f10a2000640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f108c06c8f0 0x7f108c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:33.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.640+0000 7f10a2000640 1 -- 172.21.15.102:0/596999099 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7165+0+0 (secure 0 0 0) 0x7f109808b4a0 con 0x7f10ac0cbd60 2024-04-20T21:28:33.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.640+0000 7f10a9e00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f108c06c8f0 0x7f108c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:33.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.641+0000 7f10a9e00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f108c06c8f0 0x7f108c06edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f109c036000 tx=0x7f109c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:33.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.646+0000 7f10a2000640 1 -- 172.21.15.102:0/596999099 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1098058d40 con 0x7f10ac0cbd60 2024-04-20T21:28:33.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.909+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10ac0cc4a0 con 0x7f10ac0cbd60 2024-04-20T21:28:33.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.911+0000 7f10a2000640 1 -- 172.21.15.102:0/596999099 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6856 (secure 0 0 0) 0x7f109805d040 con 0x7f10ac0cbd60 2024-04-20T21:28:33.910 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:33.911 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5659,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:26:30.825500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5533":{"gid":5533,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":41,"addr":"172.21.15.102:6833/2239658043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2239658043},{"type":"v1","addr":"172.21.15.102:6833","nonce":2239658043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5533,"qdb_cluster":[5533]},"id":1}]} 2024-04-20T21:28:33.911 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f108c06c8f0 msgr2=0x7f108c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f108c06c8f0 0x7f108c06edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f109c036000 tx=0x7f109c008030 comp rx=0 tx=0).stop 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10ac0cbd60 msgr2=0x7f10ac0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10ac0cbd60 0x7f10ac0b4e70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f109800c430 tx=0x7f109800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 shutdown_connections 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10ac0cdb30 0x7f10ac1f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10ac0cbd60 0x7f10ac0b4e70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f108c06c8f0 0x7f108c06edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 --2- 172.21.15.102:0/596999099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10ac0cb980 0x7f10ac15e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.913+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 >> 172.21.15.102:0/596999099 conn(0x7f10ac075080 msgr2=0x7f10ac150350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:33.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.914+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 shutdown_connections 2024-04-20T21:28:33.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:33.914+0000 7f10b0c00640 1 -- 172.21.15.102:0/596999099 wait complete. 2024-04-20T21:28:33.931 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:28:35.931 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:36.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.140+0000 7ff844a00640 1 Processor -- start 2024-04-20T21:28:36.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.140+0000 7ff844a00640 1 -- start start 2024-04-20T21:28:36.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff844a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff840155110 0x7ff84006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff844a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff84006f3d0 0x7ff84015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff844a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff84015f3c0 con 0x7ff8400c4040 2024-04-20T21:28:36.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff844a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff8400c7c30 con 0x7ff840155110 2024-04-20T21:28:36.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff844a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff84006ea00 con 0x7ff84006f3d0 2024-04-20T21:28:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff837e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff84006f3d0 0x7ff84015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff83e800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff840155110 0x7ff84006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff83f200640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff8400c4040 0x7ff84015eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43310/0 (socket says 172.21.15.102:43310) 2024-04-20T21:28:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.142+0000 7ff83f200640 1 -- 172.21.15.102:0/3699344955 learned_addr learned my addr 172.21.15.102:0/3699344955 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.143+0000 7ff83de00640 1 -- 172.21.15.102:0/3699344955 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269955432 0 0) 0x7ff84015f3c0 con 0x7ff8400c4040 2024-04-20T21:28:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.143+0000 7ff83e800640 1 -- 172.21.15.102:0/3699344955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff84006f3d0 msgr2=0x7ff84015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.143+0000 7ff83e800640 1 --2- 172.21.15.102:0/3699344955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff84006f3d0 0x7ff84015e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.143+0000 7ff83e800640 1 -- 172.21.15.102:0/3699344955 >> v1:172.21.15.193:6789/0 conn(0x7ff8400c4040 legacy=0x7ff84015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.143+0000 7ff83e800640 1 -- 172.21.15.102:0/3699344955 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84006e6d0 con 0x7ff840155110 2024-04-20T21:28:36.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.143+0000 7ff837e00640 1 --2- 172.21.15.102:0/3699344955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff84006f3d0 0x7ff84015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:36.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.144+0000 7ff83e800640 1 --2- 172.21.15.102:0/3699344955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff840155110 0x7ff84006ee90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8400c7db0 tx=0x7ff83002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68cfd714a7255d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:36.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.144+0000 7ff83de00640 1 -- 172.21.15.102:0/3699344955 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830034070 con 0x7ff840155110 2024-04-20T21:28:36.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.144+0000 7ff83de00640 1 -- 172.21.15.102:0/3699344955 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff830030800 con 0x7ff840155110 2024-04-20T21:28:36.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.144+0000 7ff83de00640 1 -- 172.21.15.102:0/3699344955 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830030bd0 con 0x7ff840155110 2024-04-20T21:28:36.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.145+0000 7ff844a00640 1 -- 172.21.15.102:0/3699344955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff840155110 msgr2=0x7ff84006ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.145+0000 7ff844a00640 1 --2- 172.21.15.102:0/3699344955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff840155110 0x7ff84006ee90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8400c7db0 tx=0x7ff83002b940 comp rx=0 tx=0).stop 2024-04-20T21:28:36.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.145+0000 7ff844a00640 1 -- 172.21.15.102:0/3699344955 shutdown_connections 2024-04-20T21:28:36.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.145+0000 7ff844a00640 1 --2- 172.21.15.102:0/3699344955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff84006f3d0 0x7ff84015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.145+0000 7ff844a00640 1 --2- 172.21.15.102:0/3699344955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff840155110 0x7ff84006ee90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.145+0000 7ff844a00640 1 -- 172.21.15.102:0/3699344955 >> 172.21.15.102:0/3699344955 conn(0x7ff8400cc820 msgr2=0x7ff8400ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:36.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.145+0000 7ff844a00640 1 -- 172.21.15.102:0/3699344955 shutdown_connections 2024-04-20T21:28:36.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.145+0000 7ff844a00640 1 -- 172.21.15.102:0/3699344955 wait complete. 2024-04-20T21:28:36.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.146+0000 7ff844a00640 1 Processor -- start 2024-04-20T21:28:36.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.147+0000 7ff844a00640 1 -- start start 2024-04-20T21:28:36.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.147+0000 7ff844a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff84006f3d0 0x7ff8401e4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.147+0000 7ff844a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8400c4040 0x7ff840151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.147+0000 7ff844a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff840155110 0x7ff84015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff844a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401554f0 con 0x7ff84006f3d0 2024-04-20T21:28:36.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff844a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400c8160 con 0x7ff8400c4040 2024-04-20T21:28:36.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff844a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff840155670 con 0x7ff840155110 2024-04-20T21:28:36.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff837e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8400c4040 0x7ff840151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff837e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8400c4040 0x7ff840151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:33536/0 (socket says 172.21.15.102:33536) 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff837e00640 1 -- 172.21.15.102:0/3459433835 learned_addr learned my addr 172.21.15.102:0/3459433835 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff83f200640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff840155110 0x7ff84015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff83e800640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff84006f3d0 0x7ff8401e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff837e00640 1 -- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff840155110 msgr2=0x7ff84015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff837e00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff840155110 0x7ff84015c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff837e00640 1 -- 172.21.15.102:0/3459433835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff84006f3d0 msgr2=0x7ff8401e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff837e00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff84006f3d0 0x7ff8401e4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.148+0000 7ff837e00640 1 -- 172.21.15.102:0/3459433835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8401e48b0 con 0x7ff8400c4040 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.149+0000 7ff837e00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8400c4040 0x7ff840151d60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff82800ec10 tx=0x7ff82800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.149+0000 7ff836a00640 1 -- 172.21.15.102:0/3459433835 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828017070 con 0x7ff8400c4040 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.149+0000 7ff836a00640 1 -- 172.21.15.102:0/3459433835 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82800cc30 con 0x7ff8400c4040 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.149+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8401f1120 con 0x7ff8400c4040 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.149+0000 7ff836a00640 1 -- 172.21.15.102:0/3459433835 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828013470 con 0x7ff8400c4040 2024-04-20T21:28:36.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.149+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401f16e0 con 0x7ff8400c4040 2024-04-20T21:28:36.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.150+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84004fd60 con 0x7ff8400c4040 2024-04-20T21:28:36.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.151+0000 7ff836a00640 1 -- 172.21.15.102:0/3459433835 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff82801a070 con 0x7ff8400c4040 2024-04-20T21:28:36.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.152+0000 7ff836a00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.152+0000 7ff836a00640 1 -- 172.21.15.102:0/3459433835 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7ff82808c0f0 con 0x7ff8400c4040 2024-04-20T21:28:36.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.152+0000 7ff83e800640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.153+0000 7ff83e800640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff81c06c8f0 0x7ff81c06edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff830036000 tx=0x7ff830008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:36.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.157+0000 7ff836a00640 1 -- 172.21.15.102:0/3459433835 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff828059790 con 0x7ff8400c4040 2024-04-20T21:28:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.397+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8400c83e0 con 0x7ff8400c4040 2024-04-20T21:28:36.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.398+0000 7ff836a00640 1 -- 172.21.15.102:0/3459433835 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6020 (secure 0 0 0) 0x7ff82805da90 con 0x7ff8400c4040 2024-04-20T21:28:36.397 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:36.397 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:35.626253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:36.398 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 38 2024-04-20T21:28:36.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.400+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff81c06c8f0 msgr2=0x7ff81c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.400+0000 7ff844a00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff81c06c8f0 0x7ff81c06edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff830036000 tx=0x7ff830008030 comp rx=0 tx=0).stop 2024-04-20T21:28:36.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.400+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8400c4040 msgr2=0x7ff840151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.400+0000 7ff844a00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8400c4040 0x7ff840151d60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff82800ec10 tx=0x7ff82800c520 comp rx=0 tx=0).stop 2024-04-20T21:28:36.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.401+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 shutdown_connections 2024-04-20T21:28:36.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.401+0000 7ff844a00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff840155110 0x7ff84015c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.401+0000 7ff844a00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.401+0000 7ff844a00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8400c4040 0x7ff840151d60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.401+0000 7ff844a00640 1 --2- 172.21.15.102:0/3459433835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff84006f3d0 0x7ff8401e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.401+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 >> 172.21.15.102:0/3459433835 conn(0x7ff8400cc820 msgr2=0x7ff840067730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:36.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.401+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 shutdown_connections 2024-04-20T21:28:36.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.401+0000 7ff844a00640 1 -- 172.21.15.102:0/3459433835 wait complete. 2024-04-20T21:28:36.420 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-04-20T21:28:36.420 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:28:36.420 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:36.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.617+0000 7f49e6a00640 1 Processor -- start 2024-04-20T21:28:36.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.617+0000 7f49e6a00640 1 -- start start 2024-04-20T21:28:36.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.618+0000 7f49e6a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f49e00c5f30 0x7f49e00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.618+0000 7f49e6a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e006ef60 con 0x7f49e0151ab0 2024-04-20T21:28:36.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.618+0000 7f49e6a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e006f240 con 0x7f49e01516d0 2024-04-20T21:28:36.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.618+0000 7f49e6a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f49e00c4dd0 con 0x7f49e00c5f30 2024-04-20T21:28:36.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.618+0000 7f49e4c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f49e0151ab0 0x7f49e00c5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:39300/0 (socket says 172.21.15.102:39300) 2024-04-20T21:28:36.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.618+0000 7f49e4c00640 1 -- 172.21.15.102:0/3769902346 learned_addr learned my addr 172.21.15.102:0/3769902346 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:36.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.619+0000 7f49e6000640 1 --2- 172.21.15.102:0/3769902346 >> v2:172.21.15.193:3300/0 conn(0x7f49e00c5f30 0x7f49e00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.619+0000 7f49dfe00640 1 -- 172.21.15.102:0/3769902346 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190411928 0 0) 0x7f49e006ef60 con 0x7f49e0151ab0 2024-04-20T21:28:36.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.619+0000 7f49dfe00640 1 -- 172.21.15.102:0/3769902346 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49c8003520 con 0x7f49e0151ab0 2024-04-20T21:28:36.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.619+0000 7f49e6000640 1 -- 172.21.15.102:0/3769902346 >> v1:172.21.15.193:6789/0 conn(0x7f49e01516d0 legacy=0x7f49e00c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.619+0000 7f49dfe00640 1 -- 172.21.15.102:0/3769902346 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772211866 0 0) 0x7f49e006f240 con 0x7f49e01516d0 2024-04-20T21:28:36.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.619+0000 7f49e6000640 1 -- 172.21.15.102:0/3769902346 >> v1:172.21.15.102:6789/0 conn(0x7f49e0151ab0 legacy=0x7f49e00c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.619+0000 7f49e6000640 1 -- 172.21.15.102:0/3769902346 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e006f520 con 0x7f49e00c5f30 2024-04-20T21:28:36.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.620+0000 7f49e6000640 1 --2- 172.21.15.102:0/3769902346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 0x7f49e00cd310 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f49e00c4f50 tx=0x7f49d802b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f853ccfc2a224759 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:36.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.620+0000 7f49dfe00640 1 -- 172.21.15.102:0/3769902346 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8034070 con 0x7f49e00c5f30 2024-04-20T21:28:36.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49dfe00640 1 -- 172.21.15.102:0/3769902346 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8030890 con 0x7f49e00c5f30 2024-04-20T21:28:36.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49dfe00640 1 -- 172.21.15.102:0/3769902346 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8030c40 con 0x7f49e00c5f30 2024-04-20T21:28:36.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49e6a00640 1 -- 172.21.15.102:0/3769902346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 msgr2=0x7f49e00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49e6a00640 1 --2- 172.21.15.102:0/3769902346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 0x7f49e00cd310 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f49e00c4f50 tx=0x7f49d802b9d0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49e6a00640 1 -- 172.21.15.102:0/3769902346 shutdown_connections 2024-04-20T21:28:36.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49e6a00640 1 --2- 172.21.15.102:0/3769902346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 0x7f49e00cd310 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49e6a00640 1 -- 172.21.15.102:0/3769902346 >> 172.21.15.102:0/3769902346 conn(0x7f49e00cc820 msgr2=0x7f49e00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:36.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49e6a00640 1 -- 172.21.15.102:0/3769902346 shutdown_connections 2024-04-20T21:28:36.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.621+0000 7f49e6a00640 1 -- 172.21.15.102:0/3769902346 wait complete. 2024-04-20T21:28:36.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.623+0000 7f49e6a00640 1 Processor -- start 2024-04-20T21:28:36.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.623+0000 7f49e6a00640 1 -- start start 2024-04-20T21:28:36.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 0x7f49e00cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49e01516d0 0x7f49e0072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49e0151ab0 0x7f49e014d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f49e00c4a40 con 0x7f49e0151ab0 2024-04-20T21:28:36.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49e00c5300 con 0x7f49e00c5f30 2024-04-20T21:28:36.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f49e00c4bc0 con 0x7f49e01516d0 2024-04-20T21:28:36.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49e0151ab0 0x7f49e014d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49e0151ab0 0x7f49e014d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33132/0 (socket says 172.21.15.102:33132) 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e6000640 1 -- 172.21.15.102:0/2752925840 learned_addr learned my addr 172.21.15.102:0/2752925840 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e4c00640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49e01516d0 0x7f49e0072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.624+0000 7f49e5600640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 0x7f49e00cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49e6000640 1 -- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49e01516d0 msgr2=0x7f49e0072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49e6000640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49e01516d0 0x7f49e0072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49e6000640 1 -- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 msgr2=0x7f49e00cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49e6000640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 0x7f49e00cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49e6000640 1 -- 172.21.15.102:0/2752925840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e0073070 con 0x7f49e0151ab0 2024-04-20T21:28:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49e6000640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49e0151ab0 0x7f49e014d200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009890 tx=0x7f49d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:36.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49dd600640 1 -- 172.21.15.102:0/2752925840 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8048070 con 0x7f49e0151ab0 2024-04-20T21:28:36.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49dd600640 1 -- 172.21.15.102:0/2752925840 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8030690 con 0x7f49e0151ab0 2024-04-20T21:28:36.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.625+0000 7f49dd600640 1 -- 172.21.15.102:0/2752925840 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8034040 con 0x7f49e0151ab0 2024-04-20T21:28:36.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.626+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e0072950 con 0x7f49e0151ab0 2024-04-20T21:28:36.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.626+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e00735f0 con 0x7f49e0151ab0 2024-04-20T21:28:36.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.627+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e004fd60 con 0x7f49e0151ab0 2024-04-20T21:28:36.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.628+0000 7f49dd600640 1 -- 172.21.15.102:0/2752925840 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49d804b070 con 0x7f49e0151ab0 2024-04-20T21:28:36.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.629+0000 7f49dd600640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f49c006c8f0 0x7f49c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:36.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.629+0000 7f49dd600640 1 -- 172.21.15.102:0/2752925840 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f49d80adfd0 con 0x7f49e0151ab0 2024-04-20T21:28:36.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.630+0000 7f49e5600640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f49c006c8f0 0x7f49c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:36.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.630+0000 7f49e5600640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f49c006c8f0 0x7f49c006edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f49d40012e0 tx=0x7f49d4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:36.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.634+0000 7f49dd600640 1 -- 172.21.15.102:0/2752925840 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f49d807b670 con 0x7f49e0151ab0 2024-04-20T21:28:36.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.876+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49e0071e10 con 0x7f49e0151ab0 2024-04-20T21:28:36.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.877+0000 7f49dd600640 1 -- 172.21.15.102:0/2752925840 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6020 (secure 0 0 0) 0x7f49d807f970 con 0x7f49e0151ab0 2024-04-20T21:28:36.876 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:36.877 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:35.626253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:36.877 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 38 2024-04-20T21:28:36.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f49c006c8f0 msgr2=0x7f49c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f49c006c8f0 0x7f49c006edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f49d40012e0 tx=0x7f49d4009040 comp rx=0 tx=0).stop 2024-04-20T21:28:36.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49e0151ab0 msgr2=0x7f49e014d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:36.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49e0151ab0 0x7f49e014d200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009890 tx=0x7f49d80046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 shutdown_connections 2024-04-20T21:28:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49e0151ab0 0x7f49e014d200 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49e01516d0 0x7f49e0072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f49c006c8f0 0x7f49c006edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 --2- 172.21.15.102:0/2752925840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49e00c5f30 0x7f49e00cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 >> 172.21.15.102:0/2752925840 conn(0x7f49e00cc820 msgr2=0x7f49e00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 shutdown_connections 2024-04-20T21:28:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:36.880+0000 7f49e6a00640 1 -- 172.21.15.102:0/2752925840 wait complete. 2024-04-20T21:28:36.898 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 38, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 38, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5659": { "addr": "172.21.15.102:6835/36459307", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 36459307, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 36459307, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5659, "incarnation": 38, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:28:35.626253+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5659 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:28:38.899 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:39.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.110+0000 7f53a8400640 1 Processor -- start 2024-04-20T21:28:39.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.110+0000 7f53a8400640 1 -- start start 2024-04-20T21:28:39.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.111+0000 7f53a8400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f53a014fa50 0x7f53a00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:39.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.111+0000 7f53a8400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a01536d0 con 0x7f53a0073a10 2024-04-20T21:28:39.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.111+0000 7f53a8400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a00c76c0 con 0x7f53a00c9eb0 2024-04-20T21:28:39.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.111+0000 7f53a8400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f53a00c7960 con 0x7f53a014fa50 2024-04-20T21:28:39.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.112+0000 7f53a5a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f53a014fa50 0x7f53a00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:39.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.112+0000 7f53a5000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f53a00c9eb0 0x7f53a00c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43086/0 (socket says 172.21.15.102:43086) 2024-04-20T21:28:39.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.112+0000 7f53a5000640 1 -- 172.21.15.102:0/1873385083 learned_addr learned my addr 172.21.15.102:0/1873385083 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:39.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.112+0000 7f53a5a00640 1 -- 172.21.15.102:0/1873385083 >> v1:172.21.15.193:6789/0 conn(0x7f53a00c9eb0 legacy=0x7f53a00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:39.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.112+0000 7f53a5a00640 1 -- 172.21.15.102:0/1873385083 >> v1:172.21.15.102:6789/0 conn(0x7f53a0073a10 legacy=0x7f53a0163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:39.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.112+0000 7f53a5a00640 1 -- 172.21.15.102:0/1873385083 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a00c3d90 con 0x7f53a014fa50 2024-04-20T21:28:39.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.112+0000 7f539fe00640 1 -- 172.21.15.102:0/1873385083 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162910251 0 0) 0x7f5398001490 con 0x7f53a0073a10 2024-04-20T21:28:39.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.113+0000 7f53a5a00640 1 --2- 172.21.15.102:0/1873385083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a014fa50 0x7f53a00c4830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f53a00c7890 tx=0x7f539402b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9f48b9fc425b620 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:39.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.114+0000 7f539fe00640 1 -- 172.21.15.102:0/1873385083 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394034070 con 0x7f53a014fa50 2024-04-20T21:28:39.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.114+0000 7f539fe00640 1 -- 172.21.15.102:0/1873385083 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5394030890 con 0x7f53a014fa50 2024-04-20T21:28:39.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.114+0000 7f539fe00640 1 -- 172.21.15.102:0/1873385083 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394030c60 con 0x7f53a014fa50 2024-04-20T21:28:39.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.115+0000 7f53a8400640 1 -- 172.21.15.102:0/1873385083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a014fa50 msgr2=0x7f53a00c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:39.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.115+0000 7f53a8400640 1 --2- 172.21.15.102:0/1873385083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a014fa50 0x7f53a00c4830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f53a00c7890 tx=0x7f539402b9d0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.115+0000 7f53a8400640 1 -- 172.21.15.102:0/1873385083 shutdown_connections 2024-04-20T21:28:39.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.115+0000 7f53a8400640 1 --2- 172.21.15.102:0/1873385083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a014fa50 0x7f53a00c4830 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.115+0000 7f53a8400640 1 -- 172.21.15.102:0/1873385083 >> 172.21.15.102:0/1873385083 conn(0x7f53a00c9390 msgr2=0x7f53a00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:39.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.115+0000 7f53a8400640 1 -- 172.21.15.102:0/1873385083 shutdown_connections 2024-04-20T21:28:39.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.115+0000 7f53a8400640 1 -- 172.21.15.102:0/1873385083 wait complete. 2024-04-20T21:28:39.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.116+0000 7f53a8400640 1 Processor -- start 2024-04-20T21:28:39.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.117+0000 7f53a8400640 1 -- start start 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.117+0000 7f53a8400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0073a10 0x7f53a01f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.117+0000 7f53a8400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53a00c9eb0 0x7f53a0158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.117+0000 7f53a8400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a014fa50 0x7f53a00725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.117+0000 7f53a8400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53a00c4030 con 0x7f53a014fa50 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.117+0000 7f53a8400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53a00c7c00 con 0x7f53a0073a10 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.117+0000 7f53a8400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f53a00c4310 con 0x7f53a00c9eb0 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0073a10 0x7f53a01f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a014fa50 0x7f53a00725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0073a10 0x7f53a01f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:33866/0 (socket says 172.21.15.102:33866) 2024-04-20T21:28:39.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5a00640 1 -- 172.21.15.102:0/4227897310 learned_addr learned my addr 172.21.15.102:0/4227897310 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:39.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5000640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53a00c9eb0 0x7f53a0158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5a00640 1 -- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53a00c9eb0 msgr2=0x7f53a0158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5a00640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53a00c9eb0 0x7f53a0158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5a00640 1 -- 172.21.15.102:0/4227897310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a014fa50 msgr2=0x7f53a00725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5a00640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a014fa50 0x7f53a00725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5a00640 1 -- 172.21.15.102:0/4227897310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a0071130 con 0x7f53a0073a10 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a5000640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53a00c9eb0 0x7f53a0158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.118+0000 7f53a6400640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a014fa50 0x7f53a00725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.119+0000 7f53a5a00640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0073a10 0x7f53a01f2170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5394000c00 tx=0x7f53940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.119+0000 7f539d600640 1 -- 172.21.15.102:0/4227897310 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394047070 con 0x7f53a0073a10 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.119+0000 7f539d600640 1 -- 172.21.15.102:0/4227897310 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5394039c90 con 0x7f53a0073a10 2024-04-20T21:28:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.119+0000 7f539d600640 1 -- 172.21.15.102:0/4227897310 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394034040 con 0x7f53a0073a10 2024-04-20T21:28:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.119+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a0071410 con 0x7f53a0073a10 2024-04-20T21:28:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.119+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a0071dc0 con 0x7f53a0073a10 2024-04-20T21:28:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.120+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a0074340 con 0x7f53a0073a10 2024-04-20T21:28:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.121+0000 7f539d600640 1 -- 172.21.15.102:0/4227897310 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f539404a070 con 0x7f53a0073a10 2024-04-20T21:28:39.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.122+0000 7f539d600640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f538006c920 0x7f538006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.122+0000 7f539d600640 1 -- 172.21.15.102:0/4227897310 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f53940adff0 con 0x7f53a0073a10 2024-04-20T21:28:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.123+0000 7f53a5000640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f538006c920 0x7f538006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.123+0000 7f53a5000640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f538006c920 0x7f538006ede0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53a00c5580 tx=0x7f538c007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:39.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.127+0000 7f539d600640 1 -- 172.21.15.102:0/4227897310 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f539407b880 con 0x7f53a0073a10 2024-04-20T21:28:39.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.368+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53a01568e0 con 0x7f53a0073a10 2024-04-20T21:28:39.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.369+0000 7f539d600640 1 -- 172.21.15.102:0/4227897310 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6020 (secure 0 0 0) 0x7f539407fb80 con 0x7f53a0073a10 2024-04-20T21:28:39.368 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:39.369 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:35.626253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:39.369 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 38 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.371+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f538006c920 msgr2=0x7f538006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.371+0000 7f53a8400640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f538006c920 0x7f538006ede0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53a00c5580 tx=0x7f538c007e80 comp rx=0 tx=0).stop 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.371+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0073a10 msgr2=0x7f53a01f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.371+0000 7f53a8400640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0073a10 0x7f53a01f2170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5394000c00 tx=0x7f53940046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.372+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 shutdown_connections 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.372+0000 7f53a8400640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a014fa50 0x7f53a00725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.372+0000 7f53a8400640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53a00c9eb0 0x7f53a0158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.372+0000 7f53a8400640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f538006c920 0x7f538006ede0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.372+0000 7f53a8400640 1 --2- 172.21.15.102:0/4227897310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0073a10 0x7f53a01f2170 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:39.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.372+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 >> 172.21.15.102:0/4227897310 conn(0x7f53a00c9390 msgr2=0x7f53a014dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:39.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.372+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 shutdown_connections 2024-04-20T21:28:39.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:39.372+0000 7f53a8400640 1 -- 172.21.15.102:0/4227897310 wait complete. 2024-04-20T21:28:39.390 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:28:41.390 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:41.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.573+0000 7f7b70e00640 1 Processor -- start 2024-04-20T21:28:41.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.573+0000 7f7b70e00640 1 -- start start 2024-04-20T21:28:41.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.574+0000 7f7b70e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:41.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.574+0000 7f7b70e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6c15ef60 con 0x7f7b6c0cb6a0 2024-04-20T21:28:41.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.574+0000 7f7b70e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6c15f240 con 0x7f7b6c0cc470 2024-04-20T21:28:41.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.574+0000 7f7b70e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c0c5ca0 con 0x7f7b6c155530 2024-04-20T21:28:41.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.574+0000 7f7b6be00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7b6c0cb6a0 0x7f7b6c1659d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49998/0 (socket says 172.21.15.102:49998) 2024-04-20T21:28:41.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.574+0000 7f7b6be00640 1 -- 172.21.15.102:0/1829021203 learned_addr learned my addr 172.21.15.102:0/1829021203 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:41.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.574+0000 7f7b6b400640 1 --2- 172.21.15.102:0/1829021203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:41.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.575+0000 7f7b6a000640 1 -- 172.21.15.102:0/1829021203 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995038679 0 0) 0x7f7b6c15ef60 con 0x7f7b6c0cb6a0 2024-04-20T21:28:41.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.575+0000 7f7b6a000640 1 -- 172.21.15.102:0/1829021203 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b50003520 con 0x7f7b6c0cb6a0 2024-04-20T21:28:41.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.575+0000 7f7b6a000640 1 -- 172.21.15.102:0/1829021203 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597002891 0 0) 0x7f7b6c15f240 con 0x7f7b6c0cc470 2024-04-20T21:28:41.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.575+0000 7f7b6a000640 1 -- 172.21.15.102:0/1829021203 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b6c15ef60 con 0x7f7b6c0cc470 2024-04-20T21:28:41.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.575+0000 7f7b6b400640 1 -- 172.21.15.102:0/1829021203 >> v1:172.21.15.193:6789/0 conn(0x7f7b6c0cc470 legacy=0x7f7b6c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:41.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.575+0000 7f7b6b400640 1 -- 172.21.15.102:0/1829021203 >> v1:172.21.15.102:6789/0 conn(0x7f7b6c0cb6a0 legacy=0x7f7b6c1659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:41.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.575+0000 7f7b6b400640 1 -- 172.21.15.102:0/1829021203 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c15f520 con 0x7f7b6c155530 2024-04-20T21:28:41.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.576+0000 7f7b6b400640 1 --2- 172.21.15.102:0/1829021203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c15e640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0097d0 tx=0x7f7b5c02b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c0b1cb30ae8b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:41.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.576+0000 7f7b6a000640 1 -- 172.21.15.102:0/1829021203 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c034070 con 0x7f7b6c155530 2024-04-20T21:28:41.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.576+0000 7f7b6a000640 1 -- 172.21.15.102:0/1829021203 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c030890 con 0x7f7b6c155530 2024-04-20T21:28:41.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.576+0000 7f7b6a000640 1 -- 172.21.15.102:0/1829021203 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c030c40 con 0x7f7b6c155530 2024-04-20T21:28:41.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.577+0000 7f7b70e00640 1 -- 172.21.15.102:0/1829021203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 msgr2=0x7f7b6c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:41.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.577+0000 7f7b70e00640 1 --2- 172.21.15.102:0/1829021203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c15e640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0097d0 tx=0x7f7b5c02b9d0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.577+0000 7f7b70e00640 1 -- 172.21.15.102:0/1829021203 shutdown_connections 2024-04-20T21:28:41.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.577+0000 7f7b70e00640 1 --2- 172.21.15.102:0/1829021203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c15e640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.577+0000 7f7b70e00640 1 -- 172.21.15.102:0/1829021203 >> 172.21.15.102:0/1829021203 conn(0x7f7b6c075080 msgr2=0x7f7b6c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:41.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.578+0000 7f7b70e00640 1 -- 172.21.15.102:0/1829021203 shutdown_connections 2024-04-20T21:28:41.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.578+0000 7f7b70e00640 1 -- 172.21.15.102:0/1829021203 wait complete. 2024-04-20T21:28:41.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.580+0000 7f7b70e00640 1 Processor -- start 2024-04-20T21:28:41.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.580+0000 7f7b70e00640 1 -- start start 2024-04-20T21:28:41.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b70e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c0cb6a0 0x7f7b6c1474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:41.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b70e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c0cc470 0x7f7b6c0b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:41.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b6aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c0cc470 0x7f7b6c0b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:41.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b6b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c0cb6a0 0x7f7b6c1474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:41.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b6aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c0cc470 0x7f7b6c0b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:33878/0 (socket says 172.21.15.102:33878) 2024-04-20T21:28:41.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b6b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c0cb6a0 0x7f7b6c1474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51922/0 (socket says 172.21.15.102:51922) 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b70e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c1f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c0cd280 con 0x7f7b6c0cb6a0 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b6aa00640 1 -- 172.21.15.102:0/1800254994 learned_addr learned my addr 172.21.15.102:0/1800254994 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c0cd5f0 con 0x7f7b6c0cc470 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.584+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c0ccbe0 con 0x7f7b6c155530 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.585+0000 7f7b6be00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c1f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.585+0000 7f7b6aa00640 1 -- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 msgr2=0x7f7b6c1f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.585+0000 7f7b6aa00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c1f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.585+0000 7f7b6aa00640 1 -- 172.21.15.102:0/1800254994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c0cb6a0 msgr2=0x7f7b6c1474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.585+0000 7f7b6aa00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c0cb6a0 0x7f7b6c1474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.585+0000 7f7b6aa00640 1 -- 172.21.15.102:0/1800254994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c1e5d10 con 0x7f7b6c0cc470 2024-04-20T21:28:41.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.585+0000 7f7b6aa00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c0cc470 0x7f7b6c0b4b40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c400 tx=0x7f7b5800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:41.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.586+0000 7f7b63400640 1 -- 172.21.15.102:0/1800254994 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58017070 con 0x7f7b6c0cc470 2024-04-20T21:28:41.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.586+0000 7f7b63400640 1 -- 172.21.15.102:0/1800254994 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b580052d0 con 0x7f7b6c0cc470 2024-04-20T21:28:41.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.586+0000 7f7b63400640 1 -- 172.21.15.102:0/1800254994 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58003830 con 0x7f7b6c0cc470 2024-04-20T21:28:41.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.586+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1f5700 con 0x7f7b6c0cc470 2024-04-20T21:28:41.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.586+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1e6270 con 0x7f7b6c0cc470 2024-04-20T21:28:41.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.587+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c04fd60 con 0x7f7b6c0cc470 2024-04-20T21:28:41.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.589+0000 7f7b63400640 1 -- 172.21.15.102:0/1800254994 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b5801a070 con 0x7f7b6c0cc470 2024-04-20T21:28:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.590+0000 7f7b63400640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b4c06c8f0 0x7f7b4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.590+0000 7f7b63400640 1 -- 172.21.15.102:0/1800254994 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f7b5808b4d0 con 0x7f7b6c0cc470 2024-04-20T21:28:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.590+0000 7f7b6b400640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b4c06c8f0 0x7f7b4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:41.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.591+0000 7f7b6b400640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b4c06c8f0 0x7f7b4c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c036000 tx=0x7f7b5c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:41.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.595+0000 7f7b63400640 1 -- 172.21.15.102:0/1800254994 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b6c1e5d10 con 0x7f7b6c0cc470 2024-04-20T21:28:41.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.860+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b6c153e00 con 0x7f7b6c0cc470 2024-04-20T21:28:41.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.861+0000 7f7b63400640 1 -- 172.21.15.102:0/1800254994 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6020 (secure 0 0 0) 0x7f7b58058ca0 con 0x7f7b6c0cc470 2024-04-20T21:28:41.861 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:41.861 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:35.626253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:41.861 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 38 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b4c06c8f0 msgr2=0x7f7b4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b4c06c8f0 0x7f7b4c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c036000 tx=0x7f7b5c03c040 comp rx=0 tx=0).stop 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c0cc470 msgr2=0x7f7b6c0b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c0cc470 0x7f7b6c0b4b40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c400 tx=0x7f7b5800c8c0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 shutdown_connections 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7b6c155530 0x7f7b6c1f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b4c06c8f0 0x7f7b4c06edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c0cc470 0x7f7b6c0b4b40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 --2- 172.21.15.102:0/1800254994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c0cb6a0 0x7f7b6c1474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 >> 172.21.15.102:0/1800254994 conn(0x7f7b6c075080 msgr2=0x7f7b6c14a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.864+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 shutdown_connections 2024-04-20T21:28:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:41.865+0000 7f7b70e00640 1 -- 172.21.15.102:0/1800254994 wait complete. 2024-04-20T21:28:41.881 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:28:43.882 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:44.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.056+0000 7fc1e4400640 1 Processor -- start 2024-04-20T21:28:44.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.056+0000 7fc1e4400640 1 -- start start 2024-04-20T21:28:44.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.057+0000 7fc1e4400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1dc14fa50 0x7fc1dc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:44.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.057+0000 7fc1e4400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc1dc0c9eb0 0x7fc1dc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e4400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1dc1536d0 con 0x7fc1dc073a10 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e4400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc1dc159500 con 0x7fc1dc14fa50 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e4400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc1dc0c76c0 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e1a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1dc14fa50 0x7fc1dc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e1a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1dc14fa50 0x7fc1dc0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51936/0 (socket says 172.21.15.102:51936) 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e1a00640 1 -- 172.21.15.102:0/3666915723 learned_addr learned my addr 172.21.15.102:0/3666915723 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e1000640 1 --2- 172.21.15.102:0/3666915723 >> v2:172.21.15.193:3300/0 conn(0x7fc1dc0c9eb0 0x7fc1dc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e1a00640 1 -- 172.21.15.102:0/3666915723 >> v2:172.21.15.193:3300/0 conn(0x7fc1dc0c9eb0 msgr2=0x7fc1dc158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e1a00640 1 --2- 172.21.15.102:0/3666915723 >> v2:172.21.15.193:3300/0 conn(0x7fc1dc0c9eb0 0x7fc1dc158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e1a00640 1 -- 172.21.15.102:0/3666915723 >> v1:172.21.15.193:6789/0 conn(0x7fc1dc073a10 legacy=0x7fc1dc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:44.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.058+0000 7fc1e1a00640 1 -- 172.21.15.102:0/3666915723 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc0c7960 con 0x7fc1dc14fa50 2024-04-20T21:28:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.060+0000 7fc1e1a00640 1 --2- 172.21.15.102:0/3666915723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc14fa50 0x7fc1dc0c4830 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc06a220 tx=0x7fc1d002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f77d422aa6c776 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.060+0000 7fc1dbe00640 1 -- 172.21.15.102:0/3666915723 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d002f850 con 0x7fc1dc14fa50 2024-04-20T21:28:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.060+0000 7fc1dbe00640 1 -- 172.21.15.102:0/3666915723 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d002f9f0 con 0x7fc1dc14fa50 2024-04-20T21:28:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.060+0000 7fc1dbe00640 1 -- 172.21.15.102:0/3666915723 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d002fdc0 con 0x7fc1dc14fa50 2024-04-20T21:28:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.061+0000 7fc1e4400640 1 -- 172.21.15.102:0/3666915723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc14fa50 msgr2=0x7fc1dc0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.061+0000 7fc1e4400640 1 --2- 172.21.15.102:0/3666915723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc14fa50 0x7fc1dc0c4830 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc06a220 tx=0x7fc1d002ba30 comp rx=0 tx=0).stop 2024-04-20T21:28:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.061+0000 7fc1e4400640 1 -- 172.21.15.102:0/3666915723 shutdown_connections 2024-04-20T21:28:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.061+0000 7fc1e4400640 1 --2- 172.21.15.102:0/3666915723 >> v2:172.21.15.193:3300/0 conn(0x7fc1dc0c9eb0 0x7fc1dc158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.061+0000 7fc1e4400640 1 --2- 172.21.15.102:0/3666915723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc14fa50 0x7fc1dc0c4830 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.061+0000 7fc1e4400640 1 -- 172.21.15.102:0/3666915723 >> 172.21.15.102:0/3666915723 conn(0x7fc1dc0c9390 msgr2=0x7fc1dc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:44.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.061+0000 7fc1e4400640 1 -- 172.21.15.102:0/3666915723 shutdown_connections 2024-04-20T21:28:44.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.061+0000 7fc1e4400640 1 -- 172.21.15.102:0/3666915723 wait complete. 2024-04-20T21:28:44.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.063+0000 7fc1e4400640 1 Processor -- start 2024-04-20T21:28:44.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.063+0000 7fc1e4400640 1 -- start start 2024-04-20T21:28:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e4400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1dc073a10 0x7fc1dc1f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e4400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc0c9eb0 0x7fc1dc149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e4400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1dc14fa50 0x7fc1dc070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e4400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc0c7c00 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e4400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc0c3d90 con 0x7fc1dc14fa50 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e4400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc1dc0c4030 con 0x7fc1dc073a10 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc0c9eb0 0x7fc1dc149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc0c9eb0 0x7fc1dc149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51946/0 (socket says 172.21.15.102:51946) 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e1000640 1 -- 172.21.15.102:0/2303460220 learned_addr learned my addr 172.21.15.102:0/2303460220 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e1a00640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1dc073a10 0x7fc1dc1f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.064+0000 7fc1e2400640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1dc14fa50 0x7fc1dc070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.065+0000 7fc1e1000640 1 -- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1dc073a10 msgr2=0x7fc1dc1f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.065+0000 7fc1e1000640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1dc073a10 0x7fc1dc1f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.065+0000 7fc1e1000640 1 -- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1dc14fa50 msgr2=0x7fc1dc070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.065+0000 7fc1e1000640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1dc14fa50 0x7fc1dc070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.065+0000 7fc1e1000640 1 -- 172.21.15.102:0/2303460220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc0c2830 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.065+0000 7fc1e1a00640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1dc073a10 0x7fc1dc1f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.065+0000 7fc1e1000640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc0c9eb0 0x7fc1dc149650 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800e9e0 tx=0x7fc1c800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.065+0000 7fc1d9600640 1 -- 172.21.15.102:0/2303460220 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8017070 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.066+0000 7fc1d9600640 1 -- 172.21.15.102:0/2303460220 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c800c9d0 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.066+0000 7fc1d9600640 1 -- 172.21.15.102:0/2303460220 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800cdc0 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.066+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc14e260 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.066+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc0c2dd0 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.067+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc0c2fd0 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.068+0000 7fc1d9600640 1 -- 172.21.15.102:0/2303460220 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1c801a070 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.069+0000 7fc1d9600640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc1bc06c920 0x7fc1bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:44.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.069+0000 7fc1d9600640 1 -- 172.21.15.102:0/2303460220 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7fc1c808c040 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.069+0000 7fc1e1a00640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc1bc06c920 0x7fc1bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:44.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.070+0000 7fc1e1a00640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc1bc06c920 0x7fc1bc06ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0036000 tx=0x7fc1d0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:44.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.074+0000 7fc1d9600640 1 -- 172.21.15.102:0/2303460220 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc1dc0c2fd0 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.306+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1dc0c56b0 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.307+0000 7fc1d9600640 1 -- 172.21.15.102:0/2303460220 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6020 (secure 0 0 0) 0x7fc1dc0c56b0 con 0x7fc1dc0c9eb0 2024-04-20T21:28:44.307 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:44.307 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:35.626253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:44.307 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 38 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc1bc06c920 msgr2=0x7fc1bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc1bc06c920 0x7fc1bc06ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0036000 tx=0x7fc1d0008030 comp rx=0 tx=0).stop 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc0c9eb0 msgr2=0x7fc1dc149650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc0c9eb0 0x7fc1dc149650 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800e9e0 tx=0x7fc1c800eea0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 shutdown_connections 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc1bc06c920 0x7fc1bc06ede0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1dc14fa50 0x7fc1dc070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1dc0c9eb0 0x7fc1dc149650 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 --2- 172.21.15.102:0/2303460220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1dc073a10 0x7fc1dc1f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.309+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 >> 172.21.15.102:0/2303460220 conn(0x7fc1dc0c9390 msgr2=0x7fc1dc06c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:44.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.310+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 shutdown_connections 2024-04-20T21:28:44.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:44.310+0000 7fc1e4400640 1 -- 172.21.15.102:0/2303460220 wait complete. 2024-04-20T21:28:44.326 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:28:46.327 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:46.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.538+0000 7fbfff600640 1 Processor -- start 2024-04-20T21:28:46.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.538+0000 7fbfff600640 1 -- start start 2024-04-20T21:28:46.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.540+0000 7fbfff600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbff815ea20 0x7fbff806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:46.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.540+0000 7fbfff600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbff815ee00 0x7fbff80710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:46.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.540+0000 7fbfff600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff80c8f20 con 0x7fbff815f2b0 2024-04-20T21:28:46.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.540+0000 7fbfff600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fbff806cb80 con 0x7fbff815ee00 2024-04-20T21:28:46.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.540+0000 7fbfff600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fbff806ce60 con 0x7fbff815ea20 2024-04-20T21:28:46.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.540+0000 7fbff7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbff815ee00 0x7fbff80710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:46.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.540+0000 7fbffd600640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbff815f2b0 0x7fbff80c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50006/0 (socket says 172.21.15.102:50006) 2024-04-20T21:28:46.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.540+0000 7fbffd600640 1 -- 172.21.15.102:0/878207834 learned_addr learned my addr 172.21.15.102:0/878207834 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:46.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.541+0000 7fbffcc00640 1 --2- 172.21.15.102:0/878207834 >> v2:172.21.15.193:3300/0 conn(0x7fbff815ea20 0x7fbff806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:46.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.541+0000 7fbff7400640 1 -- 172.21.15.102:0/878207834 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1235025698 0 0) 0x7fbff80c8f20 con 0x7fbff815f2b0 2024-04-20T21:28:46.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.541+0000 7fbff7400640 1 -- 172.21.15.102:0/878207834 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfe0003520 con 0x7fbff815f2b0 2024-04-20T21:28:46.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.541+0000 7fbff7e00640 1 -- 172.21.15.102:0/878207834 >> v2:172.21.15.193:3300/0 conn(0x7fbff815ea20 msgr2=0x7fbff806d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:46.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.541+0000 7fbff7e00640 1 --2- 172.21.15.102:0/878207834 >> v2:172.21.15.193:3300/0 conn(0x7fbff815ea20 0x7fbff806d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.541+0000 7fbff7e00640 1 -- 172.21.15.102:0/878207834 >> v1:172.21.15.102:6789/0 conn(0x7fbff815f2b0 legacy=0x7fbff80c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:46.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.541+0000 7fbff7e00640 1 -- 172.21.15.102:0/878207834 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff806d140 con 0x7fbff815ee00 2024-04-20T21:28:46.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.541+0000 7fbffcc00640 1 --2- 172.21.15.102:0/878207834 >> v2:172.21.15.193:3300/0 conn(0x7fbff815ea20 0x7fbff806d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:46.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.542+0000 7fbff7e00640 1 --2- 172.21.15.102:0/878207834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815ee00 0x7fbff80710a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbff80cd8d0 tx=0x7fbfe402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad2cc6a3f8b0a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:46.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.542+0000 7fbff7400640 1 -- 172.21.15.102:0/878207834 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4034070 con 0x7fbff815ee00 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.542+0000 7fbff7400640 1 -- 172.21.15.102:0/878207834 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe4030610 con 0x7fbff815ee00 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.542+0000 7fbff7400640 1 -- 172.21.15.102:0/878207834 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40309e0 con 0x7fbff815ee00 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.543+0000 7fbfff600640 1 -- 172.21.15.102:0/878207834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815ee00 msgr2=0x7fbff80710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.543+0000 7fbfff600640 1 --2- 172.21.15.102:0/878207834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815ee00 0x7fbff80710a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbff80cd8d0 tx=0x7fbfe402b960 comp rx=0 tx=0).stop 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.543+0000 7fbfff600640 1 -- 172.21.15.102:0/878207834 shutdown_connections 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.543+0000 7fbfff600640 1 --2- 172.21.15.102:0/878207834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815ee00 0x7fbff80710a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.543+0000 7fbfff600640 1 --2- 172.21.15.102:0/878207834 >> v2:172.21.15.193:3300/0 conn(0x7fbff815ea20 0x7fbff806d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.543+0000 7fbfff600640 1 -- 172.21.15.102:0/878207834 >> 172.21.15.102:0/878207834 conn(0x7fbff80c9390 msgr2=0x7fbff80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.543+0000 7fbfff600640 1 -- 172.21.15.102:0/878207834 shutdown_connections 2024-04-20T21:28:46.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.543+0000 7fbfff600640 1 -- 172.21.15.102:0/878207834 wait complete. 2024-04-20T21:28:46.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.545+0000 7fbfff600640 1 Processor -- start 2024-04-20T21:28:46.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.545+0000 7fbfff600640 1 -- start start 2024-04-20T21:28:46.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.546+0000 7fbfff600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff815ea20 0x7fbff81f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:46.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.546+0000 7fbfff600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbff815ee00 0x7fbff814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:46.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.546+0000 7fbfff600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815f2b0 0x7fbff8150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:46.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.546+0000 7fbfff600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbff80c8c80 con 0x7fbff815f2b0 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.546+0000 7fbfff600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbff80c8b00 con 0x7fbff815ea20 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.546+0000 7fbfff600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbff8154e80 con 0x7fbff815ee00 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbff7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbff815ee00 0x7fbff814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbff7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbff815ee00 0x7fbff814fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:41854/0 (socket says 172.21.15.102:41854) 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbff7e00640 1 -- 172.21.15.102:0/290261564 learned_addr learned my addr 172.21.15.102:0/290261564 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbffd600640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815f2b0 0x7fbff8150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbffcc00640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff815ea20 0x7fbff81f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbffcc00640 1 -- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbff815ee00 msgr2=0x7fbff814fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbffcc00640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbff815ee00 0x7fbff814fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbffcc00640 1 -- 172.21.15.102:0/290261564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815f2b0 msgr2=0x7fbff8150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbffcc00640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815f2b0 0x7fbff8150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbffcc00640 1 -- 172.21.15.102:0/290261564 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff80c5c80 con 0x7fbff815ea20 2024-04-20T21:28:46.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.547+0000 7fbffcc00640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff815ea20 0x7fbff81f13e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00e960 tx=0x7fbfec00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.548+0000 7fbff4c00640 1 -- 172.21.15.102:0/290261564 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec017070 con 0x7fbff815ea20 2024-04-20T21:28:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.548+0000 7fbff4c00640 1 -- 172.21.15.102:0/290261564 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec00ca00 con 0x7fbff815ea20 2024-04-20T21:28:46.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.548+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff80c5f20 con 0x7fbff815ea20 2024-04-20T21:28:46.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.548+0000 7fbff4c00640 1 -- 172.21.15.102:0/290261564 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00cdf0 con 0x7fbff815ea20 2024-04-20T21:28:46.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.548+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff80c64e0 con 0x7fbff815ea20 2024-04-20T21:28:46.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.549+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff8074340 con 0x7fbff815ea20 2024-04-20T21:28:46.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.550+0000 7fbff4c00640 1 -- 172.21.15.102:0/290261564 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfec01a070 con 0x7fbff815ea20 2024-04-20T21:28:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.551+0000 7fbff4c00640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbfd806c8f0 0x7fbfd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.551+0000 7fbff4c00640 1 -- 172.21.15.102:0/290261564 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7fbfec08c370 con 0x7fbff815ea20 2024-04-20T21:28:46.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.551+0000 7fbff7e00640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbfd806c8f0 0x7fbfd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:46.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.552+0000 7fbff7e00640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbfd806c8f0 0x7fbfd806edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbff8152d60 tx=0x7fbfe403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:46.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.556+0000 7fbff4c00640 1 -- 172.21.15.102:0/290261564 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbfec059aa0 con 0x7fbff815ea20 2024-04-20T21:28:46.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.799+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbff80c9ef0 con 0x7fbff815ea20 2024-04-20T21:28:46.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.800+0000 7fbff4c00640 1 -- 172.21.15.102:0/290261564 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+6024 (secure 0 0 0) 0x7fbfec05dda0 con 0x7fbff815ea20 2024-04-20T21:28:46.800 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:46.800 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:44.584294+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:46.800 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 39 2024-04-20T21:28:46.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.802+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbfd806c8f0 msgr2=0x7fbfd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:46.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.802+0000 7fbfff600640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbfd806c8f0 0x7fbfd806edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbff8152d60 tx=0x7fbfe403b040 comp rx=0 tx=0).stop 2024-04-20T21:28:46.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff815ea20 msgr2=0x7fbff81f13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:46.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff815ea20 0x7fbff81f13e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00e960 tx=0x7fbfec00ee20 comp rx=0 tx=0).stop 2024-04-20T21:28:46.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 shutdown_connections 2024-04-20T21:28:46.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff815f2b0 0x7fbff8150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbff815ee00 0x7fbff814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbfd806c8f0 0x7fbfd806edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 --2- 172.21.15.102:0/290261564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbff815ea20 0x7fbff81f13e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:46.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 >> 172.21.15.102:0/290261564 conn(0x7fbff80c9390 msgr2=0x7fbff806f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:46.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 shutdown_connections 2024-04-20T21:28:46.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:46.803+0000 7fbfff600640 1 -- 172.21.15.102:0/290261564 wait complete. 2024-04-20T21:28:46.822 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:28:48.823 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:49.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.028+0000 7f2f77200640 1 Processor -- start 2024-04-20T21:28:49.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.028+0000 7f2f77200640 1 -- start start 2024-04-20T21:28:49.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.030+0000 7f2f77200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f70155110 0x7f2f7006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.030+0000 7f2f77200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f7015f3c0 con 0x7f2f700c4040 2024-04-20T21:28:49.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.030+0000 7f2f77200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f7006e6d0 con 0x7f2f7006f3d0 2024-04-20T21:28:49.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.030+0000 7f2f77200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2f700c7c30 con 0x7f2f70155110 2024-04-20T21:28:49.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.030+0000 7f2f76800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2f700c4040 0x7f2f70165910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57806/0 (socket says 172.21.15.102:57806) 2024-04-20T21:28:49.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.030+0000 7f2f76800640 1 -- 172.21.15.102:0/3551867642 learned_addr learned my addr 172.21.15.102:0/3551867642 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:49.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.030+0000 7f2f75e00640 1 --2- 172.21.15.102:0/3551867642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f70155110 0x7f2f7006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.031+0000 7f2f74a00640 1 -- 172.21.15.102:0/3551867642 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 63330543 0 0) 0x7f2f7015f3c0 con 0x7f2f700c4040 2024-04-20T21:28:49.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.031+0000 7f2f74a00640 1 -- 172.21.15.102:0/3551867642 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f58003520 con 0x7f2f700c4040 2024-04-20T21:28:49.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.031+0000 7f2f74a00640 1 -- 172.21.15.102:0/3551867642 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 582048061 0 0) 0x7f2f7006e6d0 con 0x7f2f7006f3d0 2024-04-20T21:28:49.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.031+0000 7f2f75e00640 1 -- 172.21.15.102:0/3551867642 >> v1:172.21.15.193:6789/0 conn(0x7f2f7006f3d0 legacy=0x7f2f7015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.031+0000 7f2f75e00640 1 -- 172.21.15.102:0/3551867642 >> v1:172.21.15.102:6789/0 conn(0x7f2f700c4040 legacy=0x7f2f70165910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.031+0000 7f2f75e00640 1 -- 172.21.15.102:0/3551867642 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f700c7e80 con 0x7f2f70155110 2024-04-20T21:28:49.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.032+0000 7f2f75e00640 1 --2- 172.21.15.102:0/3551867642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f70155110 0x7f2f7006ee90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f640097d0 tx=0x7f2f6402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59fa407d1158816 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:49.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.032+0000 7f2f74a00640 1 -- 172.21.15.102:0/3551867642 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64034070 con 0x7f2f70155110 2024-04-20T21:28:49.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.032+0000 7f2f74a00640 1 -- 172.21.15.102:0/3551867642 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f64030880 con 0x7f2f70155110 2024-04-20T21:28:49.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.032+0000 7f2f74a00640 1 -- 172.21.15.102:0/3551867642 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64030c50 con 0x7f2f70155110 2024-04-20T21:28:49.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.033+0000 7f2f77200640 1 -- 172.21.15.102:0/3551867642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f70155110 msgr2=0x7f2f7006ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.033+0000 7f2f77200640 1 --2- 172.21.15.102:0/3551867642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f70155110 0x7f2f7006ee90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f640097d0 tx=0x7f2f6402b9c0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.033+0000 7f2f77200640 1 -- 172.21.15.102:0/3551867642 shutdown_connections 2024-04-20T21:28:49.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.033+0000 7f2f77200640 1 --2- 172.21.15.102:0/3551867642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f70155110 0x7f2f7006ee90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.033+0000 7f2f77200640 1 -- 172.21.15.102:0/3551867642 >> 172.21.15.102:0/3551867642 conn(0x7f2f700cc820 msgr2=0x7f2f700ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:49.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.033+0000 7f2f77200640 1 -- 172.21.15.102:0/3551867642 shutdown_connections 2024-04-20T21:28:49.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.033+0000 7f2f77200640 1 -- 172.21.15.102:0/3551867642 wait complete. 2024-04-20T21:28:49.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.035+0000 7f2f77200640 1 Processor -- start 2024-04-20T21:28:49.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.035+0000 7f2f77200640 1 -- start start 2024-04-20T21:28:49.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.035+0000 7f2f77200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7006f3d0 0x7f2f701f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.035+0000 7f2f77200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f700c4040 0x7f2f7014aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f77200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f70155110 0x7f2f7015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f77200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701554f0 con 0x7f2f70155110 2024-04-20T21:28:49.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f77200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7006ea00 con 0x7f2f700c4040 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f77200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2f70155670 con 0x7f2f7006f3d0 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7006f3d0 0x7f2f701f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7006f3d0 0x7f2f701f48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:39172/0 (socket says 172.21.15.102:39172) 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75e00640 1 -- 172.21.15.102:0/1537307082 learned_addr learned my addr 172.21.15.102:0/1537307082 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75400640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f700c4040 0x7f2f7014aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f76800640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f70155110 0x7f2f7015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75400640 1 -- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7006f3d0 msgr2=0x7f2f701f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75400640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7006f3d0 0x7f2f701f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75400640 1 -- 172.21.15.102:0/1537307082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f70155110 msgr2=0x7f2f7015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75400640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f70155110 0x7f2f7015cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75400640 1 -- 172.21.15.102:0/1537307082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f701f0a80 con 0x7f2f700c4040 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.036+0000 7f2f75e00640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7006f3d0 0x7f2f701f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.037+0000 7f2f75400640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f700c4040 0x7f2f7014aab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2f7006e8a0 tx=0x7f2f5c00c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:49.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.037+0000 7f2f6a000640 1 -- 172.21.15.102:0/1537307082 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c017070 con 0x7f2f700c4040 2024-04-20T21:28:49.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.037+0000 7f2f6a000640 1 -- 172.21.15.102:0/1537307082 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c0052f0 con 0x7f2f700c4040 2024-04-20T21:28:49.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.037+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f701f18a0 con 0x7f2f700c4040 2024-04-20T21:28:49.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.037+0000 7f2f6a000640 1 -- 172.21.15.102:0/1537307082 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c003850 con 0x7f2f700c4040 2024-04-20T21:28:49.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.037+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f701f1000 con 0x7f2f700c4040 2024-04-20T21:28:49.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.038+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f701547f0 con 0x7f2f700c4040 2024-04-20T21:28:49.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.039+0000 7f2f6a000640 1 -- 172.21.15.102:0/1537307082 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f5c01a070 con 0x7f2f700c4040 2024-04-20T21:28:49.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.040+0000 7f2f6a000640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f5006c920 0x7f2f5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.040+0000 7f2f6a000640 1 -- 172.21.15.102:0/1537307082 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f2f5c08b4f0 con 0x7f2f700c4040 2024-04-20T21:28:49.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.040+0000 7f2f75e00640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f5006c920 0x7f2f5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.040 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.041+0000 7f2f75e00640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f5006c920 0x7f2f5006ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2f6402ba00 tx=0x7f2f6403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:49.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.045+0000 7f2f6a000640 1 -- 172.21.15.102:0/1537307082 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2f701547f0 con 0x7f2f700c4040 2024-04-20T21:28:49.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.278+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f7004fd60 con 0x7f2f700c4040 2024-04-20T21:28:49.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.279+0000 7f2f6a000640 1 -- 172.21.15.102:0/1537307082 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+6024 (secure 0 0 0) 0x7f2f5c058b90 con 0x7f2f700c4040 2024-04-20T21:28:49.279 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:49.279 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:44.584294+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:49.279 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 39 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f5006c920 msgr2=0x7f2f5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f5006c920 0x7f2f5006ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2f6402ba00 tx=0x7f2f6403b040 comp rx=0 tx=0).stop 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f700c4040 msgr2=0x7f2f7014aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f700c4040 0x7f2f7014aab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2f7006e8a0 tx=0x7f2f5c00c890 comp rx=0 tx=0).stop 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 shutdown_connections 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f70155110 0x7f2f7015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f5006c920 0x7f2f5006ede0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f700c4040 0x7f2f7014aab0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 --2- 172.21.15.102:0/1537307082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7006f3d0 0x7f2f701f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 >> 172.21.15.102:0/1537307082 conn(0x7f2f700cc820 msgr2=0x7f2f70153d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.281+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 shutdown_connections 2024-04-20T21:28:49.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:49.282+0000 7f2f77200640 1 -- 172.21.15.102:0/1537307082 wait complete. 2024-04-20T21:28:49.299 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 39, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 39, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5659": { "addr": "172.21.15.102:6835/36459307", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 36459307, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 36459307, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5659, "incarnation": 38, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 33 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:28:44.584294+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5659 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:28:51.300 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:51.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.510+0000 7f0ae0800640 1 Processor -- start 2024-04-20T21:28:51.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.510+0000 7f0ae0800640 1 -- start start 2024-04-20T21:28:51.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0ae0800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8152170 0x7f0ad80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:51.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0ae0800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0ad8158770 0x7f0ad8070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:51.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0ae0800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad8071f70 con 0x7f0ad8158b50 2024-04-20T21:28:51.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0ae0800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad80c6dc0 con 0x7f0ad8158770 2024-04-20T21:28:51.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0ae0800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad8072210 con 0x7f0ad8152170 2024-04-20T21:28:51.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0add400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0ad8158770 0x7f0ad8070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:51.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0add400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0ad8158770 0x7f0ad8070df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52042/0 (socket says 172.21.15.102:52042) 2024-04-20T21:28:51.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0add400640 1 -- 172.21.15.102:0/2972786160 learned_addr learned my addr 172.21.15.102:0/2972786160 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:51.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.512+0000 7f0adde00640 1 --2- 172.21.15.102:0/2972786160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8152170 0x7f0ad80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:51.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.513+0000 7f0add400640 1 -- 172.21.15.102:0/2972786160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8152170 msgr2=0x7f0ad80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:51.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.513+0000 7f0add400640 1 --2- 172.21.15.102:0/2972786160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8152170 0x7f0ad80c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.513+0000 7f0add400640 1 -- 172.21.15.102:0/2972786160 >> v1:172.21.15.193:6789/0 conn(0x7f0ad8158b50 legacy=0x7f0ad806c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:51.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.513+0000 7f0add400640 1 -- 172.21.15.102:0/2972786160 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad80724b0 con 0x7f0ad8158770 2024-04-20T21:28:51.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.514+0000 7f0add400640 1 --2- 172.21.15.102:0/2972786160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8158770 0x7f0ad8070df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8072680 tx=0x7f0ac802dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edec37be26d21d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:51.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.514+0000 7f0adca00640 1 -- 172.21.15.102:0/2972786160 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac803b070 con 0x7f0ad8158770 2024-04-20T21:28:51.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.514+0000 7f0adca00640 1 -- 172.21.15.102:0/2972786160 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac802e9d0 con 0x7f0ad8158770 2024-04-20T21:28:51.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.514+0000 7f0adca00640 1 -- 172.21.15.102:0/2972786160 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac802eda0 con 0x7f0ad8158770 2024-04-20T21:28:51.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.515+0000 7f0ae0800640 1 -- 172.21.15.102:0/2972786160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8158770 msgr2=0x7f0ad8070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:51.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.515+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2972786160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8158770 0x7f0ad8070df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8072680 tx=0x7f0ac802dc20 comp rx=0 tx=0).stop 2024-04-20T21:28:51.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.515+0000 7f0ae0800640 1 -- 172.21.15.102:0/2972786160 shutdown_connections 2024-04-20T21:28:51.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.515+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2972786160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8158770 0x7f0ad8070df0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.515+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2972786160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8152170 0x7f0ad80c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.515+0000 7f0ae0800640 1 -- 172.21.15.102:0/2972786160 >> 172.21.15.102:0/2972786160 conn(0x7f0ad80c9390 msgr2=0x7f0ad80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:51.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.515+0000 7f0ae0800640 1 -- 172.21.15.102:0/2972786160 shutdown_connections 2024-04-20T21:28:51.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.515+0000 7f0ae0800640 1 -- 172.21.15.102:0/2972786160 wait complete. 2024-04-20T21:28:51.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.517+0000 7f0ae0800640 1 Processor -- start 2024-04-20T21:28:51.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.517+0000 7f0ae0800640 1 -- start start 2024-04-20T21:28:51.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0ae0800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8152170 0x7f0ad8157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:51.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0ae0800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8158770 0x7f0ad81f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0ae0800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad8158b50 0x7f0ad81f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0ae0800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad80c9f20 con 0x7f0ad8152170 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0ae0800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8158250 con 0x7f0ad8158b50 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0ae0800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad8157f20 con 0x7f0ad8158770 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0adde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8152170 0x7f0ad8157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0add400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8158770 0x7f0ad81f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0ade800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad8158b50 0x7f0ad81f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0add400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8158770 0x7f0ad81f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:39202/0 (socket says 172.21.15.102:39202) 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0add400640 1 -- 172.21.15.102:0/2747204336 learned_addr learned my addr 172.21.15.102:0/2747204336 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0add400640 1 -- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad8158b50 msgr2=0x7f0ad81f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0add400640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad8158b50 0x7f0ad81f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.518+0000 7f0add400640 1 -- 172.21.15.102:0/2747204336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8152170 msgr2=0x7f0ad8157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0add400640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8152170 0x7f0ad8157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0add400640 1 -- 172.21.15.102:0/2747204336 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad815a540 con 0x7f0ad8158770 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0adde00640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8152170 0x7f0ad8157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0add400640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8158770 0x7f0ad81f1910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ac802dc60 tx=0x7f0ac80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0ad2000640 1 -- 172.21.15.102:0/2747204336 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8046070 con 0x7f0ad8158770 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0ad2000640 1 -- 172.21.15.102:0/2747204336 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac8036d80 con 0x7f0ad8158770 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad81f4b80 con 0x7f0ad8158770 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0ad2000640 1 -- 172.21.15.102:0/2747204336 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac803b070 con 0x7f0ad8158770 2024-04-20T21:28:51.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.519+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad81f5140 con 0x7f0ad8158770 2024-04-20T21:28:51.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.520+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad8074340 con 0x7f0ad8158770 2024-04-20T21:28:51.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.521+0000 7f0ad2000640 1 -- 172.21.15.102:0/2747204336 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ac8049070 con 0x7f0ad8158770 2024-04-20T21:28:51.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.522+0000 7f0ad2000640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:51.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.522+0000 7f0ad2000640 1 -- 172.21.15.102:0/2747204336 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f0ac80ad9c0 con 0x7f0ad8158770 2024-04-20T21:28:51.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.523+0000 7f0adde00640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:51.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.523+0000 7f0adde00640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009f30 tx=0x7f0acc009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:51.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.527+0000 7f0ad2000640 1 -- 172.21.15.102:0/2747204336 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ac807b0a0 con 0x7f0ad8158770 2024-04-20T21:28:51.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.768+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ad8155d60 con 0x7f0ad8158770 2024-04-20T21:28:51.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.769+0000 7f0ad2000640 1 -- 172.21.15.102:0/2747204336 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+6021 (secure 0 0 0) 0x7f0ac807f3a0 con 0x7f0ad8158770 2024-04-20T21:28:51.769 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:51.769 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:49.774151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:51.769 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 40 2024-04-20T21:28:51.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 msgr2=0x7f0abc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:51.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009f30 tx=0x7f0acc009190 comp rx=0 tx=0).stop 2024-04-20T21:28:51.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8158770 msgr2=0x7f0ad81f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:51.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8158770 0x7f0ad81f1910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ac802dc60 tx=0x7f0ac80046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 shutdown_connections 2024-04-20T21:28:51.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad8158b50 0x7f0ad81f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ad8158770 0x7f0ad81f1910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 --2- 172.21.15.102:0/2747204336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8152170 0x7f0ad8157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 >> 172.21.15.102:0/2747204336 conn(0x7f0ad80c9390 msgr2=0x7f0ad8156550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:51.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 shutdown_connections 2024-04-20T21:28:51.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:51.772+0000 7f0ae0800640 1 -- 172.21.15.102:0/2747204336 wait complete. 2024-04-20T21:28:51.791 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:28:53.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:54.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.003+0000 7f244d600640 1 Processor -- start 2024-04-20T21:28:54.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.003+0000 7f244d600640 1 -- start start 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.004+0000 7f244d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24480c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.004+0000 7f244d600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2448071a70 0x7f244806d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.004+0000 7f244d600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2448071040 con 0x7f24480c7290 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.004+0000 7f244d600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f24480c9200 con 0x7f2448071a70 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.004+0000 7f244d600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2448070df0 con 0x7f2448151c90 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f2446a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24480c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f243e000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f24480c7290 0x7f2448158240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:59670/0 (socket says 172.21.15.102:59670) 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f243e000640 1 -- 172.21.15.102:0/3566227023 learned_addr learned my addr 172.21.15.102:0/3566227023 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f2447400640 1 --2- 172.21.15.102:0/3566227023 >> v2:172.21.15.193:3300/0 conn(0x7f2448071a70 0x7f244806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f2446000640 1 -- 172.21.15.102:0/3566227023 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2048184462 0 0) 0x7f2448071040 con 0x7f24480c7290 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f2446a00640 1 -- 172.21.15.102:0/3566227023 >> v2:172.21.15.193:3300/0 conn(0x7f2448071a70 msgr2=0x7f244806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f2446a00640 1 --2- 172.21.15.102:0/3566227023 >> v2:172.21.15.193:3300/0 conn(0x7f2448071a70 0x7f244806d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f2446a00640 1 -- 172.21.15.102:0/3566227023 >> v1:172.21.15.193:6789/0 conn(0x7f24480c7290 legacy=0x7f2448158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.005+0000 7f2446a00640 1 -- 172.21.15.102:0/3566227023 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2448071320 con 0x7f2448151c90 2024-04-20T21:28:54.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.006+0000 7f2446a00640 1 --2- 172.21.15.102:0/3566227023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24480c8cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f24380097a0 tx=0x7f243802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599c2fbad393df92 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f2446000640 1 -- 172.21.15.102:0/3566227023 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438034070 con 0x7f2448151c90 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f2446000640 1 -- 172.21.15.102:0/3566227023 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2438030610 con 0x7f2448151c90 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f2446000640 1 -- 172.21.15.102:0/3566227023 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380309e0 con 0x7f2448151c90 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f244d600640 1 -- 172.21.15.102:0/3566227023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 msgr2=0x7f24480c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f244d600640 1 --2- 172.21.15.102:0/3566227023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24480c8cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f24380097a0 tx=0x7f243802b960 comp rx=0 tx=0).stop 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f244d600640 1 -- 172.21.15.102:0/3566227023 shutdown_connections 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f244d600640 1 --2- 172.21.15.102:0/3566227023 >> v2:172.21.15.193:3300/0 conn(0x7f2448071a70 0x7f244806d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f244d600640 1 --2- 172.21.15.102:0/3566227023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24480c8cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.007+0000 7f244d600640 1 -- 172.21.15.102:0/3566227023 >> 172.21.15.102:0/3566227023 conn(0x7f24480c9390 msgr2=0x7f24480c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.008+0000 7f244d600640 1 -- 172.21.15.102:0/3566227023 shutdown_connections 2024-04-20T21:28:54.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.008+0000 7f244d600640 1 -- 172.21.15.102:0/3566227023 wait complete. 2024-04-20T21:28:54.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.009+0000 7f244d600640 1 Processor -- start 2024-04-20T21:28:54.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.009+0000 7f244d600640 1 -- start start 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.010+0000 7f244d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2448071a70 0x7f24481462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.010+0000 7f244d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24480c7290 0x7f244815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.010+0000 7f244d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24481f0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.010+0000 7f244d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2448157a40 con 0x7f2448071a70 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.010+0000 7f244d600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2448157db0 con 0x7f24480c7290 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.010+0000 7f244d600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f24480ca1f0 con 0x7f2448151c90 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.010+0000 7f243e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24480c7290 0x7f244815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f2446a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2448071a70 0x7f24481462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f2447400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24481f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f243e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24480c7290 0x7f244815c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:47534/0 (socket says 172.21.15.102:47534) 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f243e000640 1 -- 172.21.15.102:0/3103783706 learned_addr learned my addr 172.21.15.102:0/3103783706 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f243e000640 1 -- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 msgr2=0x7f24481f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f243e000640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24481f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f243e000640 1 -- 172.21.15.102:0/3103783706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2448071a70 msgr2=0x7f24481462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f243e000640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2448071a70 0x7f24481462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f243e000640 1 -- 172.21.15.102:0/3103783706 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244815a7a0 con 0x7f24480c7290 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f2446a00640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2448071a70 0x7f24481462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.011+0000 7f243e000640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24480c7290 0x7f244815c9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f243000c4b0 tx=0x7f243000c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.012+0000 7f243f400640 1 -- 172.21.15.102:0/3103783706 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430018070 con 0x7f24480c7290 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.012+0000 7f243f400640 1 -- 172.21.15.102:0/3103783706 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2430007500 con 0x7f24480c7290 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.012+0000 7f243f400640 1 -- 172.21.15.102:0/3103783706 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430004a20 con 0x7f24480c7290 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.012+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244815aa40 con 0x7f24480c7290 2024-04-20T21:28:54.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.012+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244815b000 con 0x7f24480c7290 2024-04-20T21:28:54.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.013+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f244815b330 con 0x7f24480c7290 2024-04-20T21:28:54.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.014+0000 7f243f400640 1 -- 172.21.15.102:0/3103783706 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2430004bc0 con 0x7f24480c7290 2024-04-20T21:28:54.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.015+0000 7f243f400640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.015+0000 7f243f400640 1 -- 172.21.15.102:0/3103783706 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f243008b460 con 0x7f24480c7290 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.015+0000 7f2446a00640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.016+0000 7f2446a00640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f242406c8f0 0x7f242406edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2438036000 tx=0x7f243803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.019+0000 7f243f400640 1 -- 172.21.15.102:0/3103783706 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2430058bc0 con 0x7f24480c7290 2024-04-20T21:28:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.261+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24480c8aa0 con 0x7f24480c7290 2024-04-20T21:28:54.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.263+0000 7f243f400640 1 -- 172.21.15.102:0/3103783706 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+6021 (secure 0 0 0) 0x7f243005cec0 con 0x7f24480c7290 2024-04-20T21:28:54.262 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:54.262 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:49.774151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:54.263 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 40 2024-04-20T21:28:54.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f242406c8f0 msgr2=0x7f242406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f242406c8f0 0x7f242406edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2438036000 tx=0x7f243803b040 comp rx=0 tx=0).stop 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24480c7290 msgr2=0x7f244815c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24480c7290 0x7f244815c9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f243000c4b0 tx=0x7f243000c970 comp rx=0 tx=0).stop 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 shutdown_connections 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2448151c90 0x7f24481f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24480c7290 0x7f244815c9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 --2- 172.21.15.102:0/3103783706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2448071a70 0x7f24481462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 >> 172.21.15.102:0/3103783706 conn(0x7f24480c9390 msgr2=0x7f24481544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 shutdown_connections 2024-04-20T21:28:54.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:54.266+0000 7f244d600640 1 -- 172.21.15.102:0/3103783706 wait complete. 2024-04-20T21:28:54.284 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:28:56.285 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:56.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.492+0000 7f9d92400640 1 Processor -- start 2024-04-20T21:28:56.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.493+0000 7f9d92400640 1 -- start start 2024-04-20T21:28:56.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.494+0000 7f9d92400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d8c1655e0 0x7f9d8c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:56.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.494+0000 7f9d92400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9d8c0c68e0 0x7f9d8c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:56.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.494+0000 7f9d92400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d8c1659c0 con 0x7f9d8c0c8600 2024-04-20T21:28:56.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.494+0000 7f9d92400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9d8c0c7df0 con 0x7f9d8c1655e0 2024-04-20T21:28:56.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.494+0000 7f9d92400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9d8c0c80d0 con 0x7f9d8c0c68e0 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.494+0000 7f9d91000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d8c1655e0 0x7f9d8c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.494+0000 7f9d91000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d8c1655e0 0x7f9d8c06e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52066/0 (socket says 172.21.15.102:52066) 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.494+0000 7f9d91000640 1 -- 172.21.15.102:0/1387449508 learned_addr learned my addr 172.21.15.102:0/1387449508 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.495+0000 7f9d8be00640 1 --2- 172.21.15.102:0/1387449508 >> v2:172.21.15.193:3300/0 conn(0x7f9d8c0c68e0 0x7f9d8c06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.495+0000 7f9d8b400640 1 -- 172.21.15.102:0/1387449508 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987742709 0 0) 0x7f9d8c1659c0 con 0x7f9d8c0c8600 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.562+0000 7f9d8b400640 1 -- 172.21.15.102:0/1387449508 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d74003500 con 0x7f9d8c0c8600 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.562+0000 7f9d91000640 1 -- 172.21.15.102:0/1387449508 >> v2:172.21.15.193:3300/0 conn(0x7f9d8c0c68e0 msgr2=0x7f9d8c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.562+0000 7f9d91000640 1 --2- 172.21.15.102:0/1387449508 >> v2:172.21.15.193:3300/0 conn(0x7f9d8c0c68e0 0x7f9d8c06eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.563+0000 7f9d91000640 1 -- 172.21.15.102:0/1387449508 >> v1:172.21.15.193:6789/0 conn(0x7f9d8c0c8600 legacy=0x7f9d8c16b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.562+0000 7f9d8b400640 1 -- 172.21.15.102:0/1387449508 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3171198844 0 0) 0x7f9d74003500 con 0x7f9d8c0c8600 2024-04-20T21:28:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.563+0000 7f9d91000640 1 -- 172.21.15.102:0/1387449508 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c0c3460 con 0x7f9d8c1655e0 2024-04-20T21:28:56.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.565+0000 7f9d91000640 1 --2- 172.21.15.102:0/1387449508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c1655e0 0x7f9d8c06e9a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0c3bf0 tx=0x7f9d80008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873aac489d213cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:56.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.566+0000 7f9d8b400640 1 -- 172.21.15.102:0/1387449508 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84001670 con 0x7f9d8c1655e0 2024-04-20T21:28:56.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.566+0000 7f9d8b400640 1 -- 172.21.15.102:0/1387449508 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8002cca0 con 0x7f9d8c1655e0 2024-04-20T21:28:56.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.566+0000 7f9d8b400640 1 -- 172.21.15.102:0/1387449508 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000a2f0 con 0x7f9d8c1655e0 2024-04-20T21:28:56.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.567+0000 7f9d92400640 1 -- 172.21.15.102:0/1387449508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c1655e0 msgr2=0x7f9d8c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:56.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.567+0000 7f9d92400640 1 --2- 172.21.15.102:0/1387449508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c1655e0 0x7f9d8c06e9a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0c3bf0 tx=0x7f9d80008800 comp rx=0 tx=0).stop 2024-04-20T21:28:56.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.567+0000 7f9d92400640 1 -- 172.21.15.102:0/1387449508 shutdown_connections 2024-04-20T21:28:56.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.568+0000 7f9d92400640 1 --2- 172.21.15.102:0/1387449508 >> v2:172.21.15.193:3300/0 conn(0x7f9d8c0c68e0 0x7f9d8c06eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.568+0000 7f9d92400640 1 --2- 172.21.15.102:0/1387449508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c1655e0 0x7f9d8c06e9a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.568+0000 7f9d92400640 1 -- 172.21.15.102:0/1387449508 >> 172.21.15.102:0/1387449508 conn(0x7f9d8c0cc820 msgr2=0x7f9d8c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:56.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.568+0000 7f9d92400640 1 -- 172.21.15.102:0/1387449508 shutdown_connections 2024-04-20T21:28:56.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.568+0000 7f9d92400640 1 -- 172.21.15.102:0/1387449508 wait complete. 2024-04-20T21:28:56.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.571+0000 7f9d92400640 1 Processor -- start 2024-04-20T21:28:56.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.571+0000 7f9d92400640 1 -- start start 2024-04-20T21:28:56.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.572+0000 7f9d92400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c0c8600 0x7f9d8c0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d92400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8c1655e0 0x7f9d8c16af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d92400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8c0c5af0 0x7f9d8c200e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d92400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c06c2a0 con 0x7f9d8c0c8600 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d92400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c06c120 con 0x7f9d8c1655e0 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d92400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8c16b930 con 0x7f9d8c0c5af0 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d91000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c0c8600 0x7f9d8c0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d91000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c0c8600 0x7f9d8c0c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52078/0 (socket says 172.21.15.102:52078) 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d91000640 1 -- 172.21.15.102:0/3696318509 learned_addr learned my addr 172.21.15.102:0/3696318509 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d8be00640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8c1655e0 0x7f9d8c16af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.573+0000 7f9d91a00640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8c0c5af0 0x7f9d8c200e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d91000640 1 -- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8c0c5af0 msgr2=0x7f9d8c200e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:56.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d91000640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8c0c5af0 0x7f9d8c200e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d91000640 1 -- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8c1655e0 msgr2=0x7f9d8c16af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:56.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d91000640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8c1655e0 0x7f9d8c16af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d91000640 1 -- 172.21.15.102:0/3696318509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c169df0 con 0x7f9d8c0c8600 2024-04-20T21:28:56.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d8be00640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8c1655e0 0x7f9d8c16af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:56.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d91000640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c0c8600 0x7f9d8c0c93b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000b340 tx=0x7f9d80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:56.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d88c00640 1 -- 172.21.15.102:0/3696318509 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80044020 con 0x7f9d8c0c8600 2024-04-20T21:28:56.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d88c00640 1 -- 172.21.15.102:0/3696318509 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8002ce50 con 0x7f9d8c0c8600 2024-04-20T21:28:56.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d88c00640 1 -- 172.21.15.102:0/3696318509 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d800074f0 con 0x7f9d8c0c8600 2024-04-20T21:28:56.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.574+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c16a090 con 0x7f9d8c0c8600 2024-04-20T21:28:56.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.575+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c16a650 con 0x7f9d8c0c8600 2024-04-20T21:28:56.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.576+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c06a0c0 con 0x7f9d8c0c8600 2024-04-20T21:28:56.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.577+0000 7f9d88c00640 1 -- 172.21.15.102:0/3696318509 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d80047020 con 0x7f9d8c0c8600 2024-04-20T21:28:56.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.578+0000 7f9d88c00640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d6c06c920 0x7f9d6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:56.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.579+0000 7f9d88c00640 1 -- 172.21.15.102:0/3696318509 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f9d800aead0 con 0x7f9d8c0c8600 2024-04-20T21:28:56.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.579+0000 7f9d8be00640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d6c06c920 0x7f9d6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:56.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.579+0000 7f9d8be00640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d6c06c920 0x7f9d6c06ede0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0c9e00 tx=0x7f9d7800f040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:56.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.584+0000 7f9d88c00640 1 -- 172.21.15.102:0/3696318509 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d8c06a0c0 con 0x7f9d8c0c8600 2024-04-20T21:28:56.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.829+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d8c04fd60 con 0x7f9d8c0c8600 2024-04-20T21:28:56.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.830+0000 7f9d88c00640 1 -- 172.21.15.102:0/3696318509 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+6021 (secure 0 0 0) 0x7f9d8004e070 con 0x7f9d8c0c8600 2024-04-20T21:28:56.830 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:56.830 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:49.774151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:56.830 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 40 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d6c06c920 msgr2=0x7f9d6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d6c06c920 0x7f9d6c06ede0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0c9e00 tx=0x7f9d7800f040 comp rx=0 tx=0).stop 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c0c8600 msgr2=0x7f9d8c0c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c0c8600 0x7f9d8c0c93b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000b340 tx=0x7f9d80004a40 comp rx=0 tx=0).stop 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 shutdown_connections 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8c0c5af0 0x7f9d8c200e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d6c06c920 0x7f9d6c06ede0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8c1655e0 0x7f9d8c16af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 --2- 172.21.15.102:0/3696318509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d8c0c8600 0x7f9d8c0c93b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 >> 172.21.15.102:0/3696318509 conn(0x7f9d8c0cc820 msgr2=0x7f9d8c160180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 shutdown_connections 2024-04-20T21:28:56.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:56.833+0000 7f9d92400640 1 -- 172.21.15.102:0/3696318509 wait complete. 2024-04-20T21:28:56.851 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:28:56.981 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:56.989 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:58.852 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:59.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.060+0000 7f6365800640 1 Processor -- start 2024-04-20T21:28:59.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.060+0000 7f6365800640 1 -- start start 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.061+0000 7f6365800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6360152170 0x7f63600c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.061+0000 7f6365800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6360158b80 0x7f63601628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.061+0000 7f6365800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6360071f70 con 0x7f6360158770 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.061+0000 7f6365800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f63600c6dc0 con 0x7f6360158b80 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.061+0000 7f6365800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6360072210 con 0x7f6360152170 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6360158b80 0x7f63601628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635ea00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6360152170 0x7f63600c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635e000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6360158770 0x7f6360070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43700/0 (socket says 172.21.15.102:43700) 2024-04-20T21:28:59.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635e000640 1 -- 172.21.15.102:0/1341945739 learned_addr learned my addr 172.21.15.102:0/1341945739 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:28:59.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635f400640 1 -- 172.21.15.102:0/1341945739 >> v2:172.21.15.193:3300/0 conn(0x7f6360152170 msgr2=0x7f63600c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:59.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635f400640 1 --2- 172.21.15.102:0/1341945739 >> v2:172.21.15.193:3300/0 conn(0x7f6360152170 0x7f63600c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635f400640 1 -- 172.21.15.102:0/1341945739 >> v1:172.21.15.193:6789/0 conn(0x7f6360158770 legacy=0x7f6360070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:59.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635f400640 1 -- 172.21.15.102:0/1341945739 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63600724b0 con 0x7f6360158b80 2024-04-20T21:28:59.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.062+0000 7f635d600640 1 -- 172.21.15.102:0/1341945739 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3656708841 0 0) 0x7f6360071f70 con 0x7f6360158770 2024-04-20T21:28:59.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.063+0000 7f635f400640 1 --2- 172.21.15.102:0/1341945739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360158b80 0x7f63601628a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f636006c2f0 tx=0x7f635802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6514c2f9d7c7b88c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:59.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f635d600640 1 -- 172.21.15.102:0/1341945739 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358039070 con 0x7f6360158b80 2024-04-20T21:28:59.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f635d600640 1 -- 172.21.15.102:0/1341945739 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635802cd00 con 0x7f6360158b80 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f635d600640 1 -- 172.21.15.102:0/1341945739 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358034660 con 0x7f6360158b80 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f6365800640 1 -- 172.21.15.102:0/1341945739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360158b80 msgr2=0x7f63601628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f6365800640 1 --2- 172.21.15.102:0/1341945739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360158b80 0x7f63601628a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f636006c2f0 tx=0x7f635802bf50 comp rx=0 tx=0).stop 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f6365800640 1 -- 172.21.15.102:0/1341945739 shutdown_connections 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f6365800640 1 --2- 172.21.15.102:0/1341945739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360158b80 0x7f63601628a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f6365800640 1 --2- 172.21.15.102:0/1341945739 >> v2:172.21.15.193:3300/0 conn(0x7f6360152170 0x7f63600c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f6365800640 1 -- 172.21.15.102:0/1341945739 >> 172.21.15.102:0/1341945739 conn(0x7f63600c9390 msgr2=0x7f63600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f6365800640 1 -- 172.21.15.102:0/1341945739 shutdown_connections 2024-04-20T21:28:59.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.064+0000 7f6365800640 1 -- 172.21.15.102:0/1341945739 wait complete. 2024-04-20T21:28:59.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.066+0000 7f6365800640 1 Processor -- start 2024-04-20T21:28:59.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.066+0000 7f6365800640 1 -- start start 2024-04-20T21:28:59.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f6365800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360152170 0x7f63600c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f6365800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6360158770 0x7f63600c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f6365800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6360158b80 0x7f63601463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f6365800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63600c9f20 con 0x7f6360152170 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f6365800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6360158250 con 0x7f6360158770 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f6365800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6360157f20 con 0x7f6360158b80 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f635f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6360158b80 0x7f63601463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f635e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6360158770 0x7f63600c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f635f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6360158b80 0x7f63601463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:49986/0 (socket says 172.21.15.102:49986) 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f635f400640 1 -- 172.21.15.102:0/566124501 learned_addr learned my addr 172.21.15.102:0/566124501 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.067+0000 7f635ea00640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360152170 0x7f63600c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f635e000640 1 -- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6360158b80 msgr2=0x7f63601463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:59.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f635e000640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6360158b80 0x7f63601463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f635e000640 1 -- 172.21.15.102:0/566124501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360152170 msgr2=0x7f63600c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f635e000640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360152170 0x7f63600c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f635e000640 1 -- 172.21.15.102:0/566124501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63601532c0 con 0x7f6360158770 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f635f400640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6360158b80 0x7f63601463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f635e000640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6360158770 0x7f63600c7c30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f634c00c3e0 tx=0x7f634c00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f6356a00640 1 -- 172.21.15.102:0/566124501 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c017070 con 0x7f6360158770 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f6356a00640 1 -- 172.21.15.102:0/566124501 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f634c005d00 con 0x7f6360158770 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6360153560 con 0x7f6360158770 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f6356a00640 1 -- 172.21.15.102:0/566124501 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c007310 con 0x7f6360158770 2024-04-20T21:28:59.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.068+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63601555e0 con 0x7f6360158770 2024-04-20T21:28:59.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.069+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6360074340 con 0x7f6360158770 2024-04-20T21:28:59.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.070+0000 7f6356a00640 1 -- 172.21.15.102:0/566124501 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f634c01a070 con 0x7f6360158770 2024-04-20T21:28:59.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.071+0000 7f6356a00640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f634006c8f0 0x7f634006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:59.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.071+0000 7f6356a00640 1 -- 172.21.15.102:0/566124501 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f634c08b510 con 0x7f6360158770 2024-04-20T21:28:59.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.072+0000 7f635ea00640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f634006c8f0 0x7f634006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:59.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.072+0000 7f635ea00640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f634006c8f0 0x7f634006edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6350009b40 tx=0x7f6350009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:59.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.076+0000 7f6356a00640 1 -- 172.21.15.102:0/566124501 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f634c058c40 con 0x7f6360158770 2024-04-20T21:28:59.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.318+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6360155840 con 0x7f6360158770 2024-04-20T21:28:59.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.319+0000 7f6356a00640 1 -- 172.21.15.102:0/566124501 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6027 (secure 0 0 0) 0x7f634c05cf40 con 0x7f6360158770 2024-04-20T21:28:59.319 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:28:59.319 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:56.167998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:28:59.319 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 41 2024-04-20T21:28:59.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f634006c8f0 msgr2=0x7f634006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:59.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f634006c8f0 0x7f634006edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6350009b40 tx=0x7f6350009190 comp rx=0 tx=0).stop 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6360158770 msgr2=0x7f63600c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6360158770 0x7f63600c7c30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f634c00c3e0 tx=0x7f634c00c8a0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 shutdown_connections 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6360158b80 0x7f63601463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f634006c8f0 0x7f634006edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6360158770 0x7f63600c7c30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 --2- 172.21.15.102:0/566124501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6360152170 0x7f63600c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 >> 172.21.15.102:0/566124501 conn(0x7f63600c9390 msgr2=0x7f6360073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 shutdown_connections 2024-04-20T21:28:59.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:28:59.322+0000 7f6365800640 1 -- 172.21.15.102:0/566124501 wait complete. 2024-04-20T21:28:59.341 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:29:01.342 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.551+0000 7f7a03c00640 1 Processor -- start 2024-04-20T21:29:01.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.551+0000 7f7a03c00640 1 -- start start 2024-04-20T21:29:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f7a03c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f79fc0c9eb0 0x7f79fc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f7a03c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f7a03c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc1536d0 con 0x7f79fc14fa50 2024-04-20T21:29:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f7a03c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f79fc159500 con 0x7f79fc0c9eb0 2024-04-20T21:29:01.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f7a03c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0c76c0 con 0x7f79fc073a10 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f79fbe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f79fc0c9eb0 0x7f79fc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f7a01c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f7a01200640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f79fc14fa50 0x7f79fc0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43710/0 (socket says 172.21.15.102:43710) 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.553+0000 7f7a01200640 1 -- 172.21.15.102:0/2612557767 learned_addr learned my addr 172.21.15.102:0/2612557767 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.554+0000 7f79fbe00640 1 -- 172.21.15.102:0/2612557767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 msgr2=0x7f79fc158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.554+0000 7f79fbe00640 1 --2- 172.21.15.102:0/2612557767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.554+0000 7f79fbe00640 1 -- 172.21.15.102:0/2612557767 >> v1:172.21.15.193:6789/0 conn(0x7f79fc14fa50 legacy=0x7f79fc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.554+0000 7f79fbe00640 1 -- 172.21.15.102:0/2612557767 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc0c7960 con 0x7f79fc0c9eb0 2024-04-20T21:29:01.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.554+0000 7f79fb400640 1 -- 172.21.15.102:0/2612557767 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4197291772 0 0) 0x7f79ec001110 con 0x7f79fc14fa50 2024-04-20T21:29:01.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.554+0000 7f7a01c00640 1 --2- 172.21.15.102:0/2612557767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:29:01.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.555+0000 7f79fbe00640 1 --2- 172.21.15.102:0/2612557767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc0c4830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79e80097a0 tx=0x7f79e802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9d6e9723198e502 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.555+0000 7f79fb400640 1 -- 172.21.15.102:0/2612557767 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8034070 con 0x7f79fc0c9eb0 2024-04-20T21:29:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.555+0000 7f79fb400640 1 -- 172.21.15.102:0/2612557767 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e8030800 con 0x7f79fc0c9eb0 2024-04-20T21:29:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.555+0000 7f79fb400640 1 -- 172.21.15.102:0/2612557767 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8030bd0 con 0x7f79fc0c9eb0 2024-04-20T21:29:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.556+0000 7f7a03c00640 1 -- 172.21.15.102:0/2612557767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 msgr2=0x7f79fc0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.556+0000 7f7a03c00640 1 --2- 172.21.15.102:0/2612557767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc0c4830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79e80097a0 tx=0x7f79e802b940 comp rx=0 tx=0).stop 2024-04-20T21:29:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.556+0000 7f7a03c00640 1 -- 172.21.15.102:0/2612557767 shutdown_connections 2024-04-20T21:29:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.556+0000 7f7a03c00640 1 --2- 172.21.15.102:0/2612557767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.556+0000 7f7a03c00640 1 --2- 172.21.15.102:0/2612557767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc0c4830 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.556+0000 7f7a03c00640 1 -- 172.21.15.102:0/2612557767 >> 172.21.15.102:0/2612557767 conn(0x7f79fc0c9390 msgr2=0x7f79fc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.556+0000 7f7a03c00640 1 -- 172.21.15.102:0/2612557767 shutdown_connections 2024-04-20T21:29:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.556+0000 7f7a03c00640 1 -- 172.21.15.102:0/2612557767 wait complete. 2024-04-20T21:29:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.558+0000 7f7a03c00640 1 Processor -- start 2024-04-20T21:29:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.558+0000 7f7a03c00640 1 -- start start 2024-04-20T21:29:01.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a03c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a03c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc1f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a03c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79fc14fa50 0x7f79fc1f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a03c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0c7c00 con 0x7f79fc14fa50 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a03c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0c3d90 con 0x7f79fc0c9eb0 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a03c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0c4030 con 0x7f79fc073a10 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a01200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f79fbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc1f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a01200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50002/0 (socket says 172.21.15.102:50002) 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f79fbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc1f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:49012/0 (socket says 172.21.15.102:49012) 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a01200640 1 -- 172.21.15.102:0/1521915302 learned_addr learned my addr 172.21.15.102:0/1521915302 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.559+0000 7f7a01c00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79fc14fa50 0x7f79fc1f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f7a01200640 1 -- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 msgr2=0x7f79fc1f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f7a01200640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc1f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f7a01200640 1 -- 172.21.15.102:0/1521915302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79fc14fa50 msgr2=0x7f79fc1f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f7a01200640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79fc14fa50 0x7f79fc1f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f7a01200640 1 -- 172.21.15.102:0/1521915302 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc156160 con 0x7f79fc073a10 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f79fbe00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc1f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f7a01200640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c780 tx=0x7f79ec00cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f79f8c00640 1 -- 172.21.15.102:0/1521915302 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec018070 con 0x7f79fc073a10 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f79f8c00640 1 -- 172.21.15.102:0/1521915302 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec0072d0 con 0x7f79fc073a10 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc0c37d0 con 0x7f79fc073a10 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.560+0000 7f79f8c00640 1 -- 172.21.15.102:0/1521915302 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0047e0 con 0x7f79fc073a10 2024-04-20T21:29:01.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.561+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc1563d0 con 0x7f79fc073a10 2024-04-20T21:29:01.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.562+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc0c56b0 con 0x7f79fc073a10 2024-04-20T21:29:01.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.563+0000 7f79f8c00640 1 -- 172.21.15.102:0/1521915302 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79ec01b070 con 0x7f79fc073a10 2024-04-20T21:29:01.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.563+0000 7f79f8c00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79dc06c8f0 0x7f79dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:01.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.564+0000 7f79f8c00640 1 -- 172.21.15.102:0/1521915302 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f79ec08b3a0 con 0x7f79fc073a10 2024-04-20T21:29:01.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.564+0000 7f79fbe00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79dc06c8f0 0x7f79dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:01.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.564+0000 7f79fbe00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79dc06c8f0 0x7f79dc06edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f79e8036000 tx=0x7f79e803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:01.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.568+0000 7f79f8c00640 1 -- 172.21.15.102:0/1521915302 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f79fc0c56b0 con 0x7f79fc073a10 2024-04-20T21:29:01.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.816+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79fc0c5990 con 0x7f79fc073a10 2024-04-20T21:29:01.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.817+0000 7f79f8c00640 1 -- 172.21.15.102:0/1521915302 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6027 (secure 0 0 0) 0x7f79fc0c5990 con 0x7f79fc073a10 2024-04-20T21:29:01.817 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:29:01.817 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:56.167998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:29:01.817 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 41 2024-04-20T21:29:01.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79dc06c8f0 msgr2=0x7f79dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79dc06c8f0 0x7f79dc06edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f79e8036000 tx=0x7f79e803b040 comp rx=0 tx=0).stop 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 msgr2=0x7f79fc158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c780 tx=0x7f79ec00cb80 comp rx=0 tx=0).stop 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 shutdown_connections 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79fc14fa50 0x7f79fc1f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79dc06c8f0 0x7f79dc06edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79fc0c9eb0 0x7f79fc1f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 --2- 172.21.15.102:0/1521915302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79fc073a10 0x7f79fc158be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 >> 172.21.15.102:0/1521915302 conn(0x7f79fc0c9390 msgr2=0x7f79fc0c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 shutdown_connections 2024-04-20T21:29:01.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:01.820+0000 7f7a03c00640 1 -- 172.21.15.102:0/1521915302 wait complete. 2024-04-20T21:29:01.838 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 41, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 41, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5659": { "addr": "172.21.15.102:6835/36459307", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 36459307, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 36459307, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5659, "incarnation": 38, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:clientreplay", "state_seq": 37 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:28:56.167998+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5659 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:29:03.839 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:04.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.039+0000 7fbf64a00640 1 Processor -- start 2024-04-20T21:29:04.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.039+0000 7fbf64a00640 1 -- start start 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.041+0000 7fbf64a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf600c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.041+0000 7fbf64a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbf60158770 0x7fbf60070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.041+0000 7fbf64a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbf60158b50 0x7fbf60071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.041+0000 7fbf64a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fbf600c6dc0 con 0x7fbf60158b50 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.041+0000 7fbf64a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fbf60071f70 con 0x7fbf60158770 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.041+0000 7fbf64a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbf60072210 con 0x7fbf60152170 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.041+0000 7fbf5de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf600c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf5e800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbf60158b50 0x7fbf60071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf5de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf600c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50004/0 (socket says 172.21.15.102:50004) 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf5de00640 1 -- 172.21.15.102:0/2887465757 learned_addr learned my addr 172.21.15.102:0/2887465757 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:29:04.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf57e00640 1 --2- 172.21.15.102:0/2887465757 >> v2:172.21.15.193:3300/0 conn(0x7fbf60158770 0x7fbf60070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf5de00640 1 -- 172.21.15.102:0/2887465757 >> v2:172.21.15.193:3300/0 conn(0x7fbf60158770 msgr2=0x7fbf60070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:04.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf5de00640 1 --2- 172.21.15.102:0/2887465757 >> v2:172.21.15.193:3300/0 conn(0x7fbf60158770 0x7fbf60070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf5de00640 1 -- 172.21.15.102:0/2887465757 >> v2:172.21.15.102:3300/0 conn(0x7fbf60158b50 msgr2=0x7fbf60071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:04.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf5de00640 1 --2- 172.21.15.102:0/2887465757 >> v2:172.21.15.102:3300/0 conn(0x7fbf60158b50 0x7fbf60071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.042+0000 7fbf5de00640 1 -- 172.21.15.102:0/2887465757 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf600724b0 con 0x7fbf60152170 2024-04-20T21:29:04.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.043+0000 7fbf5de00640 1 --2- 172.21.15.102:0/2887465757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf600c8db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf60070a20 tx=0x7fbf5002e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d69b50de034a66cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.044+0000 7fbf5d400640 1 -- 172.21.15.102:0/2887465757 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5003b070 con 0x7fbf60152170 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.044+0000 7fbf5d400640 1 -- 172.21.15.102:0/2887465757 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf50036440 con 0x7fbf60152170 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.044+0000 7fbf5d400640 1 -- 172.21.15.102:0/2887465757 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50036810 con 0x7fbf60152170 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.044+0000 7fbf64a00640 1 -- 172.21.15.102:0/2887465757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 msgr2=0x7fbf600c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.044+0000 7fbf64a00640 1 --2- 172.21.15.102:0/2887465757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf600c8db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf60070a20 tx=0x7fbf5002e160 comp rx=0 tx=0).stop 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.045+0000 7fbf64a00640 1 -- 172.21.15.102:0/2887465757 shutdown_connections 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.045+0000 7fbf64a00640 1 --2- 172.21.15.102:0/2887465757 >> v2:172.21.15.102:3300/0 conn(0x7fbf60158b50 0x7fbf60071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.045+0000 7fbf64a00640 1 --2- 172.21.15.102:0/2887465757 >> v2:172.21.15.193:3300/0 conn(0x7fbf60158770 0x7fbf60070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.045+0000 7fbf64a00640 1 --2- 172.21.15.102:0/2887465757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf600c8db0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.045+0000 7fbf64a00640 1 -- 172.21.15.102:0/2887465757 >> 172.21.15.102:0/2887465757 conn(0x7fbf600c9390 msgr2=0x7fbf600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:04.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.045+0000 7fbf64a00640 1 -- 172.21.15.102:0/2887465757 shutdown_connections 2024-04-20T21:29:04.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.045+0000 7fbf64a00640 1 -- 172.21.15.102:0/2887465757 wait complete. 2024-04-20T21:29:04.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.047+0000 7fbf64a00640 1 Processor -- start 2024-04-20T21:29:04.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.047+0000 7fbf64a00640 1 -- start start 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf64a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf6014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf64a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf60158770 0x7fbf601f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf64a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60158b50 0x7fbf601f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf64a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf600c9f20 con 0x7fbf60158b50 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf64a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60158250 con 0x7fbf60158770 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf64a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbf60157f20 con 0x7fbf60152170 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf60158770 0x7fbf601f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf5de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf6014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf5de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf6014e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50008/0 (socket says 172.21.15.102:50008) 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf5de00640 1 -- 172.21.15.102:0/812381370 learned_addr learned my addr 172.21.15.102:0/812381370 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf60158770 0x7fbf601f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:49026/0 (socket says 172.21.15.102:49026) 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.048+0000 7fbf5e800640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60158b50 0x7fbf601f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf5de00640 1 -- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf60158770 msgr2=0x7fbf601f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf5de00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf60158770 0x7fbf601f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf5de00640 1 -- 172.21.15.102:0/812381370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60158b50 msgr2=0x7fbf601f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf5de00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60158b50 0x7fbf601f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf5de00640 1 -- 172.21.15.102:0/812381370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf600c8090 con 0x7fbf60152170 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf57e00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf60158770 0x7fbf601f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:29:04.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf5de00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf6014e390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbf5002e1a0 tx=0x7fbf500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:04.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf56000640 1 -- 172.21.15.102:0/812381370 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50046070 con 0x7fbf60152170 2024-04-20T21:29:04.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf56000640 1 -- 172.21.15.102:0/812381370 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5003d050 con 0x7fbf60152170 2024-04-20T21:29:04.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf600c8370 con 0x7fbf60152170 2024-04-20T21:29:04.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.049+0000 7fbf56000640 1 -- 172.21.15.102:0/812381370 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5003b030 con 0x7fbf60152170 2024-04-20T21:29:04.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.050+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf601f6ca0 con 0x7fbf60152170 2024-04-20T21:29:04.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.051+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf601f6fd0 con 0x7fbf60152170 2024-04-20T21:29:04.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.052+0000 7fbf56000640 1 -- 172.21.15.102:0/812381370 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf50049070 con 0x7fbf60152170 2024-04-20T21:29:04.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.053+0000 7fbf56000640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.053+0000 7fbf56000640 1 -- 172.21.15.102:0/812381370 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7fbf500adb40 con 0x7fbf60152170 2024-04-20T21:29:04.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.053+0000 7fbf57e00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.054+0000 7fbf57e00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbf601f2f40 tx=0x7fbf48009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.057+0000 7fbf56000640 1 -- 172.21.15.102:0/812381370 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbf5007b270 con 0x7fbf60152170 2024-04-20T21:29:04.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.368+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf60074340 con 0x7fbf60152170 2024-04-20T21:29:04.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.369+0000 7fbf56000640 1 -- 172.21.15.102:0/812381370 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6027 (secure 0 0 0) 0x7fbf5007f570 con 0x7fbf60152170 2024-04-20T21:29:04.369 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:29:04.369 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:28:56.167998+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:29:04.369 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 41 2024-04-20T21:29:04.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 msgr2=0x7fbf3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:04.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbf601f2f40 tx=0x7fbf48009190 comp rx=0 tx=0).stop 2024-04-20T21:29:04.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 msgr2=0x7fbf6014e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:04.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf6014e390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbf5002e1a0 tx=0x7fbf500046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 shutdown_connections 2024-04-20T21:29:04.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60158b50 0x7fbf601f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf60158770 0x7fbf601f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 --2- 172.21.15.102:0/812381370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf60152170 0x7fbf6014e390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 >> 172.21.15.102:0/812381370 conn(0x7fbf600c9390 msgr2=0x7fbf60072ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 shutdown_connections 2024-04-20T21:29:04.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:04.373+0000 7fbf64a00640 1 -- 172.21.15.102:0/812381370 wait complete. 2024-04-20T21:29:04.395 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:29:06.396 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:06.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.596+0000 7f0463400640 1 Processor -- start 2024-04-20T21:29:06.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.596+0000 7f0463400640 1 -- start start 2024-04-20T21:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.597+0000 7f0463400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0cd940 0x7f045c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.597+0000 7f0463400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f045c0ce310 0x7f045c072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0463400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f045c15f940 con 0x7f045c0c7900 2024-04-20T21:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0463400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f045c15eb60 con 0x7f045c0ce310 2024-04-20T21:29:06.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0463400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f045c15ee00 con 0x7f045c0cd940 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0461400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f045c0ce310 0x7f045c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f045be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0cd940 0x7f045c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0461400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f045c0ce310 0x7f045c072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:49032/0 (socket says 172.21.15.102:49032) 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0461400640 1 -- 172.21.15.102:0/1635899454 learned_addr learned my addr 172.21.15.102:0/1635899454 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0460a00640 1 --1- 172.21.15.102:0/1635899454 >> v1:172.21.15.193:6789/0 conn(0x7f045c0c7900 0x7f045c074ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43720/0 (socket says 172.21.15.102:43720) 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0461400640 1 -- 172.21.15.102:0/1635899454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0cd940 msgr2=0x7f045c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0461400640 1 --2- 172.21.15.102:0/1635899454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0cd940 0x7f045c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0461400640 1 -- 172.21.15.102:0/1635899454 >> v1:172.21.15.193:6789/0 conn(0x7f045c0c7900 legacy=0x7f045c074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.598+0000 7f0461400640 1 -- 172.21.15.102:0/1635899454 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c15f0a0 con 0x7f045c0ce310 2024-04-20T21:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.600+0000 7f0461400640 1 --2- 172.21.15.102:0/1635899454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0ce310 0x7f045c072230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0454009e50 tx=0x7f045402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=550cc99a20dd725a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.600+0000 7f045b400640 1 -- 172.21.15.102:0/1635899454 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045402cb00 con 0x7f045c0ce310 2024-04-20T21:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.600+0000 7f045b400640 1 -- 172.21.15.102:0/1635899454 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045402cca0 con 0x7f045c0ce310 2024-04-20T21:29:06.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.600+0000 7f045b400640 1 -- 172.21.15.102:0/1635899454 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454034610 con 0x7f045c0ce310 2024-04-20T21:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.601+0000 7f0463400640 1 -- 172.21.15.102:0/1635899454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0ce310 msgr2=0x7f045c072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.601+0000 7f0463400640 1 --2- 172.21.15.102:0/1635899454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0ce310 0x7f045c072230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0454009e50 tx=0x7f045402bf60 comp rx=0 tx=0).stop 2024-04-20T21:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.601+0000 7f0463400640 1 -- 172.21.15.102:0/1635899454 shutdown_connections 2024-04-20T21:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.601+0000 7f0463400640 1 --2- 172.21.15.102:0/1635899454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0ce310 0x7f045c072230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.601+0000 7f0463400640 1 --2- 172.21.15.102:0/1635899454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0cd940 0x7f045c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.601+0000 7f0463400640 1 -- 172.21.15.102:0/1635899454 >> 172.21.15.102:0/1635899454 conn(0x7f045c06dad0 msgr2=0x7f045c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.601+0000 7f0463400640 1 -- 172.21.15.102:0/1635899454 shutdown_connections 2024-04-20T21:29:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.601+0000 7f0463400640 1 -- 172.21.15.102:0/1635899454 wait complete. 2024-04-20T21:29:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.603+0000 7f0463400640 1 Processor -- start 2024-04-20T21:29:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.603+0000 7f0463400640 1 -- start start 2024-04-20T21:29:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0463400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0c7900 0x7f045c150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0463400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0cd940 0x7f045c0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0463400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f045c0ce310 0x7f045c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0463400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f045c0755b0 con 0x7f045c0ce310 2024-04-20T21:29:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0463400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f045c15f660 con 0x7f045c0cd940 2024-04-20T21:29:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0463400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f045c075730 con 0x7f045c0c7900 2024-04-20T21:29:06.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0461400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f045c0ce310 0x7f045c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:06.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0461400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f045c0ce310 0x7f045c0caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53030/0 (socket says 172.21.15.102:53030) 2024-04-20T21:29:06.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0461400640 1 -- 172.21.15.102:0/3236554855 learned_addr learned my addr 172.21.15.102:0/3236554855 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:29:06.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f0460a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0c7900 0x7f045c150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:06.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.604+0000 7f045be00640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0cd940 0x7f045c0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:06.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0461400640 1 -- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0c7900 msgr2=0x7f045c150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:06.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0461400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0c7900 0x7f045c150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0461400640 1 -- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0cd940 msgr2=0x7f045c0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:06.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0461400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0cd940 0x7f045c0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0461400640 1 -- 172.21.15.102:0/3236554855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c0c8d80 con 0x7f045c0ce310 2024-04-20T21:29:06.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0460a00640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0c7900 0x7f045c150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:29:06.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0461400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f045c0ce310 0x7f045c0caac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0454009e50 tx=0x7f0454004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:06.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0458c00640 1 -- 172.21.15.102:0/3236554855 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454043020 con 0x7f045c0ce310 2024-04-20T21:29:06.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0458c00640 1 -- 172.21.15.102:0/3236554855 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045402ce50 con 0x7f045c0ce310 2024-04-20T21:29:06.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c1f4300 con 0x7f045c0ce310 2024-04-20T21:29:06.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.605+0000 7f0458c00640 1 -- 172.21.15.102:0/3236554855 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454009510 con 0x7f045c0ce310 2024-04-20T21:29:06.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.606+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c1f48c0 con 0x7f045c0ce310 2024-04-20T21:29:06.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.607+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f045c074340 con 0x7f045c0ce310 2024-04-20T21:29:06.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.608+0000 7f0458c00640 1 -- 172.21.15.102:0/3236554855 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0454046020 con 0x7f045c0ce310 2024-04-20T21:29:06.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.609+0000 7f0458c00640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f043c06c8f0 0x7f043c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:06.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.609+0000 7f0458c00640 1 -- 172.21.15.102:0/3236554855 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f04540ae6f0 con 0x7f045c0ce310 2024-04-20T21:29:06.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.609+0000 7f0460a00640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f043c06c8f0 0x7f043c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:06.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.610+0000 7f0460a00640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f043c06c8f0 0x7f043c06edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0450007fd0 tx=0x7f0450007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:06.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.614+0000 7f0458c00640 1 -- 172.21.15.102:0/3236554855 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f045404d030 con 0x7f045c0ce310 2024-04-20T21:29:06.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.860+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f045c154390 con 0x7f045c0ce310 2024-04-20T21:29:06.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.862+0000 7f0458c00640 1 -- 172.21.15.102:0/3236554855 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6028 (secure 0 0 0) 0x7f045c0c8d80 con 0x7f045c0ce310 2024-04-20T21:29:06.861 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:29:06.861 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:29:06.861 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 42 2024-04-20T21:29:06.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f043c06c8f0 msgr2=0x7f043c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:06.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f043c06c8f0 0x7f043c06edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0450007fd0 tx=0x7f0450007e80 comp rx=0 tx=0).stop 2024-04-20T21:29:06.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f045c0ce310 msgr2=0x7f045c0caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:06.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f045c0ce310 0x7f045c0caac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0454009e50 tx=0x7f0454004a40 comp rx=0 tx=0).stop 2024-04-20T21:29:06.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 shutdown_connections 2024-04-20T21:29:06.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f045c0ce310 0x7f045c0caac0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f043c06c8f0 0x7f043c06edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f045c0cd940 0x7f045c0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.864+0000 7f0463400640 1 --2- 172.21.15.102:0/3236554855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f045c0c7900 0x7f045c150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:06.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.865+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 >> 172.21.15.102:0/3236554855 conn(0x7f045c06dad0 msgr2=0x7f045c15c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:06.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.865+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 shutdown_connections 2024-04-20T21:29:06.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:06.865+0000 7f0463400640 1 -- 172.21.15.102:0/3236554855 wait complete. 2024-04-20T21:29:06.882 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=5533 2024-04-20T21:29:06.882 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.a 2024-04-20T21:29:11.883 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-04-20T21:29:11.883 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:29:11.883 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:29:11.888 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:29:11.888 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:12.059 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-20T21:29:12.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.094+0000 7f9f92000640 1 Processor -- start 2024-04-20T21:29:12.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.094+0000 7f9f92000640 1 -- start start 2024-04-20T21:29:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.095+0000 7f9f92000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9f8c0c7350 0x7f9f8c0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.095+0000 7f9f92000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9f8c0c8670 0x7f9f8c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.095+0000 7f9f92000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c0c7730 con 0x7f9f8c0716d0 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.095+0000 7f9f92000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c0ca5a0 con 0x7f9f8c0c7350 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.095+0000 7f9f92000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c15b460 con 0x7f9f8c0c8670 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f8b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9f8c0c7350 0x7f9f8c0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f82a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9f8c0716d0 0x7f9f8c06b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:44412/0 (socket says 172.21.15.102:44412) 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f8b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9f8c0c7350 0x7f9f8c0c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47224/0 (socket says 172.21.15.102:47224) 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f82a00640 1 -- 172.21.15.102:0/72145454 learned_addr learned my addr 172.21.15.102:0/72145454 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f8be00640 1 --2- 172.21.15.102:0/72145454 >> v2:172.21.15.193:3300/0 conn(0x7f9f8c0c8670 0x7f9f8c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f8b400640 1 -- 172.21.15.102:0/72145454 >> v2:172.21.15.193:3300/0 conn(0x7f9f8c0c8670 msgr2=0x7f9f8c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f8b400640 1 --2- 172.21.15.102:0/72145454 >> v2:172.21.15.193:3300/0 conn(0x7f9f8c0c8670 0x7f9f8c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f8b400640 1 -- 172.21.15.102:0/72145454 >> v1:172.21.15.102:6789/0 conn(0x7f9f8c0716d0 legacy=0x7f9f8c06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f8aa00640 1 -- 172.21.15.102:0/72145454 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677942328 0 0) 0x7f9f8c0c7730 con 0x7f9f8c0716d0 2024-04-20T21:29:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.096+0000 7f9f8b400640 1 -- 172.21.15.102:0/72145454 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c15b740 con 0x7f9f8c0c7350 2024-04-20T21:29:12.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.097+0000 7f9f8b400640 1 --2- 172.21.15.102:0/72145454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c7350 0x7f9f8c0c9ef0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c15f9a0 tx=0x7f9f7c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caaffd8ca5b6561 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:12.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f8aa00640 1 -- 172.21.15.102:0/72145454 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c039070 con 0x7f9f8c0c7350 2024-04-20T21:29:12.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f8aa00640 1 -- 172.21.15.102:0/72145454 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c02cd00 con 0x7f9f8c0c7350 2024-04-20T21:29:12.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f8aa00640 1 -- 172.21.15.102:0/72145454 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c034660 con 0x7f9f8c0c7350 2024-04-20T21:29:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f92000640 1 -- 172.21.15.102:0/72145454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c7350 msgr2=0x7f9f8c0c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f92000640 1 --2- 172.21.15.102:0/72145454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c7350 0x7f9f8c0c9ef0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c15f9a0 tx=0x7f9f7c02bf50 comp rx=0 tx=0).stop 2024-04-20T21:29:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f92000640 1 -- 172.21.15.102:0/72145454 shutdown_connections 2024-04-20T21:29:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f92000640 1 --2- 172.21.15.102:0/72145454 >> v2:172.21.15.193:3300/0 conn(0x7f9f8c0c8670 0x7f9f8c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f92000640 1 --2- 172.21.15.102:0/72145454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c7350 0x7f9f8c0c9ef0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.098+0000 7f9f92000640 1 -- 172.21.15.102:0/72145454 >> 172.21.15.102:0/72145454 conn(0x7f9f8c06dd40 msgr2=0x7f9f8c074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.099+0000 7f9f92000640 1 -- 172.21.15.102:0/72145454 shutdown_connections 2024-04-20T21:29:12.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.099+0000 7f9f92000640 1 -- 172.21.15.102:0/72145454 wait complete. 2024-04-20T21:29:12.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.100+0000 7f9f92000640 1 Processor -- start 2024-04-20T21:29:12.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.100+0000 7f9f92000640 1 -- start start 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f92000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f8c0716d0 0x7f9f8c06f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f92000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f8c0c7350 0x7f9f8c1f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f92000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c8670 0x7f9f8c1f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f92000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15bfe0 con 0x7f9f8c0c8670 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f92000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15bd00 con 0x7f9f8c0c7350 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f92000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c06c050 con 0x7f9f8c0716d0 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c8670 0x7f9f8c1f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c8670 0x7f9f8c1f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47226/0 (socket says 172.21.15.102:47226) 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.101+0000 7f9f8be00640 1 -- 172.21.15.102:0/936522867 learned_addr learned my addr 172.21.15.102:0/936522867 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f82a00640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f8c0c7350 0x7f9f8c1f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:12.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f8b400640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f8c0716d0 0x7f9f8c06f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f8b400640 1 -- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f8c0c7350 msgr2=0x7f9f8c1f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f8b400640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f8c0c7350 0x7f9f8c1f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f8b400640 1 -- 172.21.15.102:0/936522867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c8670 msgr2=0x7f9f8c1f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f8b400640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c8670 0x7f9f8c1f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f8b400640 1 -- 172.21.15.102:0/936522867 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c15a580 con 0x7f9f8c0716d0 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f8be00640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c8670 0x7f9f8c1f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f8b400640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f8c0716d0 0x7f9f8c06f340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c06c600 tx=0x7f9f7c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f83e00640 1 -- 172.21.15.102:0/936522867 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c044020 con 0x7f9f8c0716d0 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.102+0000 7f9f83e00640 1 -- 172.21.15.102:0/936522867 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c03a040 con 0x7f9f8c0716d0 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.103+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c0cc8e0 con 0x7f9f8c0716d0 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.103+0000 7f9f83e00640 1 -- 172.21.15.102:0/936522867 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c039050 con 0x7f9f8c0716d0 2024-04-20T21:29:12.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.103+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c0cd1a0 con 0x7f9f8c0716d0 2024-04-20T21:29:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.104+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c074340 con 0x7f9f8c0716d0 2024-04-20T21:29:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.105+0000 7f9f83e00640 1 -- 172.21.15.102:0/936522867 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f7c003b90 con 0x7f9f8c0716d0 2024-04-20T21:29:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.105+0000 7f9f83e00640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.106+0000 7f9f83e00640 1 -- 172.21.15.102:0/936522867 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f9f7c0ae7b0 con 0x7f9f8c0716d0 2024-04-20T21:29:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.106+0000 7f9f82a00640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.106+0000 7f9f82a00640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6806c8f0 0x7f9f6806edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c13ce80 tx=0x7f9f74007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.110+0000 7f9f83e00640 1 -- 172.21.15.102:0/936522867 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9f7c04e020 con 0x7f9f8c0716d0 2024-04-20T21:29:12.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.350+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f8c0ca9b0 con 0x7f9f8c0716d0 2024-04-20T21:29:12.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.351+0000 7f9f83e00640 1 -- 172.21.15.102:0/936522867 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6028 (secure 0 0 0) 0x7f9f7c07bf70 con 0x7f9f8c0716d0 2024-04-20T21:29:12.351 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:29:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:29:12.351 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 42 2024-04-20T21:29:12.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.354+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6806c8f0 msgr2=0x7f9f6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:12.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.354+0000 7f9f92000640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6806c8f0 0x7f9f6806edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c13ce80 tx=0x7f9f74007d50 comp rx=0 tx=0).stop 2024-04-20T21:29:12.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.354+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f8c0716d0 msgr2=0x7f9f8c06f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:12.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.354+0000 7f9f92000640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f8c0716d0 0x7f9f8c06f340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c06c600 tx=0x7f9f7c004a40 comp rx=0 tx=0).stop 2024-04-20T21:29:12.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.355+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 shutdown_connections 2024-04-20T21:29:12.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.355+0000 7f9f92000640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c0c8670 0x7f9f8c1f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.355+0000 7f9f92000640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.355+0000 7f9f92000640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f8c0c7350 0x7f9f8c1f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.355+0000 7f9f92000640 1 --2- 172.21.15.102:0/936522867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f8c0716d0 0x7f9f8c06f340 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:12.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.355+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 >> 172.21.15.102:0/936522867 conn(0x7f9f8c06dd40 msgr2=0x7f9f8c071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:12.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.355+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 shutdown_connections 2024-04-20T21:29:12.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:12.355+0000 7f9f92000640 1 -- 172.21.15.102:0/936522867 wait complete. 2024-04-20T21:29:12.372 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-04-20T21:29:14.373 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.595+0000 7fafe3200640 1 Processor -- start 2024-04-20T21:29:14.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.595+0000 7fafe3200640 1 -- start start 2024-04-20T21:29:14.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafe3200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe40ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:14.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafe3200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fafe40c67d0 0x7fafe40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:14.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafe3200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe40ca310 con 0x7fafe40c95c0 2024-04-20T21:29:14.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafe3200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fafe40c6340 con 0x7fafe40c67d0 2024-04-20T21:29:14.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafe3200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fafe40ca5f0 con 0x7fafe40c99a0 2024-04-20T21:29:14.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafd9400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe40ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:14.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafe2800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fafe40c67d0 0x7fafe40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:14.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafe1e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fafe40c95c0 0x7fafe40c8050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:35124/0 (socket says 172.21.15.102:35124) 2024-04-20T21:29:14.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.597+0000 7fafe1e00640 1 -- 172.21.15.102:0/2527467436 learned_addr learned my addr 172.21.15.102:0/2527467436 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:29:14.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.598+0000 7fafe2800640 1 -- 172.21.15.102:0/2527467436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 msgr2=0x7fafe40ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:14.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.598+0000 7fafe2800640 1 --2- 172.21.15.102:0/2527467436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe40ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.598+0000 7fafe2800640 1 -- 172.21.15.102:0/2527467436 >> v1:172.21.15.193:6789/0 conn(0x7fafe40c95c0 legacy=0x7fafe40c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:14.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.598+0000 7fafe2800640 1 -- 172.21.15.102:0/2527467436 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe40ca8d0 con 0x7fafe40c67d0 2024-04-20T21:29:14.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.598+0000 7fafd9400640 1 --2- 172.21.15.102:0/2527467436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe40ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:29:14.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.598+0000 7fafe1400640 1 -- 172.21.15.102:0/2527467436 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1076441854 0 0) 0x7fafe40ca310 con 0x7fafe40c95c0 2024-04-20T21:29:14.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.599+0000 7fafe2800640 1 --2- 172.21.15.102:0/2527467436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 0x7fafe40cee00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0097a0 tx=0x7fafdc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3276ca4d6015030 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:14.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.599+0000 7fafe1400640 1 -- 172.21.15.102:0/2527467436 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0315a0 con 0x7fafe40c67d0 2024-04-20T21:29:14.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.599+0000 7fafe1400640 1 -- 172.21.15.102:0/2527467436 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc037070 con 0x7fafe40c67d0 2024-04-20T21:29:14.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.599+0000 7fafe1400640 1 -- 172.21.15.102:0/2527467436 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc031960 con 0x7fafe40c67d0 2024-04-20T21:29:14.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.600+0000 7fafe3200640 1 -- 172.21.15.102:0/2527467436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 msgr2=0x7fafe40cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:14.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.600+0000 7fafe3200640 1 --2- 172.21.15.102:0/2527467436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 0x7fafe40cee00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0097a0 tx=0x7fafdc02b840 comp rx=0 tx=0).stop 2024-04-20T21:29:14.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.600+0000 7fafe3200640 1 -- 172.21.15.102:0/2527467436 shutdown_connections 2024-04-20T21:29:14.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.600+0000 7fafe3200640 1 --2- 172.21.15.102:0/2527467436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 0x7fafe40cee00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.600+0000 7fafe3200640 1 --2- 172.21.15.102:0/2527467436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe40ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.600+0000 7fafe3200640 1 -- 172.21.15.102:0/2527467436 >> 172.21.15.102:0/2527467436 conn(0x7fafe4074ea0 msgr2=0x7fafe4075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:14.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.600+0000 7fafe3200640 1 -- 172.21.15.102:0/2527467436 shutdown_connections 2024-04-20T21:29:14.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.600+0000 7fafe3200640 1 -- 172.21.15.102:0/2527467436 wait complete. 2024-04-20T21:29:14.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.602+0000 7fafe3200640 1 Processor -- start 2024-04-20T21:29:14.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.602+0000 7fafe3200640 1 -- start start 2024-04-20T21:29:14.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafe3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 0x7fafe406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:14.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafe3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafe40c95c0 0x7fafe415a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:14.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafe3200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe41f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:14.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafe3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fafe40c9fb0 con 0x7fafe40c95c0 2024-04-20T21:29:14.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafe3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fafe40c5a80 con 0x7fafe40c67d0 2024-04-20T21:29:14.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafe3200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fafe4152c60 con 0x7fafe40c99a0 2024-04-20T21:29:14.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafd9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafe40c95c0 0x7fafe415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:14.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafe1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 0x7fafe406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:14.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafe2800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe41f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:14.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafd9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafe40c95c0 0x7fafe415a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47230/0 (socket says 172.21.15.102:47230) 2024-04-20T21:29:14.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.603+0000 7fafd9400640 1 -- 172.21.15.102:0/109060200 learned_addr learned my addr 172.21.15.102:0/109060200 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafe2800640 1 -- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 msgr2=0x7fafe406d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafe2800640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 0x7fafe406d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafe2800640 1 -- 172.21.15.102:0/109060200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafe40c95c0 msgr2=0x7fafe415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafe2800640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafe40c95c0 0x7fafe415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafe2800640 1 -- 172.21.15.102:0/109060200 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe415b400 con 0x7fafe40c99a0 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafe2800640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe41f3dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fafdc033000 tx=0x7fafdc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafdaa00640 1 -- 172.21.15.102:0/109060200 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc032630 con 0x7fafe40c99a0 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafdaa00640 1 -- 172.21.15.102:0/109060200 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc0327f0 con 0x7fafe40c99a0 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafdaa00640 1 -- 172.21.15.102:0/109060200 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc032be0 con 0x7fafe40c99a0 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.604+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe4158f90 con 0x7fafe40c99a0 2024-04-20T21:29:14.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.605+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe4159230 con 0x7fafe40c99a0 2024-04-20T21:29:14.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.606+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe40ca310 con 0x7fafe40c99a0 2024-04-20T21:29:14.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.607+0000 7fafdaa00640 1 -- 172.21.15.102:0/109060200 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafdc04e430 con 0x7fafe40c99a0 2024-04-20T21:29:14.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.608+0000 7fafdaa00640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafc006c920 0x7fafc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:14.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.608+0000 7fafdaa00640 1 -- 172.21.15.102:0/109060200 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7fafdc02f030 con 0x7fafe40c99a0 2024-04-20T21:29:14.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.608+0000 7fafe1e00640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafc006c920 0x7fafc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:14.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.609+0000 7fafe1e00640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafc006c920 0x7fafc006ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fafd40012e0 tx=0x7fafd4001700 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:14.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.612+0000 7fafdaa00640 1 -- 172.21.15.102:0/109060200 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fafe40ca310 con 0x7fafe40c99a0 2024-04-20T21:29:14.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.875+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafe404fd60 con 0x7fafe40c99a0 2024-04-20T21:29:14.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.876+0000 7fafdaa00640 1 -- 172.21.15.102:0/109060200 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6856 (secure 0 0 0) 0x7fafdc040070 con 0x7fafe40c99a0 2024-04-20T21:29:14.876 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:29:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:29:14.876 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 43 2024-04-20T21:29:14.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafc006c920 msgr2=0x7fafc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:14.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafc006c920 0x7fafc006ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fafd40012e0 tx=0x7fafd4001700 comp rx=0 tx=0).stop 2024-04-20T21:29:14.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 msgr2=0x7fafe41f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:14.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe41f3dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fafdc033000 tx=0x7fafdc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 shutdown_connections 2024-04-20T21:29:14.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fafe40c99a0 0x7fafe41f3dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafe40c95c0 0x7fafe415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fafc006c920 0x7fafc006ede0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 --2- 172.21.15.102:0/109060200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafe40c67d0 0x7fafe406d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:14.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 >> 172.21.15.102:0/109060200 conn(0x7fafe4074ea0 msgr2=0x7fafe4150d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:14.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 shutdown_connections 2024-04-20T21:29:14.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:29:14.879+0000 7fafe3200640 1 -- 172.21.15.102:0/109060200 wait complete. 2024-04-20T21:29:14.896 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-04-20T21:29:14.896 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 117 secs before thrashing 2024-04-20T21:29:15.876 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:29:15.877+0000 7f34cb400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:29:27.152 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:27.160 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:31.454 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 2% (374/12485) Updating files: 3% (375/12485) Updating files: 3% (443/12485) Updating files: 4% (500/12485) Updating files: 4% (584/12485) Updating files: 5% (625/12485) Updating files: 5% (717/12485) Updating files: 6% (750/12485) Updating files: 6% (834/12485) Updating files: 7% (874/12485) Updating files: 7% (974/12485) Updating files: 8% (999/12485) Updating files: 9% (1124/12485) Updating files: 9% (1126/12485) Updating files: 9% (1221/12485) Updating files: 10% (1249/12485) Updating files: 10% (1342/12485) Updating files: 11% (1374/12485) Updating files: 12% (1499/12485) Updating files: 12% (1503/12485) Updating files: 13% (1624/12485) Updating files: 13% (1635/12485) Updating files: 13% (1659/12485) Updating files: 14% (1748/12485) Updating files: 14% (1808/12485) Updating files: 15% (1873/12485) Updating files: 15% (1928/12485) Updating files: 16% (1998/12485) Updating files: 16% (2105/12485) Updating files: 17% (2123/12485) Updating files: 18% (2248/12485) Updating files: 18% (2311/12485) Updating files: 19% (2373/12485) Updating files: 19% (2457/12485) Updating files: 20% (2497/12485) Updating files: 20% (2577/12485) Updating files: 21% (2622/12485) Updating files: 21% (2667/12485) Updating files: 22% (2747/12485) Updating files: 22% (2773/12485) Updating files: 23% (2872/12485) Updating files: 23% (2976/12485) Updating files: 24% (2997/12485) Updating files: 24% (3110/12485) Updating files: 25% (3122/12485) Updating files: 25% (3189/12485) Updating files: 25% (3195/12485) Updating files: 26% (3247/12485) Updating files: 26% (3265/12485) Updating files: 26% (3359/12485) Updating files: 27% (3371/12485) Updating files: 27% (3430/12485) Updating files: 28% (3496/12485) Updating files: 28% (3563/12485) Updating files: 29% (3621/12485) Updating files: 29% (3688/12485) Updating files: 30% (3746/12485) Updating files: 31% (3871/12485) Updating files: 31% (3879/12485) Updating files: 31% (3944/12485) Updating files: 32% (3996/12485) Updating files: 32% (4095/12485) Updating files: 33% (4121/12485) Updating files: 33% (4140/12485) Updating files: 33% (4201/12485) Updating files: 34% (4245/12485) Updating files: 34% (4320/12485) Updating files: 35% (4370/12485) Updating files: 35% (4422/12485) Updating files: 36% (4495/12485) Updating files: 36% (4580/12485) Updating files: 36% (4608/12485) Updating files: 37% (4620/12485) Updating files: 38% (4745/12485) Updating files: 38% (4823/12485) Updating files: 39% (4870/12485) Updating files: 39% (4898/12485) Updating files: 39% (4945/12485) Updating files: 39% (4967/12485) Updating files: 40% (4994/12485) Updating files: 40% (5046/12485) Updating files: 40% (5048/12485) Updating files: 40% (5060/12485) Updating files: 41% (5119/12485) Updating files: 41% (5241/12485) Updating files: 42% (5244/12485) Updating files: 42% (5336/12485) Updating files: 43% (5369/12485) Updating files: 43% (5482/12485) Updating files: 44% (5494/12485) Updating files: 45% (5619/12485) Updating files: 45% (5680/12485) Updating files: 45% (5737/12485) Updating files: 46% (5744/12485) Updating files: 46% (5810/12485) Updating files: 46% (5824/12485) Updating files: 47% (5868/12485) Updating files: 47% (5931/12485) Updating files: 48% (5993/12485) Updating files: 48% (6034/12485) Updating files: 49% (6118/12485) Updating files: 49% (6141/12485) Updating files: 50% (6243/12485) Updating files: 50% (6305/12485) Updating files: 50% (6333/12485) Updating files: 51% (6368/12485) Updating files: 51% (6471/12485) Updating files: 52% (6493/12485) Updating files: 52% (6549/12485) Updating files: 53% (6618/12485) Updating files: 53% (6690/12485) Updating files: 54% (6742/12485) Updating files: 54% (6788/12485) Updating files: 55% (6867/12485) Updating files: 55% (6960/12485) Updating files: 56% (6992/12485) Updating files: 56% (7013/12485) Updating files: 57% (7117/12485) Updating files: 57% (7236/12485) Updating files: 58% (7242/12485) Updating files: 58% (7346/12485) Updating files: 59% (7367/12485) Updating files: 59% (7468/12485) Updating files: 59% (7469/12485) Updating files: 60% (7491/12485) Updating files: 61% (7616/12485) Updating files: 61% (7712/12485) Updating files: 62% (7741/12485) Updating files: 63% (7866/12485) Updating files: 63% (7907/12485) Updating files: 63% (7980/12485) Updating files: 64% (7991/12485) Updating files: 64% (8087/12485) Updating files: 65% (8116/12485) Updating files: 66% (8241/12485) Updating files: 66% (8302/12485) Updating files: 67% (8365/12485) Updating files: 67% (8377/12485) Updating files: 67% (8420/12485) Updating files: 67% (8488/12485) Updating files: 68% (8490/12485) Updating files: 68% (8499/12485) Updating files: 69% (8615/12485) Updating files: 69% (8643/12485) Updating files: 69% (8699/12485) Updating files: 70% (8740/12485) Updating files: 70% (8779/12485) Updating files: 70% (8825/12485) Updating files: 71% (8865/12485) Updating files: 71% (8909/12485) Updating files: 71% (8987/12485) Updating files: 72% (8990/12485) Updating files: 72% (9083/12485) Updating files: 72% (9097/12485) Updating files: 73% (9115/12485) Updating files: 74% (9239/12485) Updating files: 74% (9342/12485) Updating files: 74% (9356/12485) Updating files: 75% (9364/12485) Updating files: 75% (9435/12485) Updating files: 76% (9489/12485) Updating files: 76% (9526/12485) Updating files: 77% (9614/12485) Updating files: 77% (9698/12485) Updating files: 78% (9739/12485) Updating files: 78% (9835/12485) Updating files: 79% (9864/12485) Updating files: 79% (9887/12485) Updating files: 80% (9988/12485) Updating files: 80% (10022/12485) Updating files: 81% (10113/12485) Updating files: 81% (10126/12485) Updating files: 81% (10198/12485) Updating files: 82% (10238/12485) Updating files: 82% (10242/12485) Updating files: 83% (10363/12485) Updating files: 83% (10407/12485) Updating files: 83% (10408/12485) Updating files: 83% (10412/12485) Updating files: 84% (10488/12485) Updating files: 84% (10537/12485) Updating files: 85% (10613/12485) Updating files: 85% (10661/12485) Updating files: 86% (10738/12485) Updating files: 86% (10739/12485) Updating files: 86% (10811/12485) Updating files: 87% (10862/12485) Updating files: 88% (10987/12485) Updating files: 88% (11096/12485) Updating files: 89% (11112/12485) Updating files: 89% (11118/12485) Updating files: 90% (11237/12485) Updating files: 90% (11287/12485) Updating files: 91% (11362/12485) Updating files: 91% (11389/12485) Updating files: 92% (11487/12485) Updating files: 92% (11500/12485) Updating files: 92% (11531/12485) Updating files: 92% (11599/12485) Updating files: 93% (11612/12485) Updating files: 94% (11736/12485) Updating files: 94% (11766/12485) Updating files: 94% (11834/12485) Updating files: 95% (11861/12485) Updating files: 96% (11986/12485) Updating files: 96% (11992/12485) Updating files: 96% (12006/12485) Updating files: 96% (12050/12485) Updating files: 97% (12111/12485) Updating files: 98% (12236/12485) Updating files: 98% (12244/12485) Updating files: 99% (12361/12485) Updating files: 99% (12413/12485) Updating files: 100% (12485/12485) Updating files: 100% (12485/12485), done. 2024-04-20T21:29:32.038 INFO:tasks.workunit.client.0.smithi102.stderr:+ trap - EXIT 2024-04-20T21:29:32.038 INFO:tasks.workunit.client.0.smithi102.stderr:+ cd ceph 2024-04-20T21:29:32.040 INFO:tasks.workunit.client.0.smithi102.stderr:++ seq 1 90 2024-04-20T21:29:32.043 INFO:tasks.workunit.client.0.smithi102.stderr:+ versions='1 2024-04-20T21:29:32.043 INFO:tasks.workunit.client.0.smithi102.stderr:2 2024-04-20T21:29:32.043 INFO:tasks.workunit.client.0.smithi102.stderr:3 2024-04-20T21:29:32.043 INFO:tasks.workunit.client.0.smithi102.stderr:4 2024-04-20T21:29:32.043 INFO:tasks.workunit.client.0.smithi102.stderr:5 2024-04-20T21:29:32.043 INFO:tasks.workunit.client.0.smithi102.stderr:6 2024-04-20T21:29:32.043 INFO:tasks.workunit.client.0.smithi102.stderr:7 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:8 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:9 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:10 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:11 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:12 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:13 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:14 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:15 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:16 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:17 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:18 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:19 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:20 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:21 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:22 2024-04-20T21:29:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:23 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:24 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:25 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:26 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:27 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:28 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:29 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:30 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:31 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:32 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:33 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:34 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:35 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:36 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:37 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:38 2024-04-20T21:29:32.045 INFO:tasks.workunit.client.0.smithi102.stderr:39 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:40 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:41 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:42 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:43 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:44 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:45 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:46 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:47 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:48 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:49 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:50 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:51 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:52 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:53 2024-04-20T21:29:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:54 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:55 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:56 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:57 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:58 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:59 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:60 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:61 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:62 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:63 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:64 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:65 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:66 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:67 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:68 2024-04-20T21:29:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:69 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:70 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:71 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:72 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:73 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:74 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:75 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:76 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:77 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:78 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:79 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:80 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:81 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:82 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:83 2024-04-20T21:29:32.048 INFO:tasks.workunit.client.0.smithi102.stderr:84 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:85 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:86 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:87 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:88 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:89 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:90' 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 48 ']' 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.1 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.1 2024-04-20T21:29:32.049 INFO:tasks.workunit.client.0.smithi102.stdout:v0.1 2024-04-20T21:29:32.050 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.1 2024-04-20T21:29:57.299 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:57.308 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:27.549 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:27.560 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:57.693 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:57.702 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:11.897 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:12.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.121+0000 7fa489800640 1 Processor -- start 2024-04-20T21:31:12.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.121+0000 7fa489800640 1 -- start start 2024-04-20T21:31:12.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.122+0000 7fa489800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa4840b1f40 0x7fa48415c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.122+0000 7fa489800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b2320 0x7fa48415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.122+0000 7fa489800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa4840c7a40 0x7fa48415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.122+0000 7fa489800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa4840c2190 con 0x7fa4840b1f40 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.122+0000 7fa489800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa4840caa30 con 0x7fa4840c7a40 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.122+0000 7fa489800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa4840c98f0 con 0x7fa4840b2320 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b2320 0x7fa48415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa488e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa4840c7a40 0x7fa48415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b2320 0x7fa48415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:59242/0 (socket says 172.21.15.102:59242) 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483400640 1 -- 172.21.15.102:0/1948261982 learned_addr learned my addr 172.21.15.102:0/1948261982 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483e00640 1 --2- 172.21.15.102:0/1948261982 >> v2:172.21.15.102:3300/0 conn(0x7fa4840b1f40 0x7fa48415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483400640 1 -- 172.21.15.102:0/1948261982 >> v2:172.21.15.193:3300/0 conn(0x7fa4840c7a40 msgr2=0x7fa48415ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483400640 1 --2- 172.21.15.102:0/1948261982 >> v2:172.21.15.193:3300/0 conn(0x7fa4840c7a40 0x7fa48415ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483400640 1 -- 172.21.15.102:0/1948261982 >> v2:172.21.15.102:3300/0 conn(0x7fa4840b1f40 msgr2=0x7fa48415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:12.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483400640 1 --2- 172.21.15.102:0/1948261982 >> v2:172.21.15.102:3300/0 conn(0x7fa4840b1f40 0x7fa48415c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa483400640 1 -- 172.21.15.102:0/1948261982 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48415f250 con 0x7fa4840b2320 2024-04-20T21:31:12.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.123+0000 7fa488e00640 1 --2- 172.21.15.102:0/1948261982 >> v2:172.21.15.193:3300/0 conn(0x7fa4840c7a40 0x7fa48415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.124+0000 7fa483400640 1 --2- 172.21.15.102:0/1948261982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b2320 0x7fa48415e720 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4840c3a70 tx=0x7fa47002b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478178576c02dfbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.125+0000 7fa482a00640 1 -- 172.21.15.102:0/1948261982 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470034070 con 0x7fa4840b2320 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.125+0000 7fa482a00640 1 -- 172.21.15.102:0/1948261982 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470030860 con 0x7fa4840b2320 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.125+0000 7fa482a00640 1 -- 172.21.15.102:0/1948261982 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470030c30 con 0x7fa4840b2320 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.125+0000 7fa489800640 1 -- 172.21.15.102:0/1948261982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b2320 msgr2=0x7fa48415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.125+0000 7fa489800640 1 --2- 172.21.15.102:0/1948261982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b2320 0x7fa48415e720 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4840c3a70 tx=0x7fa47002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.126+0000 7fa489800640 1 -- 172.21.15.102:0/1948261982 shutdown_connections 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.126+0000 7fa489800640 1 --2- 172.21.15.102:0/1948261982 >> v2:172.21.15.193:3300/0 conn(0x7fa4840c7a40 0x7fa48415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.126+0000 7fa489800640 1 --2- 172.21.15.102:0/1948261982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b2320 0x7fa48415e720 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.126+0000 7fa489800640 1 --2- 172.21.15.102:0/1948261982 >> v2:172.21.15.102:3300/0 conn(0x7fa4840b1f40 0x7fa48415c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.126+0000 7fa489800640 1 -- 172.21.15.102:0/1948261982 >> 172.21.15.102:0/1948261982 conn(0x7fa4840c9b40 msgr2=0x7fa4840c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.126+0000 7fa489800640 1 -- 172.21.15.102:0/1948261982 shutdown_connections 2024-04-20T21:31:12.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.126+0000 7fa489800640 1 -- 172.21.15.102:0/1948261982 wait complete. 2024-04-20T21:31:12.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.128+0000 7fa489800640 1 Processor -- start 2024-04-20T21:31:12.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.129+0000 7fa489800640 1 -- start start 2024-04-20T21:31:12.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa489800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b1f40 0x7fa484071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa489800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4840b2320 0x7fa4840cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa489800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4840c7a40 0x7fa4840cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa489800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840c9320 con 0x7fa4840c7a40 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa489800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840c9610 con 0x7fa4840b2320 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa489800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa48415cb60 con 0x7fa4840b1f40 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa488e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4840c7a40 0x7fa4840cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa488e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4840c7a40 0x7fa4840cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43210/0 (socket says 172.21.15.102:43210) 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa488e00640 1 -- 172.21.15.102:0/1240440209 learned_addr learned my addr 172.21.15.102:0/1240440209 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa483e00640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b1f40 0x7fa484071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.130+0000 7fa483400640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4840b2320 0x7fa4840cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.131+0000 7fa488e00640 1 -- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b1f40 msgr2=0x7fa484071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.131+0000 7fa488e00640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b1f40 0x7fa484071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.131+0000 7fa488e00640 1 -- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4840b2320 msgr2=0x7fa4840cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.131+0000 7fa488e00640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4840b2320 0x7fa4840cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.131+0000 7fa488e00640 1 -- 172.21.15.102:0/1240440209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa484073260 con 0x7fa4840c7a40 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.131+0000 7fa483e00640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b1f40 0x7fa484071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.131+0000 7fa488e00640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4840c7a40 0x7fa4840cca50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00efc0 tx=0x7fa47c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.132+0000 7fa47be00640 1 -- 172.21.15.102:0/1240440209 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c018020 con 0x7fa4840c7a40 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.132+0000 7fa47be00640 1 -- 172.21.15.102:0/1240440209 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c014480 con 0x7fa4840c7a40 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.132+0000 7fa47be00640 1 -- 172.21.15.102:0/1240440209 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c014870 con 0x7fa4840c7a40 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.132+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4841f4e50 con 0x7fa4840c7a40 2024-04-20T21:31:12.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.132+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841f5410 con 0x7fa4840c7a40 2024-04-20T21:31:12.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.133+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48404fd60 con 0x7fa4840c7a40 2024-04-20T21:31:12.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.135+0000 7fa47be00640 1 -- 172.21.15.102:0/1240440209 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa47c005d10 con 0x7fa4840c7a40 2024-04-20T21:31:12.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.136+0000 7fa47be00640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa46406c920 0x7fa46406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.136+0000 7fa47be00640 1 -- 172.21.15.102:0/1240440209 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7fa47c08cd50 con 0x7fa4840c7a40 2024-04-20T21:31:12.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.136+0000 7fa483e00640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa46406c920 0x7fa46406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.137+0000 7fa483e00640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa46406c920 0x7fa46406ede0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa474009eb0 tx=0x7fa474009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.141+0000 7fa47be00640 1 -- 172.21.15.102:0/1240440209 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa47c022030 con 0x7fa4840c7a40 2024-04-20T21:31:12.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.395+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4840ca810 con 0x7fa4840c7a40 2024-04-20T21:31:12.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.396+0000 7fa47be00640 1 -- 172.21.15.102:0/1240440209 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6856 (secure 0 0 0) 0x7fa47c010040 con 0x7fa4840c7a40 2024-04-20T21:31:12.395 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:31:12.396 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 43 2024-04-20T21:31:12.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa46406c920 msgr2=0x7fa46406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa46406c920 0x7fa46406ede0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa474009eb0 tx=0x7fa474009190 comp rx=0 tx=0).stop 2024-04-20T21:31:12.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4840c7a40 msgr2=0x7fa4840cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4840c7a40 0x7fa4840cca50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00efc0 tx=0x7fa47c00c490 comp rx=0 tx=0).stop 2024-04-20T21:31:12.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 shutdown_connections 2024-04-20T21:31:12.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4840c7a40 0x7fa4840cca50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa46406c920 0x7fa46406ede0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4840b2320 0x7fa4840cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 --2- 172.21.15.102:0/1240440209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4840b1f40 0x7fa484071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 >> 172.21.15.102:0/1240440209 conn(0x7fa4840c9b40 msgr2=0x7fa484067730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 shutdown_connections 2024-04-20T21:31:12.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.399+0000 7fa489800640 1 -- 172.21.15.102:0/1240440209 wait complete. 2024-04-20T21:31:12.416 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:31:12.417 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:31:12.417 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:12.454 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:31:12.454 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:12.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.698+0000 7f1e5bc00640 1 Processor -- start 2024-04-20T21:31:12.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.698+0000 7f1e5bc00640 1 -- start start 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e5bc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e540c8db0 0x7f1e54070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e5bc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1e54071330 0x7f1e541628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e5bc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e54158e80 con 0x7f1e54152170 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e5bc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1e540c6dc0 con 0x7f1e54071330 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e5bc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1e54071f70 con 0x7f1e540c8db0 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e59c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1e54071330 0x7f1e541628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e59c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1e54071330 0x7f1e541628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43212/0 (socket says 172.21.15.102:43212) 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e59c00640 1 -- 172.21.15.102:0/1599984655 learned_addr learned my addr 172.21.15.102:0/1599984655 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.699+0000 7f1e4be00640 1 --2- 172.21.15.102:0/1599984655 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e540c8db0 0x7f1e54070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.700+0000 7f1e59c00640 1 -- 172.21.15.102:0/1599984655 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e540c8db0 msgr2=0x7f1e54070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:12.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.700+0000 7f1e59c00640 1 --2- 172.21.15.102:0/1599984655 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e540c8db0 0x7f1e54070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.700+0000 7f1e59c00640 1 -- 172.21.15.102:0/1599984655 >> v1:172.21.15.193:6789/0 conn(0x7f1e54152170 legacy=0x7f1e54158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.700+0000 7f1e59c00640 1 -- 172.21.15.102:0/1599984655 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e54072210 con 0x7f1e54071330 2024-04-20T21:31:12.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.701+0000 7f1e59c00640 1 --2- 172.21.15.102:0/1599984655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54071330 0x7f1e541628a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d150 tx=0x7f1e4c02e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c375639011e4d09b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.701+0000 7f1e53e00640 1 -- 172.21.15.102:0/1599984655 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00b0d0 con 0x7f1e54071330 2024-04-20T21:31:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.701+0000 7f1e53e00640 1 -- 172.21.15.102:0/1599984655 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4c00b270 con 0x7f1e54071330 2024-04-20T21:31:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.701+0000 7f1e53e00640 1 -- 172.21.15.102:0/1599984655 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c009600 con 0x7f1e54071330 2024-04-20T21:31:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.702+0000 7f1e5bc00640 1 -- 172.21.15.102:0/1599984655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54071330 msgr2=0x7f1e541628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.702+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/1599984655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54071330 0x7f1e541628a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d150 tx=0x7f1e4c02e160 comp rx=0 tx=0).stop 2024-04-20T21:31:12.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.702+0000 7f1e5bc00640 1 -- 172.21.15.102:0/1599984655 shutdown_connections 2024-04-20T21:31:12.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.702+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/1599984655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54071330 0x7f1e541628a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.702+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/1599984655 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e540c8db0 0x7f1e54070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.702+0000 7f1e5bc00640 1 -- 172.21.15.102:0/1599984655 >> 172.21.15.102:0/1599984655 conn(0x7f1e540c9390 msgr2=0x7f1e540c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.702+0000 7f1e5bc00640 1 -- 172.21.15.102:0/1599984655 shutdown_connections 2024-04-20T21:31:12.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.702+0000 7f1e5bc00640 1 -- 172.21.15.102:0/1599984655 wait complete. 2024-04-20T21:31:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.704+0000 7f1e5bc00640 1 Processor -- start 2024-04-20T21:31:12.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.704+0000 7f1e5bc00640 1 -- start start 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e5bc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e54071330 0x7f1e540cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e5bc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e540c8db0 0x7f1e541590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e5bc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54152170 0x7f1e5406e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e5bc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1e540c9190 con 0x7f1e54152170 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e5bc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e540724b0 con 0x7f1e540c8db0 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e5bc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1e54158250 con 0x7f1e54071330 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54152170 0x7f1e5406e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54152170 0x7f1e5406e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43224/0 (socket says 172.21.15.102:43224) 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59c00640 1 -- 172.21.15.102:0/364878033 learned_addr learned my addr 172.21.15.102:0/364878033 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59200640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e54071330 0x7f1e540cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e4be00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e540c8db0 0x7f1e541590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59c00640 1 -- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e54071330 msgr2=0x7f1e540cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59c00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e54071330 0x7f1e540cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59c00640 1 -- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e540c8db0 msgr2=0x7f1e541590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59c00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e540c8db0 0x7f1e541590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.705+0000 7f1e59c00640 1 -- 172.21.15.102:0/364878033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e540cb6e0 con 0x7f1e54152170 2024-04-20T21:31:12.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.706+0000 7f1e59c00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54152170 0x7f1e5406e7b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c02e770 tx=0x7f1e4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.706+0000 7f1e51600640 1 -- 172.21.15.102:0/364878033 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00b0d0 con 0x7f1e54152170 2024-04-20T21:31:12.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.706+0000 7f1e51600640 1 -- 172.21.15.102:0/364878033 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4c045070 con 0x7f1e54152170 2024-04-20T21:31:12.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.706+0000 7f1e51600640 1 -- 172.21.15.102:0/364878033 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00b720 con 0x7f1e54152170 2024-04-20T21:31:12.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.706+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e541f5020 con 0x7f1e54152170 2024-04-20T21:31:12.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.706+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e541f12b0 con 0x7f1e54152170 2024-04-20T21:31:12.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.708+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e541f10a0 con 0x7f1e54152170 2024-04-20T21:31:12.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.709+0000 7f1e51600640 1 -- 172.21.15.102:0/364878033 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e4c02ed10 con 0x7f1e54152170 2024-04-20T21:31:12.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.710+0000 7f1e51600640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1e3006c920 0x7f1e3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.710+0000 7f1e51600640 1 -- 172.21.15.102:0/364878033 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f1e4c0ae550 con 0x7f1e54152170 2024-04-20T21:31:12.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.710+0000 7f1e59200640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1e3006c920 0x7f1e3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.711+0000 7f1e59200640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1e3006c920 0x7f1e3006ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e440016e0 tx=0x7f1e44009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.715+0000 7f1e51600640 1 -- 172.21.15.102:0/364878033 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1e4c04e020 con 0x7f1e54152170 2024-04-20T21:31:12.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.966+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e54074340 con 0x7f1e54152170 2024-04-20T21:31:12.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.967+0000 7f1e51600640 1 -- 172.21.15.102:0/364878033 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6856 (secure 0 0 0) 0x7f1e4c082090 con 0x7f1e54152170 2024-04-20T21:31:12.967 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:31:12.967 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 43 2024-04-20T21:31:12.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1e3006c920 msgr2=0x7f1e3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1e3006c920 0x7f1e3006ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e440016e0 tx=0x7f1e44009040 comp rx=0 tx=0).stop 2024-04-20T21:31:12.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54152170 msgr2=0x7f1e5406e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54152170 0x7f1e5406e7b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c02e770 tx=0x7f1e4c004a40 comp rx=0 tx=0).stop 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 shutdown_connections 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e54152170 0x7f1e5406e7b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1e3006c920 0x7f1e3006ede0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e540c8db0 0x7f1e541590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 --2- 172.21.15.102:0/364878033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e54071330 0x7f1e540cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 >> 172.21.15.102:0/364878033 conn(0x7f1e540c9390 msgr2=0x7f1e540677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 shutdown_connections 2024-04-20T21:31:12.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:12.971+0000 7f1e5bc00640 1 -- 172.21.15.102:0/364878033 wait complete. 2024-04-20T21:31:12.992 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:31:14.993 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:15.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.201+0000 7f2c4c600640 1 Processor -- start 2024-04-20T21:31:15.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.201+0000 7f2c4c600640 1 -- start start 2024-04-20T21:31:15.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c4c600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2c4414fa50 0x7f2c440c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c4c600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c440c9eb0 0x7f2c44158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c4c600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c441536d0 con 0x7f2c44073a10 2024-04-20T21:31:15.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c4c600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f2c44159500 con 0x7f2c4414fa50 2024-04-20T21:31:15.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c4c600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2c440c76c0 con 0x7f2c440c9eb0 2024-04-20T21:31:15.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c49c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2c4414fa50 0x7f2c440c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c4a600640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2c44073a10 0x7f2c440c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45908/0 (socket says 172.21.15.102:45908) 2024-04-20T21:31:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c4a600640 1 -- 172.21.15.102:0/3025079932 learned_addr learned my addr 172.21.15.102:0/3025079932 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.203+0000 7f2c49200640 1 --2- 172.21.15.102:0/3025079932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c440c9eb0 0x7f2c44158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.204+0000 7f2c49c00640 1 -- 172.21.15.102:0/3025079932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c440c9eb0 msgr2=0x7f2c44158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.204+0000 7f2c49c00640 1 --2- 172.21.15.102:0/3025079932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c440c9eb0 0x7f2c44158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.204+0000 7f2c49c00640 1 -- 172.21.15.102:0/3025079932 >> v1:172.21.15.102:6789/0 conn(0x7f2c44073a10 legacy=0x7f2c440c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.204+0000 7f2c43e00640 1 -- 172.21.15.102:0/3025079932 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613596231 0 0) 0x7f2c3c001490 con 0x7f2c44073a10 2024-04-20T21:31:15.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.204+0000 7f2c49c00640 1 -- 172.21.15.102:0/3025079932 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c440c7960 con 0x7f2c4414fa50 2024-04-20T21:31:15.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.205+0000 7f2c49c00640 1 --2- 172.21.15.102:0/3025079932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c4414fa50 0x7f2c440c4830 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c440c7b30 tx=0x7f2c3802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe51d55263d4b85a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:15.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c43e00640 1 -- 172.21.15.102:0/3025079932 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38035020 con 0x7f2c4414fa50 2024-04-20T21:31:15.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c43e00640 1 -- 172.21.15.102:0/3025079932 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c38031890 con 0x7f2c4414fa50 2024-04-20T21:31:15.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c43e00640 1 -- 172.21.15.102:0/3025079932 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38031c60 con 0x7f2c4414fa50 2024-04-20T21:31:15.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c4c600640 1 -- 172.21.15.102:0/3025079932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c4414fa50 msgr2=0x7f2c440c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c4c600640 1 --2- 172.21.15.102:0/3025079932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c4414fa50 0x7f2c440c4830 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c440c7b30 tx=0x7f2c3802b930 comp rx=0 tx=0).stop 2024-04-20T21:31:15.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c4c600640 1 -- 172.21.15.102:0/3025079932 shutdown_connections 2024-04-20T21:31:15.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c4c600640 1 --2- 172.21.15.102:0/3025079932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c440c9eb0 0x7f2c44158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c4c600640 1 --2- 172.21.15.102:0/3025079932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c4414fa50 0x7f2c440c4830 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c4c600640 1 -- 172.21.15.102:0/3025079932 >> 172.21.15.102:0/3025079932 conn(0x7f2c440c9390 msgr2=0x7f2c440c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:15.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c4c600640 1 -- 172.21.15.102:0/3025079932 shutdown_connections 2024-04-20T21:31:15.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.206+0000 7f2c4c600640 1 -- 172.21.15.102:0/3025079932 wait complete. 2024-04-20T21:31:15.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.208+0000 7f2c4c600640 1 Processor -- start 2024-04-20T21:31:15.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.208+0000 7f2c4c600640 1 -- start start 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c4c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c44073a10 0x7f2c44158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c4c600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c440c9eb0 0x7f2c44156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c4c600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c4414fa50 0x7f2c441427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c4c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440c7c00 con 0x7f2c44073a10 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c4c600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440c3d90 con 0x7f2c440c9eb0 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c4c600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2c440c4030 con 0x7f2c4414fa50 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c49c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c44073a10 0x7f2c44158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c49c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c44073a10 0x7f2c44158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43246/0 (socket says 172.21.15.102:43246) 2024-04-20T21:31:15.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c49c00640 1 -- 172.21.15.102:0/372327630 learned_addr learned my addr 172.21.15.102:0/372327630 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:15.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c4a600640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c4414fa50 0x7f2c441427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.209+0000 7f2c49200640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c440c9eb0 0x7f2c44156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c49c00640 1 -- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c4414fa50 msgr2=0x7f2c441427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c49c00640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c4414fa50 0x7f2c441427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c49c00640 1 -- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c440c9eb0 msgr2=0x7f2c44156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c49c00640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c440c9eb0 0x7f2c44156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c49c00640 1 -- 172.21.15.102:0/372327630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c441e24b0 con 0x7f2c44073a10 2024-04-20T21:31:15.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c49c00640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c44073a10 0x7f2c44158ed0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009890 tx=0x7f2c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:15.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c41600640 1 -- 172.21.15.102:0/372327630 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38035040 con 0x7f2c44073a10 2024-04-20T21:31:15.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c41600640 1 -- 172.21.15.102:0/372327630 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c38030d20 con 0x7f2c44073a10 2024-04-20T21:31:15.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c41600640 1 -- 172.21.15.102:0/372327630 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380466c0 con 0x7f2c44073a10 2024-04-20T21:31:15.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.210+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c441e2750 con 0x7f2c44073a10 2024-04-20T21:31:15.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.211+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c44158090 con 0x7f2c44073a10 2024-04-20T21:31:15.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.212+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4406db10 con 0x7f2c44073a10 2024-04-20T21:31:15.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.213+0000 7f2c41600640 1 -- 172.21.15.102:0/372327630 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c38038020 con 0x7f2c44073a10 2024-04-20T21:31:15.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.214+0000 7f2c41600640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c2406c8f0 0x7f2c2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.214+0000 7f2c41600640 1 -- 172.21.15.102:0/372327630 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f2c380ae6a0 con 0x7f2c44073a10 2024-04-20T21:31:15.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.214+0000 7f2c49200640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c2406c8f0 0x7f2c2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.215+0000 7f2c49200640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c2406c8f0 0x7f2c2406edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c4406e950 tx=0x7f2c30009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:15.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.218+0000 7f2c41600640 1 -- 172.21.15.102:0/372327630 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2c3802d040 con 0x7f2c44073a10 2024-04-20T21:31:15.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.459+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c440c7320 con 0x7f2c44073a10 2024-04-20T21:31:15.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.460+0000 7f2c41600640 1 -- 172.21.15.102:0/372327630 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6856 (secure 0 0 0) 0x7f2c3807bf00 con 0x7f2c44073a10 2024-04-20T21:31:15.460 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:15.460 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:31:15.460 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 43 2024-04-20T21:31:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.463+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c2406c8f0 msgr2=0x7f2c2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.463+0000 7f2c4c600640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c2406c8f0 0x7f2c2406edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c4406e950 tx=0x7f2c30009190 comp rx=0 tx=0).stop 2024-04-20T21:31:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.463+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c44073a10 msgr2=0x7f2c44158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.463+0000 7f2c4c600640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c44073a10 0x7f2c44158ed0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009890 tx=0x7f2c380046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.464+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 shutdown_connections 2024-04-20T21:31:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.464+0000 7f2c4c600640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c4414fa50 0x7f2c441427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.464+0000 7f2c4c600640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2c2406c8f0 0x7f2c2406edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.464+0000 7f2c4c600640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c440c9eb0 0x7f2c44156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.464+0000 7f2c4c600640 1 --2- 172.21.15.102:0/372327630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c44073a10 0x7f2c44158ed0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.464+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 >> 172.21.15.102:0/372327630 conn(0x7f2c440c9390 msgr2=0x7f2c440c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.464+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 shutdown_connections 2024-04-20T21:31:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:15.464+0000 7f2c4c600640 1 -- 172.21.15.102:0/372327630 wait complete. 2024-04-20T21:31:15.483 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:31:17.484 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:17.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.697+0000 7f6c9a600640 1 Processor -- start 2024-04-20T21:31:17.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.697+0000 7f6c9a600640 1 -- start start 2024-04-20T21:31:17.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.698+0000 7f6c9a600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c940c75a0 0x7f6c940c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:17.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c9a600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c94073c70 0x7f6c940c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c9a600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c94153560 con 0x7f6c94153180 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c9a600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6c940c7a70 con 0x7f6c940c75a0 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c9a600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6c940c7d30 con 0x7f6c94073c70 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c8be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c940c75a0 0x7f6c940c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c8be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c940c75a0 0x7f6c940c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43258/0 (socket says 172.21.15.102:43258) 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c8be00640 1 -- 172.21.15.102:0/1852683741 learned_addr learned my addr 172.21.15.102:0/1852683741 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c99c00640 1 --2- 172.21.15.102:0/1852683741 >> v2:172.21.15.193:3300/0 conn(0x7f6c94073c70 0x7f6c940c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c8be00640 1 -- 172.21.15.102:0/1852683741 >> v2:172.21.15.193:3300/0 conn(0x7f6c94073c70 msgr2=0x7f6c940c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c8be00640 1 --2- 172.21.15.102:0/1852683741 >> v2:172.21.15.193:3300/0 conn(0x7f6c94073c70 0x7f6c940c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c8be00640 1 -- 172.21.15.102:0/1852683741 >> v1:172.21.15.193:6789/0 conn(0x7f6c94153180 legacy=0x7f6c94158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:17.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.699+0000 7f6c8be00640 1 -- 172.21.15.102:0/1852683741 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9415c4d0 con 0x7f6c940c75a0 2024-04-20T21:31:17.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.700+0000 7f6c8be00640 1 --2- 172.21.15.102:0/1852683741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940c5f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c7980 tx=0x7f6c8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc4a6fac84e37e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:17.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.701+0000 7f6c93e00640 1 -- 172.21.15.102:0/1852683741 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84034070 con 0x7f6c940c75a0 2024-04-20T21:31:17.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.701+0000 7f6c93e00640 1 -- 172.21.15.102:0/1852683741 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c84030860 con 0x7f6c940c75a0 2024-04-20T21:31:17.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.701+0000 7f6c93e00640 1 -- 172.21.15.102:0/1852683741 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84030c30 con 0x7f6c940c75a0 2024-04-20T21:31:17.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.701+0000 7f6c9a600640 1 -- 172.21.15.102:0/1852683741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 msgr2=0x7f6c940c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:17.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.701+0000 7f6c9a600640 1 --2- 172.21.15.102:0/1852683741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940c5f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c7980 tx=0x7f6c8402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.702+0000 7f6c9a600640 1 -- 172.21.15.102:0/1852683741 shutdown_connections 2024-04-20T21:31:17.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.702+0000 7f6c9a600640 1 --2- 172.21.15.102:0/1852683741 >> v2:172.21.15.193:3300/0 conn(0x7f6c94073c70 0x7f6c940c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.702+0000 7f6c9a600640 1 --2- 172.21.15.102:0/1852683741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940c5f30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.702+0000 7f6c9a600640 1 -- 172.21.15.102:0/1852683741 >> 172.21.15.102:0/1852683741 conn(0x7f6c940cc820 msgr2=0x7f6c940ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:17.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.702+0000 7f6c9a600640 1 -- 172.21.15.102:0/1852683741 shutdown_connections 2024-04-20T21:31:17.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.702+0000 7f6c9a600640 1 -- 172.21.15.102:0/1852683741 wait complete. 2024-04-20T21:31:17.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.703+0000 7f6c9a600640 1 Processor -- start 2024-04-20T21:31:17.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.703+0000 7f6c9a600640 1 -- start start 2024-04-20T21:31:17.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.704+0000 7f6c9a600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c94073c70 0x7f6c94156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:17.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.704+0000 7f6c9a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:17.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.704+0000 7f6c9a600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c94153180 0x7f6c94141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.704+0000 7f6c9a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6c94159550 con 0x7f6c940c75a0 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.704+0000 7f6c9a600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9415ca90 con 0x7f6c94153180 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.704+0000 7f6c9a600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9415d290 con 0x7f6c94073c70 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43266/0 (socket says 172.21.15.102:43266) 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c8be00640 1 -- 172.21.15.102:0/3719755103 learned_addr learned my addr 172.21.15.102:0/3719755103 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c99c00640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c94153180 0x7f6c94141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c99200640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c94073c70 0x7f6c94156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c8be00640 1 -- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c94073c70 msgr2=0x7f6c94156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c8be00640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c94073c70 0x7f6c94156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c8be00640 1 -- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c94153180 msgr2=0x7f6c94141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c8be00640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c94153180 0x7f6c94141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c8be00640 1 -- 172.21.15.102:0/3719755103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c94147b60 con 0x7f6c940c75a0 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.705+0000 7f6c99c00640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c94153180 0x7f6c94141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:17.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.706+0000 7f6c8be00640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940713c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6c8402b9e0 tx=0x7f6c840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:17.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.706+0000 7f6c91600640 1 -- 172.21.15.102:0/3719755103 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84034040 con 0x7f6c940c75a0 2024-04-20T21:31:17.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.706+0000 7f6c91600640 1 -- 172.21.15.102:0/3719755103 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c84039d90 con 0x7f6c940c75a0 2024-04-20T21:31:17.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.706+0000 7f6c91600640 1 -- 172.21.15.102:0/3719755103 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84043610 con 0x7f6c940c75a0 2024-04-20T21:31:17.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.706+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c94147e40 con 0x7f6c940c75a0 2024-04-20T21:31:17.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.706+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c94146620 con 0x7f6c940c75a0 2024-04-20T21:31:17.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.707+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c9404fd60 con 0x7f6c940c75a0 2024-04-20T21:31:17.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.708+0000 7f6c91600640 1 -- 172.21.15.102:0/3719755103 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c8404a060 con 0x7f6c940c75a0 2024-04-20T21:31:17.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.709+0000 7f6c91600640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:17.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.709+0000 7f6c91600640 1 -- 172.21.15.102:0/3719755103 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f6c840ae1a0 con 0x7f6c940c75a0 2024-04-20T21:31:17.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.709+0000 7f6c99200640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:17.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.710+0000 7f6c99200640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0016e0 tx=0x7f6c7c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:17.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.714+0000 7f6c91600640 1 -- 172.21.15.102:0/3719755103 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c8403e070 con 0x7f6c940c75a0 2024-04-20T21:31:17.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.957+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c940c3690 con 0x7f6c940c75a0 2024-04-20T21:31:17.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.958+0000 7f6c91600640 1 -- 172.21.15.102:0/3719755103 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6856 (secure 0 0 0) 0x7f6c8407b8d0 con 0x7f6c940c75a0 2024-04-20T21:31:17.957 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:17.958 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:31:17.958 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 43 2024-04-20T21:31:17.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.960+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 msgr2=0x7f6c7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:17.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.960+0000 7f6c9a600640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0016e0 tx=0x7f6c7c009040 comp rx=0 tx=0).stop 2024-04-20T21:31:17.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.960+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 msgr2=0x7f6c940713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.960+0000 7f6c9a600640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940713c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6c8402b9e0 tx=0x7f6c840046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.961+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 shutdown_connections 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.961+0000 7f6c9a600640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.961+0000 7f6c9a600640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c94153180 0x7f6c94141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.961+0000 7f6c9a600640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c940c75a0 0x7f6c940713c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.961+0000 7f6c9a600640 1 --2- 172.21.15.102:0/3719755103 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c94073c70 0x7f6c94156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.961+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 >> 172.21.15.102:0/3719755103 conn(0x7f6c940cc820 msgr2=0x7f6c94148310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.961+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 shutdown_connections 2024-04-20T21:31:17.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:17.961+0000 7f6c9a600640 1 -- 172.21.15.102:0/3719755103 wait complete. 2024-04-20T21:31:17.980 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:31:19.981 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:20.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.190+0000 7f0dd3e00640 1 Processor -- start 2024-04-20T21:31:20.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.190+0000 7f0dd3e00640 1 -- start start 2024-04-20T21:31:20.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc14fa50 0x7f0dcc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0dcc073a10 0x7f0dcc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd3e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dcc1536d0 con 0x7f0dcc0c9eb0 2024-04-20T21:31:20.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd3e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0dcc159500 con 0x7f0dcc073a10 2024-04-20T21:31:20.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc0c76c0 con 0x7f0dcc14fa50 2024-04-20T21:31:20.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd0a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0dcc0c9eb0 0x7f0dcc0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46698/0 (socket says 172.21.15.102:46698) 2024-04-20T21:31:20.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd0a00640 1 -- 172.21.15.102:0/4119997486 learned_addr learned my addr 172.21.15.102:0/4119997486 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:20.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd1e00640 1 --2- 172.21.15.102:0/4119997486 >> v2:172.21.15.193:3300/0 conn(0x7f0dcc073a10 0x7f0dcc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.192+0000 7f0dd1400640 1 --2- 172.21.15.102:0/4119997486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc14fa50 0x7f0dcc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.193+0000 7f0dcbe00640 1 -- 172.21.15.102:0/4119997486 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683899135 0 0) 0x7f0dbc001110 con 0x7f0dcc0c9eb0 2024-04-20T21:31:20.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.193+0000 7f0dd1e00640 1 -- 172.21.15.102:0/4119997486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc14fa50 msgr2=0x7f0dcc0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:20.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.193+0000 7f0dd1e00640 1 --2- 172.21.15.102:0/4119997486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc14fa50 0x7f0dcc0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.193+0000 7f0dd1e00640 1 -- 172.21.15.102:0/4119997486 >> v1:172.21.15.102:6789/0 conn(0x7f0dcc0c9eb0 legacy=0x7f0dcc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.193+0000 7f0dd1e00640 1 -- 172.21.15.102:0/4119997486 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc0c7960 con 0x7f0dcc073a10 2024-04-20T21:31:20.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.194+0000 7f0dd1e00640 1 --2- 172.21.15.102:0/4119997486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc073a10 0x7f0dcc158fc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009d90 tx=0x7f0dc402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d139eab71ddce500 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:20.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.194+0000 7f0dcbe00640 1 -- 172.21.15.102:0/4119997486 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4039070 con 0x7f0dcc073a10 2024-04-20T21:31:20.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.194+0000 7f0dcbe00640 1 -- 172.21.15.102:0/4119997486 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc402ccf0 con 0x7f0dcc073a10 2024-04-20T21:31:20.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.194+0000 7f0dcbe00640 1 -- 172.21.15.102:0/4119997486 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4034660 con 0x7f0dcc073a10 2024-04-20T21:31:20.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.195+0000 7f0dd3e00640 1 -- 172.21.15.102:0/4119997486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc073a10 msgr2=0x7f0dcc158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.195+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/4119997486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc073a10 0x7f0dcc158fc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009d90 tx=0x7f0dc402bf40 comp rx=0 tx=0).stop 2024-04-20T21:31:20.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.195+0000 7f0dd3e00640 1 -- 172.21.15.102:0/4119997486 shutdown_connections 2024-04-20T21:31:20.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.195+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/4119997486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc073a10 0x7f0dcc158fc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.195+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/4119997486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc14fa50 0x7f0dcc0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.195+0000 7f0dd3e00640 1 -- 172.21.15.102:0/4119997486 >> 172.21.15.102:0/4119997486 conn(0x7f0dcc0c9390 msgr2=0x7f0dcc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:20.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.195+0000 7f0dd3e00640 1 -- 172.21.15.102:0/4119997486 shutdown_connections 2024-04-20T21:31:20.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.195+0000 7f0dd3e00640 1 -- 172.21.15.102:0/4119997486 wait complete. 2024-04-20T21:31:20.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.197+0000 7f0dd3e00640 1 Processor -- start 2024-04-20T21:31:20.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.197+0000 7f0dd3e00640 1 -- start start 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dcc073a10 0x7f0dcc149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc0c9eb0 0x7f0dcc0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc14fa50 0x7f0dcc1e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc0c7c00 con 0x7f0dcc073a10 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc0c3d90 con 0x7f0dcc14fa50 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc0c4030 con 0x7f0dcc0c9eb0 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dcc073a10 0x7f0dcc149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dcc073a10 0x7f0dcc149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33472/0 (socket says 172.21.15.102:33472) 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd1400640 1 -- 172.21.15.102:0/1242302685 learned_addr learned my addr 172.21.15.102:0/1242302685 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd0a00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc0c9eb0 0x7f0dcc0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd1e00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc14fa50 0x7f0dcc1e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd1400640 1 -- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc0c9eb0 msgr2=0x7f0dcc0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd1400640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc0c9eb0 0x7f0dcc0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd1400640 1 -- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc14fa50 msgr2=0x7f0dcc1e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.198+0000 7f0dd1400640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc14fa50 0x7f0dcc1e5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.199+0000 7f0dd1400640 1 -- 172.21.15.102:0/1242302685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc14a110 con 0x7f0dcc073a10 2024-04-20T21:31:20.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.199+0000 7f0dd1e00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc14fa50 0x7f0dcc1e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:20.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.199+0000 7f0dd1400640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dcc073a10 0x7f0dcc149a60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000e8a0 tx=0x7f0dc000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:20.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.199+0000 7f0dc9600640 1 -- 172.21.15.102:0/1242302685 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0017070 con 0x7f0dcc073a10 2024-04-20T21:31:20.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.199+0000 7f0dc9600640 1 -- 172.21.15.102:0/1242302685 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc000ee90 con 0x7f0dcc073a10 2024-04-20T21:31:20.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.199+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc0cbe20 con 0x7f0dcc073a10 2024-04-20T21:31:20.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.199+0000 7f0dc9600640 1 -- 172.21.15.102:0/1242302685 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc000cb00 con 0x7f0dcc073a10 2024-04-20T21:31:20.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.199+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc0cc3e0 con 0x7f0dcc073a10 2024-04-20T21:31:20.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.200+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc0cc620 con 0x7f0dcc073a10 2024-04-20T21:31:20.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.202+0000 7f0dc9600640 1 -- 172.21.15.102:0/1242302685 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dc001a070 con 0x7f0dcc073a10 2024-04-20T21:31:20.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.203+0000 7f0dc9600640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0db006c920 0x7f0db006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.203+0000 7f0dc9600640 1 -- 172.21.15.102:0/1242302685 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f0dc008c2b0 con 0x7f0dcc073a10 2024-04-20T21:31:20.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.203+0000 7f0dd0a00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0db006c920 0x7f0db006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.204+0000 7f0dd0a00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0db006c920 0x7f0db006ede0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc06e7f0 tx=0x7f0dbc007970 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:20.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.207+0000 7f0dc9600640 1 -- 172.21.15.102:0/1242302685 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0dcc0cc620 con 0x7f0dcc073a10 2024-04-20T21:31:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.450+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dcc074340 con 0x7f0dcc073a10 2024-04-20T21:31:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.451+0000 7f0dc9600640 1 -- 172.21.15.102:0/1242302685 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6856 (secure 0 0 0) 0x7f0dc005a0c0 con 0x7f0dcc073a10 2024-04-20T21:31:20.450 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:20.450 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:31:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 43 2024-04-20T21:31:20.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0db006c920 msgr2=0x7f0db006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0db006c920 0x7f0db006ede0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc06e7f0 tx=0x7f0dbc007970 comp rx=0 tx=0).stop 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dcc073a10 msgr2=0x7f0dcc149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dcc073a10 0x7f0dcc149a60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000e8a0 tx=0x7f0dc000ed60 comp rx=0 tx=0).stop 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 shutdown_connections 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0db006c920 0x7f0db006ede0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dcc14fa50 0x7f0dcc1e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dcc0c9eb0 0x7f0dcc0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 --2- 172.21.15.102:0/1242302685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dcc073a10 0x7f0dcc149a60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.453+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 >> 172.21.15.102:0/1242302685 conn(0x7f0dcc0c9390 msgr2=0x7f0dcc06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:20.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.454+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 shutdown_connections 2024-04-20T21:31:20.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:20.454+0000 7f0dd3e00640 1 -- 172.21.15.102:0/1242302685 wait complete. 2024-04-20T21:31:20.472 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:31:22.473 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:22.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.689+0000 7f3115200640 1 Processor -- start 2024-04-20T21:31:22.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.689+0000 7f3115200640 1 -- start start 2024-04-20T21:31:22.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.690+0000 7f3115200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31100c8db0 0x7f3110070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.690+0000 7f3115200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3110158e80 con 0x7f3110152170 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f3115200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31100c91c0 con 0x7f3110071330 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f3115200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f31100c6dc0 con 0x7f31100c8db0 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310de00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3110152170 0x7f3110158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46706/0 (socket says 172.21.15.102:46706) 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310de00640 1 -- 172.21.15.102:0/2815909578 learned_addr learned my addr 172.21.15.102:0/2815909578 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310d400640 1 --2- 172.21.15.102:0/2815909578 >> v2:172.21.15.102:3300/0 conn(0x7f31100c8db0 0x7f3110070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310ca00640 1 -- 172.21.15.102:0/2815909578 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283026888 0 0) 0x7f3110158e80 con 0x7f3110152170 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310ca00640 1 -- 172.21.15.102:0/2815909578 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30f4005730 con 0x7f3110152170 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310ca00640 1 -- 172.21.15.102:0/2815909578 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301581922 0 0) 0x7f31100c91c0 con 0x7f3110071330 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310d400640 1 -- 172.21.15.102:0/2815909578 >> v1:172.21.15.193:6789/0 conn(0x7f3110071330 legacy=0x7f31101628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:22.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310d400640 1 -- 172.21.15.102:0/2815909578 >> v1:172.21.15.102:6789/0 conn(0x7f3110152170 legacy=0x7f3110158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:22.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.691+0000 7f310d400640 1 -- 172.21.15.102:0/2815909578 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3110071f70 con 0x7f31100c8db0 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f310d400640 1 --2- 172.21.15.102:0/2815909578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110070df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0097a0 tx=0x7f30fc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73027e9685815938 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f310ca00640 1 -- 172.21.15.102:0/2815909578 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc034070 con 0x7f31100c8db0 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f310ca00640 1 -- 172.21.15.102:0/2815909578 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30fc030610 con 0x7f31100c8db0 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f310ca00640 1 -- 172.21.15.102:0/2815909578 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc0309e0 con 0x7f31100c8db0 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f3115200640 1 -- 172.21.15.102:0/2815909578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 msgr2=0x7f3110070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f3115200640 1 --2- 172.21.15.102:0/2815909578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110070df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0097a0 tx=0x7f30fc02b960 comp rx=0 tx=0).stop 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f3115200640 1 -- 172.21.15.102:0/2815909578 shutdown_connections 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f3115200640 1 --2- 172.21.15.102:0/2815909578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110070df0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.693+0000 7f3115200640 1 -- 172.21.15.102:0/2815909578 >> 172.21.15.102:0/2815909578 conn(0x7f31100c9390 msgr2=0x7f31100c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.694+0000 7f3115200640 1 -- 172.21.15.102:0/2815909578 shutdown_connections 2024-04-20T21:31:22.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.694+0000 7f3115200640 1 -- 172.21.15.102:0/2815909578 wait complete. 2024-04-20T21:31:22.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.695+0000 7f3115200640 1 Processor -- start 2024-04-20T21:31:22.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.695+0000 7f3115200640 1 -- start start 2024-04-20T21:31:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.696+0000 7f3115200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3110071330 0x7f311015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.696+0000 7f3115200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.696+0000 7f3115200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3110152170 0x7f3110147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.696+0000 7f3115200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31100724b0 con 0x7f31100c8db0 2024-04-20T21:31:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.696+0000 7f3115200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3110072210 con 0x7f3110152170 2024-04-20T21:31:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.696+0000 7f3115200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3110158250 con 0x7f3110071330 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33486/0 (socket says 172.21.15.102:33486) 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310d400640 1 -- 172.21.15.102:0/3104315184 learned_addr learned my addr 172.21.15.102:0/3104315184 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310de00640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3110071330 0x7f311015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310e800640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3110152170 0x7f3110147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310d400640 1 -- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3110071330 msgr2=0x7f311015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310d400640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3110071330 0x7f311015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310d400640 1 -- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3110152170 msgr2=0x7f3110147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310d400640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3110152170 0x7f3110147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.697+0000 7f310d400640 1 -- 172.21.15.102:0/3104315184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31100cb340 con 0x7f31100c8db0 2024-04-20T21:31:22.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.698+0000 7f310d400640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110147a10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0098c0 tx=0x7f30fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:22.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.698+0000 7f3106000640 1 -- 172.21.15.102:0/3104315184 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc049070 con 0x7f31100c8db0 2024-04-20T21:31:22.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.698+0000 7f3106000640 1 -- 172.21.15.102:0/3104315184 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30fc030410 con 0x7f31100c8db0 2024-04-20T21:31:22.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.698+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3110151100 con 0x7f31100c8db0 2024-04-20T21:31:22.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.698+0000 7f3106000640 1 -- 172.21.15.102:0/3104315184 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc034040 con 0x7f31100c8db0 2024-04-20T21:31:22.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.698+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31101ed7e0 con 0x7f31100c8db0 2024-04-20T21:31:22.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.699+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3110074340 con 0x7f31100c8db0 2024-04-20T21:31:22.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.700+0000 7f3106000640 1 -- 172.21.15.102:0/3104315184 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30fc04c070 con 0x7f31100c8db0 2024-04-20T21:31:22.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.701+0000 7f3106000640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30f006c8f0 0x7f30f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:22.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.702+0000 7f3106000640 1 -- 172.21.15.102:0/3104315184 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f30fc0adbe0 con 0x7f31100c8db0 2024-04-20T21:31:22.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.702+0000 7f310de00640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30f006c8f0 0x7f30f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:22.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.703+0000 7f310de00640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30f006c8f0 0x7f30f006edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3100005880 tx=0x7f3100009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:22.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.706+0000 7f3106000640 1 -- 172.21.15.102:0/3104315184 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f30fc07b3a0 con 0x7f31100c8db0 2024-04-20T21:31:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.949+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f311014fef0 con 0x7f31100c8db0 2024-04-20T21:31:22.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.950+0000 7f3106000640 1 -- 172.21.15.102:0/3104315184 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6856 (secure 0 0 0) 0x7f30fc07f6a0 con 0x7f31100c8db0 2024-04-20T21:31:22.950 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:22.950 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:31:22.950 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 43 2024-04-20T21:31:22.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30f006c8f0 msgr2=0x7f30f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:22.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30f006c8f0 0x7f30f006edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3100005880 tx=0x7f3100009040 comp rx=0 tx=0).stop 2024-04-20T21:31:22.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 msgr2=0x7f3110147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:22.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110147a10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0098c0 tx=0x7f30fc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:22.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 shutdown_connections 2024-04-20T21:31:22.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30f006c8f0 0x7f30f006edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:22.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3110152170 0x7f3110147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:22.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31100c8db0 0x7f3110147a10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:22.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 --2- 172.21.15.102:0/3104315184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3110071330 0x7f311015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:22.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 >> 172.21.15.102:0/3104315184 conn(0x7f31100c9390 msgr2=0x7f311006ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:22.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 shutdown_connections 2024-04-20T21:31:22.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:22.953+0000 7f3115200640 1 -- 172.21.15.102:0/3104315184 wait complete. 2024-04-20T21:31:22.972 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:31:24.973 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.179+0000 7f3cfb400640 1 Processor -- start 2024-04-20T21:31:25.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.179+0000 7f3cfb400640 1 -- start start 2024-04-20T21:31:25.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.181+0000 7f3cfb400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3cf40cb4c0 0x7f3cf40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.181+0000 7f3cfb400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf415b250 con 0x7f3cf41555e0 2024-04-20T21:31:25.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.181+0000 7f3cfb400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf415ea90 con 0x7f3cf415b750 2024-04-20T21:31:25.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.181+0000 7f3cfb400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf40cc840 con 0x7f3cf40cb4c0 2024-04-20T21:31:25.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.181+0000 7f3cf8a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3cf41555e0 0x7f3cf40cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46714/0 (socket says 172.21.15.102:46714) 2024-04-20T21:31:25.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.181+0000 7f3cf8a00640 1 -- 172.21.15.102:0/2054052343 learned_addr learned my addr 172.21.15.102:0/2054052343 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:25.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.181+0000 7f3cf3e00640 1 --2- 172.21.15.102:0/2054052343 >> v2:172.21.15.193:3300/0 conn(0x7f3cf40cb4c0 0x7f3cf40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.182+0000 7f3cf3400640 1 -- 172.21.15.102:0/2054052343 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965917262 0 0) 0x7f3cf415ea90 con 0x7f3cf415b750 2024-04-20T21:31:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.182+0000 7f3cf3400640 1 -- 172.21.15.102:0/2054052343 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cd8003520 con 0x7f3cf415b750 2024-04-20T21:31:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.182+0000 7f3cf3400640 1 -- 172.21.15.102:0/2054052343 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954422712 0 0) 0x7f3cf415b250 con 0x7f3cf41555e0 2024-04-20T21:31:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.182+0000 7f3cf3400640 1 -- 172.21.15.102:0/2054052343 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cf415ea90 con 0x7f3cf41555e0 2024-04-20T21:31:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.182+0000 7f3cf3e00640 1 -- 172.21.15.102:0/2054052343 >> v1:172.21.15.193:6789/0 conn(0x7f3cf415b750 legacy=0x7f3cf415bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.182+0000 7f3cf3e00640 1 -- 172.21.15.102:0/2054052343 >> v1:172.21.15.102:6789/0 conn(0x7f3cf41555e0 legacy=0x7f3cf40cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.182+0000 7f3cf3e00640 1 -- 172.21.15.102:0/2054052343 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf415ed70 con 0x7f3cf40cb4c0 2024-04-20T21:31:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.183+0000 7f3cf3e00640 1 --2- 172.21.15.102:0/2054052343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf40cb4c0 0x7f3cf40cc300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009d90 tx=0x7f3cdc02bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59126b5eac46086d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.183+0000 7f3cf3400640 1 -- 172.21.15.102:0/2054052343 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc039070 con 0x7f3cf40cb4c0 2024-04-20T21:31:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cf3400640 1 -- 172.21.15.102:0/2054052343 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cdc02cd80 con 0x7f3cf40cb4c0 2024-04-20T21:31:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cf3400640 1 -- 172.21.15.102:0/2054052343 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc034660 con 0x7f3cf40cb4c0 2024-04-20T21:31:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cfb400640 1 -- 172.21.15.102:0/2054052343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf40cb4c0 msgr2=0x7f3cf40cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cfb400640 1 --2- 172.21.15.102:0/2054052343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf40cb4c0 0x7f3cf40cc300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009d90 tx=0x7f3cdc02bfd0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cfb400640 1 -- 172.21.15.102:0/2054052343 shutdown_connections 2024-04-20T21:31:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cfb400640 1 --2- 172.21.15.102:0/2054052343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf40cb4c0 0x7f3cf40cc300 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cfb400640 1 -- 172.21.15.102:0/2054052343 >> 172.21.15.102:0/2054052343 conn(0x7f3cf40754d0 msgr2=0x7f3cf4072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cfb400640 1 -- 172.21.15.102:0/2054052343 shutdown_connections 2024-04-20T21:31:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.184+0000 7f3cfb400640 1 -- 172.21.15.102:0/2054052343 wait complete. 2024-04-20T21:31:25.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.186+0000 7f3cfb400640 1 Processor -- start 2024-04-20T21:31:25.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.186+0000 7f3cfb400640 1 -- start start 2024-04-20T21:31:25.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cfb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf40cb4c0 0x7f3cf41496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cfb400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3cf41555e0 0x7f3cf4149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cfb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf415b750 0x7f3cf4143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cfb400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf415e840 con 0x7f3cf40cb4c0 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cfb400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4073a90 con 0x7f3cf415b750 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cfb400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf41559c0 con 0x7f3cf41555e0 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cf8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf40cb4c0 0x7f3cf41496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cf9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf415b750 0x7f3cf4143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cf3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3cf41555e0 0x7f3cf4149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cf9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf415b750 0x7f3cf4143990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:57564/0 (socket says 172.21.15.102:57564) 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cf8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf40cb4c0 0x7f3cf41496d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33492/0 (socket says 172.21.15.102:33492) 2024-04-20T21:31:25.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.187+0000 7f3cf9400640 1 -- 172.21.15.102:0/271268763 learned_addr learned my addr 172.21.15.102:0/271268763 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:25.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.188+0000 7f3cf3e00640 1 -- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf415b750 msgr2=0x7f3cf4143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.188+0000 7f3cf3e00640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf415b750 0x7f3cf4143990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.188+0000 7f3cf3e00640 1 -- 172.21.15.102:0/271268763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf40cb4c0 msgr2=0x7f3cf41496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.188+0000 7f3cf3e00640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf40cb4c0 0x7f3cf41496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.188+0000 7f3cf3e00640 1 -- 172.21.15.102:0/271268763 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf415d4b0 con 0x7f3cf41555e0 2024-04-20T21:31:25.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.188+0000 7f3cf3e00640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3cf41555e0 0x7f3cf4149c10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc033000 tx=0x7f3cdc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:25.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.188+0000 7f3cf0c00640 1 -- 172.21.15.102:0/271268763 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc039050 con 0x7f3cf41555e0 2024-04-20T21:31:25.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.189+0000 7f3cf0c00640 1 -- 172.21.15.102:0/271268763 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cdc045070 con 0x7f3cf41555e0 2024-04-20T21:31:25.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.189+0000 7f3cf0c00640 1 -- 172.21.15.102:0/271268763 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc009450 con 0x7f3cf41555e0 2024-04-20T21:31:25.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.189+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf41f4080 con 0x7f3cf41555e0 2024-04-20T21:31:25.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.189+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf41f4620 con 0x7f3cf41555e0 2024-04-20T21:31:25.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.190+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf4068c30 con 0x7f3cf41555e0 2024-04-20T21:31:25.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.191+0000 7f3cf0c00640 1 -- 172.21.15.102:0/271268763 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cdc03e870 con 0x7f3cf41555e0 2024-04-20T21:31:25.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.192+0000 7f3cf0c00640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3cd006c8f0 0x7f3cd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.192+0000 7f3cf0c00640 1 -- 172.21.15.102:0/271268763 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 7251+0+0 (secure 0 0 0) 0x7f3cdc0aebe0 con 0x7f3cf41555e0 2024-04-20T21:31:25.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.192+0000 7f3cf8a00640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3cd006c8f0 0x7f3cd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.193+0000 7f3cf8a00640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3cd006c8f0 0x7f3cd006edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40011e0 tx=0x7f3ce4007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.197+0000 7f3cf0c00640 1 -- 172.21.15.102:0/271268763 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3cf415d4b0 con 0x7f3cf41555e0 2024-04-20T21:31:25.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.440+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cf41425b0 con 0x7f3cf41555e0 2024-04-20T21:31:25.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.441+0000 7f3cf0c00640 1 -- 172.21.15.102:0/271268763 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6856 (secure 0 0 0) 0x7f3cdc07c4c0 con 0x7f3cf41555e0 2024-04-20T21:31:25.440 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:25.441 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:29:06.006115+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5659":{"gid":5659,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":40,"addr":"172.21.15.102:6835/36459307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":36459307},{"type":"v1","addr":"172.21.15.102:6835","nonce":36459307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5659,"qdb_cluster":[5659]},"id":1}]} 2024-04-20T21:31:25.441 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 43 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.444+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3cd006c8f0 msgr2=0x7f3cd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.444+0000 7f3cfb400640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3cd006c8f0 0x7f3cd006edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40011e0 tx=0x7f3ce4007d30 comp rx=0 tx=0).stop 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.444+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3cf41555e0 msgr2=0x7f3cf4149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.444+0000 7f3cfb400640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3cf41555e0 0x7f3cf4149c10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc033000 tx=0x7f3cdc004a40 comp rx=0 tx=0).stop 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.445+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 shutdown_connections 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.445+0000 7f3cfb400640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3cd006c8f0 0x7f3cd006edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.445+0000 7f3cfb400640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cf415b750 0x7f3cf4143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.445+0000 7f3cfb400640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3cf41555e0 0x7f3cf4149c10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.445+0000 7f3cfb400640 1 --2- 172.21.15.102:0/271268763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf40cb4c0 0x7f3cf41496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.445+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 >> 172.21.15.102:0/271268763 conn(0x7f3cf40754d0 msgr2=0x7f3cf415bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.445+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 shutdown_connections 2024-04-20T21:31:25.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:25.445+0000 7f3cfb400640 1 -- 172.21.15.102:0/271268763 wait complete. 2024-04-20T21:31:25.463 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:31:27.464 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:27.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.673+0000 7ff785c00640 1 Processor -- start 2024-04-20T21:31:27.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.673+0000 7ff785c00640 1 -- start start 2024-04-20T21:31:27.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.675+0000 7ff785c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff780152170 0x7ff7800c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.675+0000 7ff785c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff780071f70 con 0x7ff780158b80 2024-04-20T21:31:27.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.675+0000 7ff785c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff780072210 con 0x7ff780158770 2024-04-20T21:31:27.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.675+0000 7ff785c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff7800c6dc0 con 0x7ff780152170 2024-04-20T21:31:27.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.675+0000 7ff77f400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff780158b80 0x7ff7801628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46728/0 (socket says 172.21.15.102:46728) 2024-04-20T21:31:27.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.675+0000 7ff77f400640 1 -- 172.21.15.102:0/954269863 learned_addr learned my addr 172.21.15.102:0/954269863 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:27.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.675+0000 7ff77ea00640 1 --2- 172.21.15.102:0/954269863 >> v2:172.21.15.102:3300/0 conn(0x7ff780152170 0x7ff7800c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.676+0000 7ff77ea00640 1 -- 172.21.15.102:0/954269863 >> v1:172.21.15.193:6789/0 conn(0x7ff780158770 legacy=0x7ff780070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.676+0000 7ff77ea00640 1 -- 172.21.15.102:0/954269863 >> v1:172.21.15.102:6789/0 conn(0x7ff780158b80 legacy=0x7ff7801628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.676+0000 7ff77d600640 1 -- 172.21.15.102:0/954269863 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2503056580 0 0) 0x7ff780072210 con 0x7ff780158770 2024-04-20T21:31:27.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.676+0000 7ff77ea00640 1 -- 172.21.15.102:0/954269863 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7800724b0 con 0x7ff780152170 2024-04-20T21:31:27.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.677+0000 7ff77ea00640 1 --2- 172.21.15.102:0/954269863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff7800c8db0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff778009d90 tx=0x7ff77802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f3149c45d0c667 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:27.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.677+0000 7ff77d600640 1 -- 172.21.15.102:0/954269863 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778039070 con 0x7ff780152170 2024-04-20T21:31:27.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.677+0000 7ff77d600640 1 -- 172.21.15.102:0/954269863 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77802cb40 con 0x7ff780152170 2024-04-20T21:31:27.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.677+0000 7ff77d600640 1 -- 172.21.15.102:0/954269863 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778034440 con 0x7ff780152170 2024-04-20T21:31:27.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.678+0000 7ff785c00640 1 -- 172.21.15.102:0/954269863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 msgr2=0x7ff7800c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.678+0000 7ff785c00640 1 --2- 172.21.15.102:0/954269863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff7800c8db0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff778009d90 tx=0x7ff77802be10 comp rx=0 tx=0).stop 2024-04-20T21:31:27.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.678+0000 7ff785c00640 1 -- 172.21.15.102:0/954269863 shutdown_connections 2024-04-20T21:31:27.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.678+0000 7ff785c00640 1 --2- 172.21.15.102:0/954269863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff7800c8db0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.678+0000 7ff785c00640 1 -- 172.21.15.102:0/954269863 >> 172.21.15.102:0/954269863 conn(0x7ff7800c9390 msgr2=0x7ff7800c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:27.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.678+0000 7ff785c00640 1 -- 172.21.15.102:0/954269863 shutdown_connections 2024-04-20T21:31:27.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.678+0000 7ff785c00640 1 -- 172.21.15.102:0/954269863 wait complete. 2024-04-20T21:31:27.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.679+0000 7ff785c00640 1 Processor -- start 2024-04-20T21:31:27.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.680+0000 7ff785c00640 1 -- start start 2024-04-20T21:31:27.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.680+0000 7ff785c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff780153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff785c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff780158770 0x7ff7801e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff785c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff780158b80 0x7ff7801f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff785c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7800c9f20 con 0x7ff780152170 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff785c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff780158250 con 0x7ff780158770 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff785c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff780157f20 con 0x7ff780158b80 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff780153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff780158770 0x7ff7801e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff780153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33512/0 (socket says 172.21.15.102:33512) 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77ea00640 1 -- 172.21.15.102:0/1534700707 learned_addr learned my addr 172.21.15.102:0/1534700707 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77f400640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff780158b80 0x7ff7801f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77ea00640 1 -- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff780158b80 msgr2=0x7ff7801f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77ea00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff780158b80 0x7ff7801f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77ea00640 1 -- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff780158770 msgr2=0x7ff7801e59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77ea00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff780158770 0x7ff7801e59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.681+0000 7ff77ea00640 1 -- 172.21.15.102:0/1534700707 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff780153f50 con 0x7ff780152170 2024-04-20T21:31:27.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.682+0000 7ff77ea00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff780153a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff778037040 tx=0x7ff778004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:27.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.682+0000 7ff776a00640 1 -- 172.21.15.102:0/1534700707 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778044020 con 0x7ff780152170 2024-04-20T21:31:27.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.682+0000 7ff776a00640 1 -- 172.21.15.102:0/1534700707 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77803f670 con 0x7ff780152170 2024-04-20T21:31:27.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.682+0000 7ff776a00640 1 -- 172.21.15.102:0/1534700707 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778039040 con 0x7ff780152170 2024-04-20T21:31:27.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.682+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78014e150 con 0x7ff780152170 2024-04-20T21:31:27.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.682+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7801f4cb0 con 0x7ff780152170 2024-04-20T21:31:27.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.683+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff780074340 con 0x7ff780152170 2024-04-20T21:31:27.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.684+0000 7ff776a00640 1 -- 172.21.15.102:0/1534700707 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff778005d10 con 0x7ff780152170 2024-04-20T21:31:27.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.685+0000 7ff776a00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff75c06c8f0 0x7ff75c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.685+0000 7ff776a00640 1 -- 172.21.15.102:0/1534700707 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7ff7780aed60 con 0x7ff780152170 2024-04-20T21:31:27.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.685+0000 7ff77e000640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff75c06c8f0 0x7ff75c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.686+0000 7ff77e000640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff75c06c8f0 0x7ff75c06edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff7800c7c10 tx=0x7ff770009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:27.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.693+0000 7ff776a00640 1 -- 172.21.15.102:0/1534700707 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff77804e070 con 0x7ff780152170 2024-04-20T21:31:27.810 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:27.818 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:27.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.936+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7800c3460 con 0x7ff780152170 2024-04-20T21:31:27.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.937+0000 7ff776a00640 1 -- 172.21.15.102:0/1534700707 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6026 (secure 0 0 0) 0x7ff77807c440 con 0x7ff780152170 2024-04-20T21:31:27.937 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:27.937 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:25.671089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:27.937 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 44 2024-04-20T21:31:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff75c06c8f0 msgr2=0x7ff75c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff75c06c8f0 0x7ff75c06edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff7800c7c10 tx=0x7ff770009040 comp rx=0 tx=0).stop 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 msgr2=0x7ff780153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff780153a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff778037040 tx=0x7ff778004a40 comp rx=0 tx=0).stop 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 shutdown_connections 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff780158b80 0x7ff7801f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff75c06c8f0 0x7ff75c06edb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff780158770 0x7ff7801e59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 --2- 172.21.15.102:0/1534700707 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780152170 0x7ff780153a10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 >> 172.21.15.102:0/1534700707 conn(0x7ff7800c9390 msgr2=0x7ff78006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 shutdown_connections 2024-04-20T21:31:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:27.940+0000 7ff785c00640 1 -- 172.21.15.102:0/1534700707 wait complete. 2024-04-20T21:31:27.959 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:31:27.959 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:31:27.959 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:28.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.160+0000 7fea93000640 1 Processor -- start 2024-04-20T21:31:28.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.160+0000 7fea93000640 1 -- start start 2024-04-20T21:31:28.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.162+0000 7fea93000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fea8c0c75a0 0x7fea8c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.162+0000 7fea93000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8c153560 con 0x7fea8c153180 2024-04-20T21:31:28.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.162+0000 7fea93000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8c0c7a70 con 0x7fea8c073c70 2024-04-20T21:31:28.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.162+0000 7fea93000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fea8c0c7d30 con 0x7fea8c0c75a0 2024-04-20T21:31:28.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.162+0000 7fea91200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fea8c0c75a0 0x7fea8c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.162+0000 7fea91c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fea8c153180 0x7fea8c158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58918/0 (socket says 172.21.15.102:58918) 2024-04-20T21:31:28.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.162+0000 7fea91c00640 1 -- 172.21.15.102:0/249635048 learned_addr learned my addr 172.21.15.102:0/249635048 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.163+0000 7fea91200640 1 -- 172.21.15.102:0/249635048 >> v1:172.21.15.193:6789/0 conn(0x7fea8c073c70 legacy=0x7fea8c1637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.163+0000 7fea91200640 1 -- 172.21.15.102:0/249635048 >> v1:172.21.15.102:6789/0 conn(0x7fea8c153180 legacy=0x7fea8c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.163+0000 7fea91200640 1 -- 172.21.15.102:0/249635048 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c15c4d0 con 0x7fea8c0c75a0 2024-04-20T21:31:28.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.164+0000 7fea91200640 1 --2- 172.21.15.102:0/249635048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0c75a0 0x7fea8c0c5f30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0097a0 tx=0x7fea7c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501e1e3192296796 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.164+0000 7fea8be00640 1 -- 172.21.15.102:0/249635048 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c034070 con 0x7fea8c0c75a0 2024-04-20T21:31:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.164+0000 7fea8be00640 1 -- 172.21.15.102:0/249635048 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7c030610 con 0x7fea8c0c75a0 2024-04-20T21:31:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.164+0000 7fea8be00640 1 -- 172.21.15.102:0/249635048 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0309e0 con 0x7fea8c0c75a0 2024-04-20T21:31:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.164+0000 7fea93000640 1 -- 172.21.15.102:0/249635048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0c75a0 msgr2=0x7fea8c0c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.164+0000 7fea93000640 1 --2- 172.21.15.102:0/249635048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0c75a0 0x7fea8c0c5f30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0097a0 tx=0x7fea7c02b960 comp rx=0 tx=0).stop 2024-04-20T21:31:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.165+0000 7fea93000640 1 -- 172.21.15.102:0/249635048 shutdown_connections 2024-04-20T21:31:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.165+0000 7fea93000640 1 --2- 172.21.15.102:0/249635048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0c75a0 0x7fea8c0c5f30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.165+0000 7fea93000640 1 -- 172.21.15.102:0/249635048 >> 172.21.15.102:0/249635048 conn(0x7fea8c0cc820 msgr2=0x7fea8c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.165+0000 7fea93000640 1 -- 172.21.15.102:0/249635048 shutdown_connections 2024-04-20T21:31:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.165+0000 7fea93000640 1 -- 172.21.15.102:0/249635048 wait complete. 2024-04-20T21:31:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.166+0000 7fea93000640 1 Processor -- start 2024-04-20T21:31:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.166+0000 7fea93000640 1 -- start start 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.167+0000 7fea93000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea8c073c70 0x7fea8c1ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.167+0000 7fea93000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea8c0c75a0 0x7fea8c1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.167+0000 7fea93000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c153180 0x7fea8c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.167+0000 7fea93000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c159550 con 0x7fea8c153180 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.167+0000 7fea93000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c15ca90 con 0x7fea8c073c70 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.167+0000 7fea93000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fea8c15d290 con 0x7fea8c0c75a0 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea8c073c70 0x7fea8c1ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea92600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c153180 0x7fea8c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea8c0c75a0 0x7fea8c1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea8c073c70 0x7fea8c1ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51046/0 (socket says 172.21.15.102:51046) 2024-04-20T21:31:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91c00640 1 -- 172.21.15.102:0/2784723824 learned_addr learned my addr 172.21.15.102:0/2784723824 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:28.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91200640 1 -- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea8c073c70 msgr2=0x7fea8c1ef6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91200640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea8c073c70 0x7fea8c1ef6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91200640 1 -- 172.21.15.102:0/2784723824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c153180 msgr2=0x7fea8c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91200640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c153180 0x7fea8c0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91200640 1 -- 172.21.15.102:0/2784723824 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c159330 con 0x7fea8c0c75a0 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea91c00640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea8c073c70 0x7fea8c1ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.168+0000 7fea92600640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c153180 0x7fea8c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.169+0000 7fea91200640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea8c0c75a0 0x7fea8c1576c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea7c033000 tx=0x7fea7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.169+0000 7fea89600640 1 -- 172.21.15.102:0/2784723824 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c034040 con 0x7fea8c0c75a0 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.169+0000 7fea89600640 1 -- 172.21.15.102:0/2784723824 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7c03acc0 con 0x7fea8c0c75a0 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.169+0000 7fea89600640 1 -- 172.21.15.102:0/2784723824 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c044630 con 0x7fea8c0c75a0 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.169+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8c159050 con 0x7fea8c0c75a0 2024-04-20T21:31:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.169+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8c1eeea0 con 0x7fea8c0c75a0 2024-04-20T21:31:28.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.170+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8c04fd60 con 0x7fea8c0c75a0 2024-04-20T21:31:28.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.171+0000 7fea89600640 1 -- 172.21.15.102:0/2784723824 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea7c04b060 con 0x7fea8c0c75a0 2024-04-20T21:31:28.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.172+0000 7fea89600640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea7006c920 0x7fea7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.172+0000 7fea89600640 1 -- 172.21.15.102:0/2784723824 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7fea7c0adf90 con 0x7fea8c0c75a0 2024-04-20T21:31:28.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.172+0000 7fea91c00640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea7006c920 0x7fea7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.173+0000 7fea91c00640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea7006c920 0x7fea7006ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea80005880 tx=0x7fea80009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:28.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.177+0000 7fea89600640 1 -- 172.21.15.102:0/2784723824 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fea7c03f070 con 0x7fea8c0c75a0 2024-04-20T21:31:28.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.425+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea8c0c3690 con 0x7fea8c0c75a0 2024-04-20T21:31:28.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.426+0000 7fea89600640 1 -- 172.21.15.102:0/2784723824 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6026 (secure 0 0 0) 0x7fea7c07b670 con 0x7fea8c0c75a0 2024-04-20T21:31:28.425 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:28.426 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:25.671089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:28.426 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 44 2024-04-20T21:31:28.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea7006c920 msgr2=0x7fea7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea7006c920 0x7fea7006ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea80005880 tx=0x7fea80009040 comp rx=0 tx=0).stop 2024-04-20T21:31:28.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea8c0c75a0 msgr2=0x7fea8c1576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea8c0c75a0 0x7fea8c1576c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea7c033000 tx=0x7fea7c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 shutdown_connections 2024-04-20T21:31:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c153180 0x7fea8c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea8c0c75a0 0x7fea8c1576c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fea7006c920 0x7fea7006ede0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 --2- 172.21.15.102:0/2784723824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea8c073c70 0x7fea8c1ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 >> 172.21.15.102:0/2784723824 conn(0x7fea8c0cc820 msgr2=0x7fea8c1422c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 shutdown_connections 2024-04-20T21:31:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:28.429+0000 7fea93000640 1 -- 172.21.15.102:0/2784723824 wait complete. 2024-04-20T21:31:28.448 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 44, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 44, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5342": { "addr": "172.21.15.102:6833/1043030921", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 1043030921, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 1043030921, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5342, "incarnation": 44, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:31:25.671089+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5342 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:31:30.449 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:30.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.659+0000 7f0938a00640 1 Processor -- start 2024-04-20T21:31:30.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.659+0000 7f0938a00640 1 -- start start 2024-04-20T21:31:30.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0938a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f093415ea20 0x7f093406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0938a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f093415ee00 0x7f09340710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0938a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09340c8f20 con 0x7f093415f2b0 2024-04-20T21:31:30.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0938a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f093406cb80 con 0x7f093415ea20 2024-04-20T21:31:30.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0938a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f093406ce60 con 0x7f093415ee00 2024-04-20T21:31:30.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0931e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f093415ea20 0x7f093406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0931e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f093415ea20 0x7f093406d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38114/0 (socket says 172.21.15.102:38114) 2024-04-20T21:31:30.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0931e00640 1 -- 172.21.15.102:0/14473004 learned_addr learned my addr 172.21.15.102:0/14473004 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:30.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.661+0000 7f0929400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f093415ee00 0x7f09340710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.662+0000 7f0931400640 1 -- 172.21.15.102:0/14473004 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2913877697 0 0) 0x7f09340c8f20 con 0x7f093415f2b0 2024-04-20T21:31:30.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.662+0000 7f0931e00640 1 -- 172.21.15.102:0/14473004 >> v2:172.21.15.193:3300/0 conn(0x7f093415ee00 msgr2=0x7f09340710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:30.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.662+0000 7f0931e00640 1 --2- 172.21.15.102:0/14473004 >> v2:172.21.15.193:3300/0 conn(0x7f093415ee00 0x7f09340710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.662+0000 7f0931e00640 1 -- 172.21.15.102:0/14473004 >> v1:172.21.15.193:6789/0 conn(0x7f093415f2b0 legacy=0x7f09340c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.662+0000 7f0931e00640 1 -- 172.21.15.102:0/14473004 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093406d140 con 0x7f093415ea20 2024-04-20T21:31:30.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.663+0000 7f0931e00640 1 --2- 172.21.15.102:0/14473004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ea20 0x7f093406d940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f093406c2f0 tx=0x7f092402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91778eee3adfcb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:30.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.663+0000 7f0931400640 1 -- 172.21.15.102:0/14473004 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092402f850 con 0x7f093415ea20 2024-04-20T21:31:30.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.663+0000 7f0931400640 1 -- 172.21.15.102:0/14473004 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092402f9f0 con 0x7f093415ea20 2024-04-20T21:31:30.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.663+0000 7f0931400640 1 -- 172.21.15.102:0/14473004 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092402fdc0 con 0x7f093415ea20 2024-04-20T21:31:30.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.664+0000 7f0938a00640 1 -- 172.21.15.102:0/14473004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ea20 msgr2=0x7f093406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.664+0000 7f0938a00640 1 --2- 172.21.15.102:0/14473004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ea20 0x7f093406d940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f093406c2f0 tx=0x7f092402ba30 comp rx=0 tx=0).stop 2024-04-20T21:31:30.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.664+0000 7f0938a00640 1 -- 172.21.15.102:0/14473004 shutdown_connections 2024-04-20T21:31:30.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.664+0000 7f0938a00640 1 --2- 172.21.15.102:0/14473004 >> v2:172.21.15.193:3300/0 conn(0x7f093415ee00 0x7f09340710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.664+0000 7f0938a00640 1 --2- 172.21.15.102:0/14473004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ea20 0x7f093406d940 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.664+0000 7f0938a00640 1 -- 172.21.15.102:0/14473004 >> 172.21.15.102:0/14473004 conn(0x7f09340c9390 msgr2=0x7f09340c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:30.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.664+0000 7f0938a00640 1 -- 172.21.15.102:0/14473004 shutdown_connections 2024-04-20T21:31:30.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.665+0000 7f0938a00640 1 -- 172.21.15.102:0/14473004 wait complete. 2024-04-20T21:31:30.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.666+0000 7f0938a00640 1 Processor -- start 2024-04-20T21:31:30.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.667+0000 7f0938a00640 1 -- start start 2024-04-20T21:31:30.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.667+0000 7f0938a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f093415ea20 0x7f09340c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.667+0000 7f0938a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ee00 0x7f093415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.667+0000 7f0938a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f093415f2b0 0x7f093415db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.667+0000 7f0938a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f09340c8c80 con 0x7f093415ee00 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.667+0000 7f0938a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09340c8b00 con 0x7f093415ea20 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.667+0000 7f0938a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0934154e80 con 0x7f093415f2b0 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0929400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ee00 0x7f093415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0929400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ee00 0x7f093415d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38128/0 (socket says 172.21.15.102:38128) 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0929400640 1 -- 172.21.15.102:0/1490995938 learned_addr learned my addr 172.21.15.102:0/1490995938 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0931e00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f093415ea20 0x7f09340c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0932800640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f093415f2b0 0x7f093415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0929400640 1 -- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f093415f2b0 msgr2=0x7f093415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0929400640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f093415f2b0 0x7f093415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0929400640 1 -- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f093415ea20 msgr2=0x7f09340c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0929400640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f093415ea20 0x7f09340c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.668+0000 7f0929400640 1 -- 172.21.15.102:0/1490995938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093414efd0 con 0x7f093415ee00 2024-04-20T21:31:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.669+0000 7f0929400640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ee00 0x7f093415d650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9b0 tx=0x7f091c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.669+0000 7f092aa00640 1 -- 172.21.15.102:0/1490995938 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c017070 con 0x7f093415ee00 2024-04-20T21:31:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.669+0000 7f092aa00640 1 -- 172.21.15.102:0/1490995938 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c00c9a0 con 0x7f093415ee00 2024-04-20T21:31:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.669+0000 7f092aa00640 1 -- 172.21.15.102:0/1490995938 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c00cd70 con 0x7f093415ee00 2024-04-20T21:31:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.669+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093414ea50 con 0x7f093415ee00 2024-04-20T21:31:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.669+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09341510f0 con 0x7f093415ee00 2024-04-20T21:31:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.670+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0934074340 con 0x7f093415ee00 2024-04-20T21:31:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.671+0000 7f092aa00640 1 -- 172.21.15.102:0/1490995938 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f091c01a070 con 0x7f093415ee00 2024-04-20T21:31:30.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.672+0000 7f092aa00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f091006c920 0x7f091006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.673+0000 7f092aa00640 1 -- 172.21.15.102:0/1490995938 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f091c08c0c0 con 0x7f093415ee00 2024-04-20T21:31:30.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.673+0000 7f0931e00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f091006c920 0x7f091006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.674+0000 7f0931e00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f091006c920 0x7f091006ede0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0924036000 tx=0x7f0924008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:30.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.677+0000 7f092aa00640 1 -- 172.21.15.102:0/1490995938 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f091c059830 con 0x7f093415ee00 2024-04-20T21:31:30.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.979+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f093414a620 con 0x7f093415ee00 2024-04-20T21:31:30.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.979+0000 7f092aa00640 1 -- 172.21.15.102:0/1490995938 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6026 (secure 0 0 0) 0x7f091c05db30 con 0x7f093415ee00 2024-04-20T21:31:30.979 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:30.979 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:25.671089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:30.979 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 44 2024-04-20T21:31:30.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f091006c920 msgr2=0x7f091006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f091006c920 0x7f091006ede0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0924036000 tx=0x7f0924008030 comp rx=0 tx=0).stop 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ee00 msgr2=0x7f093415d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ee00 0x7f093415d650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9b0 tx=0x7f091c00ee70 comp rx=0 tx=0).stop 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 shutdown_connections 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f093415f2b0 0x7f093415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f093415ee00 0x7f093415d650 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f091006c920 0x7f091006ede0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 --2- 172.21.15.102:0/1490995938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f093415ea20 0x7f09340c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 >> 172.21.15.102:0/1490995938 conn(0x7f09340c9390 msgr2=0x7f0934152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 shutdown_connections 2024-04-20T21:31:30.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:30.982+0000 7f0938a00640 1 -- 172.21.15.102:0/1490995938 wait complete. 2024-04-20T21:31:31.001 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:31:33.002 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:33.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.193+0000 7f2903400640 1 Processor -- start 2024-04-20T21:31:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.193+0000 7f2903400640 1 -- start start 2024-04-20T21:31:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.194+0000 7f2903400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f28fc0c7330 0x7f28fc159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.194+0000 7f2903400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0c7710 0x7f28fc070220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.194+0000 7f2903400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f28fc0cd940 0x7f28fc152f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2903400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f28fc0cde80 con 0x7f28fc0cd940 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2903400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f28fc0ce140 con 0x7f28fc0c7330 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2903400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f28fc0ce420 con 0x7f28fc0c7710 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2901400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f28fc0cd940 0x7f28fc152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0c7710 0x7f28fc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2901400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f28fc0cd940 0x7f28fc152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38138/0 (socket says 172.21.15.102:38138) 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2901400640 1 -- 172.21.15.102:0/1059283572 learned_addr learned my addr 172.21.15.102:0/1059283572 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2900a00640 1 --2- 172.21.15.102:0/1059283572 >> v2:172.21.15.193:3300/0 conn(0x7f28fc0c7330 0x7f28fc159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2901400640 1 -- 172.21.15.102:0/1059283572 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0c7710 msgr2=0x7f28fc070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2901400640 1 --2- 172.21.15.102:0/1059283572 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0c7710 0x7f28fc070220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2901400640 1 -- 172.21.15.102:0/1059283572 >> v2:172.21.15.193:3300/0 conn(0x7f28fc0c7330 msgr2=0x7f28fc159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2901400640 1 --2- 172.21.15.102:0/1059283572 >> v2:172.21.15.193:3300/0 conn(0x7f28fc0c7330 0x7f28fc159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.195+0000 7f2901400640 1 -- 172.21.15.102:0/1059283572 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc0ce700 con 0x7f28fc0cd940 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.196+0000 7f2901400640 1 --2- 172.21.15.102:0/1059283572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0cd940 0x7f28fc152f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0cd910 tx=0x7f28f402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeaa403ebc53d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.197+0000 7f28fbe00640 1 -- 172.21.15.102:0/1059283572 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f402cb00 con 0x7f28fc0cd940 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.197+0000 7f28fbe00640 1 -- 172.21.15.102:0/1059283572 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f402cca0 con 0x7f28fc0cd940 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.197+0000 7f28fbe00640 1 -- 172.21.15.102:0/1059283572 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4034610 con 0x7f28fc0cd940 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.197+0000 7f2903400640 1 -- 172.21.15.102:0/1059283572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0cd940 msgr2=0x7f28fc152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.197+0000 7f2903400640 1 --2- 172.21.15.102:0/1059283572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0cd940 0x7f28fc152f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0cd910 tx=0x7f28f402bf60 comp rx=0 tx=0).stop 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.198+0000 7f2903400640 1 -- 172.21.15.102:0/1059283572 shutdown_connections 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.198+0000 7f2903400640 1 --2- 172.21.15.102:0/1059283572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0cd940 0x7f28fc152f40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.198+0000 7f2903400640 1 --2- 172.21.15.102:0/1059283572 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0c7710 0x7f28fc070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.198+0000 7f2903400640 1 --2- 172.21.15.102:0/1059283572 >> v2:172.21.15.193:3300/0 conn(0x7f28fc0c7330 0x7f28fc159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.198+0000 7f2903400640 1 -- 172.21.15.102:0/1059283572 >> 172.21.15.102:0/1059283572 conn(0x7f28fc06dad0 msgr2=0x7f28fc06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:33.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.198+0000 7f2903400640 1 -- 172.21.15.102:0/1059283572 shutdown_connections 2024-04-20T21:31:33.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.198+0000 7f2903400640 1 -- 172.21.15.102:0/1059283572 wait complete. 2024-04-20T21:31:33.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.199+0000 7f2903400640 1 Processor -- start 2024-04-20T21:31:33.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.199+0000 7f2903400640 1 -- start start 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.200+0000 7f2903400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28fc0c7330 0x7f28fc14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.200+0000 7f2903400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0c7710 0x7f28fc1f1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.200+0000 7f2903400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0cd940 0x7f28fc151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.200+0000 7f2903400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc1591b0 con 0x7f28fc0c7710 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.200+0000 7f2903400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc073e30 con 0x7f28fc0c7330 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.200+0000 7f2903400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f28fc15c7a0 con 0x7f28fc0cd940 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.200+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0c7710 0x7f28fc1f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0c7710 0x7f28fc1f1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38150/0 (socket says 172.21.15.102:38150) 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f3e00640 1 -- 172.21.15.102:0/2930371494 learned_addr learned my addr 172.21.15.102:0/2930371494 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f2900a00640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28fc0c7330 0x7f28fc14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f2901400640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0cd940 0x7f28fc151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f3e00640 1 -- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0cd940 msgr2=0x7f28fc151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f3e00640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0cd940 0x7f28fc151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f3e00640 1 -- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28fc0c7330 msgr2=0x7f28fc14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f3e00640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28fc0c7330 0x7f28fc14fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f3e00640 1 -- 172.21.15.102:0/2930371494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc1f1840 con 0x7f28fc0c7710 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f2900a00640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28fc0c7330 0x7f28fc14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f3e00640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0c7710 0x7f28fc1f1300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f28e400e9e0 tx=0x7f28e400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.201+0000 7f28f9600640 1 -- 172.21.15.102:0/2930371494 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4017070 con 0x7f28fc0c7710 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.202+0000 7f28f9600640 1 -- 172.21.15.102:0/2930371494 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e400c9d0 con 0x7f28fc0c7710 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.202+0000 7f28f9600640 1 -- 172.21.15.102:0/2930371494 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e400cdc0 con 0x7f28fc0c7710 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.202+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28fc1f1ad0 con 0x7f28fc0c7710 2024-04-20T21:31:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.202+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28fc1f2090 con 0x7f28fc0c7710 2024-04-20T21:31:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.203+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28fc074340 con 0x7f28fc0c7710 2024-04-20T21:31:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.204+0000 7f28f9600640 1 -- 172.21.15.102:0/2930371494 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28e401a070 con 0x7f28fc0c7710 2024-04-20T21:31:33.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.205+0000 7f28f9600640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f28d806c8f0 0x7f28d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.205+0000 7f28f9600640 1 -- 172.21.15.102:0/2930371494 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f28e408c070 con 0x7f28fc0c7710 2024-04-20T21:31:33.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.206+0000 7f2900a00640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f28d806c8f0 0x7f28d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.206+0000 7f2900a00640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f28d806c8f0 0x7f28d806edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009e10 tx=0x7f28ec009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.210+0000 7f28f9600640 1 -- 172.21.15.102:0/2930371494 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f28e4059880 con 0x7f28fc0c7710 2024-04-20T21:31:33.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.448+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28fc150460 con 0x7f28fc0c7710 2024-04-20T21:31:33.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.448+0000 7f28f9600640 1 -- 172.21.15.102:0/2930371494 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6026 (secure 0 0 0) 0x7f28e405db80 con 0x7f28fc0c7710 2024-04-20T21:31:33.448 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:33.448 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:25.671089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:33.448 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 44 2024-04-20T21:31:33.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f28d806c8f0 msgr2=0x7f28d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f28d806c8f0 0x7f28d806edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009e10 tx=0x7f28ec009190 comp rx=0 tx=0).stop 2024-04-20T21:31:33.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0c7710 msgr2=0x7f28fc1f1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0c7710 0x7f28fc1f1300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f28e400e9e0 tx=0x7f28e400eea0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 shutdown_connections 2024-04-20T21:31:33.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28fc0cd940 0x7f28fc151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc0c7710 0x7f28fc1f1300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f28d806c8f0 0x7f28d806edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 --2- 172.21.15.102:0/2930371494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28fc0c7330 0x7f28fc14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 >> 172.21.15.102:0/2930371494 conn(0x7f28fc06dad0 msgr2=0x7f28fc06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:33.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 shutdown_connections 2024-04-20T21:31:33.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:33.451+0000 7f2903400640 1 -- 172.21.15.102:0/2930371494 wait complete. 2024-04-20T21:31:33.466 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:31:35.468 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:35.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.650+0000 7f6a1fc00640 1 Processor -- start 2024-04-20T21:31:35.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.650+0000 7f6a1fc00640 1 -- start start 2024-04-20T21:31:35.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1fc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6a1814fa50 0x7f6a180c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1fc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a18073a10 0x7f6a18158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1fc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a181536d0 con 0x7f6a180c9eb0 2024-04-20T21:31:35.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1fc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6a18159500 con 0x7f6a1814fa50 2024-04-20T21:31:35.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1fc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6a180c76c0 con 0x7f6a18073a10 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1dc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a18073a10 0x7f6a18158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1d200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6a1814fa50 0x7f6a180c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1dc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a18073a10 0x7f6a18158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35878/0 (socket says 172.21.15.102:35878) 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a17e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6a180c9eb0 0x7f6a180c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:40716/0 (socket says 172.21.15.102:40716) 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.652+0000 7f6a1dc00640 1 -- 172.21.15.102:0/2592359368 learned_addr learned my addr 172.21.15.102:0/2592359368 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.653+0000 7f6a1d200640 1 -- 172.21.15.102:0/2592359368 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a18073a10 msgr2=0x7f6a18158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.653+0000 7f6a1d200640 1 --2- 172.21.15.102:0/2592359368 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a18073a10 0x7f6a18158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.653+0000 7f6a1d200640 1 -- 172.21.15.102:0/2592359368 >> v1:172.21.15.193:6789/0 conn(0x7f6a180c9eb0 legacy=0x7f6a180c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.653+0000 7f6a1d200640 1 -- 172.21.15.102:0/2592359368 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a180c7960 con 0x7f6a1814fa50 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.653+0000 7f6a1dc00640 1 --2- 172.21.15.102:0/2592359368 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a18073a10 0x7f6a18158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:35.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.653+0000 7f6a17400640 1 -- 172.21.15.102:0/2592359368 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4199978107 0 0) 0x7f6a04001110 con 0x7f6a180c9eb0 2024-04-20T21:31:35.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.654+0000 7f6a1d200640 1 --2- 172.21.15.102:0/2592359368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 0x7f6a180c4830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0097a0 tx=0x7f6a0c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af40099e6d71a445 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.654+0000 7f6a17400640 1 -- 172.21.15.102:0/2592359368 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c034070 con 0x7f6a1814fa50 2024-04-20T21:31:35.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.654+0000 7f6a17400640 1 -- 172.21.15.102:0/2592359368 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a0c030800 con 0x7f6a1814fa50 2024-04-20T21:31:35.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.654+0000 7f6a17400640 1 -- 172.21.15.102:0/2592359368 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c030bd0 con 0x7f6a1814fa50 2024-04-20T21:31:35.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.655+0000 7f6a1fc00640 1 -- 172.21.15.102:0/2592359368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 msgr2=0x7f6a180c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.655+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/2592359368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 0x7f6a180c4830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0097a0 tx=0x7f6a0c02b940 comp rx=0 tx=0).stop 2024-04-20T21:31:35.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.655+0000 7f6a1fc00640 1 -- 172.21.15.102:0/2592359368 shutdown_connections 2024-04-20T21:31:35.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.655+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/2592359368 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a18073a10 0x7f6a18158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.655+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/2592359368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 0x7f6a180c4830 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.655+0000 7f6a1fc00640 1 -- 172.21.15.102:0/2592359368 >> 172.21.15.102:0/2592359368 conn(0x7f6a180c9390 msgr2=0x7f6a180c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.655+0000 7f6a1fc00640 1 -- 172.21.15.102:0/2592359368 shutdown_connections 2024-04-20T21:31:35.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.655+0000 7f6a1fc00640 1 -- 172.21.15.102:0/2592359368 wait complete. 2024-04-20T21:31:35.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.657+0000 7f6a1fc00640 1 Processor -- start 2024-04-20T21:31:35.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.657+0000 7f6a1fc00640 1 -- start start 2024-04-20T21:31:35.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.658+0000 7f6a1fc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a18073a10 0x7f6a181f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.658+0000 7f6a1fc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a180c9eb0 0x7f6a181f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.658+0000 7f6a1fc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 0x7f6a181f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.658+0000 7f6a1fc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180c7c00 con 0x7f6a18073a10 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.658+0000 7f6a1fc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180c3d90 con 0x7f6a1814fa50 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.658+0000 7f6a1fc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6a180c4030 con 0x7f6a180c9eb0 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a1d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a18073a10 0x7f6a181f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a1d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a18073a10 0x7f6a181f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38152/0 (socket says 172.21.15.102:38152) 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a17e00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a180c9eb0 0x7f6a181f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a1d200640 1 -- 172.21.15.102:0/4229084174 learned_addr learned my addr 172.21.15.102:0/4229084174 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a1dc00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 0x7f6a181f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a17e00640 1 -- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 msgr2=0x7f6a181f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a17e00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 0x7f6a181f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a17e00640 1 -- 172.21.15.102:0/4229084174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a18073a10 msgr2=0x7f6a181f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a17e00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a18073a10 0x7f6a181f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a17e00640 1 -- 172.21.15.102:0/4229084174 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a181f4f70 con 0x7f6a180c9eb0 2024-04-20T21:31:35.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.659+0000 7f6a17e00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a180c9eb0 0x7f6a181f2890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c780 tx=0x7f6a0400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.660+0000 7f6a14c00640 1 -- 172.21.15.102:0/4229084174 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04018070 con 0x7f6a180c9eb0 2024-04-20T21:31:35.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.660+0000 7f6a14c00640 1 -- 172.21.15.102:0/4229084174 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a040070d0 con 0x7f6a180c9eb0 2024-04-20T21:31:35.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.660+0000 7f6a14c00640 1 -- 172.21.15.102:0/4229084174 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a040074c0 con 0x7f6a180c9eb0 2024-04-20T21:31:35.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.660+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1814da30 con 0x7f6a180c9eb0 2024-04-20T21:31:35.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.660+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a181f4c90 con 0x7f6a180c9eb0 2024-04-20T21:31:35.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.661+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a18074340 con 0x7f6a180c9eb0 2024-04-20T21:31:35.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.662+0000 7f6a14c00640 1 -- 172.21.15.102:0/4229084174 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a0401b070 con 0x7f6a180c9eb0 2024-04-20T21:31:35.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.663+0000 7f6a14c00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f69f806c8f0 0x7f69f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.663+0000 7f6a14c00640 1 -- 172.21.15.102:0/4229084174 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f6a0408b300 con 0x7f6a180c9eb0 2024-04-20T21:31:35.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.663+0000 7f6a1d200640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f69f806c8f0 0x7f69f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.664+0000 7f6a1d200640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f69f806c8f0 0x7f69f806edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c02b980 tx=0x7f6a0c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.667+0000 7f6a14c00640 1 -- 172.21.15.102:0/4229084174 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6a181f4f70 con 0x7f6a180c9eb0 2024-04-20T21:31:35.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.911+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a180c7320 con 0x7f6a180c9eb0 2024-04-20T21:31:35.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.912+0000 7f6a14c00640 1 -- 172.21.15.102:0/4229084174 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6026 (secure 0 0 0) 0x7f6a181536d0 con 0x7f6a180c9eb0 2024-04-20T21:31:35.911 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:35.911 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:25.671089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:35.911 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 44 2024-04-20T21:31:35.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f69f806c8f0 msgr2=0x7f69f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f69f806c8f0 0x7f69f806edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c02b980 tx=0x7f6a0c008030 comp rx=0 tx=0).stop 2024-04-20T21:31:35.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a180c9eb0 msgr2=0x7f6a181f2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a180c9eb0 0x7f6a181f2890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c780 tx=0x7f6a0400cc40 comp rx=0 tx=0).stop 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 shutdown_connections 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f69f806c8f0 0x7f69f806edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a1814fa50 0x7f6a181f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a180c9eb0 0x7f6a181f2890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 --2- 172.21.15.102:0/4229084174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a18073a10 0x7f6a181f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 >> 172.21.15.102:0/4229084174 conn(0x7f6a180c9390 msgr2=0x7f6a1806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 shutdown_connections 2024-04-20T21:31:35.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:35.914+0000 7f6a1fc00640 1 -- 172.21.15.102:0/4229084174 wait complete. 2024-04-20T21:31:35.931 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:31:37.932 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:38.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.139+0000 7f7472a00640 1 Processor -- start 2024-04-20T21:31:38.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.139+0000 7f7472a00640 1 -- start start 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.140+0000 7f7472a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c152170 0x7f746c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.140+0000 7f7472a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f746c158b50 0x7f746c1628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.140+0000 7f7472a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746c071f70 con 0x7f746c158770 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.140+0000 7f7472a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f746c0c6dc0 con 0x7f746c158b50 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.140+0000 7f7472a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f746c072210 con 0x7f746c152170 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f746be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c152170 0x7f746c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f7470a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f746c158b50 0x7f746c1628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f746be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c152170 0x7f746c0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:45258/0 (socket says 172.21.15.102:45258) 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f746be00640 1 -- 172.21.15.102:0/4132330429 learned_addr learned my addr 172.21.15.102:0/4132330429 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:38.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f7463400640 1 --1- 172.21.15.102:0/4132330429 >> v1:172.21.15.193:6789/0 conn(0x7f746c158770 0x7f746c070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43204/0 (socket says 172.21.15.102:43204) 2024-04-20T21:31:38.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f746be00640 1 -- 172.21.15.102:0/4132330429 >> v2:172.21.15.193:3300/0 conn(0x7f746c158b50 msgr2=0x7f746c1628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:38.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f746be00640 1 --2- 172.21.15.102:0/4132330429 >> v2:172.21.15.193:3300/0 conn(0x7f746c158b50 0x7f746c1628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f746be00640 1 -- 172.21.15.102:0/4132330429 >> v1:172.21.15.193:6789/0 conn(0x7f746c158770 legacy=0x7f746c070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.141+0000 7f746be00640 1 -- 172.21.15.102:0/4132330429 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c0724b0 con 0x7f746c152170 2024-04-20T21:31:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.142+0000 7f746be00640 1 --2- 172.21.15.102:0/4132330429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c152170 0x7f746c0c8db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f745c00d150 tx=0x7f745c02e1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ea75e29e1920d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f746b400640 1 -- 172.21.15.102:0/4132330429 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c00b0d0 con 0x7f746c152170 2024-04-20T21:31:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f746b400640 1 -- 172.21.15.102:0/4132330429 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c00b270 con 0x7f746c152170 2024-04-20T21:31:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f746b400640 1 -- 172.21.15.102:0/4132330429 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c009600 con 0x7f746c152170 2024-04-20T21:31:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f7472a00640 1 -- 172.21.15.102:0/4132330429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c152170 msgr2=0x7f746c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f7472a00640 1 --2- 172.21.15.102:0/4132330429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c152170 0x7f746c0c8db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f745c00d150 tx=0x7f745c02e1f0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f7472a00640 1 -- 172.21.15.102:0/4132330429 shutdown_connections 2024-04-20T21:31:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f7472a00640 1 --2- 172.21.15.102:0/4132330429 >> v2:172.21.15.193:3300/0 conn(0x7f746c158b50 0x7f746c1628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f7472a00640 1 --2- 172.21.15.102:0/4132330429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c152170 0x7f746c0c8db0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f7472a00640 1 -- 172.21.15.102:0/4132330429 >> 172.21.15.102:0/4132330429 conn(0x7f746c0c9390 msgr2=0x7f746c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f7472a00640 1 -- 172.21.15.102:0/4132330429 shutdown_connections 2024-04-20T21:31:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.143+0000 7f7472a00640 1 -- 172.21.15.102:0/4132330429 wait complete. 2024-04-20T21:31:38.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.145+0000 7f7472a00640 1 Processor -- start 2024-04-20T21:31:38.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.145+0000 7f7472a00640 1 -- start start 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f7472a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f746c152170 0x7f746c1552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f7472a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c158770 0x7f746c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f7472a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f746c158b50 0x7f746c1f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f7472a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f746c0c9f20 con 0x7f746c152170 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f7472a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f746c158250 con 0x7f746c158b50 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f7472a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f746c157f20 con 0x7f746c158770 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f746be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f746c152170 0x7f746c1552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f746be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f746c152170 0x7f746c1552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48110/0 (socket says 172.21.15.102:48110) 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f746be00640 1 -- 172.21.15.102:0/3527049407 learned_addr learned my addr 172.21.15.102:0/3527049407 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:38.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f7470a00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f746c158b50 0x7f746c1f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.146+0000 7f7463400640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c158770 0x7f746c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.147+0000 7f746be00640 1 -- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c158770 msgr2=0x7f746c150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.147+0000 7f746be00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c158770 0x7f746c150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.147+0000 7f746be00640 1 -- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f746c158b50 msgr2=0x7f746c1f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.147+0000 7f746be00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f746c158b50 0x7f746c1f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.147+0000 7f746be00640 1 -- 172.21.15.102:0/3527049407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c1541b0 con 0x7f746c152170 2024-04-20T21:31:38.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.147+0000 7f746be00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f746c152170 0x7f746c1552a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f745c038040 tx=0x7f745c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.147+0000 7f7468c00640 1 -- 172.21.15.102:0/3527049407 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c00b0d0 con 0x7f746c152170 2024-04-20T21:31:38.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.148+0000 7f7468c00640 1 -- 172.21.15.102:0/3527049407 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c045070 con 0x7f746c152170 2024-04-20T21:31:38.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.148+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c1f53d0 con 0x7f746c152170 2024-04-20T21:31:38.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.148+0000 7f7468c00640 1 -- 172.21.15.102:0/3527049407 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c00b720 con 0x7f746c152170 2024-04-20T21:31:38.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.148+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c152c90 con 0x7f746c152170 2024-04-20T21:31:38.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.149+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746c074340 con 0x7f746c152170 2024-04-20T21:31:38.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.150+0000 7f7468c00640 1 -- 172.21.15.102:0/3527049407 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f745c036c50 con 0x7f746c152170 2024-04-20T21:31:38.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.151+0000 7f7468c00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f744806c8f0 0x7f744806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.151+0000 7f7468c00640 1 -- 172.21.15.102:0/3527049407 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f745c0aef80 con 0x7f746c152170 2024-04-20T21:31:38.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.151+0000 7f7463400640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f744806c8f0 0x7f744806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.152+0000 7f7463400640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f744806c8f0 0x7f744806edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f746c0cc530 tx=0x7f7454007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.155+0000 7f7468c00640 1 -- 172.21.15.102:0/3527049407 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f745c07c700 con 0x7f746c152170 2024-04-20T21:31:38.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.394+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f746c155000 con 0x7f746c152170 2024-04-20T21:31:38.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.395+0000 7f7468c00640 1 -- 172.21.15.102:0/3527049407 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6030 (secure 0 0 0) 0x7f745c080a00 con 0x7f746c152170 2024-04-20T21:31:38.394 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:38.395 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:36.133520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:38.395 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 45 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f744806c8f0 msgr2=0x7f744806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f744806c8f0 0x7f744806edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f746c0cc530 tx=0x7f7454007e80 comp rx=0 tx=0).stop 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f746c152170 msgr2=0x7f746c1552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f746c152170 0x7f746c1552a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f745c038040 tx=0x7f745c004a40 comp rx=0 tx=0).stop 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 shutdown_connections 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f744806c8f0 0x7f744806edb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f746c158b50 0x7f746c1f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f746c158770 0x7f746c150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 --2- 172.21.15.102:0/3527049407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f746c152170 0x7f746c1552a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 >> 172.21.15.102:0/3527049407 conn(0x7f746c0c9390 msgr2=0x7f746c157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 shutdown_connections 2024-04-20T21:31:38.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:38.398+0000 7f7472a00640 1 -- 172.21.15.102:0/3527049407 wait complete. 2024-04-20T21:31:38.415 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:31:40.416 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:40.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.623+0000 7f51ada00640 1 Processor -- start 2024-04-20T21:31:40.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.623+0000 7f51ada00640 1 -- start start 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.625+0000 7f51ada00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f51a80c75a0 0x7f51a80c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.625+0000 7f51ada00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8073c70 0x7f51a80c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.625+0000 7f51ada00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a8153560 con 0x7f51a8153180 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.625+0000 7f51ada00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f51a80c7a70 con 0x7f51a80c75a0 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.625+0000 7f51ada00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f51a80c7d30 con 0x7f51a8073c70 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.626+0000 7f51a7e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f51a8153180 0x7f51a8158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46694/0 (socket says 172.21.15.102:46694) 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.626+0000 7f51a7e00640 1 -- 172.21.15.102:0/3033514096 learned_addr learned my addr 172.21.15.102:0/3033514096 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.626+0000 7f51a7400640 1 --2- 172.21.15.102:0/3033514096 >> v2:172.21.15.193:3300/0 conn(0x7f51a80c75a0 0x7f51a80c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.626+0000 7f51ad000640 1 --2- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8073c70 0x7f51a80c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:40.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.626+0000 7f51a6a00640 1 -- 172.21.15.102:0/3033514096 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2027210334 0 0) 0x7f51a8153560 con 0x7f51a8153180 2024-04-20T21:31:40.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.626+0000 7f51a6a00640 1 -- 172.21.15.102:0/3033514096 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f518c003520 con 0x7f51a8153180 2024-04-20T21:31:40.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.626+0000 7f51a7400640 1 -- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8073c70 msgr2=0x7f51a80c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:40.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.626+0000 7f51a7400640 1 --2- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8073c70 0x7f51a80c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.627+0000 7f51a7400640 1 -- 172.21.15.102:0/3033514096 >> v1:172.21.15.102:6789/0 conn(0x7f51a8153180 legacy=0x7f51a8158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:40.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.627+0000 7f51a7400640 1 -- 172.21.15.102:0/3033514096 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a815c4d0 con 0x7f51a80c75a0 2024-04-20T21:31:40.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.627+0000 7f51ad000640 1 --2- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8073c70 0x7f51a80c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:40.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.628+0000 7f51a7400640 1 --2- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a80c75a0 0x7f51a80c5f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c277045b9da56ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:40.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.628+0000 7f51a6a00640 1 -- 172.21.15.102:0/3033514096 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194035020 con 0x7f51a80c75a0 2024-04-20T21:31:40.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.628+0000 7f51a6a00640 1 -- 172.21.15.102:0/3033514096 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5194031810 con 0x7f51a80c75a0 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.628+0000 7f51a6a00640 1 -- 172.21.15.102:0/3033514096 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194031be0 con 0x7f51a80c75a0 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.629+0000 7f51ada00640 1 -- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a80c75a0 msgr2=0x7f51a80c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.629+0000 7f51ada00640 1 --2- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a80c75a0 0x7f51a80c5f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402b8b0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.629+0000 7f51ada00640 1 -- 172.21.15.102:0/3033514096 shutdown_connections 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.629+0000 7f51ada00640 1 --2- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8073c70 0x7f51a80c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.629+0000 7f51ada00640 1 --2- 172.21.15.102:0/3033514096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a80c75a0 0x7f51a80c5f30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.629+0000 7f51ada00640 1 -- 172.21.15.102:0/3033514096 >> 172.21.15.102:0/3033514096 conn(0x7f51a80cc820 msgr2=0x7f51a80ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.629+0000 7f51ada00640 1 -- 172.21.15.102:0/3033514096 shutdown_connections 2024-04-20T21:31:40.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.629+0000 7f51ada00640 1 -- 172.21.15.102:0/3033514096 wait complete. 2024-04-20T21:31:40.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.631+0000 7f51ada00640 1 Processor -- start 2024-04-20T21:31:40.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.631+0000 7f51ada00640 1 -- start start 2024-04-20T21:31:40.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51ada00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a8073c70 0x7f51a8153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:40.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51ada00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51a80c75a0 0x7f51a814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:40.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51ada00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8153180 0x7f51a814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:40.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51ada00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8159550 con 0x7f51a80c75a0 2024-04-20T21:31:40.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51ada00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51a815ca90 con 0x7f51a8073c70 2024-04-20T21:31:40.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51ada00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f51a815d290 con 0x7f51a8153180 2024-04-20T21:31:40.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51a80c75a0 0x7f51a814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:40.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51a80c75a0 0x7f51a814c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48112/0 (socket says 172.21.15.102:48112) 2024-04-20T21:31:40.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51a7400640 1 -- 172.21.15.102:0/719840256 learned_addr learned my addr 172.21.15.102:0/719840256 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:40.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51a7e00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a8073c70 0x7f51a8153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:40.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.632+0000 7f51ad000640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8153180 0x7f51a814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:40.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51a7400640 1 -- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8153180 msgr2=0x7f51a814a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:40.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51a7400640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8153180 0x7f51a814a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51a7400640 1 -- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a8073c70 msgr2=0x7f51a8153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:40.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51a7400640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a8073c70 0x7f51a8153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51a7400640 1 -- 172.21.15.102:0/719840256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a814bcb0 con 0x7f51a80c75a0 2024-04-20T21:31:40.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51ad000640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8153180 0x7f51a814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:40.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51a7400640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51a80c75a0 0x7f51a814c770 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f51940098c0 tx=0x7f51940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:40.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f519fe00640 1 -- 172.21.15.102:0/719840256 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194035040 con 0x7f51a80c75a0 2024-04-20T21:31:40.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f519fe00640 1 -- 172.21.15.102:0/719840256 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5194039cf0 con 0x7f51a80c75a0 2024-04-20T21:31:40.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f519fe00640 1 -- 172.21.15.102:0/719840256 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194039ea0 con 0x7f51a80c75a0 2024-04-20T21:31:40.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a80c6090 con 0x7f51a80c75a0 2024-04-20T21:31:40.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.633+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a80c6650 con 0x7f51a80c75a0 2024-04-20T21:31:40.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.635+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a8070820 con 0x7f51a80c75a0 2024-04-20T21:31:40.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.636+0000 7f519fe00640 1 -- 172.21.15.102:0/719840256 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5194038020 con 0x7f51a80c75a0 2024-04-20T21:31:40.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.637+0000 7f519fe00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f518806c8f0 0x7f518806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:40.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.637+0000 7f519fe00640 1 -- 172.21.15.102:0/719840256 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f51940adda0 con 0x7f51a80c75a0 2024-04-20T21:31:40.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.637+0000 7f51a7e00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f518806c8f0 0x7f518806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:40.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.638+0000 7f51a7e00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f518806c8f0 0x7f518806edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5198005880 tx=0x7f5198009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:40.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.641+0000 7f519fe00640 1 -- 172.21.15.102:0/719840256 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f51a8070820 con 0x7f51a80c75a0 2024-04-20T21:31:40.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.885+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51a8151d00 con 0x7f51a80c75a0 2024-04-20T21:31:40.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.886+0000 7f519fe00640 1 -- 172.21.15.102:0/719840256 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6030 (secure 0 0 0) 0x7f51a8151d00 con 0x7f51a80c75a0 2024-04-20T21:31:40.886 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:40.886 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:36.133520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:40.887 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 45 2024-04-20T21:31:40.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f518806c8f0 msgr2=0x7f518806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f518806c8f0 0x7f518806edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5198005880 tx=0x7f5198009040 comp rx=0 tx=0).stop 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51a80c75a0 msgr2=0x7f51a814c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51a80c75a0 0x7f51a814c770 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f51940098c0 tx=0x7f51940046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 shutdown_connections 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51a8153180 0x7f51a814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51a80c75a0 0x7f51a814c770 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f518806c8f0 0x7f518806edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 --2- 172.21.15.102:0/719840256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51a8073c70 0x7f51a8153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 >> 172.21.15.102:0/719840256 conn(0x7f51a80cc820 msgr2=0x7f51a806ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 shutdown_connections 2024-04-20T21:31:40.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:40.889+0000 7f51ada00640 1 -- 172.21.15.102:0/719840256 wait complete. 2024-04-20T21:31:40.907 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 45, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 45, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5342": { "addr": "172.21.15.102:6833/1043030921", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 1043030921, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 1043030921, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5342, "incarnation": 44, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 38 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:31:36.133520+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5342 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:31:42.909 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:43.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.119+0000 7fc3ea800640 1 Processor -- start 2024-04-20T21:31:43.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.119+0000 7fc3ea800640 1 -- start start 2024-04-20T21:31:43.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3ea800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc3e406c550 0x7fc3e415c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3ea800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc3e415bfd0 0x7fc3e40ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3ea800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e4073a20 con 0x7fc3e41554f0 2024-04-20T21:31:43.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3ea800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e415b510 con 0x7fc3e415bfd0 2024-04-20T21:31:43.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3ea800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e4073cc0 con 0x7fc3e406c550 2024-04-20T21:31:43.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3e3400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc3e41554f0 0x7fc3e40c98f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46696/0 (socket says 172.21.15.102:46696) 2024-04-20T21:31:43.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3e3400640 1 -- 172.21.15.102:0/1182310671 learned_addr learned my addr 172.21.15.102:0/1182310671 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:43.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3e3e00640 1 --2- 172.21.15.102:0/1182310671 >> v2:172.21.15.102:3300/0 conn(0x7fc3e415bfd0 0x7fc3e40ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3dbe00640 1 --2- 172.21.15.102:0/1182310671 >> v2:172.21.15.193:3300/0 conn(0x7fc3e406c550 0x7fc3e415c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3e2a00640 1 -- 172.21.15.102:0/1182310671 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 754382243 0 0) 0x7fc3e4073a20 con 0x7fc3e41554f0 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3e2a00640 1 -- 172.21.15.102:0/1182310671 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3c8003520 con 0x7fc3e41554f0 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3e3e00640 1 -- 172.21.15.102:0/1182310671 >> v2:172.21.15.193:3300/0 conn(0x7fc3e406c550 msgr2=0x7fc3e415c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.121+0000 7fc3e3e00640 1 --2- 172.21.15.102:0/1182310671 >> v2:172.21.15.193:3300/0 conn(0x7fc3e406c550 0x7fc3e415c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.122+0000 7fc3e3e00640 1 -- 172.21.15.102:0/1182310671 >> v1:172.21.15.102:6789/0 conn(0x7fc3e41554f0 legacy=0x7fc3e40c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.122+0000 7fc3e3e00640 1 -- 172.21.15.102:0/1182310671 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e415b7d0 con 0x7fc3e415bfd0 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.123+0000 7fc3e3e00640 1 --2- 172.21.15.102:0/1182310671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e415bfd0 0x7fc3e40ceb90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc3e406ce30 tx=0x7fc3dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb146108018df166 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.123+0000 7fc3e2a00640 1 -- 172.21.15.102:0/1182310671 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc039070 con 0x7fc3e415bfd0 2024-04-20T21:31:43.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.123+0000 7fc3e2a00640 1 -- 172.21.15.102:0/1182310671 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3dc02cc80 con 0x7fc3e415bfd0 2024-04-20T21:31:43.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.123+0000 7fc3e2a00640 1 -- 172.21.15.102:0/1182310671 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc034660 con 0x7fc3e415bfd0 2024-04-20T21:31:43.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.123+0000 7fc3ea800640 1 -- 172.21.15.102:0/1182310671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e415bfd0 msgr2=0x7fc3e40ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.123+0000 7fc3ea800640 1 --2- 172.21.15.102:0/1182310671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e415bfd0 0x7fc3e40ceb90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc3e406ce30 tx=0x7fc3dc02bed0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.124+0000 7fc3ea800640 1 -- 172.21.15.102:0/1182310671 shutdown_connections 2024-04-20T21:31:43.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.124+0000 7fc3ea800640 1 --2- 172.21.15.102:0/1182310671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e415bfd0 0x7fc3e40ceb90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.124+0000 7fc3ea800640 1 --2- 172.21.15.102:0/1182310671 >> v2:172.21.15.193:3300/0 conn(0x7fc3e406c550 0x7fc3e415c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.124+0000 7fc3ea800640 1 -- 172.21.15.102:0/1182310671 >> 172.21.15.102:0/1182310671 conn(0x7fc3e406ebf0 msgr2=0x7fc3e406efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:43.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.124+0000 7fc3ea800640 1 -- 172.21.15.102:0/1182310671 shutdown_connections 2024-04-20T21:31:43.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.124+0000 7fc3ea800640 1 -- 172.21.15.102:0/1182310671 wait complete. 2024-04-20T21:31:43.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.125+0000 7fc3ea800640 1 Processor -- start 2024-04-20T21:31:43.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.126+0000 7fc3ea800640 1 -- start start 2024-04-20T21:31:43.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.126+0000 7fc3ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e406c550 0x7fc3e41f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.126+0000 7fc3ea800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e41554f0 0x7fc3e41517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3ea800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e415bfd0 0x7fc3e40ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3ea800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e40ca000 con 0x7fc3e406c550 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3ea800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e40736e0 con 0x7fc3e41554f0 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3ea800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e41558d0 con 0x7fc3e415bfd0 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e406c550 0x7fc3e41f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e406c550 0x7fc3e41f0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48140/0 (socket says 172.21.15.102:48140) 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3400640 1 -- 172.21.15.102:0/4213253385 learned_addr learned my addr 172.21.15.102:0/4213253385 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3dbe00640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e41554f0 0x7fc3e41517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3e00640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e415bfd0 0x7fc3e40ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3400640 1 -- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e415bfd0 msgr2=0x7fc3e40ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3400640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e415bfd0 0x7fc3e40ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3400640 1 -- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e41554f0 msgr2=0x7fc3e41517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3400640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e41554f0 0x7fc3e41517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.127+0000 7fc3e3400640 1 -- 172.21.15.102:0/4213253385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e41f4f80 con 0x7fc3e406c550 2024-04-20T21:31:43.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.128+0000 7fc3e3400640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e406c550 0x7fc3e41f0a90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40010e0 tx=0x7fc3d400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:43.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.128+0000 7fc3db400640 1 -- 172.21.15.102:0/4213253385 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4017070 con 0x7fc3e406c550 2024-04-20T21:31:43.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.128+0000 7fc3db400640 1 -- 172.21.15.102:0/4213253385 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d4005d00 con 0x7fc3e406c550 2024-04-20T21:31:43.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.128+0000 7fc3db400640 1 -- 172.21.15.102:0/4213253385 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40072b0 con 0x7fc3e406c550 2024-04-20T21:31:43.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.128+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e41f4120 con 0x7fc3e406c550 2024-04-20T21:31:43.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.128+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e41f46e0 con 0x7fc3e406c550 2024-04-20T21:31:43.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.129+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e4074340 con 0x7fc3e406c550 2024-04-20T21:31:43.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.130+0000 7fc3db400640 1 -- 172.21.15.102:0/4213253385 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3d401a070 con 0x7fc3e406c550 2024-04-20T21:31:43.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.131+0000 7fc3db400640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3c006c8f0 0x7fc3c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.131+0000 7fc3db400640 1 -- 172.21.15.102:0/4213253385 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7fc3d408b540 con 0x7fc3e406c550 2024-04-20T21:31:43.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.131+0000 7fc3dbe00640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3c006c8f0 0x7fc3c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.132+0000 7fc3dbe00640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3c006c8f0 0x7fc3c006edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40c8a40 tx=0x7fc3cc009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:43.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.136+0000 7fc3db400640 1 -- 172.21.15.102:0/4213253385 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc3d4058cb0 con 0x7fc3e406c550 2024-04-20T21:31:43.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.372+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3e4159e80 con 0x7fc3e406c550 2024-04-20T21:31:43.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.374+0000 7fc3db400640 1 -- 172.21.15.102:0/4213253385 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+6027 (secure 0 0 0) 0x7fc3d405cfb0 con 0x7fc3e406c550 2024-04-20T21:31:43.373 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:43.373 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:41.458494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:rejoin","state_seq":40,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:43.373 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 46 2024-04-20T21:31:43.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3c006c8f0 msgr2=0x7fc3c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3c006c8f0 0x7fc3c006edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40c8a40 tx=0x7fc3cc009190 comp rx=0 tx=0).stop 2024-04-20T21:31:43.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e406c550 msgr2=0x7fc3e41f0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e406c550 0x7fc3e41f0a90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40010e0 tx=0x7fc3d400c960 comp rx=0 tx=0).stop 2024-04-20T21:31:43.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 shutdown_connections 2024-04-20T21:31:43.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e415bfd0 0x7fc3e40ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3c006c8f0 0x7fc3c006edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e41554f0 0x7fc3e41517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 --2- 172.21.15.102:0/4213253385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e406c550 0x7fc3e41f0a90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 >> 172.21.15.102:0/4213253385 conn(0x7fc3e406ebf0 msgr2=0x7fc3e406efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:43.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 shutdown_connections 2024-04-20T21:31:43.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:43.376+0000 7fc3ea800640 1 -- 172.21.15.102:0/4213253385 wait complete. 2024-04-20T21:31:43.395 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:31:45.396 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:45.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.605+0000 7f2205600640 1 Processor -- start 2024-04-20T21:31:45.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.605+0000 7f2205600640 1 -- start start 2024-04-20T21:31:45.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f2205600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f22000c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f2205600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22000c9eb0 0x7f2200158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f2205600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22001536d0 con 0x7f2200073a10 2024-04-20T21:31:45.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f2205600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f2200159500 con 0x7f22000c9eb0 2024-04-20T21:31:45.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f2205600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f22000c76c0 con 0x7f220014fa50 2024-04-20T21:31:45.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f21fe000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22000c9eb0 0x7f2200158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f21fea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f22000c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f21fe000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22000c9eb0 0x7f2200158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48152/0 (socket says 172.21.15.102:48152) 2024-04-20T21:31:45.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f21fe000640 1 -- 172.21.15.102:0/1225074852 learned_addr learned my addr 172.21.15.102:0/1225074852 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:45.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.607+0000 7f21ff400640 1 --1- 172.21.15.102:0/1225074852 >> v1:172.21.15.193:6789/0 conn(0x7f2200073a10 0x7f22000c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43216/0 (socket says 172.21.15.102:43216) 2024-04-20T21:31:45.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.608+0000 7f21fe000640 1 -- 172.21.15.102:0/1225074852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 msgr2=0x7f22000c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:45.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.608+0000 7f21fe000640 1 --2- 172.21.15.102:0/1225074852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f22000c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.608+0000 7f21fe000640 1 -- 172.21.15.102:0/1225074852 >> v1:172.21.15.193:6789/0 conn(0x7f2200073a10 legacy=0x7f22000c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.608+0000 7f21fe000640 1 -- 172.21.15.102:0/1225074852 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22000c7960 con 0x7f22000c9eb0 2024-04-20T21:31:45.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.608+0000 7f21fea00640 1 --2- 172.21.15.102:0/1225074852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f22000c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:45.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.608+0000 7f21fd600640 1 -- 172.21.15.102:0/1225074852 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224423082 0 0) 0x7f21f8001490 con 0x7f2200073a10 2024-04-20T21:31:45.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.609+0000 7f21fe000640 1 --2- 172.21.15.102:0/1225074852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22000c9eb0 0x7f2200158fc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22000c7b30 tx=0x7f21ec02ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fd50c97b91d58c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:45.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.609+0000 7f21fd600640 1 -- 172.21.15.102:0/1225074852 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec02f820 con 0x7f22000c9eb0 2024-04-20T21:31:45.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.609+0000 7f21fd600640 1 -- 172.21.15.102:0/1225074852 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec02f9c0 con 0x7f22000c9eb0 2024-04-20T21:31:45.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.609+0000 7f21fd600640 1 -- 172.21.15.102:0/1225074852 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec02fd90 con 0x7f22000c9eb0 2024-04-20T21:31:45.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.609+0000 7f2205600640 1 -- 172.21.15.102:0/1225074852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22000c9eb0 msgr2=0x7f2200158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.609+0000 7f2205600640 1 --2- 172.21.15.102:0/1225074852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22000c9eb0 0x7f2200158fc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22000c7b30 tx=0x7f21ec02ba00 comp rx=0 tx=0).stop 2024-04-20T21:31:45.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.610+0000 7f2205600640 1 -- 172.21.15.102:0/1225074852 shutdown_connections 2024-04-20T21:31:45.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.610+0000 7f2205600640 1 --2- 172.21.15.102:0/1225074852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22000c9eb0 0x7f2200158fc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.610+0000 7f2205600640 1 --2- 172.21.15.102:0/1225074852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f22000c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.610+0000 7f2205600640 1 -- 172.21.15.102:0/1225074852 >> 172.21.15.102:0/1225074852 conn(0x7f22000c9390 msgr2=0x7f22000c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:45.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.610+0000 7f2205600640 1 -- 172.21.15.102:0/1225074852 shutdown_connections 2024-04-20T21:31:45.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.610+0000 7f2205600640 1 -- 172.21.15.102:0/1225074852 wait complete. 2024-04-20T21:31:45.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.611+0000 7f2205600640 1 Processor -- start 2024-04-20T21:31:45.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.612+0000 7f2205600640 1 -- start start 2024-04-20T21:31:45.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.612+0000 7f2205600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200073a10 0x7f22001f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.612+0000 7f2205600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22000c9eb0 0x7f2200154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.612+0000 7f2205600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f2200154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.612+0000 7f2205600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22000c7c00 con 0x7f2200073a10 2024-04-20T21:31:45.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.612+0000 7f2205600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22000c3d90 con 0x7f22000c9eb0 2024-04-20T21:31:45.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.612+0000 7f2205600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f22000c4030 con 0x7f220014fa50 2024-04-20T21:31:45.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200073a10 0x7f22001f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200073a10 0x7f22001f0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48154/0 (socket says 172.21.15.102:48154) 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21ff400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f2200154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fea00640 1 -- 172.21.15.102:0/982747782 learned_addr learned my addr 172.21.15.102:0/982747782 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fe000640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22000c9eb0 0x7f2200154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fe000640 1 -- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 msgr2=0x7f2200154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fe000640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f2200154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fe000640 1 -- 172.21.15.102:0/982747782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200073a10 msgr2=0x7f22001f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fe000640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200073a10 0x7f22001f0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fe000640 1 -- 172.21.15.102:0/982747782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2200155110 con 0x7f22000c9eb0 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.613+0000 7f21fea00640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200073a10 0x7f22001f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.614+0000 7f21fe000640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22000c9eb0 0x7f2200154310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f21ec033000 tx=0x7f21ec004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.614+0000 7f21f6a00640 1 -- 172.21.15.102:0/982747782 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec037b40 con 0x7f22000c9eb0 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.614+0000 7f21f6a00640 1 -- 172.21.15.102:0/982747782 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec046070 con 0x7f22000c9eb0 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.614+0000 7f21f6a00640 1 -- 172.21.15.102:0/982747782 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec041400 con 0x7f22000c9eb0 2024-04-20T21:31:45.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.614+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22000caab0 con 0x7f22000c9eb0 2024-04-20T21:31:45.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.614+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22000cb070 con 0x7f22000c9eb0 2024-04-20T21:31:45.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.615+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2200070310 con 0x7f22000c9eb0 2024-04-20T21:31:45.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.616+0000 7f21f6a00640 1 -- 172.21.15.102:0/982747782 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21ec040070 con 0x7f22000c9eb0 2024-04-20T21:31:45.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.617+0000 7f21f6a00640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e006c920 0x7f21e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.617+0000 7f21f6a00640 1 -- 172.21.15.102:0/982747782 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f21ec0ae550 con 0x7f22000c9eb0 2024-04-20T21:31:45.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.617+0000 7f21fea00640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e006c920 0x7f21e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.618+0000 7f21fea00640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e006c920 0x7f21e006ede0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009560 tx=0x7f21f0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:45.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.622+0000 7f21f6a00640 1 -- 172.21.15.102:0/982747782 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2200070310 con 0x7f22000c9eb0 2024-04-20T21:31:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.868+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22000c56b0 con 0x7f22000c9eb0 2024-04-20T21:31:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.869+0000 7f21f6a00640 1 -- 172.21.15.102:0/982747782 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+6027 (secure 0 0 0) 0x7f22000c56b0 con 0x7f22000c9eb0 2024-04-20T21:31:45.868 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:45.868 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:41.458494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:rejoin","state_seq":40,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 46 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e006c920 msgr2=0x7f21e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e006c920 0x7f21e006ede0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009560 tx=0x7f21f0009190 comp rx=0 tx=0).stop 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22000c9eb0 msgr2=0x7f2200154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22000c9eb0 0x7f2200154310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f21ec033000 tx=0x7f21ec004730 comp rx=0 tx=0).stop 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 shutdown_connections 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f220014fa50 0x7f2200154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e006c920 0x7f21e006ede0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22000c9eb0 0x7f2200154310 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 --2- 172.21.15.102:0/982747782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200073a10 0x7f22001f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 >> 172.21.15.102:0/982747782 conn(0x7f22000c9390 msgr2=0x7f220014be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:45.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 shutdown_connections 2024-04-20T21:31:45.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:45.872+0000 7f2205600640 1 -- 172.21.15.102:0/982747782 wait complete. 2024-04-20T21:31:45.889 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:31:47.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:48.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.098+0000 7f3195600640 1 Processor -- start 2024-04-20T21:31:48.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.098+0000 7f3195600640 1 -- start start 2024-04-20T21:31:48.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3195600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f319014fa50 0x7f31900c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:48.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3195600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31900c9eb0 0x7f3190158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:48.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3195600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3190073a10 0x7f31900c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:48.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3195600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3190159500 con 0x7f31900c9eb0 2024-04-20T21:31:48.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3195600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f31901536d0 con 0x7f319014fa50 2024-04-20T21:31:48.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3195600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f31900c76c0 con 0x7f3190073a10 2024-04-20T21:31:48.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3186000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31900c9eb0 0x7f3190158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3186000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31900c9eb0 0x7f3190158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58578/0 (socket says 172.21.15.102:58578) 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f3186000640 1 -- 172.21.15.102:0/2366324429 learned_addr learned my addr 172.21.15.102:0/2366324429 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f318ea00640 1 --2- 172.21.15.102:0/2366324429 >> v2:172.21.15.193:3300/0 conn(0x7f319014fa50 0x7f31900c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.100+0000 7f318f400640 1 --2- 172.21.15.102:0/2366324429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3190073a10 0x7f31900c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.101+0000 7f3186000640 1 -- 172.21.15.102:0/2366324429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3190073a10 msgr2=0x7f31900c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.101+0000 7f3186000640 1 --2- 172.21.15.102:0/2366324429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3190073a10 0x7f31900c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.101+0000 7f3186000640 1 -- 172.21.15.102:0/2366324429 >> v2:172.21.15.193:3300/0 conn(0x7f319014fa50 msgr2=0x7f31900c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.101+0000 7f3186000640 1 --2- 172.21.15.102:0/2366324429 >> v2:172.21.15.193:3300/0 conn(0x7f319014fa50 0x7f31900c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.101+0000 7f3186000640 1 -- 172.21.15.102:0/2366324429 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31900c7960 con 0x7f31900c9eb0 2024-04-20T21:31:48.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.102+0000 7f3186000640 1 --2- 172.21.15.102:0/2366324429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31900c9eb0 0x7f3190158fc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f31900c7b30 tx=0x7f317802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628a2f02b2018b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.102+0000 7f318e000640 1 -- 172.21.15.102:0/2366324429 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178034070 con 0x7f31900c9eb0 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.102+0000 7f318e000640 1 -- 172.21.15.102:0/2366324429 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3178030840 con 0x7f31900c9eb0 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.102+0000 7f318e000640 1 -- 172.21.15.102:0/2366324429 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178030c10 con 0x7f31900c9eb0 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 -- 172.21.15.102:0/2366324429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31900c9eb0 msgr2=0x7f3190158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 --2- 172.21.15.102:0/2366324429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31900c9eb0 0x7f3190158fc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f31900c7b30 tx=0x7f317802b980 comp rx=0 tx=0).stop 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 -- 172.21.15.102:0/2366324429 shutdown_connections 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 --2- 172.21.15.102:0/2366324429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3190073a10 0x7f31900c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 --2- 172.21.15.102:0/2366324429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31900c9eb0 0x7f3190158fc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 --2- 172.21.15.102:0/2366324429 >> v2:172.21.15.193:3300/0 conn(0x7f319014fa50 0x7f31900c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 -- 172.21.15.102:0/2366324429 >> 172.21.15.102:0/2366324429 conn(0x7f31900c9390 msgr2=0x7f31900c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:48.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 -- 172.21.15.102:0/2366324429 shutdown_connections 2024-04-20T21:31:48.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.103+0000 7f3195600640 1 -- 172.21.15.102:0/2366324429 wait complete. 2024-04-20T21:31:48.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.105+0000 7f3195600640 1 Processor -- start 2024-04-20T21:31:48.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.105+0000 7f3195600640 1 -- start start 2024-04-20T21:31:48.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f3195600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3190073a10 0x7f3190158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:48.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f3195600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31900c9eb0 0x7f31901444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f3195600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319014fa50 0x7f319015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f3195600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31900c7c00 con 0x7f3190073a10 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f3195600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31900c3d90 con 0x7f319014fa50 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f3195600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f31900c4030 con 0x7f31900c9eb0 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3190073a10 0x7f3190158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3190073a10 0x7f3190158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58588/0 (socket says 172.21.15.102:58588) 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318ea00640 1 -- 172.21.15.102:0/4010963031 learned_addr learned my addr 172.21.15.102:0/4010963031 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f3186000640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31900c9eb0 0x7f31901444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318f400640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319014fa50 0x7f319015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318ea00640 1 -- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31900c9eb0 msgr2=0x7f31901444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:48.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318ea00640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31900c9eb0 0x7f31901444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318ea00640 1 -- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319014fa50 msgr2=0x7f319015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318ea00640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319014fa50 0x7f319015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f318ea00640 1 -- 172.21.15.102:0/4010963031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3190145940 con 0x7f3190073a10 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.106+0000 7f3186000640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31900c9eb0 0x7f31901444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.107+0000 7f318ea00640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3190073a10 0x7f3190158e50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f318000efc0 tx=0x7f318000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.107+0000 7f3187400640 1 -- 172.21.15.102:0/4010963031 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180018020 con 0x7f3190073a10 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.107+0000 7f3187400640 1 -- 172.21.15.102:0/4010963031 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3180014480 con 0x7f3190073a10 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.107+0000 7f3187400640 1 -- 172.21.15.102:0/4010963031 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180014850 con 0x7f3190073a10 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.107+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3190144f80 con 0x7f3190073a10 2024-04-20T21:31:48.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.107+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31901f29d0 con 0x7f3190073a10 2024-04-20T21:31:48.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.108+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3190074340 con 0x7f3190073a10 2024-04-20T21:31:48.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.109+0000 7f3187400640 1 -- 172.21.15.102:0/4010963031 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3180005d10 con 0x7f3190073a10 2024-04-20T21:31:48.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.110+0000 7f3187400640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f316c06c8f0 0x7f316c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:48.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.110+0000 7f3187400640 1 -- 172.21.15.102:0/4010963031 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f318008cd10 con 0x7f3190073a10 2024-04-20T21:31:48.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.110+0000 7f3186000640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f316c06c8f0 0x7f316c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:48.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.111+0000 7f3186000640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f316c06c8f0 0x7f316c06edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3178036000 tx=0x7f317803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:48.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.115+0000 7f3187400640 1 -- 172.21.15.102:0/4010963031 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3180092020 con 0x7f3190073a10 2024-04-20T21:31:48.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.357+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31900c7320 con 0x7f3190073a10 2024-04-20T21:31:48.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.359+0000 7f3187400640 1 -- 172.21.15.102:0/4010963031 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6033 (secure 0 0 0) 0x7f3180022070 con 0x7f3190073a10 2024-04-20T21:31:48.358 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:48.358 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:46.709675+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:clientreplay","state_seq":42,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:48.359 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 47 2024-04-20T21:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.361+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f316c06c8f0 msgr2=0x7f316c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.361+0000 7f3195600640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f316c06c8f0 0x7f316c06edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3178036000 tx=0x7f317803b040 comp rx=0 tx=0).stop 2024-04-20T21:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.361+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3190073a10 msgr2=0x7f3190158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.361+0000 7f3195600640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3190073a10 0x7f3190158e50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f318000efc0 tx=0x7f318000c490 comp rx=0 tx=0).stop 2024-04-20T21:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.362+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 shutdown_connections 2024-04-20T21:31:48.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.362+0000 7f3195600640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f316c06c8f0 0x7f316c06edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.362+0000 7f3195600640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f319014fa50 0x7f319015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.362+0000 7f3195600640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31900c9eb0 0x7f31901444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.362+0000 7f3195600640 1 --2- 172.21.15.102:0/4010963031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3190073a10 0x7f3190158e50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.362+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 >> 172.21.15.102:0/4010963031 conn(0x7f31900c9390 msgr2=0x7f319014c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.362+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 shutdown_connections 2024-04-20T21:31:48.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:48.362+0000 7f3195600640 1 -- 172.21.15.102:0/4010963031 wait complete. 2024-04-20T21:31:48.381 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:31:50.382 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:50.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.595+0000 7f30c5600640 1 Processor -- start 2024-04-20T21:31:50.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.595+0000 7f30c5600640 1 -- start start 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.596+0000 7f30c5600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c014fa50 0x7f30c00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.596+0000 7f30c5600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f30c00c9eb0 0x7f30c0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.596+0000 7f30c5600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c01536d0 con 0x7f30c0073a10 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.596+0000 7f30c5600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f30c0159500 con 0x7f30c00c9eb0 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.596+0000 7f30c5600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f30c00c76c0 con 0x7f30c014fa50 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30bea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c014fa50 0x7f30c00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30be000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f30c00c9eb0 0x7f30c0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30bf400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f30c0073a10 0x7f30c00c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43126/0 (socket says 172.21.15.102:43126) 2024-04-20T21:31:50.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30bf400640 1 -- 172.21.15.102:0/3740865535 learned_addr learned my addr 172.21.15.102:0/3740865535 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:50.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30bea00640 1 -- 172.21.15.102:0/3740865535 >> v2:172.21.15.102:3300/0 conn(0x7f30c00c9eb0 msgr2=0x7f30c0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:50.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30bea00640 1 --2- 172.21.15.102:0/3740865535 >> v2:172.21.15.102:3300/0 conn(0x7f30c00c9eb0 0x7f30c0158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30bea00640 1 -- 172.21.15.102:0/3740865535 >> v1:172.21.15.102:6789/0 conn(0x7f30c0073a10 legacy=0x7f30c00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:50.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30bea00640 1 -- 172.21.15.102:0/3740865535 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c00c7960 con 0x7f30c014fa50 2024-04-20T21:31:50.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.597+0000 7f30be000640 1 --2- 172.21.15.102:0/3740865535 >> v2:172.21.15.102:3300/0 conn(0x7f30c00c9eb0 0x7f30c0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:50.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.598+0000 7f30bea00640 1 --2- 172.21.15.102:0/3740865535 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c014fa50 0x7f30c00c4830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f30c00c7b30 tx=0x7f30b802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1790d47bec9939ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:50.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30bd600640 1 -- 172.21.15.102:0/3740865535 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8034070 con 0x7f30c014fa50 2024-04-20T21:31:50.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30bd600640 1 -- 172.21.15.102:0/3740865535 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b8030800 con 0x7f30c014fa50 2024-04-20T21:31:50.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30bd600640 1 -- 172.21.15.102:0/3740865535 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8030bd0 con 0x7f30c014fa50 2024-04-20T21:31:50.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30c5600640 1 -- 172.21.15.102:0/3740865535 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c014fa50 msgr2=0x7f30c00c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:50.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30c5600640 1 --2- 172.21.15.102:0/3740865535 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c014fa50 0x7f30c00c4830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f30c00c7b30 tx=0x7f30b802b940 comp rx=0 tx=0).stop 2024-04-20T21:31:50.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30c5600640 1 -- 172.21.15.102:0/3740865535 shutdown_connections 2024-04-20T21:31:50.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30c5600640 1 --2- 172.21.15.102:0/3740865535 >> v2:172.21.15.102:3300/0 conn(0x7f30c00c9eb0 0x7f30c0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30c5600640 1 --2- 172.21.15.102:0/3740865535 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c014fa50 0x7f30c00c4830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30c5600640 1 -- 172.21.15.102:0/3740865535 >> 172.21.15.102:0/3740865535 conn(0x7f30c00c9390 msgr2=0x7f30c00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:50.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30c5600640 1 -- 172.21.15.102:0/3740865535 shutdown_connections 2024-04-20T21:31:50.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.599+0000 7f30c5600640 1 -- 172.21.15.102:0/3740865535 wait complete. 2024-04-20T21:31:50.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.601+0000 7f30c5600640 1 Processor -- start 2024-04-20T21:31:50.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.601+0000 7f30c5600640 1 -- start start 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.602+0000 7f30c5600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c0073a10 0x7f30c00cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.602+0000 7f30c5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30c00c9eb0 0x7f30c0144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.602+0000 7f30c5600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30c014fa50 0x7f30c0144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.602+0000 7f30c5600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00c7c00 con 0x7f30c00c9eb0 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.602+0000 7f30c5600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00c3d90 con 0x7f30c014fa50 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.602+0000 7f30c5600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f30c00c4030 con 0x7f30c0073a10 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c0073a10 0x7f30c00cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30c014fa50 0x7f30c0144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c0073a10 0x7f30c00cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:59038/0 (socket says 172.21.15.102:59038) 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 -- 172.21.15.102:0/1706619176 learned_addr learned my addr 172.21.15.102:0/1706619176 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:50.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30be000640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30c00c9eb0 0x7f30c0144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:50.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 -- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30c014fa50 msgr2=0x7f30c0144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:50.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30c014fa50 0x7f30c0144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 -- 172.21.15.102:0/1706619176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30c00c9eb0 msgr2=0x7f30c0144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:50.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30c00c9eb0 0x7f30c0144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 -- 172.21.15.102:0/1706619176 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c0158600 con 0x7f30c0073a10 2024-04-20T21:31:50.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bf400640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30c014fa50 0x7f30c0144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:50.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.603+0000 7f30bea00640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c0073a10 0x7f30c00cb950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30b802bf50 tx=0x7f30b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:50.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.604+0000 7f30b6a00640 1 -- 172.21.15.102:0/1706619176 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8048070 con 0x7f30c0073a10 2024-04-20T21:31:50.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.604+0000 7f30b6a00640 1 -- 172.21.15.102:0/1706619176 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b8030600 con 0x7f30c0073a10 2024-04-20T21:31:50.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.604+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c0153b70 con 0x7f30c0073a10 2024-04-20T21:31:50.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.604+0000 7f30b6a00640 1 -- 172.21.15.102:0/1706619176 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8034040 con 0x7f30c0073a10 2024-04-20T21:31:50.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.604+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c0153e10 con 0x7f30c0073a10 2024-04-20T21:31:50.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.605+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c0074340 con 0x7f30c0073a10 2024-04-20T21:31:50.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.606+0000 7f30b6a00640 1 -- 172.21.15.102:0/1706619176 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30b804b070 con 0x7f30c0073a10 2024-04-20T21:31:50.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.607+0000 7f30b6a00640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f309c06c8f0 0x7f309c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:50.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.607+0000 7f30b6a00640 1 -- 172.21.15.102:0/1706619176 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f30b80add00 con 0x7f30c0073a10 2024-04-20T21:31:50.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.607+0000 7f30be000640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f309c06c8f0 0x7f309c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:50.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.608+0000 7f30be000640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f309c06c8f0 0x7f309c06edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009560 tx=0x7f30b0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:50.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.612+0000 7f30b6a00640 1 -- 172.21.15.102:0/1706619176 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f30b80307d0 con 0x7f30c0073a10 2024-04-20T21:31:50.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.855+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30c01568e0 con 0x7f30c0073a10 2024-04-20T21:31:50.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.856+0000 7f30b6a00640 1 -- 172.21.15.102:0/1706619176 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6033 (secure 0 0 0) 0x7f30b807b3f0 con 0x7f30c0073a10 2024-04-20T21:31:50.855 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:50.855 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:46.709675+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:clientreplay","state_seq":42,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:50.856 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 47 2024-04-20T21:31:50.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.858+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f309c06c8f0 msgr2=0x7f309c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:50.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.858+0000 7f30c5600640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f309c06c8f0 0x7f309c06edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009560 tx=0x7f30b0009190 comp rx=0 tx=0).stop 2024-04-20T21:31:50.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.858+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c0073a10 msgr2=0x7f30c00cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:50.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.858+0000 7f30c5600640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c0073a10 0x7f30c00cb950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30b802bf50 tx=0x7f30b80046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.859+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 shutdown_connections 2024-04-20T21:31:50.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.859+0000 7f30c5600640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f309c06c8f0 0x7f309c06edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.859+0000 7f30c5600640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30c014fa50 0x7f30c0144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.859+0000 7f30c5600640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30c00c9eb0 0x7f30c0144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.859+0000 7f30c5600640 1 --2- 172.21.15.102:0/1706619176 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30c0073a10 0x7f30c00cb950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:50.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.859+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 >> 172.21.15.102:0/1706619176 conn(0x7f30c00c9390 msgr2=0x7f30c00c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:50.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.859+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 shutdown_connections 2024-04-20T21:31:50.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:50.859+0000 7f30c5600640 1 -- 172.21.15.102:0/1706619176 wait complete. 2024-04-20T21:31:50.876 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:31:52.877 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:53.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.083+0000 7f3ee6c00640 1 Processor -- start 2024-04-20T21:31:53.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.083+0000 7f3ee6c00640 1 -- start start 2024-04-20T21:31:53.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.084+0000 7f3ee6c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee014fa50 0x7f3ee00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:53.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.084+0000 7f3ee6c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3ee0073a10 0x7f3ee0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:53.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.084+0000 7f3ee6c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ee01536d0 con 0x7f3ee00c9eb0 2024-04-20T21:31:53.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.084+0000 7f3ee6c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3ee0159500 con 0x7f3ee0073a10 2024-04-20T21:31:53.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.084+0000 7f3ee6c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee00c76c0 con 0x7f3ee014fa50 2024-04-20T21:31:53.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.084+0000 7f3ee4c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3ee0073a10 0x7f3ee0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:53.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.084+0000 7f3edf400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3ee00c9eb0 0x7f3ee00c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43130/0 (socket says 172.21.15.102:43130) 2024-04-20T21:31:53.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.084+0000 7f3edf400640 1 -- 172.21.15.102:0/3337914000 learned_addr learned my addr 172.21.15.102:0/3337914000 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:53.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.085+0000 7f3edfe00640 1 --2- 172.21.15.102:0/3337914000 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee014fa50 0x7f3ee00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:53.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.085+0000 7f3ee4c00640 1 -- 172.21.15.102:0/3337914000 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee014fa50 msgr2=0x7f3ee00c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:53.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.085+0000 7f3ee4c00640 1 --2- 172.21.15.102:0/3337914000 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee014fa50 0x7f3ee00c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.085+0000 7f3ee4c00640 1 -- 172.21.15.102:0/3337914000 >> v1:172.21.15.102:6789/0 conn(0x7f3ee00c9eb0 legacy=0x7f3ee00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:53.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.085+0000 7f3ee4c00640 1 -- 172.21.15.102:0/3337914000 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee00c7960 con 0x7f3ee0073a10 2024-04-20T21:31:53.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.085+0000 7f3edea00640 1 -- 172.21.15.102:0/3337914000 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2132829922 0 0) 0x7f3ec8001110 con 0x7f3ee00c9eb0 2024-04-20T21:31:53.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.086+0000 7f3ee4c00640 1 --2- 172.21.15.102:0/3337914000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee0073a10 0x7f3ee0158fc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00097a0 tx=0x7f3ed002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc099b4de57803f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:53.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.086+0000 7f3edea00640 1 -- 172.21.15.102:0/3337914000 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0034070 con 0x7f3ee0073a10 2024-04-20T21:31:53.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.086+0000 7f3edea00640 1 -- 172.21.15.102:0/3337914000 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed0030610 con 0x7f3ee0073a10 2024-04-20T21:31:53.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.086+0000 7f3edea00640 1 -- 172.21.15.102:0/3337914000 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed00309e0 con 0x7f3ee0073a10 2024-04-20T21:31:53.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.087+0000 7f3ee6c00640 1 -- 172.21.15.102:0/3337914000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee0073a10 msgr2=0x7f3ee0158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:53.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.087+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/3337914000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee0073a10 0x7f3ee0158fc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00097a0 tx=0x7f3ed002b960 comp rx=0 tx=0).stop 2024-04-20T21:31:53.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.087+0000 7f3ee6c00640 1 -- 172.21.15.102:0/3337914000 shutdown_connections 2024-04-20T21:31:53.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.087+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/3337914000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee0073a10 0x7f3ee0158fc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.087+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/3337914000 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee014fa50 0x7f3ee00c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.087+0000 7f3ee6c00640 1 -- 172.21.15.102:0/3337914000 >> 172.21.15.102:0/3337914000 conn(0x7f3ee00c9390 msgr2=0x7f3ee00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:53.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.087+0000 7f3ee6c00640 1 -- 172.21.15.102:0/3337914000 shutdown_connections 2024-04-20T21:31:53.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.087+0000 7f3ee6c00640 1 -- 172.21.15.102:0/3337914000 wait complete. 2024-04-20T21:31:53.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.089+0000 7f3ee6c00640 1 Processor -- start 2024-04-20T21:31:53.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.089+0000 7f3ee6c00640 1 -- start start 2024-04-20T21:31:53.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee6c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ee0073a10 0x7f3ee00cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:53.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee6c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee00c9eb0 0x7f3ee01f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:53.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee6c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee014fa50 0x7f3ee01e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:53.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee6c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee00c7c00 con 0x7f3ee014fa50 2024-04-20T21:31:53.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee6c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee00c3d90 con 0x7f3ee0073a10 2024-04-20T21:31:53.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee6c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee00c4030 con 0x7f3ee00c9eb0 2024-04-20T21:31:53.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee014fa50 0x7f3ee01e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:53.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee014fa50 0x7f3ee01e5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58626/0 (socket says 172.21.15.102:58626) 2024-04-20T21:31:53.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.090+0000 7f3ee4c00640 1 -- 172.21.15.102:0/2132174269 learned_addr learned my addr 172.21.15.102:0/2132174269 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:53.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ee4c00640 1 -- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee00c9eb0 msgr2=0x7f3ee01f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:53.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ee4c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee00c9eb0 0x7f3ee01f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ee4c00640 1 -- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ee0073a10 msgr2=0x7f3ee00cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:53.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ee4c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ee0073a10 0x7f3ee00cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ee4c00640 1 -- 172.21.15.102:0/2132174269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee0149b40 con 0x7f3ee014fa50 2024-04-20T21:31:53.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ee4c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee014fa50 0x7f3ee01e5e30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00097a0 tx=0x7f3ed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:53.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ed7e00640 1 -- 172.21.15.102:0/2132174269 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0048070 con 0x7f3ee014fa50 2024-04-20T21:31:53.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ed7e00640 1 -- 172.21.15.102:0/2132174269 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed0030410 con 0x7f3ee014fa50 2024-04-20T21:31:53.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee0149e20 con 0x7f3ee014fa50 2024-04-20T21:31:53.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ed7e00640 1 -- 172.21.15.102:0/2132174269 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0034040 con 0x7f3ee014fa50 2024-04-20T21:31:53.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.091+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee00cadd0 con 0x7f3ee014fa50 2024-04-20T21:31:53.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.092+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee0074340 con 0x7f3ee014fa50 2024-04-20T21:31:53.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.093+0000 7f3ed7e00640 1 -- 172.21.15.102:0/2132174269 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ed004b070 con 0x7f3ee014fa50 2024-04-20T21:31:53.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.094+0000 7f3ed7e00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:53.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.094+0000 7f3ed7e00640 1 -- 172.21.15.102:0/2132174269 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f3ed00ada10 con 0x7f3ee014fa50 2024-04-20T21:31:53.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.095+0000 7f3edfe00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:53.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.096+0000 7f3edfe00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009fa0 tx=0x7f3ed800b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:53.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.099+0000 7f3ed7e00640 1 -- 172.21.15.102:0/2132174269 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3ed007b7d0 con 0x7f3ee014fa50 2024-04-20T21:31:53.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.339+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ee006eaa0 con 0x7f3ee014fa50 2024-04-20T21:31:53.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.340+0000 7f3ed7e00640 1 -- 172.21.15.102:0/2132174269 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6033 (secure 0 0 0) 0x7f3ed007b0d0 con 0x7f3ee014fa50 2024-04-20T21:31:53.340 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:53.340 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:46.709675+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:clientreplay","state_seq":42,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:31:53.340 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 47 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3ebc06c8f0 msgr2=0x7f3ebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009fa0 tx=0x7f3ed800b040 comp rx=0 tx=0).stop 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee014fa50 msgr2=0x7f3ee01e5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee014fa50 0x7f3ee01e5e30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00097a0 tx=0x7f3ed00046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 shutdown_connections 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee014fa50 0x7f3ee01e5e30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ee00c9eb0 0x7f3ee01f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 --2- 172.21.15.102:0/2132174269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ee0073a10 0x7f3ee00cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 >> 172.21.15.102:0/2132174269 conn(0x7f3ee00c9390 msgr2=0x7f3ee006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.343+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 shutdown_connections 2024-04-20T21:31:53.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:53.344+0000 7f3ee6c00640 1 -- 172.21.15.102:0/2132174269 wait complete. 2024-04-20T21:31:53.361 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 47, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 47, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5342": { "addr": "172.21.15.102:6833/1043030921", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 1043030921, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 1043030921, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5342, "incarnation": 44, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:clientreplay", "state_seq": 42 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:31:46.709675+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5342 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:31:55.363 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:55.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.563+0000 7fa14f200640 1 Processor -- start 2024-04-20T21:31:55.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.563+0000 7fa14f200640 1 -- start start 2024-04-20T21:31:55.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14f200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa150073c70 0x7fa1501647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14f200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa150153560 con 0x7fa150153180 2024-04-20T21:31:55.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14f200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1500c7a70 con 0x7fa1500c75a0 2024-04-20T21:31:55.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14f200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa1500c7d30 con 0x7fa150073c70 2024-04-20T21:31:55.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14de00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa150153180 0x7fa150158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43134/0 (socket says 172.21.15.102:43134) 2024-04-20T21:31:55.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14de00640 1 -- 172.21.15.102:0/4000277633 learned_addr learned my addr 172.21.15.102:0/4000277633 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:31:55.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14e800640 1 --2- 172.21.15.102:0/4000277633 >> v2:172.21.15.193:3300/0 conn(0x7fa150073c70 0x7fa1501647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14ca00640 1 -- 172.21.15.102:0/4000277633 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536679490 0 0) 0x7fa150153560 con 0x7fa150153180 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14ca00640 1 -- 172.21.15.102:0/4000277633 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa134003520 con 0x7fa150153180 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.565+0000 7fa14ca00640 1 -- 172.21.15.102:0/4000277633 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187537951 0 0) 0x7fa1500c7a70 con 0x7fa1500c75a0 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.566+0000 7fa14ca00640 1 -- 172.21.15.102:0/4000277633 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa150153560 con 0x7fa1500c75a0 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.566+0000 7fa14e800640 1 -- 172.21.15.102:0/4000277633 >> v1:172.21.15.193:6789/0 conn(0x7fa1500c75a0 legacy=0x7fa1500c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.566+0000 7fa14e800640 1 -- 172.21.15.102:0/4000277633 >> v1:172.21.15.102:6789/0 conn(0x7fa150153180 legacy=0x7fa150158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.566+0000 7fa14e800640 1 -- 172.21.15.102:0/4000277633 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15015c4d0 con 0x7fa150073c70 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.567+0000 7fa14e800640 1 --2- 172.21.15.102:0/4000277633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150073c70 0x7fa1501647d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa148009d90 tx=0x7fa14802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542b5ffe06aa165b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:55.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.567+0000 7fa14ca00640 1 -- 172.21.15.102:0/4000277633 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148039070 con 0x7fa150073c70 2024-04-20T21:31:55.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.567+0000 7fa14ca00640 1 -- 172.21.15.102:0/4000277633 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14802ccf0 con 0x7fa150073c70 2024-04-20T21:31:55.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.567+0000 7fa14ca00640 1 -- 172.21.15.102:0/4000277633 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148034660 con 0x7fa150073c70 2024-04-20T21:31:55.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.568+0000 7fa14f200640 1 -- 172.21.15.102:0/4000277633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150073c70 msgr2=0x7fa1501647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.568+0000 7fa14f200640 1 --2- 172.21.15.102:0/4000277633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150073c70 0x7fa1501647d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa148009d90 tx=0x7fa14802bf40 comp rx=0 tx=0).stop 2024-04-20T21:31:55.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.568+0000 7fa14f200640 1 -- 172.21.15.102:0/4000277633 shutdown_connections 2024-04-20T21:31:55.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.568+0000 7fa14f200640 1 --2- 172.21.15.102:0/4000277633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150073c70 0x7fa1501647d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.568+0000 7fa14f200640 1 -- 172.21.15.102:0/4000277633 >> 172.21.15.102:0/4000277633 conn(0x7fa1500cc820 msgr2=0x7fa1500ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:55.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.568+0000 7fa14f200640 1 -- 172.21.15.102:0/4000277633 shutdown_connections 2024-04-20T21:31:55.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.568+0000 7fa14f200640 1 -- 172.21.15.102:0/4000277633 wait complete. 2024-04-20T21:31:55.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.569+0000 7fa14f200640 1 Processor -- start 2024-04-20T21:31:55.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.570+0000 7fa14f200640 1 -- start start 2024-04-20T21:31:55.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.570+0000 7fa14f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150073c70 0x7fa1500cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.570+0000 7fa14f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500c75a0 0x7fa1501e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.570+0000 7fa14f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150153180 0x7fa1501f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.570+0000 7fa14f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa150159550 con 0x7fa150073c70 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.570+0000 7fa14f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa15015ca90 con 0x7fa150153180 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.570+0000 7fa14f200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa15015d290 con 0x7fa1500c75a0 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150073c70 0x7fa1500cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150073c70 0x7fa1500cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58630/0 (socket says 172.21.15.102:58630) 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14de00640 1 -- 172.21.15.102:0/946806941 learned_addr learned my addr 172.21.15.102:0/946806941 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14d400640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500c75a0 0x7fa1501e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14e800640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150153180 0x7fa1501f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14de00640 1 -- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500c75a0 msgr2=0x7fa1501e56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14de00640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500c75a0 0x7fa1501e56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14de00640 1 -- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150153180 msgr2=0x7fa1501f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14de00640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150153180 0x7fa1501f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.571+0000 7fa14de00640 1 -- 172.21.15.102:0/946806941 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15015aa80 con 0x7fa150073c70 2024-04-20T21:31:55.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.572+0000 7fa14de00640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150073c70 0x7fa1500cae20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c530 tx=0x7fa14000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:55.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.572+0000 7fa146000640 1 -- 172.21.15.102:0/946806941 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140017070 con 0x7fa150073c70 2024-04-20T21:31:55.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.572+0000 7fa146000640 1 -- 172.21.15.102:0/946806941 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa140005ce0 con 0x7fa150073c70 2024-04-20T21:31:55.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.572+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1500cb580 con 0x7fa150073c70 2024-04-20T21:31:55.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.572+0000 7fa146000640 1 -- 172.21.15.102:0/946806941 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140007270 con 0x7fa150073c70 2024-04-20T21:31:55.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.572+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1501eef10 con 0x7fa150073c70 2024-04-20T21:31:55.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.573+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa150070820 con 0x7fa150073c70 2024-04-20T21:31:55.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.574+0000 7fa146000640 1 -- 172.21.15.102:0/946806941 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa14001a070 con 0x7fa150073c70 2024-04-20T21:31:55.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.575+0000 7fa146000640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa13006c920 0x7fa13006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.576+0000 7fa146000640 1 -- 172.21.15.102:0/946806941 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7fa14008b410 con 0x7fa150073c70 2024-04-20T21:31:55.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.578+0000 7fa14d400640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa13006c920 0x7fa13006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.580+0000 7fa146000640 1 -- 172.21.15.102:0/946806941 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa150070820 con 0x7fa150073c70 2024-04-20T21:31:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.582+0000 7fa14d400640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa13006c920 0x7fa13006ede0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa13c007fd0 tx=0x7fa13c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:55.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.823+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa15004fd60 con 0x7fa150073c70 2024-04-20T21:31:55.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.824+0000 7fa146000640 1 -- 172.21.15.102:0/946806941 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+6034 (secure 0 0 0) 0x7fa140058b80 con 0x7fa150073c70 2024-04-20T21:31:55.823 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:31:55.823 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:31:55.823 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 48 2024-04-20T21:31:55.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.826+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa13006c920 msgr2=0x7fa13006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.826+0000 7fa14f200640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa13006c920 0x7fa13006ede0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa13c007fd0 tx=0x7fa13c009040 comp rx=0 tx=0).stop 2024-04-20T21:31:55.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.826+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150073c70 msgr2=0x7fa1500cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.826+0000 7fa14f200640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150073c70 0x7fa1500cae20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c530 tx=0x7fa14000c9f0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.827+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 shutdown_connections 2024-04-20T21:31:55.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.827+0000 7fa14f200640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa13006c920 0x7fa13006ede0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.827+0000 7fa14f200640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa150153180 0x7fa1501f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.827+0000 7fa14f200640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500c75a0 0x7fa1501e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.827+0000 7fa14f200640 1 --2- 172.21.15.102:0/946806941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa150073c70 0x7fa1500cae20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.827+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 >> 172.21.15.102:0/946806941 conn(0x7fa1500cc820 msgr2=0x7fa1500ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:55.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.827+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 shutdown_connections 2024-04-20T21:31:55.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:31:55.827+0000 7fa14f200640 1 -- 172.21.15.102:0/946806941 wait complete. 2024-04-20T21:31:55.846 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5659 2024-04-20T21:31:55.846 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.c 2024-04-20T21:31:58.103 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:58.111 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:02.846 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:32:02.847 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:32:02.847 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:32:02.852 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:32:02.852 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:02.994 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:32:03.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.026+0000 7f6c99a00640 1 Processor -- start 2024-04-20T21:32:03.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.026+0000 7f6c99a00640 1 -- start start 2024-04-20T21:32:03.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.027+0000 7f6c99a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c9414fa50 0x7f6c940c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.027+0000 7f6c99a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.027+0000 7f6c99a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c941536d0 con 0x7f6c94073a10 2024-04-20T21:32:03.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.027+0000 7f6c99a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6c94159500 con 0x7f6c9414fa50 2024-04-20T21:32:03.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.027+0000 7f6c99a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6c940c76c0 con 0x7f6c940c9eb0 2024-04-20T21:32:03.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.027+0000 7f6c92a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c9414fa50 0x7f6c940c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.027+0000 7f6c8be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.028+0000 7f6c93400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6c94073a10 0x7f6c940c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55436/0 (socket says 172.21.15.102:55436) 2024-04-20T21:32:03.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.028+0000 7f6c93400640 1 -- 172.21.15.102:0/2372582656 learned_addr learned my addr 172.21.15.102:0/2372582656 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:32:03.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.028+0000 7f6c8be00640 1 -- 172.21.15.102:0/2372582656 >> v2:172.21.15.193:3300/0 conn(0x7f6c9414fa50 msgr2=0x7f6c940c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:03.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.028+0000 7f6c8be00640 1 --2- 172.21.15.102:0/2372582656 >> v2:172.21.15.193:3300/0 conn(0x7f6c9414fa50 0x7f6c940c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.028+0000 7f6c8be00640 1 -- 172.21.15.102:0/2372582656 >> v1:172.21.15.193:6789/0 conn(0x7f6c94073a10 legacy=0x7f6c940c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.028+0000 7f6c8be00640 1 -- 172.21.15.102:0/2372582656 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c940c7960 con 0x7f6c940c9eb0 2024-04-20T21:32:03.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.028+0000 7f6c92000640 1 -- 172.21.15.102:0/2372582656 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138778370 0 0) 0x7f6c8c001490 con 0x7f6c94073a10 2024-04-20T21:32:03.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.029+0000 7f6c8be00640 1 --2- 172.21.15.102:0/2372582656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94158fc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c7b30 tx=0x7f6c7c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843a75a2231a3a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:03.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.029+0000 7f6c92000640 1 -- 172.21.15.102:0/2372582656 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c035020 con 0x7f6c940c9eb0 2024-04-20T21:32:03.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.029+0000 7f6c92000640 1 -- 172.21.15.102:0/2372582656 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c7c031890 con 0x7f6c940c9eb0 2024-04-20T21:32:03.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.029+0000 7f6c92000640 1 -- 172.21.15.102:0/2372582656 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c031c60 con 0x7f6c940c9eb0 2024-04-20T21:32:03.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.030+0000 7f6c99a00640 1 -- 172.21.15.102:0/2372582656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 msgr2=0x7f6c94158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.030+0000 7f6c99a00640 1 --2- 172.21.15.102:0/2372582656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94158fc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c7b30 tx=0x7f6c7c02b930 comp rx=0 tx=0).stop 2024-04-20T21:32:03.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.030+0000 7f6c99a00640 1 -- 172.21.15.102:0/2372582656 shutdown_connections 2024-04-20T21:32:03.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.030+0000 7f6c99a00640 1 --2- 172.21.15.102:0/2372582656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94158fc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.030+0000 7f6c99a00640 1 --2- 172.21.15.102:0/2372582656 >> v2:172.21.15.193:3300/0 conn(0x7f6c9414fa50 0x7f6c940c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.030+0000 7f6c99a00640 1 -- 172.21.15.102:0/2372582656 >> 172.21.15.102:0/2372582656 conn(0x7f6c940c9390 msgr2=0x7f6c940c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:03.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.030+0000 7f6c99a00640 1 -- 172.21.15.102:0/2372582656 shutdown_connections 2024-04-20T21:32:03.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.030+0000 7f6c99a00640 1 -- 172.21.15.102:0/2372582656 wait complete. 2024-04-20T21:32:03.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.032+0000 7f6c99a00640 1 Processor -- start 2024-04-20T21:32:03.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.032+0000 7f6c99a00640 1 -- start start 2024-04-20T21:32:03.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c99a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c94073a10 0x7f6c94156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c99a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c99a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c9414fa50 0x7f6c9414dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c99a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6c940c7c00 con 0x7f6c94073a10 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c99a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c940c3d90 con 0x7f6c9414fa50 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c99a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6c940c4030 con 0x7f6c940c9eb0 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c8be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c92a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c94073a10 0x7f6c94156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c8be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94071dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:34382/0 (socket says 172.21.15.102:34382) 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c8be00640 1 -- 172.21.15.102:0/4052619152 learned_addr learned my addr 172.21.15.102:0/4052619152 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.033+0000 7f6c93400640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c9414fa50 0x7f6c9414dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8be00640 1 -- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c9414fa50 msgr2=0x7f6c9414dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8be00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c9414fa50 0x7f6c9414dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8be00640 1 -- 172.21.15.102:0/4052619152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c94073a10 msgr2=0x7f6c94156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8be00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c94073a10 0x7f6c94156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8be00640 1 -- 172.21.15.102:0/4052619152 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c940cb8a0 con 0x7f6c940c9eb0 2024-04-20T21:32:03.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8be00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94071dc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009890 tx=0x7f6c7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:03.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8aa00640 1 -- 172.21.15.102:0/4052619152 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c035040 con 0x7f6c940c9eb0 2024-04-20T21:32:03.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8aa00640 1 -- 172.21.15.102:0/4052619152 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c7c030d20 con 0x7f6c940c9eb0 2024-04-20T21:32:03.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c8aa00640 1 -- 172.21.15.102:0/4052619152 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c0456a0 con 0x7f6c940c9eb0 2024-04-20T21:32:03.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c94072300 con 0x7f6c940c9eb0 2024-04-20T21:32:03.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.034+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c94153cc0 con 0x7f6c940c9eb0 2024-04-20T21:32:03.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.035+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c94074340 con 0x7f6c940c9eb0 2024-04-20T21:32:03.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.036+0000 7f6c8aa00640 1 -- 172.21.15.102:0/4052619152 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c7c038020 con 0x7f6c940c9eb0 2024-04-20T21:32:03.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.037+0000 7f6c8aa00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.037+0000 7f6c8aa00640 1 -- 172.21.15.102:0/4052619152 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f6c7c0ae3d0 con 0x7f6c940c9eb0 2024-04-20T21:32:03.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.038+0000 7f6c92a00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.038+0000 7f6c92a00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009b40 tx=0x7f6c84009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:03.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.042+0000 7f6c8aa00640 1 -- 172.21.15.102:0/4052619152 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c7c02d040 con 0x7f6c940c9eb0 2024-04-20T21:32:03.052 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-04-20T21:32:03.053+0000 7f7debe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:32:03.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.285+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c941568e0 con 0x7f6c940c9eb0 2024-04-20T21:32:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:32:03.285 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:32:03.286 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 48 2024-04-20T21:32:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.286+0000 7f6c8aa00640 1 -- 172.21.15.102:0/4052619152 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+6034 (secure 0 0 0) 0x7f6c7c07bb80 con 0x7f6c940c9eb0 2024-04-20T21:32:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 msgr2=0x7f6c7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009b40 tx=0x7f6c84009190 comp rx=0 tx=0).stop 2024-04-20T21:32:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 msgr2=0x7f6c94071dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94071dc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009890 tx=0x7f6c7c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 shutdown_connections 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6c7006c8f0 0x7f6c7006edb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c9414fa50 0x7f6c9414dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c940c9eb0 0x7f6c94071dc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 --2- 172.21.15.102:0/4052619152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c94073a10 0x7f6c94156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 >> 172.21.15.102:0/4052619152 conn(0x7f6c940c9390 msgr2=0x7f6c9406d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 shutdown_connections 2024-04-20T21:32:03.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:03.289+0000 7f6c99a00640 1 -- 172.21.15.102:0/4052619152 wait complete. 2024-04-20T21:32:03.438 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-20T21:32:05.439 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:05.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.653+0000 7fd2bb000640 1 Processor -- start 2024-04-20T21:32:05.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.653+0000 7fd2bb000640 1 -- start start 2024-04-20T21:32:05.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.655+0000 7fd2bb000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b4166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:05.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.655+0000 7fd2bb000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b415f3c0 con 0x7fd2b4155110 2024-04-20T21:32:05.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.655+0000 7fd2bb000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b406e6d0 con 0x7fd2b40c4040 2024-04-20T21:32:05.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.655+0000 7fd2bb000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b40c7c30 con 0x7fd2b415ec90 2024-04-20T21:32:05.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.655+0000 7fd2b9c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd2b4155110 0x7fd2b406ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50068/0 (socket says 172.21.15.102:50068) 2024-04-20T21:32:05.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.655+0000 7fd2b9c00640 1 -- 172.21.15.102:0/3581869478 learned_addr learned my addr 172.21.15.102:0/3581869478 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:32:05.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.655+0000 7fd2ba600640 1 --2- 172.21.15.102:0/3581869478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b4166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:05.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.656+0000 7fd2b3e00640 1 -- 172.21.15.102:0/3581869478 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003285191 0 0) 0x7fd2b415f3c0 con 0x7fd2b4155110 2024-04-20T21:32:05.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.656+0000 7fd2b3e00640 1 -- 172.21.15.102:0/3581869478 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd29c003520 con 0x7fd2b4155110 2024-04-20T21:32:05.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.656+0000 7fd2b3e00640 1 -- 172.21.15.102:0/3581869478 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1364877701 0 0) 0x7fd2b406e6d0 con 0x7fd2b40c4040 2024-04-20T21:32:05.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.656+0000 7fd2b3e00640 1 -- 172.21.15.102:0/3581869478 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2b415f3c0 con 0x7fd2b40c4040 2024-04-20T21:32:05.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.656+0000 7fd2ba600640 1 -- 172.21.15.102:0/3581869478 >> v1:172.21.15.193:6789/0 conn(0x7fd2b40c4040 legacy=0x7fd2b415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:05.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.656+0000 7fd2ba600640 1 -- 172.21.15.102:0/3581869478 >> v1:172.21.15.102:6789/0 conn(0x7fd2b4155110 legacy=0x7fd2b406ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:05.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.656+0000 7fd2ba600640 1 -- 172.21.15.102:0/3581869478 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b40c7e80 con 0x7fd2b415ec90 2024-04-20T21:32:05.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.657+0000 7fd2ba600640 1 --2- 172.21.15.102:0/3581869478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b4166920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009dc0 tx=0x7fd2ac02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a9e59a199a4e052 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:05.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.657+0000 7fd2b3e00640 1 -- 172.21.15.102:0/3581869478 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac039070 con 0x7fd2b415ec90 2024-04-20T21:32:05.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.657+0000 7fd2b3e00640 1 -- 172.21.15.102:0/3581869478 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac02cd50 con 0x7fd2b415ec90 2024-04-20T21:32:05.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.657+0000 7fd2b3e00640 1 -- 172.21.15.102:0/3581869478 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac034660 con 0x7fd2b415ec90 2024-04-20T21:32:05.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.658+0000 7fd2bb000640 1 -- 172.21.15.102:0/3581869478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 msgr2=0x7fd2b4166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:05.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.658+0000 7fd2bb000640 1 --2- 172.21.15.102:0/3581869478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b4166920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009dc0 tx=0x7fd2ac02bfa0 comp rx=0 tx=0).stop 2024-04-20T21:32:05.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.658+0000 7fd2bb000640 1 -- 172.21.15.102:0/3581869478 shutdown_connections 2024-04-20T21:32:05.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.658+0000 7fd2bb000640 1 --2- 172.21.15.102:0/3581869478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b4166920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:05.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.658+0000 7fd2bb000640 1 -- 172.21.15.102:0/3581869478 >> 172.21.15.102:0/3581869478 conn(0x7fd2b40cc820 msgr2=0x7fd2b40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:05.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.658+0000 7fd2bb000640 1 -- 172.21.15.102:0/3581869478 shutdown_connections 2024-04-20T21:32:05.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.658+0000 7fd2bb000640 1 -- 172.21.15.102:0/3581869478 wait complete. 2024-04-20T21:32:05.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.660+0000 7fd2bb000640 1 Processor -- start 2024-04-20T21:32:05.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.660+0000 7fd2bb000640 1 -- start start 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2bb000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2b40c4040 0x7fd2b415b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2bb000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2b4155110 0x7fd2b40ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2bb000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b40cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2bb000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b40c8160 con 0x7fd2b40c4040 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2bb000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b406ea00 con 0x7fd2b4155110 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2bb000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b406f5a0 con 0x7fd2b415ec90 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2b9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2b40c4040 0x7fd2b415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2b9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2b40c4040 0x7fd2b415b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35002/0 (socket says 172.21.15.102:35002) 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2b9c00640 1 -- 172.21.15.102:0/1445443189 learned_addr learned my addr 172.21.15.102:0/1445443189 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2b9200640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2b4155110 0x7fd2b40ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.661+0000 7fd2ba600640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b40cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:05.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b9c00640 1 -- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 msgr2=0x7fd2b40cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:05.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b9c00640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b40cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:05.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b9c00640 1 -- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2b4155110 msgr2=0x7fd2b40ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:05.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b9c00640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2b4155110 0x7fd2b40ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:05.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b9c00640 1 -- 172.21.15.102:0/1445443189 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b41f0cb0 con 0x7fd2b40c4040 2024-04-20T21:32:05.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b9c00640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2b40c4040 0x7fd2b415b340 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000c490 tx=0x7fd2a000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:05.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b1600640 1 -- 172.21.15.102:0/1445443189 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0017070 con 0x7fd2b40c4040 2024-04-20T21:32:05.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b1600640 1 -- 172.21.15.102:0/1445443189 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0005da0 con 0x7fd2b40c4040 2024-04-20T21:32:05.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2b1600640 1 -- 172.21.15.102:0/1445443189 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00072f0 con 0x7fd2b40c4040 2024-04-20T21:32:05.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b41f0f90 con 0x7fd2b40c4040 2024-04-20T21:32:05.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.662+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b41f1550 con 0x7fd2b40c4040 2024-04-20T21:32:05.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.664+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b4069340 con 0x7fd2b40c4040 2024-04-20T21:32:05.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.665+0000 7fd2b1600640 1 -- 172.21.15.102:0/1445443189 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2a001a070 con 0x7fd2b40c4040 2024-04-20T21:32:05.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.666+0000 7fd2b1600640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd29406c8f0 0x7fd29406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:05.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.666+0000 7fd2b1600640 1 -- 172.21.15.102:0/1445443189 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7fd2a008b5c0 con 0x7fd2b40c4040 2024-04-20T21:32:05.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.666+0000 7fd2b9200640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd29406c8f0 0x7fd29406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:05.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.667+0000 7fd2b9200640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd29406c8f0 0x7fd29406edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8007fd0 tx=0x7fd2a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:05.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.670+0000 7fd2b1600640 1 -- 172.21.15.102:0/1445443189 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd2b4069340 con 0x7fd2b40c4040 2024-04-20T21:32:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.922+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2b41554f0 con 0x7fd2b40c4040 2024-04-20T21:32:05.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.924+0000 7fd2b1600640 1 -- 172.21.15.102:0/1445443189 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+6862 (secure 0 0 0) 0x7fd2b41554f0 con 0x7fd2b40c4040 2024-04-20T21:32:05.923 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:32:05.923 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:32:05.924 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 49 2024-04-20T21:32:05.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd29406c8f0 msgr2=0x7fd29406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:05.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd29406c8f0 0x7fd29406edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8007fd0 tx=0x7fd2a8009040 comp rx=0 tx=0).stop 2024-04-20T21:32:05.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2b40c4040 msgr2=0x7fd2b415b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:05.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2b40c4040 0x7fd2b415b340 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000c490 tx=0x7fd2a000c950 comp rx=0 tx=0).stop 2024-04-20T21:32:05.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 shutdown_connections 2024-04-20T21:32:05.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2b415ec90 0x7fd2b40cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:05.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd29406c8f0 0x7fd29406edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:05.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2b4155110 0x7fd2b40ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:05.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 --2- 172.21.15.102:0/1445443189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2b40c4040 0x7fd2b415b340 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:05.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.927+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 >> 172.21.15.102:0/1445443189 conn(0x7fd2b40cc820 msgr2=0x7fd2b41532b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.928+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 shutdown_connections 2024-04-20T21:32:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:05.928+0000 7fd2bb000640 1 -- 172.21.15.102:0/1445443189 wait complete. 2024-04-20T21:32:05.947 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:32:05.947 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 44 secs before thrashing 2024-04-20T21:32:28.224 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:28.233 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:49.948 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:50.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.160+0000 7f220d200640 1 Processor -- start 2024-04-20T21:32:50.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.161+0000 7f220d200640 1 -- start start 2024-04-20T21:32:50.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f220d200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f220806e030 0x7f2208158a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f220d200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22080c5a60 0x7f220806d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f220d200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2208071da0 con 0x7f220806d530 2024-04-20T21:32:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f220d200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f22080c6f00 con 0x7f22080c5a60 2024-04-20T21:32:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f220d200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f22080c6010 con 0x7f220806e030 2024-04-20T21:32:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f2206800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22080c5a60 0x7f220806d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f2206800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22080c5a60 0x7f220806d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59702/0 (socket says 172.21.15.102:59702) 2024-04-20T21:32:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f2206800640 1 -- 172.21.15.102:0/1212207933 learned_addr learned my addr 172.21.15.102:0/1212207933 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:32:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.162+0000 7f21fd400640 1 --2- 172.21.15.102:0/1212207933 >> v2:172.21.15.193:3300/0 conn(0x7f220806e030 0x7f2208158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.163+0000 7f21fd400640 1 -- 172.21.15.102:0/1212207933 >> v2:172.21.15.102:3300/0 conn(0x7f22080c5a60 msgr2=0x7f220806d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.163+0000 7f2205400640 1 -- 172.21.15.102:0/1212207933 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402082381 0 0) 0x7f2208071da0 con 0x7f220806d530 2024-04-20T21:32:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.163+0000 7f21fd400640 1 --2- 172.21.15.102:0/1212207933 >> v2:172.21.15.102:3300/0 conn(0x7f22080c5a60 0x7f220806d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.163+0000 7f21fd400640 1 -- 172.21.15.102:0/1212207933 >> v1:172.21.15.193:6789/0 conn(0x7f220806d530 legacy=0x7f2208071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.163+0000 7f21fd400640 1 -- 172.21.15.102:0/1212207933 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22080c62f0 con 0x7f220806e030 2024-04-20T21:32:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.163+0000 7f2206800640 1 --2- 172.21.15.102:0/1212207933 >> v2:172.21.15.102:3300/0 conn(0x7f22080c5a60 0x7f220806d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.164+0000 7f21fd400640 1 --2- 172.21.15.102:0/1212207933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806e030 0x7f2208158a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22080cb010 tx=0x7f21f002db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42fb1796d9a8204e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.164+0000 7f2205400640 1 -- 172.21.15.102:0/1212207933 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f003b070 con 0x7f220806e030 2024-04-20T21:32:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.164+0000 7f2205400640 1 -- 172.21.15.102:0/1212207933 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f002e920 con 0x7f220806e030 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.164+0000 7f2205400640 1 -- 172.21.15.102:0/1212207933 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f002ecf0 con 0x7f220806e030 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.165+0000 7f220d200640 1 -- 172.21.15.102:0/1212207933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806e030 msgr2=0x7f2208158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.165+0000 7f220d200640 1 --2- 172.21.15.102:0/1212207933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806e030 0x7f2208158a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22080cb010 tx=0x7f21f002db70 comp rx=0 tx=0).stop 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.165+0000 7f220d200640 1 -- 172.21.15.102:0/1212207933 shutdown_connections 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.165+0000 7f220d200640 1 --2- 172.21.15.102:0/1212207933 >> v2:172.21.15.102:3300/0 conn(0x7f22080c5a60 0x7f220806d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.165+0000 7f220d200640 1 --2- 172.21.15.102:0/1212207933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806e030 0x7f2208158a60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.165+0000 7f220d200640 1 -- 172.21.15.102:0/1212207933 >> 172.21.15.102:0/1212207933 conn(0x7f22080cd160 msgr2=0x7f22080cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.165+0000 7f220d200640 1 -- 172.21.15.102:0/1212207933 shutdown_connections 2024-04-20T21:32:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.166+0000 7f220d200640 1 -- 172.21.15.102:0/1212207933 wait complete. 2024-04-20T21:32:50.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.167+0000 7f220d200640 1 Processor -- start 2024-04-20T21:32:50.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.168+0000 7f220d200640 1 -- start start 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.168+0000 7f220d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806d530 0x7f220814c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.168+0000 7f220d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220806e030 0x7f2208153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f220d200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22080c5a60 0x7f2208154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f220d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22080c6b90 con 0x7f220806e030 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f220d200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22080c68b0 con 0x7f220806d530 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f220d200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f22080c7390 con 0x7f22080c5a60 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f21fd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220806e030 0x7f2208153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f21fd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220806e030 0x7f2208153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59706/0 (socket says 172.21.15.102:59706) 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f21fd400640 1 -- 172.21.15.102:0/3253489888 learned_addr learned my addr 172.21.15.102:0/3253489888 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:32:50.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f2205e00640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806d530 0x7f220814c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f2206800640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22080c5a60 0x7f2208154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f2205e00640 1 -- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22080c5a60 msgr2=0x7f2208154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f2205e00640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22080c5a60 0x7f2208154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f2205e00640 1 -- 172.21.15.102:0/3253489888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220806e030 msgr2=0x7f2208153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f2205e00640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220806e030 0x7f2208153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f2205e00640 1 -- 172.21.15.102:0/3253489888 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081f1540 con 0x7f220806d530 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.169+0000 7f21fd400640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220806e030 0x7f2208153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.170+0000 7f2205e00640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806d530 0x7f220814c220 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c450 tx=0x7f21f800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.170+0000 7f21fea00640 1 -- 172.21.15.102:0/3253489888 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8017070 con 0x7f220806d530 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.170+0000 7f21fea00640 1 -- 172.21.15.102:0/3253489888 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f80035f0 con 0x7f220806d530 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.170+0000 7f21fea00640 1 -- 172.21.15.102:0/3253489888 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f80039e0 con 0x7f220806d530 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.170+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22081f1820 con 0x7f220806d530 2024-04-20T21:32:50.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.170+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22081f1de0 con 0x7f220806d530 2024-04-20T21:32:50.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.172+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2208068c30 con 0x7f220806d530 2024-04-20T21:32:50.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.173+0000 7f21fea00640 1 -- 172.21.15.102:0/3253489888 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21f801a070 con 0x7f220806d530 2024-04-20T21:32:50.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.174+0000 7f21fea00640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e406c920 0x7f21e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.174+0000 7f21fea00640 1 -- 172.21.15.102:0/3253489888 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f21f808b490 con 0x7f220806d530 2024-04-20T21:32:50.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.174+0000 7f21fd400640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e406c920 0x7f21e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.175+0000 7f21fd400640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e406c920 0x7f21e406ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f21f002dbb0 tx=0x7f21f0032000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:50.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.178+0000 7f21fea00640 1 -- 172.21.15.102:0/3253489888 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f21f8058c00 con 0x7f220806d530 2024-04-20T21:32:50.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.420+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22080c7630 con 0x7f220806d530 2024-04-20T21:32:50.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.422+0000 7f21fea00640 1 -- 172.21.15.102:0/3253489888 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+6862 (secure 0 0 0) 0x7f21f805cf00 con 0x7f220806d530 2024-04-20T21:32:50.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:32:50.421 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:32:50.421 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 49 2024-04-20T21:32:50.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.424+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e406c920 msgr2=0x7f21e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.424+0000 7f220d200640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e406c920 0x7f21e406ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f21f002dbb0 tx=0x7f21f0032000 comp rx=0 tx=0).stop 2024-04-20T21:32:50.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.424+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806d530 msgr2=0x7f220814c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.424+0000 7f220d200640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806d530 0x7f220814c220 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c450 tx=0x7f21f800c910 comp rx=0 tx=0).stop 2024-04-20T21:32:50.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.424+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 shutdown_connections 2024-04-20T21:32:50.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.424+0000 7f220d200640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22080c5a60 0x7f2208154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.425+0000 7f220d200640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220806e030 0x7f2208153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.425+0000 7f220d200640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f21e406c920 0x7f21e406ede0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.425+0000 7f220d200640 1 --2- 172.21.15.102:0/3253489888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f220806d530 0x7f220814c220 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.425+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 >> 172.21.15.102:0/3253489888 conn(0x7f22080cd160 msgr2=0x7f22080cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:50.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.425+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 shutdown_connections 2024-04-20T21:32:50.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.425+0000 7f220d200640 1 -- 172.21.15.102:0/3253489888 wait complete. 2024-04-20T21:32:50.443 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-04-20T21:32:50.443 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:32:50.443 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:32:50.470 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:32:50.471 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:50.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.725+0000 7efd63e00640 1 Processor -- start 2024-04-20T21:32:50.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.725+0000 7efd63e00640 1 -- start start 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.726+0000 7efd63e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd64152290 0x7efd640cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.726+0000 7efd63e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7efd640c3700 0x7efd6406e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.726+0000 7efd63e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd6415c220 con 0x7efd6406eb70 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd63e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7efd640c9180 con 0x7efd640c3700 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd63e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efd640c4840 con 0x7efd64152290 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd63400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efd6406eb70 0x7efd640c8330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53092/0 (socket says 172.21.15.102:53092) 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd62a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd64152290 0x7efd640cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd62000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7efd640c3700 0x7efd6406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd63400640 1 -- 172.21.15.102:0/2769045805 learned_addr learned my addr 172.21.15.102:0/2769045805 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:32:50.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd62a00640 1 -- 172.21.15.102:0/2769045805 >> v2:172.21.15.193:3300/0 conn(0x7efd640c3700 msgr2=0x7efd6406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:50.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd62a00640 1 --2- 172.21.15.102:0/2769045805 >> v2:172.21.15.193:3300/0 conn(0x7efd640c3700 0x7efd6406e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd62a00640 1 -- 172.21.15.102:0/2769045805 >> v1:172.21.15.102:6789/0 conn(0x7efd6406eb70 legacy=0x7efd640c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd62a00640 1 -- 172.21.15.102:0/2769045805 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd640c4560 con 0x7efd64152290 2024-04-20T21:32:50.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.727+0000 7efd62000640 1 --2- 172.21.15.102:0/2769045805 >> v2:172.21.15.193:3300/0 conn(0x7efd640c3700 0x7efd6406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:50.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.729+0000 7efd62a00640 1 --2- 172.21.15.102:0/2769045805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd64152290 0x7efd640cd310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efd54009d90 tx=0x7efd5402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e449143ee27f91dd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:50.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.729+0000 7efd61600640 1 -- 172.21.15.102:0/2769045805 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54039070 con 0x7efd64152290 2024-04-20T21:32:50.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.729+0000 7efd61600640 1 -- 172.21.15.102:0/2769045805 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5402cce0 con 0x7efd64152290 2024-04-20T21:32:50.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.729+0000 7efd61600640 1 -- 172.21.15.102:0/2769045805 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54034660 con 0x7efd64152290 2024-04-20T21:32:50.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.729+0000 7efd63e00640 1 -- 172.21.15.102:0/2769045805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd64152290 msgr2=0x7efd640cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.729+0000 7efd63e00640 1 --2- 172.21.15.102:0/2769045805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd64152290 0x7efd640cd310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efd54009d90 tx=0x7efd5402bf30 comp rx=0 tx=0).stop 2024-04-20T21:32:50.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.730+0000 7efd63e00640 1 -- 172.21.15.102:0/2769045805 shutdown_connections 2024-04-20T21:32:50.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.730+0000 7efd63e00640 1 --2- 172.21.15.102:0/2769045805 >> v2:172.21.15.193:3300/0 conn(0x7efd640c3700 0x7efd6406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.730+0000 7efd63e00640 1 --2- 172.21.15.102:0/2769045805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd64152290 0x7efd640cd310 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.730+0000 7efd63e00640 1 -- 172.21.15.102:0/2769045805 >> 172.21.15.102:0/2769045805 conn(0x7efd640cc820 msgr2=0x7efd640ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:50.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.730+0000 7efd63e00640 1 -- 172.21.15.102:0/2769045805 shutdown_connections 2024-04-20T21:32:50.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.730+0000 7efd63e00640 1 -- 172.21.15.102:0/2769045805 wait complete. 2024-04-20T21:32:50.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.731+0000 7efd63e00640 1 Processor -- start 2024-04-20T21:32:50.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.731+0000 7efd63e00640 1 -- start start 2024-04-20T21:32:50.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.732+0000 7efd63e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd6406eb70 0x7efd641e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.732+0000 7efd63e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd640c3700 0x7efd640c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.732+0000 7efd63e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd64152290 0x7efd641e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.732+0000 7efd63e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd6415ba20 con 0x7efd64152290 2024-04-20T21:32:50.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.732+0000 7efd63e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efd6415b740 con 0x7efd640c3700 2024-04-20T21:32:50.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.732+0000 7efd63e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efd6415bd00 con 0x7efd6406eb70 2024-04-20T21:32:50.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd6406eb70 0x7efd641e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd6406eb70 0x7efd641e6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:54272/0 (socket says 172.21.15.102:54272) 2024-04-20T21:32:50.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 -- 172.21.15.102:0/2943325830 learned_addr learned my addr 172.21.15.102:0/2943325830 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:32:50.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62000640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd640c3700 0x7efd640c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd63400640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd64152290 0x7efd641e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 -- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd640c3700 msgr2=0x7efd640c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd640c3700 0x7efd640c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 -- 172.21.15.102:0/2943325830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd64152290 msgr2=0x7efd641e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd64152290 0x7efd641e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 -- 172.21.15.102:0/2943325830 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd641f4b40 con 0x7efd6406eb70 2024-04-20T21:32:50.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.733+0000 7efd62a00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd6406eb70 0x7efd641e6ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efd54037040 tx=0x7efd54004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:50.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.734+0000 7efd5aa00640 1 -- 172.21.15.102:0/2943325830 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54039050 con 0x7efd6406eb70 2024-04-20T21:32:50.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.734+0000 7efd5aa00640 1 -- 172.21.15.102:0/2943325830 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd54045070 con 0x7efd6406eb70 2024-04-20T21:32:50.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.734+0000 7efd5aa00640 1 -- 172.21.15.102:0/2943325830 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5403e850 con 0x7efd6406eb70 2024-04-20T21:32:50.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.734+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd641e3020 con 0x7efd6406eb70 2024-04-20T21:32:50.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.734+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641e35e0 con 0x7efd6406eb70 2024-04-20T21:32:50.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.735+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6406a0c0 con 0x7efd6406eb70 2024-04-20T21:32:50.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.736+0000 7efd5aa00640 1 -- 172.21.15.102:0/2943325830 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd5403e9f0 con 0x7efd6406eb70 2024-04-20T21:32:50.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.737+0000 7efd5aa00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:50.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.737+0000 7efd5aa00640 1 -- 172.21.15.102:0/2943325830 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7efd540aef00 con 0x7efd6406eb70 2024-04-20T21:32:50.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.737+0000 7efd62000640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:50.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.738+0000 7efd62000640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009560 tx=0x7efd4c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:50.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.742+0000 7efd5aa00640 1 -- 172.21.15.102:0/2943325830 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efd6406a0c0 con 0x7efd6406eb70 2024-04-20T21:32:50.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.985+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd6404fd60 con 0x7efd6406eb70 2024-04-20T21:32:50.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.986+0000 7efd5aa00640 1 -- 172.21.15.102:0/2943325830 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+6862 (secure 0 0 0) 0x7efd5407c790 con 0x7efd6406eb70 2024-04-20T21:32:50.986 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:32:50.986 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:32:50.986 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 49 2024-04-20T21:32:50.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 msgr2=0x7efd4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009560 tx=0x7efd4c009190 comp rx=0 tx=0).stop 2024-04-20T21:32:50.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd6406eb70 msgr2=0x7efd641e6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:50.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd6406eb70 0x7efd641e6ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efd54037040 tx=0x7efd54004a40 comp rx=0 tx=0).stop 2024-04-20T21:32:50.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 shutdown_connections 2024-04-20T21:32:50.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd64152290 0x7efd641e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd640c3700 0x7efd640c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 --2- 172.21.15.102:0/2943325830 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd6406eb70 0x7efd641e6ec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:50.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 >> 172.21.15.102:0/2943325830 conn(0x7efd640cc820 msgr2=0x7efd640c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:50.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 shutdown_connections 2024-04-20T21:32:50.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:50.990+0000 7efd63e00640 1 -- 172.21.15.102:0/2943325830 wait complete. 2024-04-20T21:32:51.009 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:32:53.009 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:53.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.221+0000 7f1c50400640 1 Processor -- start 2024-04-20T21:32:53.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.221+0000 7f1c50400640 1 -- start start 2024-04-20T21:32:53.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c50400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1c480c8db0 0x7f1c48070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:53.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c50400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c48158e80 con 0x7f1c48152170 2024-04-20T21:32:53.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c50400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c480c91c0 con 0x7f1c48071330 2024-04-20T21:32:53.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c50400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1c480c6dc0 con 0x7f1c480c8db0 2024-04-20T21:32:53.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c4da00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1c48152170 0x7f1c48158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53094/0 (socket says 172.21.15.102:53094) 2024-04-20T21:32:53.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c4da00640 1 -- 172.21.15.102:0/1031549959 learned_addr learned my addr 172.21.15.102:0/1031549959 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:32:53.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c4d000640 1 --2- 172.21.15.102:0/1031549959 >> v2:172.21.15.193:3300/0 conn(0x7f1c480c8db0 0x7f1c48070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:53.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c47e00640 1 -- 172.21.15.102:0/1031549959 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3171912251 0 0) 0x7f1c48158e80 con 0x7f1c48152170 2024-04-20T21:32:53.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c47e00640 1 -- 172.21.15.102:0/1031549959 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c30005730 con 0x7f1c48152170 2024-04-20T21:32:53.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c47e00640 1 -- 172.21.15.102:0/1031549959 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3655754506 0 0) 0x7f1c480c91c0 con 0x7f1c48071330 2024-04-20T21:32:53.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.223+0000 7f1c47e00640 1 -- 172.21.15.102:0/1031549959 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c48158e80 con 0x7f1c48071330 2024-04-20T21:32:53.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.224+0000 7f1c4d000640 1 -- 172.21.15.102:0/1031549959 >> v1:172.21.15.193:6789/0 conn(0x7f1c48071330 legacy=0x7f1c481628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:53.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.224+0000 7f1c4d000640 1 -- 172.21.15.102:0/1031549959 >> v1:172.21.15.102:6789/0 conn(0x7f1c48152170 legacy=0x7f1c48158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:53.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.224+0000 7f1c47e00640 1 -- 172.21.15.102:0/1031549959 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 874904046 0 0) 0x7f1c30005730 con 0x7f1c48152170 2024-04-20T21:32:53.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.224+0000 7f1c4d000640 1 -- 172.21.15.102:0/1031549959 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c48071f70 con 0x7f1c480c8db0 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.225+0000 7f1c4d000640 1 --2- 172.21.15.102:0/1031549959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c480c8db0 0x7f1c48070df0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1c340097a0 tx=0x7f1c3402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17512ac14681c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.225+0000 7f1c47e00640 1 -- 172.21.15.102:0/1031549959 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34034070 con 0x7f1c480c8db0 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.225+0000 7f1c47e00640 1 -- 172.21.15.102:0/1031549959 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34030610 con 0x7f1c480c8db0 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.225+0000 7f1c47e00640 1 -- 172.21.15.102:0/1031549959 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c340309e0 con 0x7f1c480c8db0 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.226+0000 7f1c50400640 1 -- 172.21.15.102:0/1031549959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c480c8db0 msgr2=0x7f1c48070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.226+0000 7f1c50400640 1 --2- 172.21.15.102:0/1031549959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c480c8db0 0x7f1c48070df0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1c340097a0 tx=0x7f1c3402b960 comp rx=0 tx=0).stop 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.226+0000 7f1c50400640 1 -- 172.21.15.102:0/1031549959 shutdown_connections 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.226+0000 7f1c50400640 1 --2- 172.21.15.102:0/1031549959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c480c8db0 0x7f1c48070df0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.226+0000 7f1c50400640 1 -- 172.21.15.102:0/1031549959 >> 172.21.15.102:0/1031549959 conn(0x7f1c480c9390 msgr2=0x7f1c480c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:53.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.226+0000 7f1c50400640 1 -- 172.21.15.102:0/1031549959 shutdown_connections 2024-04-20T21:32:53.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.226+0000 7f1c50400640 1 -- 172.21.15.102:0/1031549959 wait complete. 2024-04-20T21:32:53.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.228+0000 7f1c50400640 1 Processor -- start 2024-04-20T21:32:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.228+0000 7f1c50400640 1 -- start start 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c50400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c48071330 0x7f1c481f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c50400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c480c8db0 0x7f1c48150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c50400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c48152170 0x7f1c4814a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c50400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1c480724b0 con 0x7f1c48152170 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c50400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48072210 con 0x7f1c48071330 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c50400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1c48158250 con 0x7f1c480c8db0 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c48152170 0x7f1c4814a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c48071330 0x7f1c481f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4d000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c480c8db0 0x7f1c48150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c48152170 0x7f1c4814a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59726/0 (socket says 172.21.15.102:59726) 2024-04-20T21:32:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4e400640 1 -- 172.21.15.102:0/1233547033 learned_addr learned my addr 172.21.15.102:0/1233547033 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4e400640 1 -- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c480c8db0 msgr2=0x7f1c48150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4e400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c480c8db0 0x7f1c48150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4e400640 1 -- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c48071330 msgr2=0x7f1c481f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4e400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c48071330 0x7f1c481f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4e400640 1 -- 172.21.15.102:0/1233547033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c481f1c30 con 0x7f1c48152170 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.229+0000 7f1c4da00640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c48071330 0x7f1c481f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.230+0000 7f1c4e400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c48152170 0x7f1c4814a6f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c710 tx=0x7f1c4000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.230+0000 7f1c45600640 1 -- 172.21.15.102:0/1233547033 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40003b70 con 0x7f1c48152170 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.230+0000 7f1c45600640 1 -- 172.21.15.102:0/1233547033 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c40003d10 con 0x7f1c48152170 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.230+0000 7f1c45600640 1 -- 172.21.15.102:0/1233547033 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000aa50 con 0x7f1c48152170 2024-04-20T21:32:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.230+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c48156080 con 0x7f1c48152170 2024-04-20T21:32:53.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.230+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c481f0ef0 con 0x7f1c48152170 2024-04-20T21:32:53.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.231+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c481565a0 con 0x7f1c48152170 2024-04-20T21:32:53.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.232+0000 7f1c45600640 1 -- 172.21.15.102:0/1233547033 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c4000abf0 con 0x7f1c48152170 2024-04-20T21:32:53.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.234+0000 7f1c45600640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c2806c8f0 0x7f1c2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:53.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.234+0000 7f1c45600640 1 -- 172.21.15.102:0/1233547033 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f1c4008bfb0 con 0x7f1c48152170 2024-04-20T21:32:53.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.234+0000 7f1c4da00640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c2806c8f0 0x7f1c2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:53.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.235+0000 7f1c4da00640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c2806c8f0 0x7f1c2806edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c005880 tx=0x7f1c3c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.239+0000 7f1c45600640 1 -- 172.21.15.102:0/1233547033 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1c481565a0 con 0x7f1c48152170 2024-04-20T21:32:53.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.479+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c4806e0f0 con 0x7f1c48152170 2024-04-20T21:32:53.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.480+0000 7f1c45600640 1 -- 172.21.15.102:0/1233547033 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+6862 (secure 0 0 0) 0x7f1c4806e0f0 con 0x7f1c48152170 2024-04-20T21:32:53.480 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:32:53.480 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:32:53.480 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 49 2024-04-20T21:32:53.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c2806c8f0 msgr2=0x7f1c2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:53.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c2806c8f0 0x7f1c2806edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c005880 tx=0x7f1c3c009040 comp rx=0 tx=0).stop 2024-04-20T21:32:53.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c48152170 msgr2=0x7f1c4814a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c48152170 0x7f1c4814a6f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c710 tx=0x7f1c4000cbd0 comp rx=0 tx=0).stop 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 shutdown_connections 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c48152170 0x7f1c4814a6f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c480c8db0 0x7f1c48150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1c2806c8f0 0x7f1c2806edb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 --2- 172.21.15.102:0/1233547033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c48071330 0x7f1c481f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 >> 172.21.15.102:0/1233547033 conn(0x7f1c480c9390 msgr2=0x7f1c4806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 shutdown_connections 2024-04-20T21:32:53.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:53.483+0000 7f1c50400640 1 -- 172.21.15.102:0/1233547033 wait complete. 2024-04-20T21:32:53.502 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:32:55.504 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:55.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.712+0000 7f7bb6e00640 1 Processor -- start 2024-04-20T21:32:55.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.712+0000 7f7bb6e00640 1 -- start start 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.713+0000 7f7bb6e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7bb00c68e0 0x7f7bb00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.713+0000 7f7bb6e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7bb00c86d0 0x7f7bb006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.713+0000 7f7bb6e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb00c8b00 con 0x7f7bb0153430 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.713+0000 7f7bb6e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb00c8dd0 con 0x7f7bb00c68e0 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.713+0000 7f7bb6e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb0159310 con 0x7f7bb00c86d0 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.714+0000 7f7ba7400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7bb00c68e0 0x7f7bb00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.714+0000 7f7bafe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7bb0153430 0x7f7bb00c95a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53100/0 (socket says 172.21.15.102:53100) 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.714+0000 7f7bafe00640 1 -- 172.21.15.102:0/2715644375 learned_addr learned my addr 172.21.15.102:0/2715644375 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:32:55.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.714+0000 7f7bb4e00640 1 --2- 172.21.15.102:0/2715644375 >> v2:172.21.15.193:3300/0 conn(0x7f7bb00c86d0 0x7f7bb006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.714+0000 7f7ba7400640 1 -- 172.21.15.102:0/2715644375 >> v2:172.21.15.193:3300/0 conn(0x7f7bb00c86d0 msgr2=0x7f7bb006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:55.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.714+0000 7f7ba7400640 1 --2- 172.21.15.102:0/2715644375 >> v2:172.21.15.193:3300/0 conn(0x7f7bb00c86d0 0x7f7bb006ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.714+0000 7f7ba7400640 1 -- 172.21.15.102:0/2715644375 >> v1:172.21.15.102:6789/0 conn(0x7f7bb0153430 legacy=0x7f7bb00c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.714+0000 7f7ba7400640 1 -- 172.21.15.102:0/2715644375 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb01595f0 con 0x7f7bb00c68e0 2024-04-20T21:32:55.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.715+0000 7f7ba7400640 1 --2- 172.21.15.102:0/2715644375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c68e0 0x7f7bb00c8190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7bb006e620 tx=0x7f7b9802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b362fa51849a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:55.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7baf400640 1 -- 172.21.15.102:0/2715644375 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98034070 con 0x7f7bb00c68e0 2024-04-20T21:32:55.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7baf400640 1 -- 172.21.15.102:0/2715644375 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b98030610 con 0x7f7bb00c68e0 2024-04-20T21:32:55.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7baf400640 1 -- 172.21.15.102:0/2715644375 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b980309e0 con 0x7f7bb00c68e0 2024-04-20T21:32:55.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7bb6e00640 1 -- 172.21.15.102:0/2715644375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c68e0 msgr2=0x7f7bb00c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/2715644375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c68e0 0x7f7bb00c8190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7bb006e620 tx=0x7f7b9802b960 comp rx=0 tx=0).stop 2024-04-20T21:32:55.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7bb6e00640 1 -- 172.21.15.102:0/2715644375 shutdown_connections 2024-04-20T21:32:55.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/2715644375 >> v2:172.21.15.193:3300/0 conn(0x7f7bb00c86d0 0x7f7bb006ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/2715644375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c68e0 0x7f7bb00c8190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.716+0000 7f7bb6e00640 1 -- 172.21.15.102:0/2715644375 >> 172.21.15.102:0/2715644375 conn(0x7f7bb0072f00 msgr2=0x7f7bb0073310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:55.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.717+0000 7f7bb6e00640 1 -- 172.21.15.102:0/2715644375 shutdown_connections 2024-04-20T21:32:55.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.717+0000 7f7bb6e00640 1 -- 172.21.15.102:0/2715644375 wait complete. 2024-04-20T21:32:55.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.718+0000 7f7bb6e00640 1 Processor -- start 2024-04-20T21:32:55.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.718+0000 7f7bb6e00640 1 -- start start 2024-04-20T21:32:55.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.719+0000 7f7bb6e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bb00c68e0 0x7f7bb01efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.719+0000 7f7bb6e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c86d0 0x7f7bb00ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.719+0000 7f7bb6e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bb0153430 0x7f7bb01482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.719+0000 7f7bb6e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0159e90 con 0x7f7bb00c86d0 2024-04-20T21:32:55.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.719+0000 7f7bb6e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0159bb0 con 0x7f7bb00c68e0 2024-04-20T21:32:55.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.719+0000 7f7bb6e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb0153810 con 0x7f7bb0153430 2024-04-20T21:32:55.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c86d0 0x7f7bb00ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7bafe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bb00c68e0 0x7f7bb01efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c86d0 0x7f7bb00ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59744/0 (socket says 172.21.15.102:59744) 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 -- 172.21.15.102:0/774232559 learned_addr learned my addr 172.21.15.102:0/774232559 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7bb4e00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bb0153430 0x7f7bb01482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 -- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bb0153430 msgr2=0x7f7bb01482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bb0153430 0x7f7bb01482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 -- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bb00c68e0 msgr2=0x7f7bb01efcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bb00c68e0 0x7f7bb01efcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 -- 172.21.15.102:0/774232559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb01f0230 con 0x7f7bb00c86d0 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.720+0000 7f7ba7400640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c86d0 0x7f7bb00ca310 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009890 tx=0x7f7b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.721+0000 7f7bacc00640 1 -- 172.21.15.102:0/774232559 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98034040 con 0x7f7bb00c86d0 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.721+0000 7f7bacc00640 1 -- 172.21.15.102:0/774232559 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9803aaa0 con 0x7f7bb00c86d0 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.721+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb01473d0 con 0x7f7bb00c86d0 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.721+0000 7f7bacc00640 1 -- 172.21.15.102:0/774232559 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b980443f0 con 0x7f7bb00c86d0 2024-04-20T21:32:55.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.721+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb01467c0 con 0x7f7bb00c86d0 2024-04-20T21:32:55.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.722+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb004fd90 con 0x7f7bb00c86d0 2024-04-20T21:32:55.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.723+0000 7f7bacc00640 1 -- 172.21.15.102:0/774232559 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b98037070 con 0x7f7bb00c86d0 2024-04-20T21:32:55.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.724+0000 7f7bacc00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b8c06c920 0x7f7b8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.724+0000 7f7bacc00640 1 -- 172.21.15.102:0/774232559 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f7b980adef0 con 0x7f7bb00c86d0 2024-04-20T21:32:55.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.725+0000 7f7bafe00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b8c06c920 0x7f7b8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.725+0000 7f7bafe00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b8c06c920 0x7f7b8c06ede0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0007fd0 tx=0x7f7ba0007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:55.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.731+0000 7f7bacc00640 1 -- 172.21.15.102:0/774232559 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7bb004fd90 con 0x7f7bb00c86d0 2024-04-20T21:32:55.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.971+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bb0068c30 con 0x7f7bb00c86d0 2024-04-20T21:32:55.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.972+0000 7f7bacc00640 1 -- 172.21.15.102:0/774232559 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+6862 (secure 0 0 0) 0x7f7b9803f070 con 0x7f7bb00c86d0 2024-04-20T21:32:55.971 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:32:55.972 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:32:55.972 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 49 2024-04-20T21:32:55.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b8c06c920 msgr2=0x7f7b8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b8c06c920 0x7f7b8c06ede0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0007fd0 tx=0x7f7ba0007e80 comp rx=0 tx=0).stop 2024-04-20T21:32:55.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c86d0 msgr2=0x7f7bb00ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c86d0 0x7f7bb00ca310 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009890 tx=0x7f7b980046c0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 shutdown_connections 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bb0153430 0x7f7bb01482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7bb00c86d0 0x7f7bb00ca310 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7b8c06c920 0x7f7b8c06ede0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 --2- 172.21.15.102:0/774232559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bb00c68e0 0x7f7bb01efcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 >> 172.21.15.102:0/774232559 conn(0x7f7bb0072f00 msgr2=0x7f7bb0150110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 shutdown_connections 2024-04-20T21:32:55.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:55.975+0000 7f7bb6e00640 1 -- 172.21.15.102:0/774232559 wait complete. 2024-04-20T21:32:55.995 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:32:57.996 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:58.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.204+0000 7fb169400640 1 Processor -- start 2024-04-20T21:32:58.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.204+0000 7fb169400640 1 -- start start 2024-04-20T21:32:58.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.206+0000 7fb169400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb1641618a0 0x7fb164163c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:58.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.206+0000 7fb169400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb164158e80 con 0x7fb164152170 2024-04-20T21:32:58.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.206+0000 7fb169400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1640c91c0 con 0x7fb1640c8db0 2024-04-20T21:32:58.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.206+0000 7fb169400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb1640c6dc0 con 0x7fb1641618a0 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.206+0000 7fb162800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb164152170 0x7fb164158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50562/0 (socket says 172.21.15.102:50562) 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.206+0000 7fb162800640 1 -- 172.21.15.102:0/3513694568 learned_addr learned my addr 172.21.15.102:0/3513694568 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb163200640 1 --2- 172.21.15.102:0/3513694568 >> v2:172.21.15.102:3300/0 conn(0x7fb1641618a0 0x7fb164163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb161400640 1 -- 172.21.15.102:0/3513694568 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1217596021 0 0) 0x7fb164158e80 con 0x7fb164152170 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb161400640 1 -- 172.21.15.102:0/3513694568 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb144003520 con 0x7fb164152170 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb161400640 1 -- 172.21.15.102:0/3513694568 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577963994 0 0) 0x7fb1640c91c0 con 0x7fb1640c8db0 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb161400640 1 -- 172.21.15.102:0/3513694568 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb164158e80 con 0x7fb1640c8db0 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb163200640 1 -- 172.21.15.102:0/3513694568 >> v1:172.21.15.193:6789/0 conn(0x7fb1640c8db0 legacy=0x7fb164070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb163200640 1 -- 172.21.15.102:0/3513694568 >> v1:172.21.15.102:6789/0 conn(0x7fb164152170 legacy=0x7fb164158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb163200640 1 -- 172.21.15.102:0/3513694568 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb164071f70 con 0x7fb1641618a0 2024-04-20T21:32:58.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.207+0000 7fb161400640 1 -- 172.21.15.102:0/3513694568 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2718357884 0 0) 0x7fb144003520 con 0x7fb164152170 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.209+0000 7fb163200640 1 --2- 172.21.15.102:0/3513694568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1641618a0 0x7fb164163c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0097a0 tx=0x7fb15c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deae42ee0aa2d959 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.209+0000 7fb161400640 1 -- 172.21.15.102:0/3513694568 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c038070 con 0x7fb1641618a0 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.209+0000 7fb161400640 1 -- 172.21.15.102:0/3513694568 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c031500 con 0x7fb1641618a0 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.209+0000 7fb161400640 1 -- 172.21.15.102:0/3513694568 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154001670 con 0x7fb1641618a0 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.209+0000 7fb169400640 1 -- 172.21.15.102:0/3513694568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1641618a0 msgr2=0x7fb164163c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.209+0000 7fb169400640 1 --2- 172.21.15.102:0/3513694568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1641618a0 0x7fb164163c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0097a0 tx=0x7fb15c02b820 comp rx=0 tx=0).stop 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.210+0000 7fb169400640 1 -- 172.21.15.102:0/3513694568 shutdown_connections 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.210+0000 7fb169400640 1 --2- 172.21.15.102:0/3513694568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1641618a0 0x7fb164163c90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:58.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.210+0000 7fb169400640 1 -- 172.21.15.102:0/3513694568 >> 172.21.15.102:0/3513694568 conn(0x7fb1640c9390 msgr2=0x7fb1640c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:58.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.210+0000 7fb169400640 1 -- 172.21.15.102:0/3513694568 shutdown_connections 2024-04-20T21:32:58.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.210+0000 7fb169400640 1 -- 172.21.15.102:0/3513694568 wait complete. 2024-04-20T21:32:58.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.211+0000 7fb169400640 1 Processor -- start 2024-04-20T21:32:58.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.212+0000 7fb169400640 1 -- start start 2024-04-20T21:32:58.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.212+0000 7fb169400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1640c8db0 0x7fb164160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb169400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb164152170 0x7fb1641f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb169400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1641618a0 0x7fb1641f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb169400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640724b0 con 0x7fb1640c8db0 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb169400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb164072210 con 0x7fb164152170 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb169400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb164158250 con 0x7fb1641618a0 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb162800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1640c8db0 0x7fb164160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb162800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1640c8db0 0x7fb164160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34242/0 (socket says 172.21.15.102:34242) 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb162800640 1 -- 172.21.15.102:0/174620972 learned_addr learned my addr 172.21.15.102:0/174620972 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:32:58.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb161e00640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb164152170 0x7fb1641f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:58.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb163200640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1641618a0 0x7fb1641f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:58.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb162800640 1 -- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1641618a0 msgr2=0x7fb1641f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:58.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb162800640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1641618a0 0x7fb1641f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:58.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb162800640 1 -- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb164152170 msgr2=0x7fb1641f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:58.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb162800640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb164152170 0x7fb1641f1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:58.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.213+0000 7fb162800640 1 -- 172.21.15.102:0/174620972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1640c8850 con 0x7fb1640c8db0 2024-04-20T21:32:58.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.214+0000 7fb162800640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1640c8db0 0x7fb164160ca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c750 tx=0x7fb15400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:58.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.214+0000 7fb15aa00640 1 -- 172.21.15.102:0/174620972 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154005ad0 con 0x7fb1640c8db0 2024-04-20T21:32:58.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.214+0000 7fb15aa00640 1 -- 172.21.15.102:0/174620972 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb154005c70 con 0x7fb1640c8db0 2024-04-20T21:32:58.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.214+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1641516b0 con 0x7fb1640c8db0 2024-04-20T21:32:58.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.214+0000 7fb15aa00640 1 -- 172.21.15.102:0/174620972 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154005e60 con 0x7fb1640c8db0 2024-04-20T21:32:58.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.214+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1641611e0 con 0x7fb1640c8db0 2024-04-20T21:32:58.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.215+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb164074340 con 0x7fb1640c8db0 2024-04-20T21:32:58.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.216+0000 7fb15aa00640 1 -- 172.21.15.102:0/174620972 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1540072b0 con 0x7fb1640c8db0 2024-04-20T21:32:58.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.217+0000 7fb15aa00640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb14006c8f0 0x7fb14006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:58.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.217+0000 7fb15aa00640 1 -- 172.21.15.102:0/174620972 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7fb15408bf20 con 0x7fb1640c8db0 2024-04-20T21:32:58.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.217+0000 7fb161e00640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb14006c8f0 0x7fb14006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:58.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.218+0000 7fb161e00640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb14006c8f0 0x7fb14006edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb1640c3d30 tx=0x7fb14c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:58.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.221+0000 7fb15aa00640 1 -- 172.21.15.102:0/174620972 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb154059750 con 0x7fb1640c8db0 2024-04-20T21:32:58.346 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:58.354 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:58.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.465+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb16406f1f0 con 0x7fb1640c8db0 2024-04-20T21:32:58.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.466+0000 7fb15aa00640 1 -- 172.21.15.102:0/174620972 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+6862 (secure 0 0 0) 0x7fb15405da50 con 0x7fb1640c8db0 2024-04-20T21:32:58.465 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:32:58.466 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:32:58.466 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 49 2024-04-20T21:32:58.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb14006c8f0 msgr2=0x7fb14006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:58.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb14006c8f0 0x7fb14006edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb1640c3d30 tx=0x7fb14c009040 comp rx=0 tx=0).stop 2024-04-20T21:32:58.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1640c8db0 msgr2=0x7fb164160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:58.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1640c8db0 0x7fb164160ca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c750 tx=0x7fb15400cc10 comp rx=0 tx=0).stop 2024-04-20T21:32:58.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 shutdown_connections 2024-04-20T21:32:58.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1641618a0 0x7fb1641f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:58.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb14006c8f0 0x7fb14006edb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:58.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb164152170 0x7fb1641f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:58.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 --2- 172.21.15.102:0/174620972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1640c8db0 0x7fb164160ca0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:58.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 >> 172.21.15.102:0/174620972 conn(0x7fb1640c9390 msgr2=0x7fb16415b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:58.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 shutdown_connections 2024-04-20T21:32:58.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:32:58.469+0000 7fb169400640 1 -- 172.21.15.102:0/174620972 wait complete. 2024-04-20T21:32:58.487 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:33:00.487 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:00.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.695+0000 7fa3fb200640 1 Processor -- start 2024-04-20T21:33:00.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.695+0000 7fa3fb200640 1 -- start start 2024-04-20T21:33:00.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.696+0000 7fa3fb200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3f40cc470 0x7fa3f40cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:00.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.696+0000 7fa3fb200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f40cdfc0 0x7fa3f40cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3fb200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f415ef60 con 0x7fa3f4155530 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3fb200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f40c5ca0 con 0x7fa3f40cc470 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3fb200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f40cd5f0 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3f9400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3f40cc470 0x7fa3f40cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3f9400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3f40cc470 0x7fa3f40cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34258/0 (socket says 172.21.15.102:34258) 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3f9400640 1 -- 172.21.15.102:0/92850386 learned_addr learned my addr 172.21.15.102:0/92850386 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3fa800640 1 --2- 172.21.15.102:0/92850386 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f40cdfc0 0x7fa3f40cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3f9400640 1 -- 172.21.15.102:0/92850386 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f40cdfc0 msgr2=0x7fa3f40cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3f9400640 1 --2- 172.21.15.102:0/92850386 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f40cdfc0 0x7fa3f40cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3f9400640 1 -- 172.21.15.102:0/92850386 >> v1:172.21.15.193:6789/0 conn(0x7fa3f4155530 legacy=0x7fa3f415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3f9400640 1 -- 172.21.15.102:0/92850386 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f415f240 con 0x7fa3f40cc470 2024-04-20T21:33:00.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.697+0000 7fa3f8a00640 1 -- 172.21.15.102:0/92850386 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4285700772 0 0) 0x7fa3f415ef60 con 0x7fa3f4155530 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.698+0000 7fa3f9400640 1 --2- 172.21.15.102:0/92850386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f40cda80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009870 tx=0x7fa3e402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346fb029cf6f233a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3f8a00640 1 -- 172.21.15.102:0/92850386 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4034070 con 0x7fa3f40cc470 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3f8a00640 1 -- 172.21.15.102:0/92850386 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e4030840 con 0x7fa3f40cc470 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3f8a00640 1 -- 172.21.15.102:0/92850386 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4030c10 con 0x7fa3f40cc470 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3fb200640 1 -- 172.21.15.102:0/92850386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 msgr2=0x7fa3f40cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3fb200640 1 --2- 172.21.15.102:0/92850386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f40cda80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009870 tx=0x7fa3e402b980 comp rx=0 tx=0).stop 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3fb200640 1 -- 172.21.15.102:0/92850386 shutdown_connections 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3fb200640 1 --2- 172.21.15.102:0/92850386 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f40cdfc0 0x7fa3f40cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3fb200640 1 --2- 172.21.15.102:0/92850386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f40cda80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3fb200640 1 -- 172.21.15.102:0/92850386 >> 172.21.15.102:0/92850386 conn(0x7fa3f4075080 msgr2=0x7fa3f4075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:00.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.699+0000 7fa3fb200640 1 -- 172.21.15.102:0/92850386 shutdown_connections 2024-04-20T21:33:00.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.700+0000 7fa3fb200640 1 -- 172.21.15.102:0/92850386 wait complete. 2024-04-20T21:33:00.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.701+0000 7fa3fb200640 1 Processor -- start 2024-04-20T21:33:00.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.701+0000 7fa3fb200640 1 -- start start 2024-04-20T21:33:00.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.702+0000 7fa3fb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f41f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:00.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.702+0000 7fa3fb200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f40cdfc0 0x7fa3f40cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:00.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.702+0000 7fa3fb200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f4155530 0x7fa3f40b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.702+0000 7fa3fb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f40cd280 con 0x7fa3f40cc470 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.702+0000 7fa3fb200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f40ccfa0 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.702+0000 7fa3fb200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f40ccbe0 con 0x7fa3f4155530 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f41f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f40cdfc0 0x7fa3f40cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f41f1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34264/0 (socket says 172.21.15.102:34264) 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9e00640 1 -- 172.21.15.102:0/3306080753 learned_addr learned my addr 172.21.15.102:0/3306080753 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3fa800640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f4155530 0x7fa3f40b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9400640 1 -- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f4155530 msgr2=0x7fa3f40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9400640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f4155530 0x7fa3f40b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9400640 1 -- 172.21.15.102:0/3306080753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 msgr2=0x7fa3f41f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9400640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f41f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9400640 1 -- 172.21.15.102:0/3306080753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f41f45e0 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3fa800640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f4155530 0x7fa3f40b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:00.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.703+0000 7fa3f9e00640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f41f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:00.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.704+0000 7fa3f9400640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f40cdfc0 0x7fa3f40cfcf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4072680 tx=0x7fa3e4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:00.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.704+0000 7fa3ee000640 1 -- 172.21.15.102:0/3306080753 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4048070 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.704+0000 7fa3ee000640 1 -- 172.21.15.102:0/3306080753 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e4030640 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.704+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f415cbc0 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.704+0000 7fa3ee000640 1 -- 172.21.15.102:0/3306080753 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4034040 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.704+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41f4b60 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.705+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f404fd60 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.706+0000 7fa3ee000640 1 -- 172.21.15.102:0/3306080753 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3e404b070 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.707+0000 7fa3ee000640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3d806c920 0x7fa3d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:00.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.707+0000 7fa3ee000640 1 -- 172.21.15.102:0/3306080753 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7fa3e40ae100 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.708+0000 7fa3f9e00640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3d806c920 0x7fa3d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:00.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.708+0000 7fa3f9e00640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3d806c920 0x7fa3d806ede0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000f60 tx=0x7fa3e8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:00.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.712+0000 7fa3ee000640 1 -- 172.21.15.102:0/3306080753 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa3e407b870 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.956+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f4148310 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.956+0000 7fa3ee000640 1 -- 172.21.15.102:0/3306080753 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+6862 (secure 0 0 0) 0x7fa3e407fb70 con 0x7fa3f40cdfc0 2024-04-20T21:33:00.956 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:00.956 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:33:00.956 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 49 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3d806c920 msgr2=0x7fa3d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3d806c920 0x7fa3d806ede0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000f60 tx=0x7fa3e8009040 comp rx=0 tx=0).stop 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f40cdfc0 msgr2=0x7fa3f40cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f40cdfc0 0x7fa3f40cfcf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4072680 tx=0x7fa3e4004730 comp rx=0 tx=0).stop 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 shutdown_connections 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3f4155530 0x7fa3f40b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3d806c920 0x7fa3d806ede0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f40cdfc0 0x7fa3f40cfcf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 --2- 172.21.15.102:0/3306080753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3f40cc470 0x7fa3f41f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 >> 172.21.15.102:0/3306080753 conn(0x7fa3f4075080 msgr2=0x7fa3f4154420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 shutdown_connections 2024-04-20T21:33:00.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:00.960+0000 7fa3fb200640 1 -- 172.21.15.102:0/3306080753 wait complete. 2024-04-20T21:33:00.980 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:33:02.981 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:03.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.189+0000 7fd0b4c00640 1 Processor -- start 2024-04-20T21:33:03.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.190+0000 7fd0b4c00640 1 -- start start 2024-04-20T21:33:03.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.191+0000 7fd0b4c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd0b0151c90 0x7fd0b00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.191+0000 7fd0b4c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b0071040 con 0x7fd0b00c7290 2024-04-20T21:33:03.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.191+0000 7fd0b4c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b0071320 con 0x7fd0b0071a70 2024-04-20T21:33:03.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.191+0000 7fd0b4c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b00c9200 con 0x7fd0b0151c90 2024-04-20T21:33:03.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.191+0000 7fd0a5400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd0b00c7290 0x7fd0b0158240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50570/0 (socket says 172.21.15.102:50570) 2024-04-20T21:33:03.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.191+0000 7fd0a5400640 1 -- 172.21.15.102:0/153745372 learned_addr learned my addr 172.21.15.102:0/153745372 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:33:03.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.191+0000 7fd0ade00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd0b0151c90 0x7fd0b00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.192+0000 7fd0ad400640 1 -- 172.21.15.102:0/153745372 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 53166002 0 0) 0x7fd0b0071040 con 0x7fd0b00c7290 2024-04-20T21:33:03.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.192+0000 7fd0ad400640 1 -- 172.21.15.102:0/153745372 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd094005730 con 0x7fd0b00c7290 2024-04-20T21:33:03.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.192+0000 7fd0ad400640 1 -- 172.21.15.102:0/153745372 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3395792657 0 0) 0x7fd0b0071320 con 0x7fd0b0071a70 2024-04-20T21:33:03.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.192+0000 7fd0ad400640 1 -- 172.21.15.102:0/153745372 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0b0071040 con 0x7fd0b0071a70 2024-04-20T21:33:03.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.192+0000 7fd0ade00640 1 -- 172.21.15.102:0/153745372 >> v1:172.21.15.193:6789/0 conn(0x7fd0b0071a70 legacy=0x7fd0b006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.192+0000 7fd0ade00640 1 -- 172.21.15.102:0/153745372 >> v1:172.21.15.102:6789/0 conn(0x7fd0b00c7290 legacy=0x7fd0b0158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.192+0000 7fd0ade00640 1 -- 172.21.15.102:0/153745372 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b0071600 con 0x7fd0b0151c90 2024-04-20T21:33:03.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.193+0000 7fd0ade00640 1 --2- 172.21.15.102:0/153745372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b00c8cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00097a0 tx=0x7fd0a002b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e729a2b721ed8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.193+0000 7fd0ad400640 1 -- 172.21.15.102:0/153745372 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a002f810 con 0x7fd0b0151c90 2024-04-20T21:33:03.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.193+0000 7fd0ad400640 1 -- 172.21.15.102:0/153745372 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a002f9b0 con 0x7fd0b0151c90 2024-04-20T21:33:03.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.193+0000 7fd0ad400640 1 -- 172.21.15.102:0/153745372 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a002fd80 con 0x7fd0b0151c90 2024-04-20T21:33:03.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.194+0000 7fd0b4c00640 1 -- 172.21.15.102:0/153745372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 msgr2=0x7fd0b00c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.194+0000 7fd0b4c00640 1 --2- 172.21.15.102:0/153745372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b00c8cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00097a0 tx=0x7fd0a002b9f0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.194+0000 7fd0b4c00640 1 -- 172.21.15.102:0/153745372 shutdown_connections 2024-04-20T21:33:03.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.194+0000 7fd0b4c00640 1 --2- 172.21.15.102:0/153745372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b00c8cc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.194+0000 7fd0b4c00640 1 -- 172.21.15.102:0/153745372 >> 172.21.15.102:0/153745372 conn(0x7fd0b00c9390 msgr2=0x7fd0b00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.194+0000 7fd0b4c00640 1 -- 172.21.15.102:0/153745372 shutdown_connections 2024-04-20T21:33:03.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.194+0000 7fd0b4c00640 1 -- 172.21.15.102:0/153745372 wait complete. 2024-04-20T21:33:03.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.196+0000 7fd0b4c00640 1 Processor -- start 2024-04-20T21:33:03.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.196+0000 7fd0b4c00640 1 -- start start 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0b4c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0b0071a70 0x7fd0b01f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0b4c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0b00c7290 0x7fd0b00721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0b4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b0148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0b4c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0157db0 con 0x7fd0b0151c90 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0b4c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0070df0 con 0x7fd0b0071a70 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0b4c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b00ca1f0 con 0x7fd0b00c7290 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ae800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b0148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ae800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b0148a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34282/0 (socket says 172.21.15.102:34282) 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ae800640 1 -- 172.21.15.102:0/2532682494 learned_addr learned my addr 172.21.15.102:0/2532682494 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:03.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ade00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0b0071a70 0x7fd0b01f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0a5400640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0b00c7290 0x7fd0b00721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ae800640 1 -- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0b00c7290 msgr2=0x7fd0b00721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ae800640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0b00c7290 0x7fd0b00721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ae800640 1 -- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0b0071a70 msgr2=0x7fd0b01f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ae800640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0b0071a70 0x7fd0b01f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.197+0000 7fd0ae800640 1 -- 172.21.15.102:0/2532682494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b015b840 con 0x7fd0b0151c90 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.198+0000 7fd0ae800640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b0148a40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c6e0 tx=0x7fd0a800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.198+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2532682494 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8003b70 con 0x7fd0b0151c90 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.198+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2532682494 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a8003d10 con 0x7fd0b0151c90 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.198+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2532682494 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80052d0 con 0x7fd0b0151c90 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.198+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b01f0f70 con 0x7fd0b0151c90 2024-04-20T21:33:03.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.198+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b01f1530 con 0x7fd0b0151c90 2024-04-20T21:33:03.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.199+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b0074340 con 0x7fd0b0151c90 2024-04-20T21:33:03.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.200+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2532682494 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0a8005470 con 0x7fd0b0151c90 2024-04-20T21:33:03.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.201+0000 7fd0a6a00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08c06c8f0 0x7fd08c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.201+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2532682494 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7fd0a808c070 con 0x7fd0b0151c90 2024-04-20T21:33:03.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.201+0000 7fd0ade00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08c06c8f0 0x7fd08c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.202+0000 7fd0ade00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08c06c8f0 0x7fd08c06edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0036000 tx=0x7fd0a0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.208+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2532682494 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd0a80596c0 con 0x7fd0b0151c90 2024-04-20T21:33:03.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.445+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0b00c51f0 con 0x7fd0b0151c90 2024-04-20T21:33:03.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.446+0000 7fd0a6a00640 1 -- 172.21.15.102:0/2532682494 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+6862 (secure 0 0 0) 0x7fd0a805d9c0 con 0x7fd0b0151c90 2024-04-20T21:33:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:03.446 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:31:54.542688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5342":{"gid":5342,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":44,"addr":"172.21.15.102:6833/1043030921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1043030921},{"type":"v1","addr":"172.21.15.102:6833","nonce":1043030921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5342,"qdb_cluster":[5342]},"id":1}]} 2024-04-20T21:33:03.446 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 49 2024-04-20T21:33:03.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08c06c8f0 msgr2=0x7fd08c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08c06c8f0 0x7fd08c06edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0036000 tx=0x7fd0a0008030 comp rx=0 tx=0).stop 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 msgr2=0x7fd0b0148a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b0148a40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c6e0 tx=0x7fd0a800cba0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 shutdown_connections 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0151c90 0x7fd0b0148a40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0b00c7290 0x7fd0b00721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd08c06c8f0 0x7fd08c06edb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 --2- 172.21.15.102:0/2532682494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0b0071a70 0x7fd0b01f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 >> 172.21.15.102:0/2532682494 conn(0x7fd0b00c9390 msgr2=0x7fd0b006efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.449+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 shutdown_connections 2024-04-20T21:33:03.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:03.450+0000 7fd0b4c00640 1 -- 172.21.15.102:0/2532682494 wait complete. 2024-04-20T21:33:03.466 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:33:05.467 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:05.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.677+0000 7f5c15e00640 1 Processor -- start 2024-04-20T21:33:05.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.677+0000 7f5c15e00640 1 -- start start 2024-04-20T21:33:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.678+0000 7f5c15e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5c1014fa50 0x7f5c100c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.678+0000 7f5c15e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c100c9eb0 0x7f5c10158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c15e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c101536d0 con 0x7f5c10073a10 2024-04-20T21:33:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c15e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5c10159500 con 0x7f5c100c9eb0 2024-04-20T21:33:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c15e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f5c100c76c0 con 0x7f5c1014fa50 2024-04-20T21:33:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c0ea00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5c1014fa50 0x7f5c100c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c0f400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f5c10073a10 0x7f5c100c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:50584/0 (socket says 172.21.15.102:50584) 2024-04-20T21:33:05.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c07e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c100c9eb0 0x7f5c10158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c0f400640 1 -- 172.21.15.102:0/2159823430 learned_addr learned my addr 172.21.15.102:0/2159823430 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:33:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c0e000640 1 -- 172.21.15.102:0/2159823430 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3926399443 0 0) 0x7f5c08001490 con 0x7f5c10073a10 2024-04-20T21:33:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.679+0000 7f5c0e000640 1 -- 172.21.15.102:0/2159823430 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bf4003520 con 0x7f5c10073a10 2024-04-20T21:33:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.680+0000 7f5c0ea00640 1 -- 172.21.15.102:0/2159823430 >> v2:172.21.15.102:3300/0 conn(0x7f5c100c9eb0 msgr2=0x7f5c10158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.680+0000 7f5c0ea00640 1 --2- 172.21.15.102:0/2159823430 >> v2:172.21.15.102:3300/0 conn(0x7f5c100c9eb0 0x7f5c10158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.680+0000 7f5c0ea00640 1 -- 172.21.15.102:0/2159823430 >> v1:172.21.15.193:6789/0 conn(0x7f5c10073a10 legacy=0x7f5c100c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.680+0000 7f5c0ea00640 1 -- 172.21.15.102:0/2159823430 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c100c7960 con 0x7f5c1014fa50 2024-04-20T21:33:05.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.680+0000 7f5c07e00640 1 --2- 172.21.15.102:0/2159823430 >> v2:172.21.15.102:3300/0 conn(0x7f5c100c9eb0 0x7f5c10158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:05.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.681+0000 7f5c0ea00640 1 --2- 172.21.15.102:0/2159823430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c1014fa50 0x7f5c100c4830 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5c100706a0 tx=0x7f5c0002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b135547d846353 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.681+0000 7f5c0e000640 1 -- 172.21.15.102:0/2159823430 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00034070 con 0x7f5c1014fa50 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.681+0000 7f5c0e000640 1 -- 172.21.15.102:0/2159823430 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c00030610 con 0x7f5c1014fa50 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.681+0000 7f5c0e000640 1 -- 172.21.15.102:0/2159823430 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000309c0 con 0x7f5c1014fa50 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.682+0000 7f5c15e00640 1 -- 172.21.15.102:0/2159823430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c1014fa50 msgr2=0x7f5c100c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.682+0000 7f5c15e00640 1 --2- 172.21.15.102:0/2159823430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c1014fa50 0x7f5c100c4830 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5c100706a0 tx=0x7f5c0002b960 comp rx=0 tx=0).stop 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.682+0000 7f5c15e00640 1 -- 172.21.15.102:0/2159823430 shutdown_connections 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.682+0000 7f5c15e00640 1 --2- 172.21.15.102:0/2159823430 >> v2:172.21.15.102:3300/0 conn(0x7f5c100c9eb0 0x7f5c10158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.682+0000 7f5c15e00640 1 --2- 172.21.15.102:0/2159823430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c1014fa50 0x7f5c100c4830 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.682+0000 7f5c15e00640 1 -- 172.21.15.102:0/2159823430 >> 172.21.15.102:0/2159823430 conn(0x7f5c100c9390 msgr2=0x7f5c100c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.682+0000 7f5c15e00640 1 -- 172.21.15.102:0/2159823430 shutdown_connections 2024-04-20T21:33:05.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.682+0000 7f5c15e00640 1 -- 172.21.15.102:0/2159823430 wait complete. 2024-04-20T21:33:05.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.684+0000 7f5c15e00640 1 Processor -- start 2024-04-20T21:33:05.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.684+0000 7f5c15e00640 1 -- start start 2024-04-20T21:33:05.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c15e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c10073a10 0x7f5c10148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c15e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c100c9eb0 0x7f5c101486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c15e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1014fa50 0x7f5c1014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c15e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c100c7c00 con 0x7f5c100c9eb0 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c15e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c100c3d90 con 0x7f5c10073a10 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c15e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5c100c4030 con 0x7f5c1014fa50 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c07e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c100c9eb0 0x7f5c101486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c07e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c100c9eb0 0x7f5c101486e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34294/0 (socket says 172.21.15.102:34294) 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c07e00640 1 -- 172.21.15.102:0/1650219351 learned_addr learned my addr 172.21.15.102:0/1650219351 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c0f400640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1014fa50 0x7f5c1014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.685+0000 7f5c0ea00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c10073a10 0x7f5c10148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c07e00640 1 -- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1014fa50 msgr2=0x7f5c1014d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c07e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1014fa50 0x7f5c1014d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c07e00640 1 -- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c10073a10 msgr2=0x7f5c10148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c07e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c10073a10 0x7f5c10148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c07e00640 1 -- 172.21.15.102:0/1650219351 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c100727d0 con 0x7f5c100c9eb0 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c0ea00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c10073a10 0x7f5c10148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c07e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c100c9eb0 0x7f5c101486e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800ec10 tx=0x7f5bf800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c07400640 1 -- 172.21.15.102:0/1650219351 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8017070 con 0x7f5c100c9eb0 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.686+0000 7f5c07400640 1 -- 172.21.15.102:0/1650219351 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf800cc30 con 0x7f5c100c9eb0 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.687+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c100cc0d0 con 0x7f5c100c9eb0 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.687+0000 7f5c07400640 1 -- 172.21.15.102:0/1650219351 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8013470 con 0x7f5c100c9eb0 2024-04-20T21:33:05.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.687+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c10070f90 con 0x7f5c100c9eb0 2024-04-20T21:33:05.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.688+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c10074340 con 0x7f5c100c9eb0 2024-04-20T21:33:05.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.689+0000 7f5c07400640 1 -- 172.21.15.102:0/1650219351 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bf801a070 con 0x7f5c100c9eb0 2024-04-20T21:33:05.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.690+0000 7f5c07400640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bec06c920 0x7f5bec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.690+0000 7f5c07400640 1 -- 172.21.15.102:0/1650219351 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7337+0+0 (secure 0 0 0) 0x7f5bf808bf80 con 0x7f5c100c9eb0 2024-04-20T21:33:05.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.690+0000 7f5c0ea00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bec06c920 0x7f5bec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.691+0000 7f5c0ea00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bec06c920 0x7f5bec06ede0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5c00036000 tx=0x7f5c0003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.694+0000 7f5c07400640 1 -- 172.21.15.102:0/1650219351 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5c100727d0 con 0x7f5c100c9eb0 2024-04-20T21:33:05.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.936+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c100c7320 con 0x7f5c100c9eb0 2024-04-20T21:33:05.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.937+0000 7f5c07400640 1 -- 172.21.15.102:0/1650219351 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+6026 (secure 0 0 0) 0x7f5bf8059790 con 0x7f5c100c9eb0 2024-04-20T21:33:05.936 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:05.936 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:05.702823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:05.937 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 50 2024-04-20T21:33:05.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bec06c920 msgr2=0x7f5bec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bec06c920 0x7f5bec06ede0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5c00036000 tx=0x7f5c0003b040 comp rx=0 tx=0).stop 2024-04-20T21:33:05.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c100c9eb0 msgr2=0x7f5c101486e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c100c9eb0 0x7f5c101486e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800ec10 tx=0x7f5bf800c520 comp rx=0 tx=0).stop 2024-04-20T21:33:05.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 shutdown_connections 2024-04-20T21:33:05.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1014fa50 0x7f5c1014d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c100c9eb0 0x7f5c101486e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bec06c920 0x7f5bec06ede0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 --2- 172.21.15.102:0/1650219351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c10073a10 0x7f5c10148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 >> 172.21.15.102:0/1650219351 conn(0x7f5c100c9390 msgr2=0x7f5c1006ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 shutdown_connections 2024-04-20T21:33:05.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:05.940+0000 7f5c15e00640 1 -- 172.21.15.102:0/1650219351 wait complete. 2024-04-20T21:33:05.958 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-04-20T21:33:05.958 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:33:05.958 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.161+0000 7f97f3e00640 1 Processor -- start 2024-04-20T21:33:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.161+0000 7f97f3e00640 1 -- start start 2024-04-20T21:33:06.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.162+0000 7f97f3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f97f4155110 0x7f97f406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.162+0000 7f97f3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f97f406f3d0 0x7f97f415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.162+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 0x7f97f415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.162+0000 7f97f3e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f97f40c7c30 con 0x7f97f406f3d0 2024-04-20T21:33:06.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.162+0000 7f97f3e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f97f406ea00 con 0x7f97f4155110 2024-04-20T21:33:06.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.162+0000 7f97f3e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f97f415f3c0 con 0x7f97f40c4040 2024-04-20T21:33:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97ea000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f97f406f3d0 0x7f97f415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97ea000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f97f406f3d0 0x7f97f415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34310/0 (socket says 172.21.15.102:34310) 2024-04-20T21:33:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97ea000640 1 -- 172.21.15.102:0/845845988 learned_addr learned my addr 172.21.15.102:0/845845988 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97f2a00640 1 --2- 172.21.15.102:0/845845988 >> v2:172.21.15.193:3300/0 conn(0x7f97f4155110 0x7f97f406ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97f3400640 1 --2- 172.21.15.102:0/845845988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 0x7f97f415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97ea000640 1 -- 172.21.15.102:0/845845988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 msgr2=0x7f97f415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97ea000640 1 --2- 172.21.15.102:0/845845988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 0x7f97f415eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97ea000640 1 -- 172.21.15.102:0/845845988 >> v2:172.21.15.193:3300/0 conn(0x7f97f4155110 msgr2=0x7f97f406ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97ea000640 1 --2- 172.21.15.102:0/845845988 >> v2:172.21.15.193:3300/0 conn(0x7f97f4155110 0x7f97f406ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.163+0000 7f97ea000640 1 -- 172.21.15.102:0/845845988 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f406e6d0 con 0x7f97f406f3d0 2024-04-20T21:33:06.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.164+0000 7f97ea000640 1 --2- 172.21.15.102:0/845845988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f97f406eb80 tx=0x7f97dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec40a1c35d9f023 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f2000640 1 -- 172.21.15.102:0/845845988 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc035020 con 0x7f97f406f3d0 2024-04-20T21:33:06.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f2000640 1 -- 172.21.15.102:0/845845988 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc031640 con 0x7f97f406f3d0 2024-04-20T21:33:06.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f2000640 1 -- 172.21.15.102:0/845845988 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc02d040 con 0x7f97f406f3d0 2024-04-20T21:33:06.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f3e00640 1 -- 172.21.15.102:0/845845988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 msgr2=0x7f97f415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f3e00640 1 --2- 172.21.15.102:0/845845988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f97f406eb80 tx=0x7f97dc02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f3e00640 1 -- 172.21.15.102:0/845845988 shutdown_connections 2024-04-20T21:33:06.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f3e00640 1 --2- 172.21.15.102:0/845845988 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 0x7f97f415eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f3e00640 1 --2- 172.21.15.102:0/845845988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e580 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f3e00640 1 --2- 172.21.15.102:0/845845988 >> v2:172.21.15.193:3300/0 conn(0x7f97f4155110 0x7f97f406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f3e00640 1 -- 172.21.15.102:0/845845988 >> 172.21.15.102:0/845845988 conn(0x7f97f40cc820 msgr2=0x7f97f40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.165+0000 7f97f3e00640 1 -- 172.21.15.102:0/845845988 shutdown_connections 2024-04-20T21:33:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.166+0000 7f97f3e00640 1 -- 172.21.15.102:0/845845988 wait complete. 2024-04-20T21:33:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.167+0000 7f97f3e00640 1 Processor -- start 2024-04-20T21:33:06.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.167+0000 7f97f3e00640 1 -- start start 2024-04-20T21:33:06.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 0x7f97f41e7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4155110 0x7f97f41f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f97f415f000 con 0x7f97f406f3d0 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97f40c8160 con 0x7f97f4155110 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f3e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f97f415f180 con 0x7f97f40c4040 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34318/0 (socket says 172.21.15.102:34318) 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.168+0000 7f97f2a00640 1 -- 172.21.15.102:0/3820170621 learned_addr learned my addr 172.21.15.102:0/3820170621 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97f3400640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4155110 0x7f97f41f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97ea000640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 0x7f97f41e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97f2a00640 1 -- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 msgr2=0x7f97f41e7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97f2a00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 0x7f97f41e7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97f2a00640 1 -- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4155110 msgr2=0x7f97f41f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97f2a00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4155110 0x7f97f41f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97f2a00640 1 -- 172.21.15.102:0/3820170621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f40cc500 con 0x7f97f406f3d0 2024-04-20T21:33:06.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97f3400640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4155110 0x7f97f41f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:06.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97f2a00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e240 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e8a0 tx=0x7f97e400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.169+0000 7f97ebe00640 1 -- 172.21.15.102:0/3820170621 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4017070 con 0x7f97f406f3d0 2024-04-20T21:33:06.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.170+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f40cbf40 con 0x7f97f406f3d0 2024-04-20T21:33:06.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.170+0000 7f97ebe00640 1 -- 172.21.15.102:0/3820170621 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e400c940 con 0x7f97f406f3d0 2024-04-20T21:33:06.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.170+0000 7f97ebe00640 1 -- 172.21.15.102:0/3820170621 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e400cd30 con 0x7f97f406f3d0 2024-04-20T21:33:06.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.170+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f41e4410 con 0x7f97f406f3d0 2024-04-20T21:33:06.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.171+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f404fd60 con 0x7f97f406f3d0 2024-04-20T21:33:06.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.172+0000 7f97ebe00640 1 -- 172.21.15.102:0/3820170621 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97e401a070 con 0x7f97f406f3d0 2024-04-20T21:33:06.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.173+0000 7f97ebe00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97d006c8f0 0x7f97d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.173+0000 7f97ebe00640 1 -- 172.21.15.102:0/3820170621 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f97e408c380 con 0x7f97f406f3d0 2024-04-20T21:33:06.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.173+0000 7f97ea000640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97d006c8f0 0x7f97d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.174+0000 7f97ea000640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97d006c8f0 0x7f97d006edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f97dc036000 tx=0x7f97dc03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.177+0000 7f97ebe00640 1 -- 172.21.15.102:0/3820170621 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f97e4059a10 con 0x7f97f406f3d0 2024-04-20T21:33:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.420+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97f41589b0 con 0x7f97f406f3d0 2024-04-20T21:33:06.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.421+0000 7f97ebe00640 1 -- 172.21.15.102:0/3820170621 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+6026 (secure 0 0 0) 0x7f97e405dd10 con 0x7f97f406f3d0 2024-04-20T21:33:06.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:06.421 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:05.702823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:06.422 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 50 2024-04-20T21:33:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.423+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97d006c8f0 msgr2=0x7f97d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.423+0000 7f97f3e00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97d006c8f0 0x7f97d006edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f97dc036000 tx=0x7f97dc03c040 comp rx=0 tx=0).stop 2024-04-20T21:33:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.423+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 msgr2=0x7f97f415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e240 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e8a0 tx=0x7f97e400ed60 comp rx=0 tx=0).stop 2024-04-20T21:33:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 shutdown_connections 2024-04-20T21:33:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f97d006c8f0 0x7f97d006edb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4155110 0x7f97f41f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f40c4040 0x7f97f41e7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 --2- 172.21.15.102:0/3820170621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f406f3d0 0x7f97f415e240 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 >> 172.21.15.102:0/3820170621 conn(0x7f97f40cc820 msgr2=0x7f97f4151230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 shutdown_connections 2024-04-20T21:33:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:06.424+0000 7f97f3e00640 1 -- 172.21.15.102:0/3820170621 wait complete. 2024-04-20T21:33:06.442 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 50, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 50, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5835": { "addr": "172.21.15.102:6835/2655303061", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 2655303061, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 2655303061, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5835, "incarnation": 50, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:33:05.702823+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5835 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:33:08.443 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:08.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.649+0000 7f876f200640 1 Processor -- start 2024-04-20T21:33:08.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.649+0000 7f876f200640 1 -- start start 2024-04-20T21:33:08.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.650+0000 7f876f200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f87700c68e0 0x7f877016b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.650+0000 7f876f200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87701659c0 con 0x7f87701655e0 2024-04-20T21:33:08.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.650+0000 7f876f200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87700c7df0 con 0x7f877006f0b0 2024-04-20T21:33:08.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.650+0000 7f876f200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f87700c80d0 con 0x7f87700c68e0 2024-04-20T21:33:08.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.651+0000 7f876de00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f87701655e0 0x7f877006e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48868/0 (socket says 172.21.15.102:48868) 2024-04-20T21:33:08.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.651+0000 7f876de00640 1 -- 172.21.15.102:0/192360584 learned_addr learned my addr 172.21.15.102:0/192360584 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:33:08.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.651+0000 7f8765400640 1 --2- 172.21.15.102:0/192360584 >> v2:172.21.15.193:3300/0 conn(0x7f87700c68e0 0x7f877016b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.651+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1774949839 0 0) 0x7f87701659c0 con 0x7f87701655e0 2024-04-20T21:33:08.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.880+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8754003500 con 0x7f87701655e0 2024-04-20T21:33:08.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.880+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2739533809 0 0) 0x7f87700c7df0 con 0x7f877006f0b0 2024-04-20T21:33:08.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87701659c0 con 0x7f877006f0b0 2024-04-20T21:33:08.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 784382452 0 0) 0x7f8754003500 con 0x7f87701655e0 2024-04-20T21:33:08.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 >> v2:172.21.15.193:3300/0 conn(0x7f87700c68e0 msgr2=0x7f877016b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:08.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 --2- 172.21.15.102:0/192360584 >> v2:172.21.15.193:3300/0 conn(0x7f87700c68e0 0x7f877016b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 >> v1:172.21.15.193:6789/0 conn(0x7f877006f0b0 legacy=0x7f877016b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87700c3460 con 0x7f87701655e0 2024-04-20T21:33:08.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7f8760001670 con 0x7f87701655e0 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 >> v1:172.21.15.102:6789/0 conn(0x7f87701655e0 legacy=0x7f877006e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8754003b60 0x7f8754003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754008dd0 0x7f875400b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f875400b700 0x7f875400daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.881+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f87700c80d0 con 0x7f875400b700 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.882+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f875400e310 con 0x7f8754008dd0 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.882+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f875400e510 con 0x7f8754003b60 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.882+0000 7f876de00640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8754003b60 0x7f8754003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.882+0000 7f876e800640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f875400b700 0x7f875400daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.882+0000 7f8765400640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754008dd0 0x7f875400b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.883+0000 7f876de00640 1 -- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754008dd0 msgr2=0x7f875400b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:08.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.883+0000 7f876de00640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754008dd0 0x7f875400b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.883+0000 7f876de00640 1 -- 172.21.15.102:0/192360584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f875400b700 msgr2=0x7f875400daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:08.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.883+0000 7f876de00640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f875400b700 0x7f875400daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.883+0000 7f876de00640 1 -- 172.21.15.102:0/192360584 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8754003500 con 0x7f8754003b60 2024-04-20T21:33:08.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.883+0000 7f876e800640 1 -- 172.21.15.102:0/192360584 reap_dead start 2024-04-20T21:33:08.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.883+0000 7f8765400640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754008dd0 0x7f875400b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:08.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.885+0000 7f876de00640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8754003b60 0x7f8754003f60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8760007fc0 tx=0x7f8754008d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83daa406ff28240 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:08.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.886+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760007be0 con 0x7f8754003b60 2024-04-20T21:33:08.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.886+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8760030070 con 0x7f8754003b60 2024-04-20T21:33:08.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.886+0000 7f876d400640 1 -- 172.21.15.102:0/192360584 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760005d70 con 0x7f8754003b60 2024-04-20T21:33:08.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.888+0000 7f876f200640 1 -- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8754003b60 msgr2=0x7f8754003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.888+0000 7f876f200640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8754003b60 0x7f8754003f60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8760007fc0 tx=0x7f8754008d60 comp rx=0 tx=0).stop 2024-04-20T21:33:08.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.888+0000 7f876f200640 1 -- 172.21.15.102:0/192360584 shutdown_connections 2024-04-20T21:33:08.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.888+0000 7f876f200640 1 --2- 172.21.15.102:0/192360584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8754003b60 0x7f8754003f60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.888+0000 7f876f200640 1 -- 172.21.15.102:0/192360584 >> 172.21.15.102:0/192360584 conn(0x7f87700cc820 msgr2=0x7f87700ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:08.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.888+0000 7f876f200640 1 -- 172.21.15.102:0/192360584 shutdown_connections 2024-04-20T21:33:08.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.888+0000 7f876f200640 1 -- 172.21.15.102:0/192360584 wait complete. 2024-04-20T21:33:08.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.893+0000 7f876f200640 1 Processor -- start 2024-04-20T21:33:08.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.893+0000 7f876f200640 1 -- start start 2024-04-20T21:33:08.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754003b60 0x7f87701597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877015a520 0x7f87701f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f877015b4f0 0x7f877008c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f877006c2a0 con 0x7f877015a520 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f877006c120 con 0x7f8754003b60 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754003b60 0x7f87701597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754003b60 0x7f87701597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:48976/0 (socket says 172.21.15.102:48976) 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876de00640 1 -- 172.21.15.102:0/489788547 learned_addr learned my addr 172.21.15.102:0/489788547 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f8765400640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877015a520 0x7f87701f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.896+0000 7f876e800640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f877015b4f0 0x7f877008c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.897+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f87700c8600 con 0x7f877015b4f0 2024-04-20T21:33:08.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.897+0000 7f876e800640 1 -- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754003b60 msgr2=0x7f87701597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.897+0000 7f876e800640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754003b60 0x7f87701597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.897+0000 7f876e800640 1 -- 172.21.15.102:0/489788547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877015a520 msgr2=0x7f87701f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.897+0000 7f876e800640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877015a520 0x7f87701f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.897+0000 7f876e800640 1 -- 172.21.15.102:0/489788547 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87701fe770 con 0x7f877015b4f0 2024-04-20T21:33:08.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.898+0000 7f876e800640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f877015b4f0 0x7f877008c8c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87700c5fb0 tx=0x7f876800ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:08.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.898+0000 7f8766a00640 1 -- 172.21.15.102:0/489788547 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876801a070 con 0x7f877015b4f0 2024-04-20T21:33:08.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.898+0000 7f8766a00640 1 -- 172.21.15.102:0/489788547 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768003c40 con 0x7f877015b4f0 2024-04-20T21:33:08.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.898+0000 7f8766a00640 1 -- 172.21.15.102:0/489788547 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768005070 con 0x7f877015b4f0 2024-04-20T21:33:08.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.898+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87701fe9e0 con 0x7f877015b4f0 2024-04-20T21:33:08.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.898+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8770158ff0 con 0x7f877015b4f0 2024-04-20T21:33:08.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.900+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f877004fd60 con 0x7f877015b4f0 2024-04-20T21:33:08.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.901+0000 7f8766a00640 1 -- 172.21.15.102:0/489788547 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8768005210 con 0x7f877015b4f0 2024-04-20T21:33:08.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.902+0000 7f8766a00640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874c06c8f0 0x7f874c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.902+0000 7f8766a00640 1 -- 172.21.15.102:0/489788547 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f876808b820 con 0x7f877015b4f0 2024-04-20T21:33:08.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.902+0000 7f876de00640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874c06c8f0 0x7f874c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.903+0000 7f876de00640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874c06c8f0 0x7f874c06edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8760000c00 tx=0x7f8760009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:08.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:08.907+0000 7f8766a00640 1 -- 172.21.15.102:0/489788547 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8768058eb0 con 0x7f877015b4f0 2024-04-20T21:33:09.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.151+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87700c66c0 con 0x7f877015b4f0 2024-04-20T21:33:09.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.152+0000 7f8766a00640 1 -- 172.21.15.102:0/489788547 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+6026 (secure 0 0 0) 0x7f876805d1b0 con 0x7f877015b4f0 2024-04-20T21:33:09.152 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:09.152 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:05.702823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:09.152 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 50 2024-04-20T21:33:09.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874c06c8f0 msgr2=0x7f874c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874c06c8f0 0x7f874c06edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8760000c00 tx=0x7f8760009030 comp rx=0 tx=0).stop 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f877015b4f0 msgr2=0x7f877008c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f877015b4f0 0x7f877008c8c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87700c5fb0 tx=0x7f876800ff70 comp rx=0 tx=0).stop 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 shutdown_connections 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f877015b4f0 0x7f877008c8c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877015a520 0x7f87701f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f874c06c8f0 0x7f874c06edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 --2- 172.21.15.102:0/489788547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8754003b60 0x7f87701597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 >> 172.21.15.102:0/489788547 conn(0x7f87700cc820 msgr2=0x7f8770160d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 shutdown_connections 2024-04-20T21:33:09.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:09.155+0000 7f876f200640 1 -- 172.21.15.102:0/489788547 wait complete. 2024-04-20T21:33:09.173 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:11.174 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:11.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.364+0000 7f5a85a00640 1 Processor -- start 2024-04-20T21:33:11.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.365+0000 7f5a85a00640 1 -- start start 2024-04-20T21:33:11.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a85a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a85a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5a800c75a0 0x7f5a801637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a85a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a80153560 con 0x7f5a80159380 2024-04-20T21:33:11.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a85a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5a800c7a70 con 0x7f5a800c75a0 2024-04-20T21:33:11.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a85a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5a800c7d30 con 0x7f5a80153180 2024-04-20T21:33:11.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a7f400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5a80159380 0x7f5a800c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48878/0 (socket says 172.21.15.102:48878) 2024-04-20T21:33:11.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a7f400640 1 -- 172.21.15.102:0/1073363167 learned_addr learned my addr 172.21.15.102:0/1073363167 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:33:11.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a7fe00640 1 --2- 172.21.15.102:0/1073363167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.366+0000 7f5a85000640 1 --2- 172.21.15.102:0/1073363167 >> v2:172.21.15.102:3300/0 conn(0x7f5a800c75a0 0x7f5a801637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.367+0000 7f5a7fe00640 1 -- 172.21.15.102:0/1073363167 >> v2:172.21.15.102:3300/0 conn(0x7f5a800c75a0 msgr2=0x7f5a801637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:11.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.367+0000 7f5a7fe00640 1 --2- 172.21.15.102:0/1073363167 >> v2:172.21.15.102:3300/0 conn(0x7f5a800c75a0 0x7f5a801637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.367+0000 7f5a7fe00640 1 -- 172.21.15.102:0/1073363167 >> v1:172.21.15.102:6789/0 conn(0x7f5a80159380 legacy=0x7f5a800c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.367+0000 7f5a7fe00640 1 -- 172.21.15.102:0/1073363167 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8015c4d0 con 0x7f5a80153180 2024-04-20T21:33:11.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.367+0000 7f5a7ea00640 1 -- 172.21.15.102:0/1073363167 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2600974129 0 0) 0x7f5a80153560 con 0x7f5a80159380 2024-04-20T21:33:11.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.369+0000 7f5a7fe00640 1 --2- 172.21.15.102:0/1073363167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80158e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5a700097a0 tx=0x7f5a7002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0013f7383e7ded server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:11.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.369+0000 7f5a7ea00640 1 -- 172.21.15.102:0/1073363167 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70035020 con 0x7f5a80153180 2024-04-20T21:33:11.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.369+0000 7f5a7ea00640 1 -- 172.21.15.102:0/1073363167 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a70031810 con 0x7f5a80153180 2024-04-20T21:33:11.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.369+0000 7f5a7ea00640 1 -- 172.21.15.102:0/1073363167 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70031be0 con 0x7f5a80153180 2024-04-20T21:33:11.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.370+0000 7f5a85a00640 1 -- 172.21.15.102:0/1073363167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 msgr2=0x7f5a80158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.370+0000 7f5a85a00640 1 --2- 172.21.15.102:0/1073363167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80158e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5a700097a0 tx=0x7f5a7002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.370+0000 7f5a85a00640 1 -- 172.21.15.102:0/1073363167 shutdown_connections 2024-04-20T21:33:11.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.370+0000 7f5a85a00640 1 --2- 172.21.15.102:0/1073363167 >> v2:172.21.15.102:3300/0 conn(0x7f5a800c75a0 0x7f5a801637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.370+0000 7f5a85a00640 1 --2- 172.21.15.102:0/1073363167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80158e40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.370+0000 7f5a85a00640 1 -- 172.21.15.102:0/1073363167 >> 172.21.15.102:0/1073363167 conn(0x7f5a800cc820 msgr2=0x7f5a800ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:11.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.370+0000 7f5a85a00640 1 -- 172.21.15.102:0/1073363167 shutdown_connections 2024-04-20T21:33:11.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.370+0000 7f5a85a00640 1 -- 172.21.15.102:0/1073363167 wait complete. 2024-04-20T21:33:11.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.372+0000 7f5a85a00640 1 Processor -- start 2024-04-20T21:33:11.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.372+0000 7f5a85a00640 1 -- start start 2024-04-20T21:33:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a85a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a800c75a0 0x7f5a8015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a85a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a85a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a80159380 0x7f5a801472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a85a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8015d290 con 0x7f5a800c75a0 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a85a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8015ca90 con 0x7f5a80159380 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a85a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5a800c6be0 con 0x7f5a80153180 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a800c75a0 0x7f5a8015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a800c75a0 0x7f5a8015b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53930/0 (socket says 172.21.15.102:53930) 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a7fe00640 1 -- 172.21.15.102:0/165921112 learned_addr learned my addr 172.21.15.102:0/165921112 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a7f400640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.373+0000 7f5a85000640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a80159380 0x7f5a801472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a7fe00640 1 -- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 msgr2=0x7f5a80146990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a7fe00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80146990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a7fe00640 1 -- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a80159380 msgr2=0x7f5a801472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a7fe00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a80159380 0x7f5a801472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a7fe00640 1 -- 172.21.15.102:0/165921112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8015a110 con 0x7f5a800c75a0 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a7f400640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a7fe00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a800c75a0 0x7f5a8015b450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5a70034040 tx=0x7f5a700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a77e00640 1 -- 172.21.15.102:0/165921112 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70041070 con 0x7f5a800c75a0 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a77e00640 1 -- 172.21.15.102:0/165921112 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a7003ac80 con 0x7f5a800c75a0 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a77e00640 1 -- 172.21.15.102:0/165921112 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70035040 con 0x7f5a800c75a0 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.374+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8015a3b0 con 0x7f5a800c75a0 2024-04-20T21:33:11.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.375+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8015a970 con 0x7f5a800c75a0 2024-04-20T21:33:11.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.376+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8004fd60 con 0x7f5a800c75a0 2024-04-20T21:33:11.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.378+0000 7f5a77e00640 1 -- 172.21.15.102:0/165921112 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a70038020 con 0x7f5a800c75a0 2024-04-20T21:33:11.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.379+0000 7f5a77e00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a6006c920 0x7f5a6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.379+0000 7f5a77e00640 1 -- 172.21.15.102:0/165921112 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f5a700adc30 con 0x7f5a800c75a0 2024-04-20T21:33:11.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.379+0000 7f5a7f400640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a6006c920 0x7f5a6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.380+0000 7f5a7f400640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a6006c920 0x7f5a6006ede0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a8015be60 tx=0x7f5a6c007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:11.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.388+0000 7f5a77e00640 1 -- 172.21.15.102:0/165921112 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5a7007b360 con 0x7f5a800c75a0 2024-04-20T21:33:11.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.631+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a80149240 con 0x7f5a800c75a0 2024-04-20T21:33:11.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.632+0000 7f5a77e00640 1 -- 172.21.15.102:0/165921112 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+6026 (secure 0 0 0) 0x7f5a7007f660 con 0x7f5a800c75a0 2024-04-20T21:33:11.631 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:05.702823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:11.632 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 50 2024-04-20T21:33:11.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.635+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a6006c920 msgr2=0x7f5a6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.635+0000 7f5a85a00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a6006c920 0x7f5a6006ede0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a8015be60 tx=0x7f5a6c007d50 comp rx=0 tx=0).stop 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.635+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a800c75a0 msgr2=0x7f5a8015b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.635+0000 7f5a85a00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a800c75a0 0x7f5a8015b450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5a70034040 tx=0x7f5a700046c0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.636+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 shutdown_connections 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.636+0000 7f5a85a00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5a6006c920 0x7f5a6006ede0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.636+0000 7f5a85a00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a80159380 0x7f5a801472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.636+0000 7f5a85a00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a80153180 0x7f5a80146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.636+0000 7f5a85a00640 1 --2- 172.21.15.102:0/165921112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a800c75a0 0x7f5a8015b450 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.636+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 >> 172.21.15.102:0/165921112 conn(0x7f5a800cc820 msgr2=0x7f5a80073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.636+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 shutdown_connections 2024-04-20T21:33:11.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:11.637+0000 7f5a85a00640 1 -- 172.21.15.102:0/165921112 wait complete. 2024-04-20T21:33:11.653 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:13.654 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:13.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.836+0000 7fdaa7e00640 1 Processor -- start 2024-04-20T21:33:13.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.836+0000 7fdaa7e00640 1 -- start start 2024-04-20T21:33:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa8155530 0x7fdaa815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa7e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdaa80cb6a0 0x7fdaa80cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa7e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa815ef60 con 0x7fdaa80cc470 2024-04-20T21:33:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa7e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa80c5ca0 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa7e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa80cd5f0 con 0x7fdaa8155530 2024-04-20T21:33:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa6000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdaa80cc470 0x7fdaa80cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48892/0 (socket says 172.21.15.102:48892) 2024-04-20T21:33:13.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa6000640 1 -- 172.21.15.102:0/1806246740 learned_addr learned my addr 172.21.15.102:0/1806246740 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:33:13.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa6a00640 1 --2- 172.21.15.102:0/1806246740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa8155530 0x7fdaa815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.838+0000 7fdaa7400640 1 --2- 172.21.15.102:0/1806246740 >> v2:172.21.15.193:3300/0 conn(0x7fdaa80cb6a0 0x7fdaa80cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.839+0000 7fdaa5600640 1 -- 172.21.15.102:0/1806246740 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919215278 0 0) 0x7fdaa815ef60 con 0x7fdaa80cc470 2024-04-20T21:33:13.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.839+0000 7fdaa6a00640 1 -- 172.21.15.102:0/1806246740 >> v2:172.21.15.193:3300/0 conn(0x7fdaa80cb6a0 msgr2=0x7fdaa80cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:13.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.839+0000 7fdaa6a00640 1 --2- 172.21.15.102:0/1806246740 >> v2:172.21.15.193:3300/0 conn(0x7fdaa80cb6a0 0x7fdaa80cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.839+0000 7fdaa6a00640 1 -- 172.21.15.102:0/1806246740 >> v1:172.21.15.102:6789/0 conn(0x7fdaa80cc470 legacy=0x7fdaa80cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.839+0000 7fdaa6a00640 1 -- 172.21.15.102:0/1806246740 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa815f240 con 0x7fdaa8155530 2024-04-20T21:33:13.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.839+0000 7fdaa7400640 1 --2- 172.21.15.102:0/1806246740 >> v2:172.21.15.193:3300/0 conn(0x7fdaa80cb6a0 0x7fdaa80cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:13.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.840+0000 7fdaa6a00640 1 --2- 172.21.15.102:0/1806246740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa8155530 0x7fdaa815e640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda980097a0 tx=0x7fda9802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0610dbc0617de0b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:13.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.840+0000 7fdaa5600640 1 -- 172.21.15.102:0/1806246740 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98034070 con 0x7fdaa8155530 2024-04-20T21:33:13.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.840+0000 7fdaa5600640 1 -- 172.21.15.102:0/1806246740 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda98030800 con 0x7fdaa8155530 2024-04-20T21:33:13.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.840+0000 7fdaa5600640 1 -- 172.21.15.102:0/1806246740 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98030bd0 con 0x7fdaa8155530 2024-04-20T21:33:13.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.841+0000 7fdaa7e00640 1 -- 172.21.15.102:0/1806246740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa8155530 msgr2=0x7fdaa815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.841+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/1806246740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa8155530 0x7fdaa815e640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda980097a0 tx=0x7fda9802b940 comp rx=0 tx=0).stop 2024-04-20T21:33:13.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.841+0000 7fdaa7e00640 1 -- 172.21.15.102:0/1806246740 shutdown_connections 2024-04-20T21:33:13.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.841+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/1806246740 >> v2:172.21.15.193:3300/0 conn(0x7fdaa80cb6a0 0x7fdaa80cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.841+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/1806246740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa8155530 0x7fdaa815e640 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.841+0000 7fdaa7e00640 1 -- 172.21.15.102:0/1806246740 >> 172.21.15.102:0/1806246740 conn(0x7fdaa8075080 msgr2=0x7fdaa8075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:13.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.841+0000 7fdaa7e00640 1 -- 172.21.15.102:0/1806246740 shutdown_connections 2024-04-20T21:33:13.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.841+0000 7fdaa7e00640 1 -- 172.21.15.102:0/1806246740 wait complete. 2024-04-20T21:33:13.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.843+0000 7fdaa7e00640 1 Processor -- start 2024-04-20T21:33:13.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.843+0000 7fdaa7e00640 1 -- start start 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa80cb6a0 0x7fdaa80cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa80cc470 0x7fdaa815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8155530 0x7fdaa81581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80cd280 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80ccfa0 con 0x7fdaa8155530 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa7e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa80ccbe0 con 0x7fdaa80cc470 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa80cb6a0 0x7fdaa80cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa80cb6a0 0x7fdaa80cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53940/0 (socket says 172.21.15.102:53940) 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6a00640 1 -- 172.21.15.102:0/302753347 learned_addr learned my addr 172.21.15.102:0/302753347 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6000640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa80cc470 0x7fdaa815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa7400640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8155530 0x7fdaa81581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6a00640 1 -- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa80cc470 msgr2=0x7fdaa815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6a00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa80cc470 0x7fdaa815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6a00640 1 -- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8155530 msgr2=0x7fdaa81581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6a00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8155530 0x7fdaa81581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.844+0000 7fdaa6a00640 1 -- 172.21.15.102:0/302753347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa80c94d0 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.845+0000 7fdaa6a00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa80cb6a0 0x7fdaa80cfbc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fda9802bf50 tx=0x7fda980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:13.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.845+0000 7fda9f400640 1 -- 172.21.15.102:0/302753347 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98034040 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.845+0000 7fda9f400640 1 -- 172.21.15.102:0/302753347 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9803ada0 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.845+0000 7fda9f400640 1 -- 172.21.15.102:0/302753347 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98006e10 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.845+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa815c5c0 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.845+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa815cb80 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.847+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa804fd60 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.847+0000 7fda9f400640 1 -- 172.21.15.102:0/302753347 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda9804b060 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.848+0000 7fda9f400640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda8806c8f0 0x7fda8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.848+0000 7fda9f400640 1 -- 172.21.15.102:0/302753347 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7fda980ae160 con 0x7fdaa80cb6a0 2024-04-20T21:33:13.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.849+0000 7fdaa6000640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda8806c8f0 0x7fda8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.849+0000 7fdaa6000640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda8806c8f0 0x7fda8806edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80b5420 tx=0x7fda94007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:13.853+0000 7fda9f400640 1 -- 172.21.15.102:0/302753347 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fda9803f070 con 0x7fdaa80cb6a0 2024-04-20T21:33:14.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.146+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaa80b6c50 con 0x7fdaa80cb6a0 2024-04-20T21:33:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.147+0000 7fda9f400640 1 -- 172.21.15.102:0/302753347 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+6026 (secure 0 0 0) 0x7fda9807bdb0 con 0x7fdaa80cb6a0 2024-04-20T21:33:14.146 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:05.702823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 50 2024-04-20T21:33:14.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda8806c8f0 msgr2=0x7fda8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda8806c8f0 0x7fda8806edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80b5420 tx=0x7fda94007d50 comp rx=0 tx=0).stop 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa80cb6a0 msgr2=0x7fdaa80cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa80cb6a0 0x7fdaa80cfbc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fda9802bf50 tx=0x7fda980046c0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 shutdown_connections 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fda8806c8f0 0x7fda8806edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8155530 0x7fdaa81581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdaa80cc470 0x7fdaa815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 --2- 172.21.15.102:0/302753347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa80cb6a0 0x7fdaa80cfbc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.150+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 >> 172.21.15.102:0/302753347 conn(0x7fdaa8075080 msgr2=0x7fdaa8075460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.151+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 shutdown_connections 2024-04-20T21:33:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:14.151+0000 7fdaa7e00640 1 -- 172.21.15.102:0/302753347 wait complete. 2024-04-20T21:33:14.173 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:16.174 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:16.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.349+0000 7f0167200640 1 Processor -- start 2024-04-20T21:33:16.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.349+0000 7f0167200640 1 -- start start 2024-04-20T21:33:16.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.351+0000 7f0167200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0160155530 0x7f016015e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.351+0000 7f0167200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016015ef60 con 0x7f01600cb6a0 2024-04-20T21:33:16.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.351+0000 7f0167200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016015f240 con 0x7f01600cc470 2024-04-20T21:33:16.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.351+0000 7f0167200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f01600c5ca0 con 0x7f0160155530 2024-04-20T21:33:16.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.351+0000 7f0166800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f01600cb6a0 0x7f01601659d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48894/0 (socket says 172.21.15.102:48894) 2024-04-20T21:33:16.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.351+0000 7f0165e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0160155530 0x7f016015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.351+0000 7f0166800640 1 -- 172.21.15.102:0/1669493567 learned_addr learned my addr 172.21.15.102:0/1669493567 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:33:16.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.352+0000 7f0165e00640 1 -- 172.21.15.102:0/1669493567 >> v1:172.21.15.193:6789/0 conn(0x7f01600cc470 legacy=0x7f01600cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.352+0000 7f0165e00640 1 -- 172.21.15.102:0/1669493567 >> v1:172.21.15.102:6789/0 conn(0x7f01600cb6a0 legacy=0x7f01601659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.352+0000 7f0164a00640 1 -- 172.21.15.102:0/1669493567 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532565683 0 0) 0x7f016015f240 con 0x7f01600cc470 2024-04-20T21:33:16.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.352+0000 7f0165e00640 1 -- 172.21.15.102:0/1669493567 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016015f520 con 0x7f0160155530 2024-04-20T21:33:16.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.353+0000 7f0165e00640 1 --2- 172.21.15.102:0/1669493567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f016015e640 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f01540097a0 tx=0x7f015402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4863232da7d621b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:16.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.353+0000 7f0164a00640 1 -- 172.21.15.102:0/1669493567 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154037070 con 0x7f0160155530 2024-04-20T21:33:16.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.353+0000 7f0164a00640 1 -- 172.21.15.102:0/1669493567 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0154030710 con 0x7f0160155530 2024-04-20T21:33:16.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.353+0000 7f0164a00640 1 -- 172.21.15.102:0/1669493567 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154030ae0 con 0x7f0160155530 2024-04-20T21:33:16.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.354+0000 7f0167200640 1 -- 172.21.15.102:0/1669493567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 msgr2=0x7f016015e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.354+0000 7f0167200640 1 --2- 172.21.15.102:0/1669493567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f016015e640 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f01540097a0 tx=0x7f015402b790 comp rx=0 tx=0).stop 2024-04-20T21:33:16.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.354+0000 7f0167200640 1 -- 172.21.15.102:0/1669493567 shutdown_connections 2024-04-20T21:33:16.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.354+0000 7f0167200640 1 --2- 172.21.15.102:0/1669493567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f016015e640 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.354+0000 7f0167200640 1 -- 172.21.15.102:0/1669493567 >> 172.21.15.102:0/1669493567 conn(0x7f0160075080 msgr2=0x7f0160075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:16.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.354+0000 7f0167200640 1 -- 172.21.15.102:0/1669493567 shutdown_connections 2024-04-20T21:33:16.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.354+0000 7f0167200640 1 -- 172.21.15.102:0/1669493567 wait complete. 2024-04-20T21:33:16.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.356+0000 7f0167200640 1 Processor -- start 2024-04-20T21:33:16.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.357+0000 7f0167200640 1 -- start start 2024-04-20T21:33:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0167200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01600cb6a0 0x7f01601f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0167200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01600cc470 0x7f01601e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0167200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f01600ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0167200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f01600cd280 con 0x7f0160155530 2024-04-20T21:33:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0167200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f01600cd5f0 con 0x7f01600cb6a0 2024-04-20T21:33:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0167200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f01600ccbe0 con 0x7f01600cc470 2024-04-20T21:33:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0166800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f01600ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0166800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f01600ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53966/0 (socket says 172.21.15.102:53966) 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0166800640 1 -- 172.21.15.102:0/181160974 learned_addr learned my addr 172.21.15.102:0/181160974 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0165e00640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01600cb6a0 0x7f01601f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.358+0000 7f0165400640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01600cc470 0x7f01601e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f0166800640 1 -- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01600cc470 msgr2=0x7f01601e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f0166800640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01600cc470 0x7f01601e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f0166800640 1 -- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01600cb6a0 msgr2=0x7f01601f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f0166800640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01600cb6a0 0x7f01601f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f0166800640 1 -- 172.21.15.102:0/181160974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0160148ba0 con 0x7f0160155530 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f0166800640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f01600ca450 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f015c00caf0 tx=0x7f015c0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f015a000640 1 -- 172.21.15.102:0/181160974 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c018070 con 0x7f0160155530 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f015a000640 1 -- 172.21.15.102:0/181160974 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c00a760 con 0x7f0160155530 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f015a000640 1 -- 172.21.15.102:0/181160974 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c00ab90 con 0x7f0160155530 2024-04-20T21:33:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.359+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0160148e80 con 0x7f0160155530 2024-04-20T21:33:16.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.360+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0160149440 con 0x7f0160155530 2024-04-20T21:33:16.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.361+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0160074340 con 0x7f0160155530 2024-04-20T21:33:16.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.362+0000 7f015a000640 1 -- 172.21.15.102:0/181160974 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f015c01b070 con 0x7f0160155530 2024-04-20T21:33:16.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.362+0000 7f015a000640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f014406c8f0 0x7f014406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.363+0000 7f015a000640 1 -- 172.21.15.102:0/181160974 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f015c08ba70 con 0x7f0160155530 2024-04-20T21:33:16.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.363+0000 7f0165e00640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f014406c8f0 0x7f014406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.363+0000 7f0165e00640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f014406c8f0 0x7f014406edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0154034000 tx=0x7f015402f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:16.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.367+0000 7f015a000640 1 -- 172.21.15.102:0/181160974 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0160074340 con 0x7f0160155530 2024-04-20T21:33:16.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.604+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f016004fd60 con 0x7f0160155530 2024-04-20T21:33:16.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.605+0000 7f015a000640 1 -- 172.21.15.102:0/181160974 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+6026 (secure 0 0 0) 0x7f015c059070 con 0x7f0160155530 2024-04-20T21:33:16.604 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:05.702823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:16.605 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 50 2024-04-20T21:33:16.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f014406c8f0 msgr2=0x7f014406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f014406c8f0 0x7f014406edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0154034000 tx=0x7f015402f000 comp rx=0 tx=0).stop 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 msgr2=0x7f01600ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f01600ca450 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f015c00caf0 tx=0x7f015c0020f0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 shutdown_connections 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0160155530 0x7f01600ca450 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01600cc470 0x7f01601e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f014406c8f0 0x7f014406edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 --2- 172.21.15.102:0/181160974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01600cb6a0 0x7f01601f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 >> 172.21.15.102:0/181160974 conn(0x7f0160075080 msgr2=0x7f01600cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.607+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 shutdown_connections 2024-04-20T21:33:16.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:16.608+0000 7f0167200640 1 -- 172.21.15.102:0/181160974 wait complete. 2024-04-20T21:33:16.622 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:18.623 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:18.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.830+0000 7fc8f7e00640 1 Processor -- start 2024-04-20T21:33:18.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.830+0000 7fc8f7e00640 1 -- start start 2024-04-20T21:33:18.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8f014fa50 0x7fc8f00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:18.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f7e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc8f00c9eb0 0x7fc8f0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:18.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f0073a10 0x7fc8f00c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:18.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f7e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f0159500 con 0x7fc8f014fa50 2024-04-20T21:33:18.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f7e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f01536d0 con 0x7fc8f00c9eb0 2024-04-20T21:33:18.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f7e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f00c76c0 con 0x7fc8f0073a10 2024-04-20T21:33:18.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f5400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8f014fa50 0x7fc8f00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:18.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f4a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc8f00c9eb0 0x7fc8f0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:18.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f4a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc8f00c9eb0 0x7fc8f0158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:47882/0 (socket says 172.21.15.102:47882) 2024-04-20T21:33:18.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f4a00640 1 -- 172.21.15.102:0/2866958923 learned_addr learned my addr 172.21.15.102:0/2866958923 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:18.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.832+0000 7fc8f5e00640 1 --2- 172.21.15.102:0/2866958923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f0073a10 0x7fc8f00c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:18.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.833+0000 7fc8f4a00640 1 -- 172.21.15.102:0/2866958923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f0073a10 msgr2=0x7fc8f00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:18.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.833+0000 7fc8f4a00640 1 --2- 172.21.15.102:0/2866958923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f0073a10 0x7fc8f00c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:18.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.833+0000 7fc8f4a00640 1 -- 172.21.15.102:0/2866958923 >> v2:172.21.15.102:3300/0 conn(0x7fc8f014fa50 msgr2=0x7fc8f00c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:18.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.833+0000 7fc8f4a00640 1 --2- 172.21.15.102:0/2866958923 >> v2:172.21.15.102:3300/0 conn(0x7fc8f014fa50 0x7fc8f00c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:18.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.833+0000 7fc8f4a00640 1 -- 172.21.15.102:0/2866958923 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f00c7960 con 0x7fc8f00c9eb0 2024-04-20T21:33:18.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.834+0000 7fc8f4a00640 1 --2- 172.21.15.102:0/2866958923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f00c9eb0 0x7fc8f0158fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00c7b30 tx=0x7fc8e002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb449f41bf579c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:18.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.834+0000 7fc8efe00640 1 -- 172.21.15.102:0/2866958923 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e002caf0 con 0x7fc8f00c9eb0 2024-04-20T21:33:18.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.834+0000 7fc8efe00640 1 -- 172.21.15.102:0/2866958923 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e002cc90 con 0x7fc8f00c9eb0 2024-04-20T21:33:18.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.834+0000 7fc8efe00640 1 -- 172.21.15.102:0/2866958923 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0034610 con 0x7fc8f00c9eb0 2024-04-20T21:33:18.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 -- 172.21.15.102:0/2866958923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f00c9eb0 msgr2=0x7fc8f0158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:18.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/2866958923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f00c9eb0 0x7fc8f0158fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00c7b30 tx=0x7fc8e002bf50 comp rx=0 tx=0).stop 2024-04-20T21:33:18.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 -- 172.21.15.102:0/2866958923 shutdown_connections 2024-04-20T21:33:18.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/2866958923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f0073a10 0x7fc8f00c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:18.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/2866958923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f00c9eb0 0x7fc8f0158fc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:18.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/2866958923 >> v2:172.21.15.102:3300/0 conn(0x7fc8f014fa50 0x7fc8f00c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:18.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 -- 172.21.15.102:0/2866958923 >> 172.21.15.102:0/2866958923 conn(0x7fc8f00c9390 msgr2=0x7fc8f00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:18.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 -- 172.21.15.102:0/2866958923 shutdown_connections 2024-04-20T21:33:18.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.835+0000 7fc8f7e00640 1 -- 172.21.15.102:0/2866958923 wait complete. 2024-04-20T21:33:18.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.837+0000 7fc8f7e00640 1 Processor -- start 2024-04-20T21:33:18.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.837+0000 7fc8f7e00640 1 -- start start 2024-04-20T21:33:18.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0073a10 0x7fc8f01f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f00c9eb0 0x7fc8f014a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8f014fa50 0x7fc8f0144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f00c7c00 con 0x7fc8f014fa50 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f00c3d90 con 0x7fc8f0073a10 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f7e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f00c4030 con 0x7fc8f00c9eb0 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8f014fa50 0x7fc8f0144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8f014fa50 0x7fc8f0144040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34122/0 (socket says 172.21.15.102:34122) 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f5e00640 1 -- 172.21.15.102:0/905555018 learned_addr learned my addr 172.21.15.102:0/905555018 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f5400640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0073a10 0x7fc8f01f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f4a00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f00c9eb0 0x7fc8f014a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f5e00640 1 -- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f00c9eb0 msgr2=0x7fc8f014a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:18.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.838+0000 7fc8f5e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f00c9eb0 0x7fc8f014a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:18.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8f5e00640 1 -- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0073a10 msgr2=0x7fc8f01f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:18.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8f5e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0073a10 0x7fc8f01f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:18.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8f5e00640 1 -- 172.21.15.102:0/905555018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f00cb950 con 0x7fc8f014fa50 2024-04-20T21:33:18.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8f5e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8f014fa50 0x7fc8f0144040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800d800 tx=0x7fc8e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:18.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8ed600640 1 -- 172.21.15.102:0/905555018 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8018020 con 0x7fc8f014fa50 2024-04-20T21:33:18.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8ed600640 1 -- 172.21.15.102:0/905555018 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e80096c0 con 0x7fc8f014fa50 2024-04-20T21:33:18.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f00cbbf0 con 0x7fc8f014fa50 2024-04-20T21:33:18.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8ed600640 1 -- 172.21.15.102:0/905555018 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8009a90 con 0x7fc8f014fa50 2024-04-20T21:33:18.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.839+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f015d3b0 con 0x7fc8f014fa50 2024-04-20T21:33:18.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.840+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f0074340 con 0x7fc8f014fa50 2024-04-20T21:33:18.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.841+0000 7fc8ed600640 1 -- 172.21.15.102:0/905555018 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8e801b020 con 0x7fc8f014fa50 2024-04-20T21:33:18.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.843+0000 7fc8ed600640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8d406c8f0 0x7fc8d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:18.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.843+0000 7fc8ed600640 1 -- 172.21.15.102:0/905555018 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7fc8e808c660 con 0x7fc8f014fa50 2024-04-20T21:33:18.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.843+0000 7fc8f5400640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8d406c8f0 0x7fc8d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:18.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.844+0000 7fc8f5400640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8d406c8f0 0x7fc8d406edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009aa0 tx=0x7fc8e4009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:18.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:18.847+0000 7fc8ed600640 1 -- 172.21.15.102:0/905555018 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc8e8022020 con 0x7fc8f014fa50 2024-04-20T21:33:19.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.088+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8f00c7320 con 0x7fc8f014fa50 2024-04-20T21:33:19.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.089+0000 7fc8ed600640 1 -- 172.21.15.102:0/905555018 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+6030 (secure 0 0 0) 0x7fc8e8059cf0 con 0x7fc8f014fa50 2024-04-20T21:33:19.088 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:19.089 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:16.559653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:19.089 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 51 2024-04-20T21:33:19.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8d406c8f0 msgr2=0x7fc8d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:19.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8d406c8f0 0x7fc8d406edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009aa0 tx=0x7fc8e4009190 comp rx=0 tx=0).stop 2024-04-20T21:33:19.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8f014fa50 msgr2=0x7fc8f0144040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:19.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8f014fa50 0x7fc8f0144040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800d800 tx=0x7fc8e800dcc0 comp rx=0 tx=0).stop 2024-04-20T21:33:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 shutdown_connections 2024-04-20T21:33:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8f014fa50 0x7fc8f0144040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f00c9eb0 0x7fc8f014a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8d406c8f0 0x7fc8d406edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 --2- 172.21.15.102:0/905555018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f0073a10 0x7fc8f01f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 >> 172.21.15.102:0/905555018 conn(0x7fc8f00c9390 msgr2=0x7fc8f006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 shutdown_connections 2024-04-20T21:33:19.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:19.093+0000 7fc8f7e00640 1 -- 172.21.15.102:0/905555018 wait complete. 2024-04-20T21:33:19.111 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 51, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 51, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5835": { "addr": "172.21.15.102:6835/2655303061", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 2655303061, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 2655303061, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5835, "incarnation": 50, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:33:16.559653+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5835 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:33:21.112 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:21.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.320+0000 7fcec0000640 1 Processor -- start 2024-04-20T21:33:21.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.321+0000 7fcec0000640 1 -- start start 2024-04-20T21:33:21.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.322+0000 7fcec0000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fceb80c9eb0 0x7fceb80c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.322+0000 7fcec0000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb8073a10 0x7fceb8158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.322+0000 7fcec0000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb81536d0 con 0x7fceb814fa50 2024-04-20T21:33:21.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.322+0000 7fcec0000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fceb8159500 con 0x7fceb80c9eb0 2024-04-20T21:33:21.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.322+0000 7fcec0000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fceb80c76c0 con 0x7fceb8073a10 2024-04-20T21:33:21.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.322+0000 7fcebd600640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fceb814fa50 0x7fceb80c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46748/0 (socket says 172.21.15.102:46748) 2024-04-20T21:33:21.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.322+0000 7fcebd600640 1 -- 172.21.15.102:0/294162121 learned_addr learned my addr 172.21.15.102:0/294162121 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:33:21.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.323+0000 7fcebe000640 1 --2- 172.21.15.102:0/294162121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb8073a10 0x7fceb8158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.323+0000 7fcebcc00640 1 --2- 172.21.15.102:0/294162121 >> v2:172.21.15.193:3300/0 conn(0x7fceb80c9eb0 0x7fceb80c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.323+0000 7fceb7e00640 1 -- 172.21.15.102:0/294162121 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4066218558 0 0) 0x7fceac001110 con 0x7fceb814fa50 2024-04-20T21:33:21.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.323+0000 7fceb7e00640 1 -- 172.21.15.102:0/294162121 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcea0003520 con 0x7fceb814fa50 2024-04-20T21:33:21.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.323+0000 7fcebcc00640 1 -- 172.21.15.102:0/294162121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb8073a10 msgr2=0x7fceb8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:21.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.323+0000 7fcebcc00640 1 --2- 172.21.15.102:0/294162121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb8073a10 0x7fceb8158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.323+0000 7fcebcc00640 1 -- 172.21.15.102:0/294162121 >> v1:172.21.15.102:6789/0 conn(0x7fceb814fa50 legacy=0x7fceb80c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.323+0000 7fcebcc00640 1 -- 172.21.15.102:0/294162121 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb80c7960 con 0x7fceb80c9eb0 2024-04-20T21:33:21.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.324+0000 7fcebcc00640 1 --2- 172.21.15.102:0/294162121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb80c9eb0 0x7fceb80c4830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcea40097a0 tx=0x7fcea402b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf83c36949184aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:21.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.325+0000 7fceb7e00640 1 -- 172.21.15.102:0/294162121 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4034070 con 0x7fceb80c9eb0 2024-04-20T21:33:21.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.325+0000 7fceb7e00640 1 -- 172.21.15.102:0/294162121 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea4030800 con 0x7fceb80c9eb0 2024-04-20T21:33:21.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.325+0000 7fceb7e00640 1 -- 172.21.15.102:0/294162121 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4030bb0 con 0x7fceb80c9eb0 2024-04-20T21:33:21.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.325+0000 7fcec0000640 1 -- 172.21.15.102:0/294162121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb80c9eb0 msgr2=0x7fceb80c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.325+0000 7fcec0000640 1 --2- 172.21.15.102:0/294162121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb80c9eb0 0x7fceb80c4830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcea40097a0 tx=0x7fcea402b940 comp rx=0 tx=0).stop 2024-04-20T21:33:21.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.326+0000 7fcec0000640 1 -- 172.21.15.102:0/294162121 shutdown_connections 2024-04-20T21:33:21.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.326+0000 7fcec0000640 1 --2- 172.21.15.102:0/294162121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb8073a10 0x7fceb8158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.326+0000 7fcec0000640 1 --2- 172.21.15.102:0/294162121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb80c9eb0 0x7fceb80c4830 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.326+0000 7fcec0000640 1 -- 172.21.15.102:0/294162121 >> 172.21.15.102:0/294162121 conn(0x7fceb80c9390 msgr2=0x7fceb80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:21.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.326+0000 7fcec0000640 1 -- 172.21.15.102:0/294162121 shutdown_connections 2024-04-20T21:33:21.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.326+0000 7fcec0000640 1 -- 172.21.15.102:0/294162121 wait complete. 2024-04-20T21:33:21.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.327+0000 7fcec0000640 1 Processor -- start 2024-04-20T21:33:21.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.328+0000 7fcec0000640 1 -- start start 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.328+0000 7fcec0000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb8073a10 0x7fceb81f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.328+0000 7fcec0000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb80c9eb0 0x7fceb81560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.328+0000 7fcec0000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb814fa50 0x7fceb815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.328+0000 7fcec0000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fceb80c7c00 con 0x7fceb814fa50 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.328+0000 7fcec0000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fceb80c3d90 con 0x7fceb8073a10 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.328+0000 7fcec0000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fceb80c4030 con 0x7fceb80c9eb0 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebcc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb80c9eb0 0x7fceb81560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb8073a10 0x7fceb81f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb814fa50 0x7fceb815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebcc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb80c9eb0 0x7fceb81560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:49980/0 (socket says 172.21.15.102:49980) 2024-04-20T21:33:21.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebcc00640 1 -- 172.21.15.102:0/1513049786 learned_addr learned my addr 172.21.15.102:0/1513049786 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebcc00640 1 -- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb8073a10 msgr2=0x7fceb81f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebcc00640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb8073a10 0x7fceb81f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebcc00640 1 -- 172.21.15.102:0/1513049786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb814fa50 msgr2=0x7fceb815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebcc00640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb814fa50 0x7fceb815a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebcc00640 1 -- 172.21.15.102:0/1513049786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb81f13f0 con 0x7fceb80c9eb0 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.329+0000 7fcebd600640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb8073a10 0x7fceb81f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.330+0000 7fcebcc00640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb80c9eb0 0x7fceb81560c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcea402b980 tx=0x7fcea40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.330+0000 7fceb5600640 1 -- 172.21.15.102:0/1513049786 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4034050 con 0x7fceb80c9eb0 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.330+0000 7fceb5600640 1 -- 172.21.15.102:0/1513049786 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea40044c0 con 0x7fceb80c9eb0 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.330+0000 7fceb5600640 1 -- 172.21.15.102:0/1513049786 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4003820 con 0x7fceb80c9eb0 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.330+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb81f16d0 con 0x7fceb80c9eb0 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.330+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb81f1c90 con 0x7fceb80c9eb0 2024-04-20T21:33:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.331+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb8074340 con 0x7fceb80c9eb0 2024-04-20T21:33:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.332+0000 7fceb5600640 1 -- 172.21.15.102:0/1513049786 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcea404a060 con 0x7fceb80c9eb0 2024-04-20T21:33:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.333+0000 7fceb5600640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce9806c920 0x7fce9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.333+0000 7fceb5600640 1 -- 172.21.15.102:0/1513049786 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7fcea40add00 con 0x7fceb80c9eb0 2024-04-20T21:33:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.333+0000 7fcebd600640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce9806c920 0x7fce9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.334+0000 7fcebd600640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce9806c920 0x7fce9806ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fceac005880 tx=0x7fceac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:21.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.338+0000 7fceb5600640 1 -- 172.21.15.102:0/1513049786 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcea407b390 con 0x7fceb80c9eb0 2024-04-20T21:33:21.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.577+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fceb80c7ee0 con 0x7fceb80c9eb0 2024-04-20T21:33:21.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.579+0000 7fceb5600640 1 -- 172.21.15.102:0/1513049786 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+6030 (secure 0 0 0) 0x7fcea407f690 con 0x7fceb80c9eb0 2024-04-20T21:33:21.577 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:21.578 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:16.559653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:21.578 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 51 2024-04-20T21:33:21.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce9806c920 msgr2=0x7fce9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce9806c920 0x7fce9806ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fceac005880 tx=0x7fceac009040 comp rx=0 tx=0).stop 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb80c9eb0 msgr2=0x7fceb81560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb80c9eb0 0x7fceb81560c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcea402b980 tx=0x7fcea40046c0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 shutdown_connections 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb814fa50 0x7fceb815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fceb80c9eb0 0x7fceb81560c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fce9806c920 0x7fce9806ede0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 --2- 172.21.15.102:0/1513049786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fceb8073a10 0x7fceb81f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 >> 172.21.15.102:0/1513049786 conn(0x7fceb80c9390 msgr2=0x7fceb80cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 shutdown_connections 2024-04-20T21:33:21.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:21.581+0000 7fcec0000640 1 -- 172.21.15.102:0/1513049786 wait complete. 2024-04-20T21:33:21.599 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:23.601 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:23.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.807+0000 7f812e000640 1 Processor -- start 2024-04-20T21:33:23.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.808+0000 7f812e000640 1 -- start start 2024-04-20T21:33:23.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f812e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f812814fa50 0x7f81280c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f812e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81281536d0 con 0x7f81280c9eb0 2024-04-20T21:33:23.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f812e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81280c76c0 con 0x7f8128073a10 2024-04-20T21:33:23.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f812e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f81280c7960 con 0x7f812814fa50 2024-04-20T21:33:23.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8126a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f81280c9eb0 0x7f81280c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46752/0 (socket says 172.21.15.102:46752) 2024-04-20T21:33:23.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8126a00640 1 -- 172.21.15.102:0/4066491504 learned_addr learned my addr 172.21.15.102:0/4066491504 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:33:23.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8127400640 1 --2- 172.21.15.102:0/4066491504 >> v2:172.21.15.193:3300/0 conn(0x7f812814fa50 0x7f81280c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8126000640 1 -- 172.21.15.102:0/4066491504 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430793265 0 0) 0x7f81280c76c0 con 0x7f8128073a10 2024-04-20T21:33:23.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8126000640 1 -- 172.21.15.102:0/4066491504 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8108003520 con 0x7f8128073a10 2024-04-20T21:33:23.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8126000640 1 -- 172.21.15.102:0/4066491504 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1560283835 0 0) 0x7f8110001110 con 0x7f81280c9eb0 2024-04-20T21:33:23.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8126000640 1 -- 172.21.15.102:0/4066491504 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81280c76c0 con 0x7f81280c9eb0 2024-04-20T21:33:23.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8127400640 1 -- 172.21.15.102:0/4066491504 >> v1:172.21.15.193:6789/0 conn(0x7f8128073a10 legacy=0x7f8128163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8127400640 1 -- 172.21.15.102:0/4066491504 >> v1:172.21.15.102:6789/0 conn(0x7f81280c9eb0 legacy=0x7f81280c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.810+0000 7f8127400640 1 -- 172.21.15.102:0/4066491504 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81280c3d90 con 0x7f812814fa50 2024-04-20T21:33:23.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.811+0000 7f8127400640 1 --2- 172.21.15.102:0/4066491504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812814fa50 0x7f81280c4830 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f81280c7890 tx=0x7f811802b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1280f210d525008c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:23.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.812+0000 7f8126000640 1 -- 172.21.15.102:0/4066491504 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118034070 con 0x7f812814fa50 2024-04-20T21:33:23.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.812+0000 7f8126000640 1 -- 172.21.15.102:0/4066491504 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8118030890 con 0x7f812814fa50 2024-04-20T21:33:23.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.812+0000 7f8126000640 1 -- 172.21.15.102:0/4066491504 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118030c60 con 0x7f812814fa50 2024-04-20T21:33:23.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.812+0000 7f812e000640 1 -- 172.21.15.102:0/4066491504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812814fa50 msgr2=0x7f81280c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.812+0000 7f812e000640 1 --2- 172.21.15.102:0/4066491504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812814fa50 0x7f81280c4830 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f81280c7890 tx=0x7f811802b9d0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.812+0000 7f812e000640 1 -- 172.21.15.102:0/4066491504 shutdown_connections 2024-04-20T21:33:23.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.812+0000 7f812e000640 1 --2- 172.21.15.102:0/4066491504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812814fa50 0x7f81280c4830 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.812+0000 7f812e000640 1 -- 172.21.15.102:0/4066491504 >> 172.21.15.102:0/4066491504 conn(0x7f81280c9390 msgr2=0x7f81280c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:23.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.813+0000 7f812e000640 1 -- 172.21.15.102:0/4066491504 shutdown_connections 2024-04-20T21:33:23.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.813+0000 7f812e000640 1 -- 172.21.15.102:0/4066491504 wait complete. 2024-04-20T21:33:23.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.814+0000 7f812e000640 1 Processor -- start 2024-04-20T21:33:23.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.814+0000 7f812e000640 1 -- start start 2024-04-20T21:33:23.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.815+0000 7f812e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8128073a10 0x7f8128072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.815+0000 7f812e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81280c9eb0 0x7f812815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.815+0000 7f812e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812814fa50 0x7f812814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.815+0000 7f812e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81280c4030 con 0x7f812814fa50 2024-04-20T21:33:23.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.815+0000 7f812e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81280c7c00 con 0x7f8128073a10 2024-04-20T21:33:23.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.815+0000 7f812e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f81280c4310 con 0x7f81280c9eb0 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.815+0000 7f8127400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8128073a10 0x7f8128072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.815+0000 7f8127e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812814fa50 0x7f812814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8128073a10 0x7f8128072920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:47946/0 (socket says 172.21.15.102:47946) 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127400640 1 -- 172.21.15.102:0/3852688744 learned_addr learned my addr 172.21.15.102:0/3852688744 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8126a00640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81280c9eb0 0x7f812815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127400640 1 -- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81280c9eb0 msgr2=0x7f812815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127400640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81280c9eb0 0x7f812815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127400640 1 -- 172.21.15.102:0/3852688744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812814fa50 msgr2=0x7f812814a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127400640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812814fa50 0x7f812814a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127400640 1 -- 172.21.15.102:0/3852688744 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812815d1b0 con 0x7f8128073a10 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127e00640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812814fa50 0x7f812814a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:23.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.816+0000 7f8127400640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8128073a10 0x7f8128072920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8118009890 tx=0x7f81180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:23.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.817+0000 7f811f400640 1 -- 172.21.15.102:0/3852688744 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118034040 con 0x7f8128073a10 2024-04-20T21:33:23.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.817+0000 7f811f400640 1 -- 172.21.15.102:0/3852688744 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811803ada0 con 0x7f8128073a10 2024-04-20T21:33:23.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.817+0000 7f811f400640 1 -- 172.21.15.102:0/3852688744 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118006e10 con 0x7f8128073a10 2024-04-20T21:33:23.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.817+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81281f4d70 con 0x7f8128073a10 2024-04-20T21:33:23.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.817+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81281f5330 con 0x7f8128073a10 2024-04-20T21:33:23.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.818+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8128069340 con 0x7f8128073a10 2024-04-20T21:33:23.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.819+0000 7f811f400640 1 -- 172.21.15.102:0/3852688744 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8118037070 con 0x7f8128073a10 2024-04-20T21:33:23.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.820+0000 7f811f400640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f810406c8f0 0x7f810406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.820+0000 7f811f400640 1 -- 172.21.15.102:0/3852688744 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f81180ae240 con 0x7f8128073a10 2024-04-20T21:33:23.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.820+0000 7f8126a00640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f810406c8f0 0x7f810406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.822+0000 7f8126a00640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f810406c8f0 0x7f810406edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f81280c29b0 tx=0x7f8110009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:23.825+0000 7f811f400640 1 -- 172.21.15.102:0/3852688744 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8128069340 con 0x7f8128073a10 2024-04-20T21:33:24.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.064+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8128159990 con 0x7f8128073a10 2024-04-20T21:33:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.066+0000 7f811f400640 1 -- 172.21.15.102:0/3852688744 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+6027 (secure 0 0 0) 0x7f811803f070 con 0x7f8128073a10 2024-04-20T21:33:24.065 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:24.065 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:22.087489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 52 2024-04-20T21:33:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f810406c8f0 msgr2=0x7f810406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f810406c8f0 0x7f810406edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f81280c29b0 tx=0x7f8110009040 comp rx=0 tx=0).stop 2024-04-20T21:33:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8128073a10 msgr2=0x7f8128072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8128073a10 0x7f8128072920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8118009890 tx=0x7f81180046c0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 shutdown_connections 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f812814fa50 0x7f812814a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81280c9eb0 0x7f812815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f810406c8f0 0x7f810406edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 --2- 172.21.15.102:0/3852688744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8128073a10 0x7f8128072920 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 >> 172.21.15.102:0/3852688744 conn(0x7f81280c9390 msgr2=0x7f81280c3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 shutdown_connections 2024-04-20T21:33:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:24.068+0000 7f812e000640 1 -- 172.21.15.102:0/3852688744 wait complete. 2024-04-20T21:33:24.085 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:26.086 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:26.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.288+0000 7fa5f0000640 1 Processor -- start 2024-04-20T21:33:26.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.288+0000 7fa5f0000640 1 -- start start 2024-04-20T21:33:26.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.289+0000 7fa5f0000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa5e815ea20 0x7fa5e806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.289+0000 7fa5f0000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa5e815ee00 0x7fa5e80710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5f0000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815f2b0 0x7fa5e80c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5f0000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e80c81e0 con 0x7fa5e815ea20 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5f0000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e80c8f20 con 0x7fa5e815ee00 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5f0000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e806cb80 con 0x7fa5e815f2b0 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ed600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa5e815ea20 0x7fa5e806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ed600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa5e815ea20 0x7fa5e806d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34148/0 (socket says 172.21.15.102:34148) 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ed600640 1 -- 172.21.15.102:0/871519362 learned_addr learned my addr 172.21.15.102:0/871519362 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ee000640 1 --2- 172.21.15.102:0/871519362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815f2b0 0x7fa5e80c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ecc00640 1 --2- 172.21.15.102:0/871519362 >> v2:172.21.15.193:3300/0 conn(0x7fa5e815ee00 0x7fa5e80710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ed600640 1 -- 172.21.15.102:0/871519362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815f2b0 msgr2=0x7fa5e80c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ed600640 1 --2- 172.21.15.102:0/871519362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815f2b0 0x7fa5e80c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ed600640 1 -- 172.21.15.102:0/871519362 >> v2:172.21.15.193:3300/0 conn(0x7fa5e815ee00 msgr2=0x7fa5e80710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ed600640 1 --2- 172.21.15.102:0/871519362 >> v2:172.21.15.193:3300/0 conn(0x7fa5e815ee00 0x7fa5e80710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.290+0000 7fa5ed600640 1 -- 172.21.15.102:0/871519362 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e806ce60 con 0x7fa5e815ea20 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.291+0000 7fa5ed600640 1 --2- 172.21.15.102:0/871519362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815ea20 0x7fa5e806d940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa5e806c2f0 tx=0x7fa5dc02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91989d4a2e9cb647 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.292+0000 7fa5e7e00640 1 -- 172.21.15.102:0/871519362 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc034070 con 0x7fa5e815ea20 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.292+0000 7fa5e7e00640 1 -- 172.21.15.102:0/871519362 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc030850 con 0x7fa5e815ea20 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.292+0000 7fa5e7e00640 1 -- 172.21.15.102:0/871519362 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc030c20 con 0x7fa5e815ea20 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.292+0000 7fa5f0000640 1 -- 172.21.15.102:0/871519362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815ea20 msgr2=0x7fa5e806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.292+0000 7fa5f0000640 1 --2- 172.21.15.102:0/871519362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815ea20 0x7fa5e806d940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa5e806c2f0 tx=0x7fa5dc02b990 comp rx=0 tx=0).stop 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.293+0000 7fa5f0000640 1 -- 172.21.15.102:0/871519362 shutdown_connections 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.293+0000 7fa5f0000640 1 --2- 172.21.15.102:0/871519362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815f2b0 0x7fa5e80c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.293+0000 7fa5f0000640 1 --2- 172.21.15.102:0/871519362 >> v2:172.21.15.193:3300/0 conn(0x7fa5e815ee00 0x7fa5e80710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.293+0000 7fa5f0000640 1 --2- 172.21.15.102:0/871519362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815ea20 0x7fa5e806d940 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.293+0000 7fa5f0000640 1 -- 172.21.15.102:0/871519362 >> 172.21.15.102:0/871519362 conn(0x7fa5e80c9390 msgr2=0x7fa5e80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.293+0000 7fa5f0000640 1 -- 172.21.15.102:0/871519362 shutdown_connections 2024-04-20T21:33:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.293+0000 7fa5f0000640 1 -- 172.21.15.102:0/871519362 wait complete. 2024-04-20T21:33:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.294+0000 7fa5f0000640 1 Processor -- start 2024-04-20T21:33:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.294+0000 7fa5f0000640 1 -- start start 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.295+0000 7fa5f0000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815ea20 0x7fa5e80c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.295+0000 7fa5f0000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5e815ee00 0x7fa5e81f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.295+0000 7fa5f0000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815f2b0 0x7fa5e80cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.295+0000 7fa5f0000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e80c8b00 con 0x7fa5e815f2b0 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.295+0000 7fa5f0000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e806d420 con 0x7fa5e815ee00 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.295+0000 7fa5f0000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e80c8c80 con 0x7fa5e815ea20 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ed600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815ea20 0x7fa5e80c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ee000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815f2b0 0x7fa5e80cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ed600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815ea20 0x7fa5e80c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50004/0 (socket says 172.21.15.102:50004) 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ed600640 1 -- 172.21.15.102:0/1647477685 learned_addr learned my addr 172.21.15.102:0/1647477685 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:26.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ecc00640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5e815ee00 0x7fa5e81f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ee000640 1 -- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815ea20 msgr2=0x7fa5e80c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ee000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815ea20 0x7fa5e80c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ee000640 1 -- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5e815ee00 msgr2=0x7fa5e81f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ee000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5e815ee00 0x7fa5e81f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ee000640 1 -- 172.21.15.102:0/1647477685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e80c3dd0 con 0x7fa5e815f2b0 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.296+0000 7fa5ed600640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815ea20 0x7fa5e80c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.297+0000 7fa5ee000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815f2b0 0x7fa5e80cbef0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000eba0 tx=0x7fa5e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.297+0000 7fa5e5600640 1 -- 172.21.15.102:0/1647477685 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0018020 con 0x7fa5e815f2b0 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.297+0000 7fa5e5600640 1 -- 172.21.15.102:0/1647477685 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e000cd20 con 0x7fa5e815f2b0 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.297+0000 7fa5e5600640 1 -- 172.21.15.102:0/1647477685 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00146c0 con 0x7fa5e815f2b0 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.297+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e80c40a0 con 0x7fa5e815f2b0 2024-04-20T21:33:26.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.297+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e80c4660 con 0x7fa5e815f2b0 2024-04-20T21:33:26.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.298+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e80729b0 con 0x7fa5e815f2b0 2024-04-20T21:33:26.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.299+0000 7fa5e5600640 1 -- 172.21.15.102:0/1647477685 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5e0005d10 con 0x7fa5e815f2b0 2024-04-20T21:33:26.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.300+0000 7fa5e5600640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa5c806c8f0 0x7fa5c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.300+0000 7fa5e5600640 1 -- 172.21.15.102:0/1647477685 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7fa5e008cda0 con 0x7fa5e815f2b0 2024-04-20T21:33:26.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.300+0000 7fa5ed600640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa5c806c8f0 0x7fa5c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.301+0000 7fa5ed600640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa5c806c8f0 0x7fa5c806edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009630 tx=0x7fa5dc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:26.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.305+0000 7fa5e5600640 1 -- 172.21.15.102:0/1647477685 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa5e0022030 con 0x7fa5e815f2b0 2024-04-20T21:33:26.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.541+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e80726d0 con 0x7fa5e815f2b0 2024-04-20T21:33:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.542+0000 7fa5e5600640 1 -- 172.21.15.102:0/1647477685 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+6027 (secure 0 0 0) 0x7fa5e0010040 con 0x7fa5e815f2b0 2024-04-20T21:33:26.542 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:26.542 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:22.087489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:26.542 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 52 2024-04-20T21:33:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa5c806c8f0 msgr2=0x7fa5c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa5c806c8f0 0x7fa5c806edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009630 tx=0x7fa5dc008030 comp rx=0 tx=0).stop 2024-04-20T21:33:26.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815f2b0 msgr2=0x7fa5e80cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815f2b0 0x7fa5e80cbef0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000eba0 tx=0x7fa5e000c490 comp rx=0 tx=0).stop 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 shutdown_connections 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5e815f2b0 0x7fa5e80cbef0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa5c806c8f0 0x7fa5c806edb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5e815ee00 0x7fa5e81f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 --2- 172.21.15.102:0/1647477685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5e815ea20 0x7fa5e80c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 >> 172.21.15.102:0/1647477685 conn(0x7fa5e80c9390 msgr2=0x7fa5e8154c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 shutdown_connections 2024-04-20T21:33:26.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:26.545+0000 7fa5f0000640 1 -- 172.21.15.102:0/1647477685 wait complete. 2024-04-20T21:33:26.563 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:28.476 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:33:28.483 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:33:28.563 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:28.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.766+0000 7fc3fac00640 1 Processor -- start 2024-04-20T21:33:28.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.766+0000 7fc3fac00640 1 -- start start 2024-04-20T21:33:28.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.767+0000 7fc3fac00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc3f4155530 0x7fc3f415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.767+0000 7fc3fac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f40cc470 0x7fc3f40cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.767+0000 7fc3fac00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f415ef60 con 0x7fc3f40cdfc0 2024-04-20T21:33:28.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.767+0000 7fc3fac00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f40c5ca0 con 0x7fc3f4155530 2024-04-20T21:33:28.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.767+0000 7fc3fac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f40cd5f0 con 0x7fc3f40cc470 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f9800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc3f4155530 0x7fc3f415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f8e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f40cc470 0x7fc3f40cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f8e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f40cc470 0x7fc3f40cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:42830/0 (socket says 172.21.15.102:42830) 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f8e00640 1 -- 172.21.15.102:0/2218416195 learned_addr learned my addr 172.21.15.102:0/2218416195 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f9800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc3f4155530 0x7fc3f415e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37688/0 (socket says 172.21.15.102:37688) 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f8e00640 1 -- 172.21.15.102:0/2218416195 >> v2:172.21.15.193:3300/0 conn(0x7fc3f4155530 msgr2=0x7fc3f415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f8e00640 1 --2- 172.21.15.102:0/2218416195 >> v2:172.21.15.193:3300/0 conn(0x7fc3f4155530 0x7fc3f415e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f8e00640 1 -- 172.21.15.102:0/2218416195 >> v1:172.21.15.102:6789/0 conn(0x7fc3f40cdfc0 legacy=0x7fc3f40cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f8e00640 1 -- 172.21.15.102:0/2218416195 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f415f240 con 0x7fc3f40cc470 2024-04-20T21:33:28.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.768+0000 7fc3f9800640 1 --2- 172.21.15.102:0/2218416195 >> v2:172.21.15.193:3300/0 conn(0x7fc3f4155530 0x7fc3f415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:28.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.769+0000 7fc3f8e00640 1 --2- 172.21.15.102:0/2218416195 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f40cc470 0x7fc3f40cda80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009870 tx=0x7fc3e002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89617f3d1a4d5718 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:28.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3f3e00640 1 -- 172.21.15.102:0/2218416195 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0034070 con 0x7fc3f40cc470 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3f3e00640 1 -- 172.21.15.102:0/2218416195 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e0030840 con 0x7fc3f40cc470 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3f3e00640 1 -- 172.21.15.102:0/2218416195 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0030c10 con 0x7fc3f40cc470 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3fac00640 1 -- 172.21.15.102:0/2218416195 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f40cc470 msgr2=0x7fc3f40cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3fac00640 1 --2- 172.21.15.102:0/2218416195 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f40cc470 0x7fc3f40cda80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009870 tx=0x7fc3e002b980 comp rx=0 tx=0).stop 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3fac00640 1 -- 172.21.15.102:0/2218416195 shutdown_connections 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3fac00640 1 --2- 172.21.15.102:0/2218416195 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f40cc470 0x7fc3f40cda80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3fac00640 1 --2- 172.21.15.102:0/2218416195 >> v2:172.21.15.193:3300/0 conn(0x7fc3f4155530 0x7fc3f415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3fac00640 1 -- 172.21.15.102:0/2218416195 >> 172.21.15.102:0/2218416195 conn(0x7fc3f4075080 msgr2=0x7fc3f4075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3fac00640 1 -- 172.21.15.102:0/2218416195 shutdown_connections 2024-04-20T21:33:28.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.770+0000 7fc3fac00640 1 -- 172.21.15.102:0/2218416195 wait complete. 2024-04-20T21:33:28.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.772+0000 7fc3fac00640 1 Processor -- start 2024-04-20T21:33:28.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.772+0000 7fc3fac00640 1 -- start start 2024-04-20T21:33:28.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.773+0000 7fc3fac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f40cc470 0x7fc3f40c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.773+0000 7fc3fac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3f40cdfc0 0x7fc3f41541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.773+0000 7fc3fac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f4155530 0x7fc3f415d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.773+0000 7fc3fac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f40cd280 con 0x7fc3f40cc470 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.773+0000 7fc3fac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f40ccfa0 con 0x7fc3f40cdfc0 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.773+0000 7fc3fac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f40ccbe0 con 0x7fc3f4155530 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.773+0000 7fc3f9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f40cc470 0x7fc3f40c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f4155530 0x7fc3f415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3f8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3f40cdfc0 0x7fc3f41541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f4155530 0x7fc3f415d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:42844/0 (socket says 172.21.15.102:42844) 2024-04-20T21:33:28.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 -- 172.21.15.102:0/58689556 learned_addr learned my addr 172.21.15.102:0/58689556 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 -- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3f40cdfc0 msgr2=0x7fc3f41541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3f40cdfc0 0x7fc3f41541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 -- 172.21.15.102:0/58689556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f40cc470 msgr2=0x7fc3f40c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f40cc470 0x7fc3f40c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 -- 172.21.15.102:0/58689556 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f41563a0 con 0x7fc3f4155530 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3f9800640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f40cc470 0x7fc3f40c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3fa200640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f4155530 0x7fc3f415d490 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c7a0 tx=0x7fc3ec00cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.774+0000 7fc3f1600640 1 -- 172.21.15.102:0/58689556 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec003b70 con 0x7fc3f4155530 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.775+0000 7fc3f1600640 1 -- 172.21.15.102:0/58689556 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ec003d30 con 0x7fc3f4155530 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.775+0000 7fc3f1600640 1 -- 172.21.15.102:0/58689556 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00aa90 con 0x7fc3f4155530 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.775+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f41560c0 con 0x7fc3f4155530 2024-04-20T21:33:28.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.775+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f41572e0 con 0x7fc3f4155530 2024-04-20T21:33:28.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.776+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f404fd60 con 0x7fc3f4155530 2024-04-20T21:33:28.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.777+0000 7fc3f1600640 1 -- 172.21.15.102:0/58689556 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3ec00ac30 con 0x7fc3f4155530 2024-04-20T21:33:28.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.778+0000 7fc3f1600640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3d406c8f0 0x7fc3d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.778+0000 7fc3f1600640 1 -- 172.21.15.102:0/58689556 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7fc3ec08c110 con 0x7fc3f4155530 2024-04-20T21:33:28.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.778+0000 7fc3f9800640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3d406c8f0 0x7fc3d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.779+0000 7fc3f9800640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3d406c8f0 0x7fc3d406edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009560 tx=0x7fc3e8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:28.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:28.782+0000 7fc3f1600640 1 -- 172.21.15.102:0/58689556 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc3ec059710 con 0x7fc3f4155530 2024-04-20T21:33:29.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.025+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3f40cc9c0 con 0x7fc3f4155530 2024-04-20T21:33:29.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.027+0000 7fc3f1600640 1 -- 172.21.15.102:0/58689556 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6033 (secure 0 0 0) 0x7fc3ec05da10 con 0x7fc3f4155530 2024-04-20T21:33:29.026 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:29.026 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:27.300683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:clientreplay","state_seq":24,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:29.026 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 53 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3d406c8f0 msgr2=0x7fc3d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3d406c8f0 0x7fc3d406edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009560 tx=0x7fc3e8009190 comp rx=0 tx=0).stop 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f4155530 msgr2=0x7fc3f415d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f4155530 0x7fc3f415d490 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c7a0 tx=0x7fc3ec00cc60 comp rx=0 tx=0).stop 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 shutdown_connections 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3f4155530 0x7fc3f415d490 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc3d406c8f0 0x7fc3d406edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3f40cdfc0 0x7fc3f41541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 --2- 172.21.15.102:0/58689556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f40cc470 0x7fc3f40c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.029+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 >> 172.21.15.102:0/58689556 conn(0x7fc3f4075080 msgr2=0x7fc3f4152190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.030+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 shutdown_connections 2024-04-20T21:33:29.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:29.030+0000 7fc3fac00640 1 -- 172.21.15.102:0/58689556 wait complete. 2024-04-20T21:33:29.048 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:31.049 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:31.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.260+0000 7f2462c00640 1 Processor -- start 2024-04-20T21:33:31.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.260+0000 7f2462c00640 1 -- start start 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.261+0000 7f2462c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f245c0716d0 0x7f245c0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.261+0000 7f2462c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f245c0c8670 0x7f245c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.261+0000 7f2462c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f245c0c7730 con 0x7f245c0c7350 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.261+0000 7f2462c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f245c0ca5a0 con 0x7f245c0c8670 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.261+0000 7f2462c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f245c15b460 con 0x7f245c0716d0 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.262+0000 7f2460c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f245c0c8670 0x7f245c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.262+0000 7f2460c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f245c0c8670 0x7f245c15e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36748/0 (socket says 172.21.15.102:36748) 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.262+0000 7f2460c00640 1 -- 172.21.15.102:0/2321341723 learned_addr learned my addr 172.21.15.102:0/2321341723 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:31.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.262+0000 7f245b400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f245c0716d0 0x7f245c0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.262+0000 7f2460c00640 1 -- 172.21.15.102:0/2321341723 >> v2:172.21.15.193:3300/0 conn(0x7f245c0716d0 msgr2=0x7f245c0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:31.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.262+0000 7f2460c00640 1 --2- 172.21.15.102:0/2321341723 >> v2:172.21.15.193:3300/0 conn(0x7f245c0716d0 0x7f245c0c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.262+0000 7f2460c00640 1 -- 172.21.15.102:0/2321341723 >> v1:172.21.15.193:6789/0 conn(0x7f245c0c7350 legacy=0x7f245c06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.262+0000 7f2460c00640 1 -- 172.21.15.102:0/2321341723 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c15b740 con 0x7f245c0c8670 2024-04-20T21:33:31.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.263+0000 7f2460c00640 1 --2- 172.21.15.102:0/2321341723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c8670 0x7f245c15e9b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f245c0ce300 tx=0x7f245402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6f4bd95ffcec4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f245aa00640 1 -- 172.21.15.102:0/2321341723 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245402cbb0 con 0x7f245c0c8670 2024-04-20T21:33:31.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f245aa00640 1 -- 172.21.15.102:0/2321341723 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245402cd50 con 0x7f245c0c8670 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f245aa00640 1 -- 172.21.15.102:0/2321341723 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454034610 con 0x7f245c0c8670 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f2462c00640 1 -- 172.21.15.102:0/2321341723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c8670 msgr2=0x7f245c15e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f2462c00640 1 --2- 172.21.15.102:0/2321341723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c8670 0x7f245c15e9b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f245c0ce300 tx=0x7f245402c010 comp rx=0 tx=0).stop 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f2462c00640 1 -- 172.21.15.102:0/2321341723 shutdown_connections 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f2462c00640 1 --2- 172.21.15.102:0/2321341723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c8670 0x7f245c15e9b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f2462c00640 1 --2- 172.21.15.102:0/2321341723 >> v2:172.21.15.193:3300/0 conn(0x7f245c0716d0 0x7f245c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f2462c00640 1 -- 172.21.15.102:0/2321341723 >> 172.21.15.102:0/2321341723 conn(0x7f245c06dd40 msgr2=0x7f245c074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f2462c00640 1 -- 172.21.15.102:0/2321341723 shutdown_connections 2024-04-20T21:33:31.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.264+0000 7f2462c00640 1 -- 172.21.15.102:0/2321341723 wait complete. 2024-04-20T21:33:31.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.266+0000 7f2462c00640 1 Processor -- start 2024-04-20T21:33:31.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.266+0000 7f2462c00640 1 -- start start 2024-04-20T21:33:31.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f2462c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f245c0716d0 0x7f245c13f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f2462c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c7350 0x7f245c0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f2462c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f245c0c8670 0x7f245c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f2462c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f245c15bfe0 con 0x7f245c0c7350 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f2462c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f245c15bd00 con 0x7f245c0c8670 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f2462c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f245c06c050 con 0x7f245c0716d0 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f245b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c7350 0x7f245c0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f245be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f245c0716d0 0x7f245c13f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f245be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f245c0716d0 0x7f245c13f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:42856/0 (socket says 172.21.15.102:42856) 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f245be00640 1 -- 172.21.15.102:0/569883341 learned_addr learned my addr 172.21.15.102:0/569883341 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f245b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c7350 0x7f245c0cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36750/0 (socket says 172.21.15.102:36750) 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.267+0000 7f2460c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f245c0c8670 0x7f245c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f245be00640 1 -- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f245c0c8670 msgr2=0x7f245c0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f245be00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f245c0c8670 0x7f245c0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f245be00640 1 -- 172.21.15.102:0/569883341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c7350 msgr2=0x7f245c0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f245be00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c7350 0x7f245c0cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f245be00640 1 -- 172.21.15.102:0/569883341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c140f80 con 0x7f245c0716d0 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f245be00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f245c0716d0 0x7f245c13f810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f244c00c400 tx=0x7f244c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f2453e00640 1 -- 172.21.15.102:0/569883341 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c0052d0 con 0x7f245c0716d0 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f2453e00640 1 -- 172.21.15.102:0/569883341 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c005490 con 0x7f245c0716d0 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f2453e00640 1 -- 172.21.15.102:0/569883341 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c007260 con 0x7f245c0716d0 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.268+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c141260 con 0x7f245c0716d0 2024-04-20T21:33:31.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.269+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c0cb4e0 con 0x7f245c0716d0 2024-04-20T21:33:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.270+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c15c220 con 0x7f245c0716d0 2024-04-20T21:33:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.270+0000 7f2453e00640 1 -- 172.21.15.102:0/569883341 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f244c007400 con 0x7f245c0716d0 2024-04-20T21:33:31.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.271+0000 7f2453e00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f243c06c8f0 0x7f243c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.271+0000 7f2453e00640 1 -- 172.21.15.102:0/569883341 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f244c08b770 con 0x7f245c0716d0 2024-04-20T21:33:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.272+0000 7f245b400640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f243c06c8f0 0x7f243c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.272+0000 7f245b400640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f243c06c8f0 0x7f243c06edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f245c15a9a0 tx=0x7f2448009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.276+0000 7f2453e00640 1 -- 172.21.15.102:0/569883341 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f245c15c220 con 0x7f245c0716d0 2024-04-20T21:33:31.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.519+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f245c0c7950 con 0x7f245c0716d0 2024-04-20T21:33:31.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.521+0000 7f2453e00640 1 -- 172.21.15.102:0/569883341 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6033 (secure 0 0 0) 0x7f245c0c7950 con 0x7f245c0716d0 2024-04-20T21:33:31.520 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:31.520 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:27.300683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:clientreplay","state_seq":24,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:31.520 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 53 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f243c06c8f0 msgr2=0x7f243c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f243c06c8f0 0x7f243c06edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f245c15a9a0 tx=0x7f2448009190 comp rx=0 tx=0).stop 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f245c0716d0 msgr2=0x7f245c13f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f245c0716d0 0x7f245c13f810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f244c00c400 tx=0x7f244c00c8c0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 shutdown_connections 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f243c06c8f0 0x7f243c06edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f245c0c8670 0x7f245c0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f245c0c7350 0x7f245c0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 --2- 172.21.15.102:0/569883341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f245c0716d0 0x7f245c13f810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 >> 172.21.15.102:0/569883341 conn(0x7f245c06dd40 msgr2=0x7f245c074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.523+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 shutdown_connections 2024-04-20T21:33:31.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:31.524+0000 7f2462c00640 1 -- 172.21.15.102:0/569883341 wait complete. 2024-04-20T21:33:31.543 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 53, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 53, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5835": { "addr": "172.21.15.102:6835/2655303061", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 2655303061, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 2655303061, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5835, "incarnation": 50, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:clientreplay", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:33:27.300683+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5835 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:33:33.545 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:33.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.807+0000 7f9466e00640 1 Processor -- start 2024-04-20T21:33:33.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.807+0000 7f9466e00640 1 -- start start 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.808+0000 7f9466e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f94600c9eb0 0x7f94600c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.808+0000 7f9466e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94601536d0 con 0x7f946014fa50 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.808+0000 7f9466e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94600c76c0 con 0x7f9460073a10 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.808+0000 7f9466e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f94600c7960 con 0x7f94600c9eb0 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.809+0000 7f945f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f94600c9eb0 0x7f94600c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.809+0000 7f945f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f94600c9eb0 0x7f94600c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37726/0 (socket says 172.21.15.102:37726) 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.809+0000 7f945f400640 1 -- 172.21.15.102:0/3991480791 learned_addr learned my addr 172.21.15.102:0/3991480791 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.809+0000 7f945f400640 1 -- 172.21.15.102:0/3991480791 >> v1:172.21.15.193:6789/0 conn(0x7f9460073a10 legacy=0x7f9460163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.809+0000 7f945f400640 1 -- 172.21.15.102:0/3991480791 >> v1:172.21.15.102:6789/0 conn(0x7f946014fa50 legacy=0x7f94600c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.809+0000 7f945f400640 1 -- 172.21.15.102:0/3991480791 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94600c3d90 con 0x7f94600c9eb0 2024-04-20T21:33:33.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.809+0000 7f945ea00640 1 -- 172.21.15.102:0/3991480791 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3872762577 0 0) 0x7f94600c76c0 con 0x7f9460073a10 2024-04-20T21:33:33.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.810+0000 7f945f400640 1 --2- 172.21.15.102:0/3991480791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94600c4830 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f94600c7890 tx=0x7f944c02bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=473df494cc120019 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:33.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.810+0000 7f945ea00640 1 -- 172.21.15.102:0/3991480791 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c02d870 con 0x7f94600c9eb0 2024-04-20T21:33:33.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.810+0000 7f945ea00640 1 -- 172.21.15.102:0/3991480791 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c02da10 con 0x7f94600c9eb0 2024-04-20T21:33:33.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.810+0000 7f945ea00640 1 -- 172.21.15.102:0/3991480791 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c02dde0 con 0x7f94600c9eb0 2024-04-20T21:33:33.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.811+0000 7f9466e00640 1 -- 172.21.15.102:0/3991480791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 msgr2=0x7f94600c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.811+0000 7f9466e00640 1 --2- 172.21.15.102:0/3991480791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94600c4830 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f94600c7890 tx=0x7f944c02bab0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.811+0000 7f9466e00640 1 -- 172.21.15.102:0/3991480791 shutdown_connections 2024-04-20T21:33:33.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.811+0000 7f9466e00640 1 --2- 172.21.15.102:0/3991480791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f94600c4830 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.811+0000 7f9466e00640 1 -- 172.21.15.102:0/3991480791 >> 172.21.15.102:0/3991480791 conn(0x7f94600c9390 msgr2=0x7f94600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:33.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.811+0000 7f9466e00640 1 -- 172.21.15.102:0/3991480791 shutdown_connections 2024-04-20T21:33:33.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.811+0000 7f9466e00640 1 -- 172.21.15.102:0/3991480791 wait complete. 2024-04-20T21:33:33.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.813+0000 7f9466e00640 1 Processor -- start 2024-04-20T21:33:33.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.813+0000 7f9466e00640 1 -- start start 2024-04-20T21:33:33.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f9466e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 0x7f94601f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f9466e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f946014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f9466e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f9460071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f9466e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94600c4030 con 0x7f946014fa50 2024-04-20T21:33:33.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f9466e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94600c7c00 con 0x7f94600c9eb0 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f9466e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f94600c4310 con 0x7f9460073a10 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f945f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f946014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f945f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f946014e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37732/0 (socket says 172.21.15.102:37732) 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f945f400640 1 -- 172.21.15.102:0/1733411314 learned_addr learned my addr 172.21.15.102:0/1733411314 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f9464e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f9460071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.814+0000 7f945fe00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 0x7f94601f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f945f400640 1 -- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 msgr2=0x7f94601f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f945f400640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 0x7f94601f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f945f400640 1 -- 172.21.15.102:0/1733411314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 msgr2=0x7f9460071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f945f400640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f9460071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f945f400640 1 -- 172.21.15.102:0/1733411314 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601489b0 con 0x7f94600c9eb0 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f9464e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f9460071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f945f400640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f946014e030 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f944c02c0c0 tx=0x7f944c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f9457e00640 1 -- 172.21.15.102:0/1733411314 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c047070 con 0x7f94600c9eb0 2024-04-20T21:33:33.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f9457e00640 1 -- 172.21.15.102:0/1733411314 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c037a40 con 0x7f94600c9eb0 2024-04-20T21:33:33.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9460148c70 con 0x7f94600c9eb0 2024-04-20T21:33:33.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f9457e00640 1 -- 172.21.15.102:0/1733411314 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c037e30 con 0x7f94600c9eb0 2024-04-20T21:33:33.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.815+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9460149230 con 0x7f94600c9eb0 2024-04-20T21:33:33.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.816+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9460069340 con 0x7f94600c9eb0 2024-04-20T21:33:33.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.817+0000 7f9457e00640 1 -- 172.21.15.102:0/1733411314 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f944c041390 con 0x7f94600c9eb0 2024-04-20T21:33:33.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.818+0000 7f9457e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f944006c920 0x7f944006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.818+0000 7f9457e00640 1 -- 172.21.15.102:0/1733411314 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f944c0adeb0 con 0x7f94600c9eb0 2024-04-20T21:33:33.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.818+0000 7f945fe00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f944006c920 0x7f944006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.819+0000 7f945fe00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f944006c920 0x7f944006ede0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9450000f60 tx=0x7f9450009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:33.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:33.822+0000 7f9457e00640 1 -- 172.21.15.102:0/1733411314 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9460069340 con 0x7f94600c9eb0 2024-04-20T21:33:34.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.061+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f946014f600 con 0x7f94600c9eb0 2024-04-20T21:33:34.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.062+0000 7f9457e00640 1 -- 172.21.15.102:0/1733411314 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6033 (secure 0 0 0) 0x7f946014f600 con 0x7f94600c9eb0 2024-04-20T21:33:34.061 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:34.061 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:27.300683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:clientreplay","state_seq":24,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:33:34.061 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 53 2024-04-20T21:33:34.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f944006c920 msgr2=0x7f944006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f944006c920 0x7f944006ede0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9450000f60 tx=0x7f9450009040 comp rx=0 tx=0).stop 2024-04-20T21:33:34.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 msgr2=0x7f946014e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f946014e030 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f944c02c0c0 tx=0x7f944c004730 comp rx=0 tx=0).stop 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 shutdown_connections 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946014fa50 0x7f9460071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f944006c920 0x7f944006ede0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94600c9eb0 0x7f946014e030 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 --2- 172.21.15.102:0/1733411314 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9460073a10 0x7f94601f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.064+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 >> 172.21.15.102:0/1733411314 conn(0x7f94600c9390 msgr2=0x7f946014baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.065+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 shutdown_connections 2024-04-20T21:33:34.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:34.065+0000 7f9466e00640 1 -- 172.21.15.102:0/1733411314 wait complete. 2024-04-20T21:33:34.081 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:33:36.082 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:36.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.295+0000 7f006ee00640 1 Processor -- start 2024-04-20T21:33:36.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.296+0000 7f006ee00640 1 -- start start 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f006ee00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0068152170 0x7f00680c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f006ee00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0068158770 0x7f0068070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f006ee00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 0x7f0068071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f006ee00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f00680c6dc0 con 0x7f0068152170 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f006ee00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0068071f70 con 0x7f0068158770 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f006ee00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0068072210 con 0x7f0068158b80 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f0067e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0068152170 0x7f00680c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f0067e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0068152170 0x7f00680c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36770/0 (socket says 172.21.15.102:36770) 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.297+0000 7f0067e00640 1 -- 172.21.15.102:0/4063741271 learned_addr learned my addr 172.21.15.102:0/4063741271 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.298+0000 7f006ce00640 1 --2- 172.21.15.102:0/4063741271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 0x7f0068071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.298+0000 7f005f400640 1 --2- 172.21.15.102:0/4063741271 >> v2:172.21.15.193:3300/0 conn(0x7f0068158770 0x7f0068070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.298+0000 7f0067e00640 1 -- 172.21.15.102:0/4063741271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 msgr2=0x7f0068071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.298+0000 7f0067e00640 1 --2- 172.21.15.102:0/4063741271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 0x7f0068071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.298+0000 7f0067e00640 1 -- 172.21.15.102:0/4063741271 >> v2:172.21.15.193:3300/0 conn(0x7f0068158770 msgr2=0x7f0068070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.298+0000 7f0067e00640 1 --2- 172.21.15.102:0/4063741271 >> v2:172.21.15.193:3300/0 conn(0x7f0068158770 0x7f0068070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.298+0000 7f0067e00640 1 -- 172.21.15.102:0/4063741271 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00680724b0 con 0x7f0068152170 2024-04-20T21:33:36.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.299+0000 7f0067e00640 1 --2- 172.21.15.102:0/4063741271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f00680c8db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0068072680 tx=0x7f005802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9411ee217a0c5d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:36.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.299+0000 7f0067400640 1 -- 172.21.15.102:0/4063741271 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058034070 con 0x7f0068152170 2024-04-20T21:33:36.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.299+0000 7f0067400640 1 -- 172.21.15.102:0/4063741271 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0058030840 con 0x7f0068152170 2024-04-20T21:33:36.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.299+0000 7f0067400640 1 -- 172.21.15.102:0/4063741271 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058030c10 con 0x7f0068152170 2024-04-20T21:33:36.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 -- 172.21.15.102:0/4063741271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 msgr2=0x7f00680c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 --2- 172.21.15.102:0/4063741271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f00680c8db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0068072680 tx=0x7f005802b980 comp rx=0 tx=0).stop 2024-04-20T21:33:36.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 -- 172.21.15.102:0/4063741271 shutdown_connections 2024-04-20T21:33:36.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 --2- 172.21.15.102:0/4063741271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 0x7f0068071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 --2- 172.21.15.102:0/4063741271 >> v2:172.21.15.193:3300/0 conn(0x7f0068158770 0x7f0068070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 --2- 172.21.15.102:0/4063741271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f00680c8db0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 -- 172.21.15.102:0/4063741271 >> 172.21.15.102:0/4063741271 conn(0x7f00680c9390 msgr2=0x7f00680c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:36.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 -- 172.21.15.102:0/4063741271 shutdown_connections 2024-04-20T21:33:36.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.300+0000 7f006ee00640 1 -- 172.21.15.102:0/4063741271 wait complete. 2024-04-20T21:33:36.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.302+0000 7f006ee00640 1 Processor -- start 2024-04-20T21:33:36.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.302+0000 7f006ee00640 1 -- start start 2024-04-20T21:33:36.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f006ee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f0068155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f006ee00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0068158770 0x7f006815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f006ee00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 0x7f00681e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f006ee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f00680c9f20 con 0x7f0068152170 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f006ee00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0068158250 con 0x7f0068158770 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f006ee00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0068157f20 con 0x7f0068158b80 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f0067e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f0068155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f0067e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f0068155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36776/0 (socket says 172.21.15.102:36776) 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f0067e00640 1 -- 172.21.15.102:0/2318316610 learned_addr learned my addr 172.21.15.102:0/2318316610 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f006ce00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 0x7f00681e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.303+0000 7f005f400640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0068158770 0x7f006815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0067e00640 1 -- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 msgr2=0x7f00681e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0067e00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 0x7f00681e1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0067e00640 1 -- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0068158770 msgr2=0x7f006815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0067e00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0068158770 0x7f006815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0067e00640 1 -- 172.21.15.102:0/2318316610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006815a1a0 con 0x7f0068152170 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0067e00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f0068155c50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0058000c00 tx=0x7f00580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0064c00640 1 -- 172.21.15.102:0/2318316610 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058034040 con 0x7f0068152170 2024-04-20T21:33:36.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0064c00640 1 -- 172.21.15.102:0/2318316610 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0058039d90 con 0x7f0068152170 2024-04-20T21:33:36.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006815a480 con 0x7f0068152170 2024-04-20T21:33:36.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f0064c00640 1 -- 172.21.15.102:0/2318316610 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580433f0 con 0x7f0068152170 2024-04-20T21:33:36.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.304+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00681e1510 con 0x7f0068152170 2024-04-20T21:33:36.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.306+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0068074340 con 0x7f0068152170 2024-04-20T21:33:36.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.307+0000 7f0064c00640 1 -- 172.21.15.102:0/2318316610 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0058037070 con 0x7f0068152170 2024-04-20T21:33:36.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.308+0000 7f0064c00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.308+0000 7f0064c00640 1 -- 172.21.15.102:0/2318316610 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f00580ae170 con 0x7f0068152170 2024-04-20T21:33:36.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.309+0000 7f005f400640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.309+0000 7f005f400640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f004406c8f0 0x7f004406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f00680c29f0 tx=0x7f0050009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:36.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.314+0000 7f0064c00640 1 -- 172.21.15.102:0/2318316610 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f005803e070 con 0x7f0068152170 2024-04-20T21:33:36.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.553+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f006806f760 con 0x7f0068152170 2024-04-20T21:33:36.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.554+0000 7f0064c00640 1 -- 172.21.15.102:0/2318316610 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+6034 (secure 0 0 0) 0x7f005807b800 con 0x7f0068152170 2024-04-20T21:33:36.554 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:36.554 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:33:36.554 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 54 2024-04-20T21:33:36.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f004406c8f0 msgr2=0x7f004406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f004406c8f0 0x7f004406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f00680c29f0 tx=0x7f0050009190 comp rx=0 tx=0).stop 2024-04-20T21:33:36.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 msgr2=0x7f0068155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f0068155c50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0058000c00 tx=0x7f00580046c0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 shutdown_connections 2024-04-20T21:33:36.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0068158b80 0x7f00681e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0068158770 0x7f006815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 --2- 172.21.15.102:0/2318316610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0068152170 0x7f0068155c50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 >> 172.21.15.102:0/2318316610 conn(0x7f00680c9390 msgr2=0x7f006806e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 shutdown_connections 2024-04-20T21:33:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:36.557+0000 7f006ee00640 1 -- 172.21.15.102:0/2318316610 wait complete. 2024-04-20T21:33:36.575 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=5342 2024-04-20T21:33:36.575 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.a 2024-04-20T21:33:43.597 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:33:43.598+0000 7f64bcc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:33:44.576 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-04-20T21:33:44.576 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:33:44.576 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:33:44.584 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:33:44.584 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:44.737 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-20T21:33:44.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.767+0000 7fb1f3c00640 1 Processor -- start 2024-04-20T21:33:44.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.767+0000 7fb1f3c00640 1 -- start start 2024-04-20T21:33:44.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f3c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec14fa50 0x7fb1ec0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:44.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f3c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb1ec0c9eb0 0x7fb1ec158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:44.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f3c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb1ec073a10 0x7fb1ec0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:44.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f3c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb1ec159500 con 0x7fb1ec073a10 2024-04-20T21:33:44.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f3c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb1ec1536d0 con 0x7fb1ec0c9eb0 2024-04-20T21:33:44.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f3c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ec0c76c0 con 0x7fb1ec14fa50 2024-04-20T21:33:44.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f1c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb1ec073a10 0x7fb1ec0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:44.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f1200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec14fa50 0x7fb1ec0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:44.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1e3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb1ec0c9eb0 0x7fb1ec158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:44.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f1c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb1ec073a10 0x7fb1ec0c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51940/0 (socket says 172.21.15.102:51940) 2024-04-20T21:33:44.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1e3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb1ec0c9eb0 0x7fb1ec158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:38298/0 (socket says 172.21.15.102:38298) 2024-04-20T21:33:44.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f1200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec14fa50 0x7fb1ec0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:40658/0 (socket says 172.21.15.102:40658) 2024-04-20T21:33:44.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.769+0000 7fb1f1c00640 1 -- 172.21.15.102:0/2460575774 learned_addr learned my addr 172.21.15.102:0/2460575774 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:44.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.770+0000 7fb1f1c00640 1 -- 172.21.15.102:0/2460575774 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec14fa50 msgr2=0x7fb1ec0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.770+0000 7fb1f1c00640 1 --2- 172.21.15.102:0/2460575774 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec14fa50 0x7fb1ec0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.770+0000 7fb1f1c00640 1 -- 172.21.15.102:0/2460575774 >> v2:172.21.15.193:3300/0 conn(0x7fb1ec0c9eb0 msgr2=0x7fb1ec158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.770+0000 7fb1f1c00640 1 --2- 172.21.15.102:0/2460575774 >> v2:172.21.15.193:3300/0 conn(0x7fb1ec0c9eb0 0x7fb1ec158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.770+0000 7fb1f1c00640 1 -- 172.21.15.102:0/2460575774 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec0c7960 con 0x7fb1ec073a10 2024-04-20T21:33:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.770+0000 7fb1e3e00640 1 --2- 172.21.15.102:0/2460575774 >> v2:172.21.15.193:3300/0 conn(0x7fb1ec0c9eb0 0x7fb1ec158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.771+0000 7fb1f1c00640 1 --2- 172.21.15.102:0/2460575774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec073a10 0x7fb1ec0c6170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec069c80 tx=0x7fb1e402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379e41ce0f6caf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.771+0000 7fb1ebe00640 1 -- 172.21.15.102:0/2460575774 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e402cb00 con 0x7fb1ec073a10 2024-04-20T21:33:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.771+0000 7fb1ebe00640 1 -- 172.21.15.102:0/2460575774 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e402cca0 con 0x7fb1ec073a10 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.771+0000 7fb1ebe00640 1 -- 172.21.15.102:0/2460575774 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4034610 con 0x7fb1ec073a10 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 -- 172.21.15.102:0/2460575774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec073a10 msgr2=0x7fb1ec0c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/2460575774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec073a10 0x7fb1ec0c6170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec069c80 tx=0x7fb1e402bf60 comp rx=0 tx=0).stop 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 -- 172.21.15.102:0/2460575774 shutdown_connections 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/2460575774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec073a10 0x7fb1ec0c6170 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/2460575774 >> v2:172.21.15.193:3300/0 conn(0x7fb1ec0c9eb0 0x7fb1ec158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/2460575774 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec14fa50 0x7fb1ec0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 -- 172.21.15.102:0/2460575774 >> 172.21.15.102:0/2460575774 conn(0x7fb1ec0c9390 msgr2=0x7fb1ec0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 -- 172.21.15.102:0/2460575774 shutdown_connections 2024-04-20T21:33:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.772+0000 7fb1f3c00640 1 -- 172.21.15.102:0/2460575774 wait complete. 2024-04-20T21:33:44.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.773+0000 7fb1f3c00640 1 Processor -- start 2024-04-20T21:33:44.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.774+0000 7fb1f3c00640 1 -- start start 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.774+0000 7fb1f3c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec073a10 0x7fb1ec070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.774+0000 7fb1f3c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec0c9eb0 0x7fb1ec0726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.774+0000 7fb1f3c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec14fa50 0x7fb1ec14caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.774+0000 7fb1f3c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec0c7c00 con 0x7fb1ec0c9eb0 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f3c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec0c3d90 con 0x7fb1ec14fa50 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f3c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ec0c4030 con 0x7fb1ec073a10 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec0c9eb0 0x7fb1ec0726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f1200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec073a10 0x7fb1ec070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec0c9eb0 0x7fb1ec0726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51944/0 (socket says 172.21.15.102:51944) 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f1200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec073a10 0x7fb1ec070df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:40664/0 (socket says 172.21.15.102:40664) 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1e3e00640 1 -- 172.21.15.102:0/4219657908 learned_addr learned my addr 172.21.15.102:0/4219657908 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:33:44.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f1c00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec14fa50 0x7fb1ec14caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f1200640 1 -- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec14fa50 msgr2=0x7fb1ec14caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f1200640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec14fa50 0x7fb1ec14caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f1200640 1 -- 172.21.15.102:0/4219657908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec0c9eb0 msgr2=0x7fb1ec0726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f1200640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec0c9eb0 0x7fb1ec0726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1f1200640 1 -- 172.21.15.102:0/4219657908 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec1f4640 con 0x7fb1ec073a10 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.775+0000 7fb1e3e00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec0c9eb0 0x7fb1ec0726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.776+0000 7fb1f1200640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec073a10 0x7fb1ec070df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00e880 tx=0x7fb1dc00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.776+0000 7fb1e9600640 1 -- 172.21.15.102:0/4219657908 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc017070 con 0x7fb1ec073a10 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.776+0000 7fb1e9600640 1 -- 172.21.15.102:0/4219657908 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1dc00c940 con 0x7fb1ec073a10 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.776+0000 7fb1e9600640 1 -- 172.21.15.102:0/4219657908 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00cd30 con 0x7fb1ec073a10 2024-04-20T21:33:44.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.776+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec1f4920 con 0x7fb1ec073a10 2024-04-20T21:33:44.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.776+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec1f4ee0 con 0x7fb1ec073a10 2024-04-20T21:33:44.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.777+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ec074340 con 0x7fb1ec073a10 2024-04-20T21:33:44.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.778+0000 7fb1e9600640 1 -- 172.21.15.102:0/4219657908 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1dc01a070 con 0x7fb1ec073a10 2024-04-20T21:33:44.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.779+0000 7fb1e9600640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb1c806c8f0 0x7fb1c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:44.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.779+0000 7fb1e9600640 1 -- 172.21.15.102:0/4219657908 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7fb1dc08c0b0 con 0x7fb1ec073a10 2024-04-20T21:33:44.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.779+0000 7fb1e3e00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb1c806c8f0 0x7fb1c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:44.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.780+0000 7fb1e3e00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb1c806c8f0 0x7fb1c806edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009560 tx=0x7fb1d4009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:44.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:44.784+0000 7fb1e9600640 1 -- 172.21.15.102:0/4219657908 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb1dc0597d0 con 0x7fb1ec073a10 2024-04-20T21:33:45.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.026+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1ec06ef20 con 0x7fb1ec073a10 2024-04-20T21:33:45.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.027+0000 7fb1e9600640 1 -- 172.21.15.102:0/4219657908 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7fb1dc05dad0 con 0x7fb1ec073a10 2024-04-20T21:33:45.026 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:33:45.027 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:33:45.027 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:33:45.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb1c806c8f0 msgr2=0x7fb1c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:45.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb1c806c8f0 0x7fb1c806edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009560 tx=0x7fb1d4009190 comp rx=0 tx=0).stop 2024-04-20T21:33:45.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec073a10 msgr2=0x7fb1ec070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec073a10 0x7fb1ec070df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00e880 tx=0x7fb1dc00ed40 comp rx=0 tx=0).stop 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 shutdown_connections 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb1c806c8f0 0x7fb1c806edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec14fa50 0x7fb1ec14caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ec0c9eb0 0x7fb1ec0726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 --2- 172.21.15.102:0/4219657908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb1ec073a10 0x7fb1ec070df0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 >> 172.21.15.102:0/4219657908 conn(0x7fb1ec0c9390 msgr2=0x7fb1ec06f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 shutdown_connections 2024-04-20T21:33:45.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:33:45.030+0000 7fb1f3c00640 1 -- 172.21.15.102:0/4219657908 wait complete. 2024-04-20T21:33:45.047 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-04-20T21:33:45.048 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 48 secs before thrashing 2024-04-20T21:33:58.588 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:33:58.597 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:28.715 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:28.725 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:33.048 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:33.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.245+0000 7fde82a00640 1 Processor -- start 2024-04-20T21:34:33.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.246+0000 7fde82a00640 1 -- start start 2024-04-20T21:34:33.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.247+0000 7fde82a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fde7c072870 0x7fde7c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.247+0000 7fde82a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c075670 0x7fde7c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.247+0000 7fde82a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c15b6c0 con 0x7fde7c0c7900 2024-04-20T21:34:33.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.247+0000 7fde82a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c073e20 con 0x7fde7c072870 2024-04-20T21:34:33.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.247+0000 7fde82a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c15b9a0 con 0x7fde7c075670 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7b400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fde7c072870 0x7fde7c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7b400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fde7c072870 0x7fde7c15cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51478/0 (socket says 172.21.15.102:51478) 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7b400640 1 -- 172.21.15.102:0/3211564131 learned_addr learned my addr 172.21.15.102:0/3211564131 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7be00640 1 --1- 172.21.15.102:0/3211564131 >> v1:172.21.15.193:6789/0 conn(0x7fde7c0c7900 0x7fde7c0cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:40030/0 (socket says 172.21.15.102:40030) 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde80a00640 1 --2- 172.21.15.102:0/3211564131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c075670 0x7fde7c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7b400640 1 -- 172.21.15.102:0/3211564131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c075670 msgr2=0x7fde7c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7aa00640 1 -- 172.21.15.102:0/3211564131 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1697267955 0 0) 0x7fde7c15b6c0 con 0x7fde7c0c7900 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7b400640 1 --2- 172.21.15.102:0/3211564131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c075670 0x7fde7c15d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7b400640 1 -- 172.21.15.102:0/3211564131 >> v1:172.21.15.193:6789/0 conn(0x7fde7c0c7900 legacy=0x7fde7c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.248+0000 7fde7b400640 1 -- 172.21.15.102:0/3211564131 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c15bc80 con 0x7fde7c072870 2024-04-20T21:34:33.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.249+0000 7fde7b400640 1 --2- 172.21.15.102:0/3211564131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c072870 0x7fde7c15cda0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fde68009870 tx=0x7fde6802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c9a3ff15350c7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:33.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.250+0000 7fde7aa00640 1 -- 172.21.15.102:0/3211564131 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68034070 con 0x7fde7c072870 2024-04-20T21:34:33.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.250+0000 7fde7aa00640 1 -- 172.21.15.102:0/3211564131 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde68030840 con 0x7fde7c072870 2024-04-20T21:34:33.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.250+0000 7fde7aa00640 1 -- 172.21.15.102:0/3211564131 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68030c10 con 0x7fde7c072870 2024-04-20T21:34:33.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.250+0000 7fde82a00640 1 -- 172.21.15.102:0/3211564131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c072870 msgr2=0x7fde7c15cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.250+0000 7fde82a00640 1 --2- 172.21.15.102:0/3211564131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c072870 0x7fde7c15cda0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fde68009870 tx=0x7fde6802b980 comp rx=0 tx=0).stop 2024-04-20T21:34:33.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.251+0000 7fde82a00640 1 -- 172.21.15.102:0/3211564131 shutdown_connections 2024-04-20T21:34:33.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.251+0000 7fde82a00640 1 --2- 172.21.15.102:0/3211564131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c075670 0x7fde7c15d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.251+0000 7fde82a00640 1 --2- 172.21.15.102:0/3211564131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c072870 0x7fde7c15cda0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.251+0000 7fde82a00640 1 -- 172.21.15.102:0/3211564131 >> 172.21.15.102:0/3211564131 conn(0x7fde7c06fa00 msgr2=0x7fde7c06fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:33.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.251+0000 7fde82a00640 1 -- 172.21.15.102:0/3211564131 shutdown_connections 2024-04-20T21:34:33.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.251+0000 7fde82a00640 1 -- 172.21.15.102:0/3211564131 wait complete. 2024-04-20T21:34:33.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.252+0000 7fde82a00640 1 Processor -- start 2024-04-20T21:34:33.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.253+0000 7fde82a00640 1 -- start start 2024-04-20T21:34:33.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde82a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c072870 0x7fde7c151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde82a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c075670 0x7fde7c1e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde82a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c0c7900 0x7fde7c1e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde82a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c15c880 con 0x7fde7c0c7900 2024-04-20T21:34:33.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde82a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c15c240 con 0x7fde7c075670 2024-04-20T21:34:33.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde82a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c15b390 con 0x7fde7c072870 2024-04-20T21:34:33.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde80a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c0c7900 0x7fde7c1e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde80a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c0c7900 0x7fde7c1e4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33946/0 (socket says 172.21.15.102:33946) 2024-04-20T21:34:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde80a00640 1 -- 172.21.15.102:0/1428140298 learned_addr learned my addr 172.21.15.102:0/1428140298 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde7b400640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c075670 0x7fde7c1e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde7be00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c072870 0x7fde7c151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde7b400640 1 -- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c072870 msgr2=0x7fde7c151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde7b400640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c072870 0x7fde7c151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde7b400640 1 -- 172.21.15.102:0/1428140298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c0c7900 msgr2=0x7fde7c1e4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.254+0000 7fde7b400640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c0c7900 0x7fde7c1e4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.255+0000 7fde7b400640 1 -- 172.21.15.102:0/1428140298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c1e32e0 con 0x7fde7c075670 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.255+0000 7fde80a00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c0c7900 0x7fde7c1e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.255+0000 7fde7b400640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c075670 0x7fde7c1e45d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0708d0 tx=0x7fde680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.255+0000 7fde73e00640 1 -- 172.21.15.102:0/1428140298 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68034040 con 0x7fde7c075670 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.255+0000 7fde73e00640 1 -- 172.21.15.102:0/1428140298 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde68039d90 con 0x7fde7c075670 2024-04-20T21:34:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.255+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c1f5090 con 0x7fde7c075670 2024-04-20T21:34:33.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.255+0000 7fde73e00640 1 -- 172.21.15.102:0/1428140298 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68043610 con 0x7fde7c075670 2024-04-20T21:34:33.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.255+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1e3dd0 con 0x7fde7c075670 2024-04-20T21:34:33.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.256+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c068c30 con 0x7fde7c075670 2024-04-20T21:34:33.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.258+0000 7fde73e00640 1 -- 172.21.15.102:0/1428140298 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde68037070 con 0x7fde7c075670 2024-04-20T21:34:33.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.259+0000 7fde73e00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5c06c920 0x7fde5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.260+0000 7fde73e00640 1 -- 172.21.15.102:0/1428140298 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7fde680ae280 con 0x7fde7c075670 2024-04-20T21:34:33.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.260+0000 7fde7be00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5c06c920 0x7fde5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.261+0000 7fde7be00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5c06c920 0x7fde5c06ede0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde6c000f60 tx=0x7fde6c007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:33.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.264+0000 7fde73e00640 1 -- 172.21.15.102:0/1428140298 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fde6803e070 con 0x7fde7c075670 2024-04-20T21:34:33.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.588+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde7c0c8340 con 0x7fde7c075670 2024-04-20T21:34:33.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.589+0000 7fde73e00640 1 -- 172.21.15.102:0/1428140298 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7fde6807ba40 con 0x7fde7c075670 2024-04-20T21:34:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:33.589 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:34:33.589 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:34:33.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5c06c920 msgr2=0x7fde5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5c06c920 0x7fde5c06ede0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde6c000f60 tx=0x7fde6c007d50 comp rx=0 tx=0).stop 2024-04-20T21:34:33.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c075670 msgr2=0x7fde7c1e45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c075670 0x7fde7c1e45d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0708d0 tx=0x7fde680046c0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 shutdown_connections 2024-04-20T21:34:33.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c0c7900 0x7fde7c1e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5c06c920 0x7fde5c06ede0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c075670 0x7fde7c1e45d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 --2- 172.21.15.102:0/1428140298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c072870 0x7fde7c151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.592+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 >> 172.21.15.102:0/1428140298 conn(0x7fde7c06fa00 msgr2=0x7fde7c06fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:33.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.593+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 shutdown_connections 2024-04-20T21:34:33.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.593+0000 7fde82a00640 1 -- 172.21.15.102:0/1428140298 wait complete. 2024-04-20T21:34:33.614 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:34:33.614 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:34:33.614 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:34:33.641 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:34:33.641 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:33.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.888+0000 7fe2ba600640 1 Processor -- start 2024-04-20T21:34:33.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.888+0000 7fe2ba600640 1 -- start start 2024-04-20T21:34:33.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.889+0000 7fe2ba600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe2b4153180 0x7fe2b4158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.889+0000 7fe2ba600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.889+0000 7fe2ba600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe2b40c75a0 0x7fe2b40c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.889+0000 7fe2ba600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b4153560 con 0x7fe2b40c75a0 2024-04-20T21:34:33.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.889+0000 7fe2ba600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b40c7a70 con 0x7fe2b4153180 2024-04-20T21:34:33.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.889+0000 7fe2ba600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b40c7d30 con 0x7fe2b4159380 2024-04-20T21:34:33.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2abe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2abe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:39156/0 (socket says 172.21.15.102:39156) 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2abe00640 1 -- 172.21.15.102:0/1298190535 learned_addr learned my addr 172.21.15.102:0/1298190535 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2b9200640 1 --2- 172.21.15.102:0/1298190535 >> v2:172.21.15.193:3300/0 conn(0x7fe2b4153180 0x7fe2b4158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2b9c00640 1 --2- 172.21.15.102:0/1298190535 >> v2:172.21.15.102:3300/0 conn(0x7fe2b40c75a0 0x7fe2b40c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2abe00640 1 -- 172.21.15.102:0/1298190535 >> v2:172.21.15.193:3300/0 conn(0x7fe2b4153180 msgr2=0x7fe2b4158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2abe00640 1 --2- 172.21.15.102:0/1298190535 >> v2:172.21.15.193:3300/0 conn(0x7fe2b4153180 0x7fe2b4158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2abe00640 1 -- 172.21.15.102:0/1298190535 >> v2:172.21.15.102:3300/0 conn(0x7fe2b40c75a0 msgr2=0x7fe2b40c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2abe00640 1 --2- 172.21.15.102:0/1298190535 >> v2:172.21.15.102:3300/0 conn(0x7fe2b40c75a0 0x7fe2b40c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.890+0000 7fe2abe00640 1 -- 172.21.15.102:0/1298190535 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b415c4d0 con 0x7fe2b4159380 2024-04-20T21:34:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.891+0000 7fe2abe00640 1 --2- 172.21.15.102:0/1298190535 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c5f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40cd860 tx=0x7fe29c02b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=977d080f4546a9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2b3e00640 1 -- 172.21.15.102:0/1298190535 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c035020 con 0x7fe2b4159380 2024-04-20T21:34:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2b3e00640 1 -- 172.21.15.102:0/1298190535 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c031870 con 0x7fe2b4159380 2024-04-20T21:34:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2b3e00640 1 -- 172.21.15.102:0/1298190535 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c031c40 con 0x7fe2b4159380 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 -- 172.21.15.102:0/1298190535 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 msgr2=0x7fe2b40c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 --2- 172.21.15.102:0/1298190535 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c5f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40cd860 tx=0x7fe29c02b910 comp rx=0 tx=0).stop 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 -- 172.21.15.102:0/1298190535 shutdown_connections 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 --2- 172.21.15.102:0/1298190535 >> v2:172.21.15.102:3300/0 conn(0x7fe2b40c75a0 0x7fe2b40c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 --2- 172.21.15.102:0/1298190535 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c5f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 --2- 172.21.15.102:0/1298190535 >> v2:172.21.15.193:3300/0 conn(0x7fe2b4153180 0x7fe2b4158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 -- 172.21.15.102:0/1298190535 >> 172.21.15.102:0/1298190535 conn(0x7fe2b40cc820 msgr2=0x7fe2b40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 -- 172.21.15.102:0/1298190535 shutdown_connections 2024-04-20T21:34:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.892+0000 7fe2ba600640 1 -- 172.21.15.102:0/1298190535 wait complete. 2024-04-20T21:34:33.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.894+0000 7fe2ba600640 1 Processor -- start 2024-04-20T21:34:33.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.894+0000 7fe2ba600640 1 -- start start 2024-04-20T21:34:33.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.895+0000 7fe2ba600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b40c75a0 0x7fe2b4158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.895+0000 7fe2ba600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2b4153180 0x7fe2b40c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.895+0000 7fe2ba600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.895+0000 7fe2ba600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b415d290 con 0x7fe2b4153180 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.895+0000 7fe2ba600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b415ca90 con 0x7fe2b40c75a0 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.895+0000 7fe2ba600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b40c6be0 con 0x7fe2b4159380 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2b4153180 0x7fe2b40c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2b9200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b40c75a0 0x7fe2b4158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2b9c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2b4153180 0x7fe2b40c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33960/0 (socket says 172.21.15.102:33960) 2024-04-20T21:34:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 -- 172.21.15.102:0/2533988418 learned_addr learned my addr 172.21.15.102:0/2533988418 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 -- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 msgr2=0x7fe2b40c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 -- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b40c75a0 msgr2=0x7fe2b4158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b40c75a0 0x7fe2b4158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 -- 172.21.15.102:0/2533988418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b4159160 con 0x7fe2b4153180 2024-04-20T21:34:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2b9200640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b40c75a0 0x7fe2b4158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.896+0000 7fe2abe00640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2b4153180 0x7fe2b40c4770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe29c000c00 tx=0x7fe29c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.897+0000 7fe2b1600640 1 -- 172.21.15.102:0/2533988418 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c035040 con 0x7fe2b4153180 2024-04-20T21:34:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.897+0000 7fe2b1600640 1 -- 172.21.15.102:0/2533988418 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c030d00 con 0x7fe2b4153180 2024-04-20T21:34:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.897+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b415a3d0 con 0x7fe2b4153180 2024-04-20T21:34:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.897+0000 7fe2b1600640 1 -- 172.21.15.102:0/2533988418 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0456a0 con 0x7fe2b4153180 2024-04-20T21:34:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.897+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41f1dc0 con 0x7fe2b4153180 2024-04-20T21:34:33.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.898+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b404fd60 con 0x7fe2b4153180 2024-04-20T21:34:33.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.899+0000 7fe2b1600640 1 -- 172.21.15.102:0/2533988418 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe29c038020 con 0x7fe2b4153180 2024-04-20T21:34:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.900+0000 7fe2b1600640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.900+0000 7fe2b1600640 1 -- 172.21.15.102:0/2533988418 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7fe29c0ae480 con 0x7fe2b4153180 2024-04-20T21:34:33.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.900+0000 7fe2b9200640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:33.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.901+0000 7fe2b9200640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe29006c920 0x7fe29006ede0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009f30 tx=0x7fe2a4009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:33.905+0000 7fe2b1600640 1 -- 172.21.15.102:0/2533988418 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe29c07ba80 con 0x7fe2b4153180 2024-04-20T21:34:34.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.152+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2b4150450 con 0x7fe2b4153180 2024-04-20T21:34:34.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.153+0000 7fe2b1600640 1 -- 172.21.15.102:0/2533988418 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7fe29c07fd80 con 0x7fe2b4153180 2024-04-20T21:34:34.153 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:34.153 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:34:34.153 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe29006c920 msgr2=0x7fe29006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe29006c920 0x7fe29006ede0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009f30 tx=0x7fe2a4009190 comp rx=0 tx=0).stop 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2b4153180 msgr2=0x7fe2b40c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2b4153180 0x7fe2b40c4770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe29c000c00 tx=0x7fe29c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 shutdown_connections 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2b4159380 0x7fe2b40c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2b4153180 0x7fe2b40c4770 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe29006c920 0x7fe29006ede0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 --2- 172.21.15.102:0/2533988418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b40c75a0 0x7fe2b4158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.156+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 >> 172.21.15.102:0/2533988418 conn(0x7fe2b40cc820 msgr2=0x7fe2b4142350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.157+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 shutdown_connections 2024-04-20T21:34:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:34.157+0000 7fe2ba600640 1 -- 172.21.15.102:0/2533988418 wait complete. 2024-04-20T21:34:34.174 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:34:36.175 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:36.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.385+0000 7f3ffb000640 1 Processor -- start 2024-04-20T21:34:36.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.385+0000 7f3ffb000640 1 -- start start 2024-04-20T21:34:36.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.386+0000 7f3ffb000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff40c9eb0 0x7f3ff40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.386+0000 7f3ffb000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff41536d0 con 0x7f3ff414fa50 2024-04-20T21:34:36.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.386+0000 7f3ffb000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff40c76c0 con 0x7f3ff4073a10 2024-04-20T21:34:36.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.386+0000 7f3ffb000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40c7960 con 0x7f3ff40c9eb0 2024-04-20T21:34:36.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff3e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3ff414fa50 0x7f3ff40c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60194/0 (socket says 172.21.15.102:60194) 2024-04-20T21:34:36.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff3e00640 1 -- 172.21.15.102:0/3345503861 learned_addr learned my addr 172.21.15.102:0/3345503861 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:34:36.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff3400640 1 --2- 172.21.15.102:0/3345503861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff40c9eb0 0x7f3ff40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff2a00640 1 -- 172.21.15.102:0/3345503861 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2707926096 0 0) 0x7f3fe4001110 con 0x7f3ff414fa50 2024-04-20T21:34:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff2a00640 1 -- 172.21.15.102:0/3345503861 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fd4003520 con 0x7f3ff414fa50 2024-04-20T21:34:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff3400640 1 -- 172.21.15.102:0/3345503861 >> v1:172.21.15.193:6789/0 conn(0x7f3ff4073a10 legacy=0x7f3ff4163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff2a00640 1 -- 172.21.15.102:0/3345503861 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537803063 0 0) 0x7f3ff40c76c0 con 0x7f3ff4073a10 2024-04-20T21:34:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff3400640 1 -- 172.21.15.102:0/3345503861 >> v1:172.21.15.102:6789/0 conn(0x7f3ff414fa50 legacy=0x7f3ff40c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.387+0000 7f3ff3400640 1 -- 172.21.15.102:0/3345503861 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40c3d90 con 0x7f3ff40c9eb0 2024-04-20T21:34:36.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.388+0000 7f3ff3400640 1 --2- 172.21.15.102:0/3345503861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff40c9eb0 0x7f3ff40c4830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40c3f60 tx=0x7f3fdc02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917bf410fbbba0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:36.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.389+0000 7f3ff2a00640 1 -- 172.21.15.102:0/3345503861 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc039070 con 0x7f3ff40c9eb0 2024-04-20T21:34:36.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.389+0000 7f3ff2a00640 1 -- 172.21.15.102:0/3345503861 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc02cd50 con 0x7f3ff40c9eb0 2024-04-20T21:34:36.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.389+0000 7f3ff2a00640 1 -- 172.21.15.102:0/3345503861 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc034660 con 0x7f3ff40c9eb0 2024-04-20T21:34:36.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.389+0000 7f3ffb000640 1 -- 172.21.15.102:0/3345503861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff40c9eb0 msgr2=0x7f3ff40c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.389+0000 7f3ffb000640 1 --2- 172.21.15.102:0/3345503861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff40c9eb0 0x7f3ff40c4830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40c3f60 tx=0x7f3fdc02bfa0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.389+0000 7f3ffb000640 1 -- 172.21.15.102:0/3345503861 shutdown_connections 2024-04-20T21:34:36.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.389+0000 7f3ffb000640 1 --2- 172.21.15.102:0/3345503861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff40c9eb0 0x7f3ff40c4830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.389+0000 7f3ffb000640 1 -- 172.21.15.102:0/3345503861 >> 172.21.15.102:0/3345503861 conn(0x7f3ff40c9390 msgr2=0x7f3ff40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:36.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.390+0000 7f3ffb000640 1 -- 172.21.15.102:0/3345503861 shutdown_connections 2024-04-20T21:34:36.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.390+0000 7f3ffb000640 1 -- 172.21.15.102:0/3345503861 wait complete. 2024-04-20T21:34:36.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.391+0000 7f3ffb000640 1 Processor -- start 2024-04-20T21:34:36.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.391+0000 7f3ffb000640 1 -- start start 2024-04-20T21:34:36.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.392+0000 7f3ffb000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff4073a10 0x7f3ff406cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.392+0000 7f3ffb000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff40c9eb0 0x7f3ff41ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.392+0000 7f3ffb000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff414fa50 0x7f3ff41ef980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.392+0000 7f3ffb000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40c4030 con 0x7f3ff4073a10 2024-04-20T21:34:36.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.392+0000 7f3ffb000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40c7c00 con 0x7f3ff40c9eb0 2024-04-20T21:34:36.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.392+0000 7f3ffb000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40c4310 con 0x7f3ff414fa50 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff4073a10 0x7f3ff406cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff4073a10 0x7f3ff406cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33962/0 (socket says 172.21.15.102:33962) 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 -- 172.21.15.102:0/2689594382 learned_addr learned my addr 172.21.15.102:0/2689594382 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3400640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff40c9eb0 0x7f3ff41ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff9000640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff414fa50 0x7f3ff41ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 -- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff414fa50 msgr2=0x7f3ff41ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff414fa50 0x7f3ff41ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 -- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff40c9eb0 msgr2=0x7f3ff41ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff40c9eb0 0x7f3ff41ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 -- 172.21.15.102:0/2689594382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff41581e0 con 0x7f3ff4073a10 2024-04-20T21:34:36.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.393+0000 7f3ff3e00640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff4073a10 0x7f3ff406cc50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c3a0 tx=0x7f3fe400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:36.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.394+0000 7f3febe00640 1 -- 172.21.15.102:0/2689594382 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4017070 con 0x7f3ff4073a10 2024-04-20T21:34:36.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.394+0000 7f3febe00640 1 -- 172.21.15.102:0/2689594382 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe4005ce0 con 0x7f3ff4073a10 2024-04-20T21:34:36.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.394+0000 7f3febe00640 1 -- 172.21.15.102:0/2689594382 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4007310 con 0x7f3ff4073a10 2024-04-20T21:34:36.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.394+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff41584c0 con 0x7f3ff4073a10 2024-04-20T21:34:36.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.394+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff4158a80 con 0x7f3ff4073a10 2024-04-20T21:34:36.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.395+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff4074340 con 0x7f3ff4073a10 2024-04-20T21:34:36.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.396+0000 7f3febe00640 1 -- 172.21.15.102:0/2689594382 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fe401a070 con 0x7f3ff4073a10 2024-04-20T21:34:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.397+0000 7f3febe00640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3fd006c8f0 0x7f3fd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.398+0000 7f3febe00640 1 -- 172.21.15.102:0/2689594382 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f3fe408b5d0 con 0x7f3ff4073a10 2024-04-20T21:34:36.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.398+0000 7f3ff3400640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3fd006c8f0 0x7f3fd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.398+0000 7f3ff3400640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3fd006c8f0 0x7f3fd006edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc036000 tx=0x7f3fdc030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:36.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.403+0000 7f3febe00640 1 -- 172.21.15.102:0/2689594382 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3ff41536d0 con 0x7f3ff4073a10 2024-04-20T21:34:36.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.642+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ff415d3c0 con 0x7f3ff4073a10 2024-04-20T21:34:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.643+0000 7f3febe00640 1 -- 172.21.15.102:0/2689594382 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7f3fe4058ce0 con 0x7f3ff4073a10 2024-04-20T21:34:36.643 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:36.643 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:34:36.643 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:34:36.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3fd006c8f0 msgr2=0x7f3fd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3fd006c8f0 0x7f3fd006edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc036000 tx=0x7f3fdc030000 comp rx=0 tx=0).stop 2024-04-20T21:34:36.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff4073a10 msgr2=0x7f3ff406cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff4073a10 0x7f3ff406cc50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c3a0 tx=0x7f3fe400c860 comp rx=0 tx=0).stop 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 shutdown_connections 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ff414fa50 0x7f3ff41ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3fd006c8f0 0x7f3fd006edb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff40c9eb0 0x7f3ff41ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 --2- 172.21.15.102:0/2689594382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff4073a10 0x7f3ff406cc50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 >> 172.21.15.102:0/2689594382 conn(0x7f3ff40c9390 msgr2=0x7f3ff414e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 shutdown_connections 2024-04-20T21:34:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:36.646+0000 7f3ffb000640 1 -- 172.21.15.102:0/2689594382 wait complete. 2024-04-20T21:34:36.663 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:34:38.664 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:38.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.871+0000 7f080a400640 1 Processor -- start 2024-04-20T21:34:38.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.871+0000 7f080a400640 1 -- start start 2024-04-20T21:34:38.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.873+0000 7f080a400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f08040cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:38.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.873+0000 7f080a400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080415ef60 con 0x7f0804155530 2024-04-20T21:34:38.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.873+0000 7f080a400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080415f240 con 0x7f08040cdfc0 2024-04-20T21:34:38.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.873+0000 7f080a400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f08040c5ca0 con 0x7f08040cc470 2024-04-20T21:34:38.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0803e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f08040cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:38.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0809000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0804155530 0x7f080415e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50320/0 (socket says 172.21.15.102:50320) 2024-04-20T21:34:38.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0809a00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f08040cdfc0 0x7f08040cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:33364/0 (socket says 172.21.15.102:33364) 2024-04-20T21:34:38.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0809000640 1 -- 172.21.15.102:0/1929413489 learned_addr learned my addr 172.21.15.102:0/1929413489 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:34:38.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0803400640 1 -- 172.21.15.102:0/1929413489 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034558720 0 0) 0x7f080415f240 con 0x7f08040cdfc0 2024-04-20T21:34:38.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0803400640 1 -- 172.21.15.102:0/1929413489 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07e4003520 con 0x7f08040cdfc0 2024-04-20T21:34:38.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0803400640 1 -- 172.21.15.102:0/1929413489 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512971433 0 0) 0x7f080415ef60 con 0x7f0804155530 2024-04-20T21:34:38.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0803400640 1 -- 172.21.15.102:0/1929413489 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f080415f240 con 0x7f0804155530 2024-04-20T21:34:38.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0803e00640 1 -- 172.21.15.102:0/1929413489 >> v1:172.21.15.193:6789/0 conn(0x7f08040cdfc0 legacy=0x7f08040cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:38.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0803e00640 1 -- 172.21.15.102:0/1929413489 >> v1:172.21.15.102:6789/0 conn(0x7f0804155530 legacy=0x7f080415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:38.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.874+0000 7f0803e00640 1 -- 172.21.15.102:0/1929413489 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080415f520 con 0x7f08040cc470 2024-04-20T21:34:38.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.875+0000 7f0803e00640 1 --2- 172.21.15.102:0/1929413489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f08040cda80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009dc0 tx=0x7f07ec02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d2e1e3f28d0b372 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:38.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.876+0000 7f0803400640 1 -- 172.21.15.102:0/1929413489 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec039070 con 0x7f08040cc470 2024-04-20T21:34:38.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.876+0000 7f0803400640 1 -- 172.21.15.102:0/1929413489 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec02ccd0 con 0x7f08040cc470 2024-04-20T21:34:38.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.876+0000 7f0803400640 1 -- 172.21.15.102:0/1929413489 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec034660 con 0x7f08040cc470 2024-04-20T21:34:38.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.876+0000 7f080a400640 1 -- 172.21.15.102:0/1929413489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 msgr2=0x7f08040cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:38.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.876+0000 7f080a400640 1 --2- 172.21.15.102:0/1929413489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f08040cda80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009dc0 tx=0x7f07ec02bf20 comp rx=0 tx=0).stop 2024-04-20T21:34:38.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.877+0000 7f080a400640 1 -- 172.21.15.102:0/1929413489 shutdown_connections 2024-04-20T21:34:38.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.877+0000 7f080a400640 1 --2- 172.21.15.102:0/1929413489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f08040cda80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:38.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.877+0000 7f080a400640 1 -- 172.21.15.102:0/1929413489 >> 172.21.15.102:0/1929413489 conn(0x7f0804075080 msgr2=0x7f0804075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:38.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.877+0000 7f080a400640 1 -- 172.21.15.102:0/1929413489 shutdown_connections 2024-04-20T21:34:38.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.877+0000 7f080a400640 1 -- 172.21.15.102:0/1929413489 wait complete. 2024-04-20T21:34:38.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.879+0000 7f080a400640 1 Processor -- start 2024-04-20T21:34:38.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.879+0000 7f080a400640 1 -- start start 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.880+0000 7f080a400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f0804152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.880+0000 7f080a400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08040cdfc0 0x7f08041f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.880+0000 7f080a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0804155530 0x7f08041f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.880+0000 7f080a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f08040cd280 con 0x7f0804155530 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.880+0000 7f080a400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08040cd5f0 con 0x7f08040cdfc0 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.880+0000 7f080a400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f08040ccbe0 con 0x7f08040cc470 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0809000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f0804152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0803e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08040cdfc0 0x7f08041f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0803e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08040cdfc0 0x7f08041f0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:59574/0 (socket says 172.21.15.102:59574) 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0803e00640 1 -- 172.21.15.102:0/3584597670 learned_addr learned my addr 172.21.15.102:0/3584597670 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:38.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0809a00640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0804155530 0x7f08041f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:38.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0803e00640 1 -- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 msgr2=0x7f0804152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:38.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0803e00640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f0804152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:38.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0803e00640 1 -- 172.21.15.102:0/3584597670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0804155530 msgr2=0x7f08041f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0803e00640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0804155530 0x7f08041f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0803e00640 1 -- 172.21.15.102:0/3584597670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080415ecd0 con 0x7f08040cdfc0 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0809000640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f0804152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.881+0000 7f0809a00640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0804155530 0x7f08041f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.882+0000 7f0803e00640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08040cdfc0 0x7f08041f0da0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f07ec037040 tx=0x7f07ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.882+0000 7f0800c00640 1 -- 172.21.15.102:0/3584597670 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec039050 con 0x7f08040cdfc0 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.882+0000 7f0800c00640 1 -- 172.21.15.102:0/3584597670 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec045070 con 0x7f08040cdfc0 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.882+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08041e63a0 con 0x7f08040cdfc0 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.882+0000 7f0800c00640 1 -- 172.21.15.102:0/3584597670 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec009450 con 0x7f08040cdfc0 2024-04-20T21:34:38.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.882+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080415e9f0 con 0x7f08040cdfc0 2024-04-20T21:34:38.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.883+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080404fd60 con 0x7f08040cdfc0 2024-04-20T21:34:38.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.884+0000 7f0800c00640 1 -- 172.21.15.102:0/3584597670 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07ec03e850 con 0x7f08040cdfc0 2024-04-20T21:34:38.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.885+0000 7f0800c00640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f07e006c8f0 0x7f07e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:38.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.885+0000 7f0800c00640 1 -- 172.21.15.102:0/3584597670 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f07ec0aeda0 con 0x7f08040cdfc0 2024-04-20T21:34:38.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.886+0000 7f0809000640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f07e006c8f0 0x7f07e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:38.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.886+0000 7f0809000640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f07e006c8f0 0x7f07e006edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f07f4005880 tx=0x7f07f4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:38.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:38.890+0000 7f0800c00640 1 -- 172.21.15.102:0/3584597670 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f07ec07c5e0 con 0x7f08040cdfc0 2024-04-20T21:34:39.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.129+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08040cc9c0 con 0x7f08040cdfc0 2024-04-20T21:34:39.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.130+0000 7f0800c00640 1 -- 172.21.15.102:0/3584597670 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7f07ec0808e0 con 0x7f08040cdfc0 2024-04-20T21:34:39.129 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:39.129 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:34:39.130 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:34:39.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f07e006c8f0 msgr2=0x7f07e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:39.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f07e006c8f0 0x7f07e006edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f07f4005880 tx=0x7f07f4009040 comp rx=0 tx=0).stop 2024-04-20T21:34:39.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08040cdfc0 msgr2=0x7f08041f0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:39.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08040cdfc0 0x7f08041f0da0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f07ec037040 tx=0x7f07ec004a40 comp rx=0 tx=0).stop 2024-04-20T21:34:39.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 shutdown_connections 2024-04-20T21:34:39.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0804155530 0x7f08041f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:39.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f07e006c8f0 0x7f07e006edb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:39.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08040cdfc0 0x7f08041f0da0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:39.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 --2- 172.21.15.102:0/3584597670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08040cc470 0x7f0804152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:39.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 >> 172.21.15.102:0/3584597670 conn(0x7f0804075080 msgr2=0x7f08040b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:39.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 shutdown_connections 2024-04-20T21:34:39.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:39.133+0000 7f080a400640 1 -- 172.21.15.102:0/3584597670 wait complete. 2024-04-20T21:34:39.153 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:34:41.153 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:41.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.354+0000 7f24ad800640 1 Processor -- start 2024-04-20T21:34:41.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.355+0000 7f24ad800640 1 -- start start 2024-04-20T21:34:41.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.356+0000 7f24ad800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f24a8152170 0x7f24a80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:41.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.356+0000 7f24ad800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a8071f70 con 0x7f24a8158b80 2024-04-20T21:34:41.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.356+0000 7f24ad800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a8072210 con 0x7f24a8158770 2024-04-20T21:34:41.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.356+0000 7f24ad800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f24a80c6dc0 con 0x7f24a8152170 2024-04-20T21:34:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a6a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f24a8152170 0x7f24a80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a7400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f24a8158b80 0x7f24a81628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50330/0 (socket says 172.21.15.102:50330) 2024-04-20T21:34:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a7400640 1 -- 172.21.15.102:0/1990155518 learned_addr learned my addr 172.21.15.102:0/1990155518 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:34:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a5600640 1 -- 172.21.15.102:0/1990155518 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1684835842 0 0) 0x7f24a8072210 con 0x7f24a8158770 2024-04-20T21:34:41.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a5600640 1 -- 172.21.15.102:0/1990155518 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f248c003520 con 0x7f24a8158770 2024-04-20T21:34:41.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a6a00640 1 -- 172.21.15.102:0/1990155518 >> v1:172.21.15.193:6789/0 conn(0x7f24a8158770 legacy=0x7f24a8070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:41.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a6a00640 1 -- 172.21.15.102:0/1990155518 >> v1:172.21.15.102:6789/0 conn(0x7f24a8158b80 legacy=0x7f24a81628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:41.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a5600640 1 -- 172.21.15.102:0/1990155518 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444065058 0 0) 0x7f24a8071f70 con 0x7f24a8158b80 2024-04-20T21:34:41.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.357+0000 7f24a6a00640 1 -- 172.21.15.102:0/1990155518 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a80724b0 con 0x7f24a8152170 2024-04-20T21:34:41.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.358+0000 7f24a6a00640 1 --2- 172.21.15.102:0/1990155518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8152170 0x7f24a80c8db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f24980097a0 tx=0x7f249802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=993ff13104ea33e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:41.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24a5600640 1 -- 172.21.15.102:0/1990155518 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249802f810 con 0x7f24a8152170 2024-04-20T21:34:41.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24a5600640 1 -- 172.21.15.102:0/1990155518 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249802f9b0 con 0x7f24a8152170 2024-04-20T21:34:41.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24a5600640 1 -- 172.21.15.102:0/1990155518 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249802fd80 con 0x7f24a8152170 2024-04-20T21:34:41.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24ad800640 1 -- 172.21.15.102:0/1990155518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8152170 msgr2=0x7f24a80c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:41.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24ad800640 1 --2- 172.21.15.102:0/1990155518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8152170 0x7f24a80c8db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f24980097a0 tx=0x7f249802b9f0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24ad800640 1 -- 172.21.15.102:0/1990155518 shutdown_connections 2024-04-20T21:34:41.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24ad800640 1 --2- 172.21.15.102:0/1990155518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8152170 0x7f24a80c8db0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24ad800640 1 -- 172.21.15.102:0/1990155518 >> 172.21.15.102:0/1990155518 conn(0x7f24a80c9390 msgr2=0x7f24a80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:41.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24ad800640 1 -- 172.21.15.102:0/1990155518 shutdown_connections 2024-04-20T21:34:41.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.359+0000 7f24ad800640 1 -- 172.21.15.102:0/1990155518 wait complete. 2024-04-20T21:34:41.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.361+0000 7f24ad800640 1 Processor -- start 2024-04-20T21:34:41.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.361+0000 7f24ad800640 1 -- start start 2024-04-20T21:34:41.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24ad800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24a8152170 0x7f24a81505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:41.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24ad800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a8158770 0x7f24a81f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:41.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24ad800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8158b80 0x7f24a81f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:41.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24ad800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f24a80c9f20 con 0x7f24a8158770 2024-04-20T21:34:41.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24ad800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8158250 con 0x7f24a8158b80 2024-04-20T21:34:41.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24ad800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f24a8157f20 con 0x7f24a8152170 2024-04-20T21:34:41.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24a6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a8158770 0x7f24a81f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:41.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24a7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8158b80 0x7f24a81f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:41.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24a6a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24a8152170 0x7f24a81505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:41.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24a6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a8158770 0x7f24a81f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38738/0 (socket says 172.21.15.102:38738) 2024-04-20T21:34:41.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.362+0000 7f24a6000640 1 -- 172.21.15.102:0/2426221195 learned_addr learned my addr 172.21.15.102:0/2426221195 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f24a6000640 1 -- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24a8152170 msgr2=0x7f24a81505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f24a6000640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24a8152170 0x7f24a81505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f24a6000640 1 -- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8158b80 msgr2=0x7f24a81f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f24a6000640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8158b80 0x7f24a81f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f24a6000640 1 -- 172.21.15.102:0/2426221195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a80c2d50 con 0x7f24a8158770 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f24a6000640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a8158770 0x7f24a81f4080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f249400c530 tx=0x7f249400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f249ea00640 1 -- 172.21.15.102:0/2426221195 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24940052d0 con 0x7f24a8158770 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f249ea00640 1 -- 172.21.15.102:0/2426221195 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2494005490 con 0x7f24a8158770 2024-04-20T21:34:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f249ea00640 1 -- 172.21.15.102:0/2426221195 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494012870 con 0x7f24a8158770 2024-04-20T21:34:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.363+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a814e080 con 0x7f24a8158770 2024-04-20T21:34:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.364+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a814e640 con 0x7f24a8158770 2024-04-20T21:34:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.365+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a80c4270 con 0x7f24a8158770 2024-04-20T21:34:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.366+0000 7f249ea00640 1 -- 172.21.15.102:0/2426221195 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2494012a10 con 0x7f24a8158770 2024-04-20T21:34:41.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.367+0000 7f249ea00640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f248806c8f0 0x7f248806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:41.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.367+0000 7f249ea00640 1 -- 172.21.15.102:0/2426221195 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f249408be00 con 0x7f24a8158770 2024-04-20T21:34:41.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.367+0000 7f24a6a00640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f248806c8f0 0x7f248806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:41.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.368+0000 7f24a6a00640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f248806c8f0 0x7f248806edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2498036000 tx=0x7f2498008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:41.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.371+0000 7f249ea00640 1 -- 172.21.15.102:0/2426221195 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f24a80c4270 con 0x7f24a8158770 2024-04-20T21:34:41.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.612+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24a8074340 con 0x7f24a8158770 2024-04-20T21:34:41.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.613+0000 7f249ea00640 1 -- 172.21.15.102:0/2426221195 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7f2494059410 con 0x7f24a8158770 2024-04-20T21:34:41.612 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:41.612 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:34:41.612 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:34:41.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.615+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f248806c8f0 msgr2=0x7f248806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:41.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.615+0000 7f24ad800640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f248806c8f0 0x7f248806edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2498036000 tx=0x7f2498008030 comp rx=0 tx=0).stop 2024-04-20T21:34:41.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.615+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a8158770 msgr2=0x7f24a81f4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:41.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.615+0000 7f24ad800640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a8158770 0x7f24a81f4080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f249400c530 tx=0x7f249400c9f0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.616+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 shutdown_connections 2024-04-20T21:34:41.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.616+0000 7f24ad800640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f248806c8f0 0x7f248806edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.616+0000 7f24ad800640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a8158b80 0x7f24a81f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.616+0000 7f24ad800640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24a8158770 0x7f24a81f4080 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.616+0000 7f24ad800640 1 --2- 172.21.15.102:0/2426221195 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24a8152170 0x7f24a81505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:41.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.616+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 >> 172.21.15.102:0/2426221195 conn(0x7f24a80c9390 msgr2=0x7f24a8072ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:41.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.616+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 shutdown_connections 2024-04-20T21:34:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:41.616+0000 7f24ad800640 1 -- 172.21.15.102:0/2426221195 wait complete. 2024-04-20T21:34:41.633 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:34:43.634 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.845+0000 7f205f200640 1 Processor -- start 2024-04-20T21:34:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.845+0000 7f205f200640 1 -- start start 2024-04-20T21:34:43.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.846+0000 7f205f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f20600c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:43.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.846+0000 7f205f200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2060153560 con 0x7f2060153180 2024-04-20T21:34:43.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.846+0000 7f205f200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20600c7a70 con 0x7f2060073c70 2024-04-20T21:34:43.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.846+0000 7f205f200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f20600c7d30 con 0x7f20600c75a0 2024-04-20T21:34:43.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205de00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2060153180 0x7f2060158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50342/0 (socket says 172.21.15.102:50342) 2024-04-20T21:34:43.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205de00640 1 -- 172.21.15.102:0/840880867 learned_addr learned my addr 172.21.15.102:0/840880867 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:34:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205d400640 1 --2- 172.21.15.102:0/840880867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f20600c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205ca00640 1 -- 172.21.15.102:0/840880867 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517994767 0 0) 0x7f2060153560 con 0x7f2060153180 2024-04-20T21:34:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205ca00640 1 -- 172.21.15.102:0/840880867 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2044003520 con 0x7f2060153180 2024-04-20T21:34:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205ca00640 1 -- 172.21.15.102:0/840880867 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1367303483 0 0) 0x7f20600c7a70 con 0x7f2060073c70 2024-04-20T21:34:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205ca00640 1 -- 172.21.15.102:0/840880867 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2060153560 con 0x7f2060073c70 2024-04-20T21:34:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205d400640 1 -- 172.21.15.102:0/840880867 >> v1:172.21.15.193:6789/0 conn(0x7f2060073c70 legacy=0x7f20601637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.847+0000 7f205d400640 1 -- 172.21.15.102:0/840880867 >> v1:172.21.15.102:6789/0 conn(0x7f2060153180 legacy=0x7f2060158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.848+0000 7f205d400640 1 -- 172.21.15.102:0/840880867 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206015c4d0 con 0x7f20600c75a0 2024-04-20T21:34:43.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.849+0000 7f205d400640 1 --2- 172.21.15.102:0/840880867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f20600c5f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f204c0097d0 tx=0x7f204c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9dab56ea41cc99 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.849+0000 7f205ca00640 1 -- 172.21.15.102:0/840880867 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c034070 con 0x7f20600c75a0 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.849+0000 7f205ca00640 1 -- 172.21.15.102:0/840880867 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c030800 con 0x7f20600c75a0 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.849+0000 7f205ca00640 1 -- 172.21.15.102:0/840880867 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c030bd0 con 0x7f20600c75a0 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.849+0000 7f205f200640 1 -- 172.21.15.102:0/840880867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 msgr2=0x7f20600c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.849+0000 7f205f200640 1 --2- 172.21.15.102:0/840880867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f20600c5f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f204c0097d0 tx=0x7f204c02b940 comp rx=0 tx=0).stop 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.850+0000 7f205f200640 1 -- 172.21.15.102:0/840880867 shutdown_connections 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.850+0000 7f205f200640 1 --2- 172.21.15.102:0/840880867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f20600c5f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.850+0000 7f205f200640 1 -- 172.21.15.102:0/840880867 >> 172.21.15.102:0/840880867 conn(0x7f20600cc820 msgr2=0x7f20600ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.850+0000 7f205f200640 1 -- 172.21.15.102:0/840880867 shutdown_connections 2024-04-20T21:34:43.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.850+0000 7f205f200640 1 -- 172.21.15.102:0/840880867 wait complete. 2024-04-20T21:34:43.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.851+0000 7f205f200640 1 Processor -- start 2024-04-20T21:34:43.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.851+0000 7f205f200640 1 -- start start 2024-04-20T21:34:43.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.852+0000 7f205f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060073c70 0x7f206015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.852+0000 7f205f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f206015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.852+0000 7f205f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2060153180 0x7f20601eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.852+0000 7f205f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2060159550 con 0x7f2060153180 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.852+0000 7f205f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f206015ca90 con 0x7f2060073c70 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.852+0000 7f205f200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f206015d290 con 0x7f20600c75a0 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060073c70 0x7f206015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f206015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2060153180 0x7f20601eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f206015bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:38936/0 (socket says 172.21.15.102:38936) 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 -- 172.21.15.102:0/1988490915 learned_addr learned my addr 172.21.15.102:0/1988490915 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:43.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060073c70 0x7f206015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:59612/0 (socket says 172.21.15.102:59612) 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 -- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060073c70 msgr2=0x7f206015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060073c70 0x7f206015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 -- 172.21.15.102:0/1988490915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2060153180 msgr2=0x7f20601eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2060153180 0x7f20601eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 -- 172.21.15.102:0/1988490915 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206015b130 con 0x7f20600c75a0 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.853+0000 7f205d400640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f206015bd30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f204c0098f0 tx=0x7f204c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.854+0000 7f2056000640 1 -- 172.21.15.102:0/1988490915 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c034040 con 0x7f20600c75a0 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.854+0000 7f2056000640 1 -- 172.21.15.102:0/1988490915 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c030e30 con 0x7f20600c75a0 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.854+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206015b410 con 0x7f20600c75a0 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.854+0000 7f2056000640 1 -- 172.21.15.102:0/1988490915 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c039d10 con 0x7f20600c75a0 2024-04-20T21:34:43.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.854+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f20601f2650 con 0x7f20600c75a0 2024-04-20T21:34:43.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.855+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2060071ee0 con 0x7f20600c75a0 2024-04-20T21:34:43.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.856+0000 7f2056000640 1 -- 172.21.15.102:0/1988490915 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f204c037070 con 0x7f20600c75a0 2024-04-20T21:34:43.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.857+0000 7f2056000640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204006c920 0x7f204006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:43.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.858+0000 7f2056000640 1 -- 172.21.15.102:0/1988490915 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f204c0ae1e0 con 0x7f20600c75a0 2024-04-20T21:34:43.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.858+0000 7f205de00640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204006c920 0x7f204006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:43.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.858+0000 7f205de00640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204006c920 0x7f204006ede0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2050000f60 tx=0x7f2050007d70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:43.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:43.862+0000 7f2056000640 1 -- 172.21.15.102:0/1988490915 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2060071ee0 con 0x7f20600c75a0 2024-04-20T21:34:44.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.103+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20600c3ae0 con 0x7f20600c75a0 2024-04-20T21:34:44.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.104+0000 7f2056000640 1 -- 172.21.15.102:0/1988490915 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7f20600c3ae0 con 0x7f20600c75a0 2024-04-20T21:34:44.103 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:44.103 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:34:44.104 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:34:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.106+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204006c920 msgr2=0x7f204006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.106+0000 7f205f200640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204006c920 0x7f204006ede0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2050000f60 tx=0x7f2050007d70 comp rx=0 tx=0).stop 2024-04-20T21:34:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.106+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 msgr2=0x7f206015bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.106+0000 7f205f200640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f206015bd30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f204c0098f0 tx=0x7f204c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:34:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.107+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 shutdown_connections 2024-04-20T21:34:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.107+0000 7f205f200640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2060153180 0x7f20601eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.107+0000 7f205f200640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20600c75a0 0x7f206015bd30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.107+0000 7f205f200640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f204006c920 0x7f204006ede0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:44.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.107+0000 7f205f200640 1 --2- 172.21.15.102:0/1988490915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060073c70 0x7f206015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:44.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.107+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 >> 172.21.15.102:0/1988490915 conn(0x7f20600cc820 msgr2=0x7f20600c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:44.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.107+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 shutdown_connections 2024-04-20T21:34:44.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:44.107+0000 7f205f200640 1 -- 172.21.15.102:0/1988490915 wait complete. 2024-04-20T21:34:44.125 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:34:46.126 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:46.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.328+0000 7f9b60400640 1 Processor -- start 2024-04-20T21:34:46.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.328+0000 7f9b60400640 1 -- start start 2024-04-20T21:34:46.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.329+0000 7f9b60400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b580c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:46.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.329+0000 7f9b60400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9b58073a10 0x7f9b58158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:46.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b60400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b581536d0 con 0x7f9b5814fa50 2024-04-20T21:34:46.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b60400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9b58159500 con 0x7f9b58073a10 2024-04-20T21:34:46.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b60400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9b580c76c0 con 0x7f9b580c9eb0 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5e400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9b58073a10 0x7f9b58158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5d000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b580c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5da00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f9b5814fa50 0x7f9b580c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:33392/0 (socket says 172.21.15.102:33392) 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5da00640 1 -- 172.21.15.102:0/2816466399 learned_addr learned my addr 172.21.15.102:0/2816466399 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b57e00640 1 -- 172.21.15.102:0/2816466399 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577615676 0 0) 0x7f9b4c001110 con 0x7f9b5814fa50 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5d000640 1 -- 172.21.15.102:0/2816466399 >> v2:172.21.15.102:3300/0 conn(0x7f9b58073a10 msgr2=0x7f9b58158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5d000640 1 --2- 172.21.15.102:0/2816466399 >> v2:172.21.15.102:3300/0 conn(0x7f9b58073a10 0x7f9b58158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5d000640 1 -- 172.21.15.102:0/2816466399 >> v1:172.21.15.193:6789/0 conn(0x7f9b5814fa50 legacy=0x7f9b580c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5d000640 1 -- 172.21.15.102:0/2816466399 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b580c7960 con 0x7f9b580c9eb0 2024-04-20T21:34:46.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.330+0000 7f9b5e400640 1 --2- 172.21.15.102:0/2816466399 >> v2:172.21.15.102:3300/0 conn(0x7f9b58073a10 0x7f9b58158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.331+0000 7f9b5d000640 1 --2- 172.21.15.102:0/2816466399 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b580c4830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b480097a0 tx=0x7f9b4802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e6465bde9176cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.332+0000 7f9b57e00640 1 -- 172.21.15.102:0/2816466399 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48034070 con 0x7f9b580c9eb0 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.332+0000 7f9b57e00640 1 -- 172.21.15.102:0/2816466399 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b48030800 con 0x7f9b580c9eb0 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.332+0000 7f9b57e00640 1 -- 172.21.15.102:0/2816466399 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48030bd0 con 0x7f9b580c9eb0 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.332+0000 7f9b60400640 1 -- 172.21.15.102:0/2816466399 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 msgr2=0x7f9b580c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.332+0000 7f9b60400640 1 --2- 172.21.15.102:0/2816466399 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b580c4830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b480097a0 tx=0x7f9b4802b940 comp rx=0 tx=0).stop 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.333+0000 7f9b60400640 1 -- 172.21.15.102:0/2816466399 shutdown_connections 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.333+0000 7f9b60400640 1 --2- 172.21.15.102:0/2816466399 >> v2:172.21.15.102:3300/0 conn(0x7f9b58073a10 0x7f9b58158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.333+0000 7f9b60400640 1 --2- 172.21.15.102:0/2816466399 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b580c4830 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.333+0000 7f9b60400640 1 -- 172.21.15.102:0/2816466399 >> 172.21.15.102:0/2816466399 conn(0x7f9b580c9390 msgr2=0x7f9b580c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:46.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.333+0000 7f9b60400640 1 -- 172.21.15.102:0/2816466399 shutdown_connections 2024-04-20T21:34:46.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.333+0000 7f9b60400640 1 -- 172.21.15.102:0/2816466399 wait complete. 2024-04-20T21:34:46.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.335+0000 7f9b60400640 1 Processor -- start 2024-04-20T21:34:46.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.335+0000 7f9b60400640 1 -- start start 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b60400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b58073a10 0x7f9b581eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b60400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b581571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b60400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5814fa50 0x7f9b58157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b60400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b580c7c00 con 0x7f9b58073a10 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b60400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b580c3d90 con 0x7f9b5814fa50 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b60400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9b580c4030 con 0x7f9b580c9eb0 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b5da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b58073a10 0x7f9b581eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b5d000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b581571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b5d000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b581571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:38948/0 (socket says 172.21.15.102:38948) 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b5e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5814fa50 0x7f9b58157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b5da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b58073a10 0x7f9b581eeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38760/0 (socket says 172.21.15.102:38760) 2024-04-20T21:34:46.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.336+0000 7f9b5d000640 1 -- 172.21.15.102:0/2865275938 learned_addr learned my addr 172.21.15.102:0/2865275938 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:46.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.337+0000 7f9b5d000640 1 -- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5814fa50 msgr2=0x7f9b58157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:46.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.337+0000 7f9b5d000640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5814fa50 0x7f9b58157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.337+0000 7f9b5d000640 1 -- 172.21.15.102:0/2865275938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b58073a10 msgr2=0x7f9b581eeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:46.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.337+0000 7f9b5d000640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b58073a10 0x7f9b581eeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.337+0000 7f9b5d000640 1 -- 172.21.15.102:0/2865275938 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5814c100 con 0x7f9b580c9eb0 2024-04-20T21:34:46.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.337+0000 7f9b5d000640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b581571d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009890 tx=0x7f9b480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:46.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.337+0000 7f9b55600640 1 -- 172.21.15.102:0/2865275938 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48048070 con 0x7f9b580c9eb0 2024-04-20T21:34:46.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.338+0000 7f9b55600640 1 -- 172.21.15.102:0/2865275938 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b48030600 con 0x7f9b580c9eb0 2024-04-20T21:34:46.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.338+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5814c3a0 con 0x7f9b580c9eb0 2024-04-20T21:34:46.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.338+0000 7f9b55600640 1 -- 172.21.15.102:0/2865275938 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48034040 con 0x7f9b580c9eb0 2024-04-20T21:34:46.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.338+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5814c940 con 0x7f9b580c9eb0 2024-04-20T21:34:46.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.339+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b580c56b0 con 0x7f9b580c9eb0 2024-04-20T21:34:46.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.340+0000 7f9b55600640 1 -- 172.21.15.102:0/2865275938 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b4804b070 con 0x7f9b580c9eb0 2024-04-20T21:34:46.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.341+0000 7f9b55600640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9b3c06c8f0 0x7f9b3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:46.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.341+0000 7f9b55600640 1 -- 172.21.15.102:0/2865275938 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f9b480ada90 con 0x7f9b580c9eb0 2024-04-20T21:34:46.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.341+0000 7f9b5da00640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9b3c06c8f0 0x7f9b3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:46.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.342+0000 7f9b5da00640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9b3c06c8f0 0x7f9b3c06edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c000f60 tx=0x7f9b4c007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:46.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.345+0000 7f9b55600640 1 -- 172.21.15.102:0/2865275938 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b580c56b0 con 0x7f9b580c9eb0 2024-04-20T21:34:46.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.582+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b581568e0 con 0x7f9b580c9eb0 2024-04-20T21:34:46.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.584+0000 7f9b55600640 1 -- 172.21.15.102:0/2865275938 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7f9b480307d0 con 0x7f9b580c9eb0 2024-04-20T21:34:46.583 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:46.583 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:34:46.583 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:34:46.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9b3c06c8f0 msgr2=0x7f9b3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9b3c06c8f0 0x7f9b3c06edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c000f60 tx=0x7f9b4c007b50 comp rx=0 tx=0).stop 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 msgr2=0x7f9b581571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b581571d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009890 tx=0x7f9b480046c0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 shutdown_connections 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9b3c06c8f0 0x7f9b3c06edb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5814fa50 0x7f9b58157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b580c9eb0 0x7f9b581571d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 --2- 172.21.15.102:0/2865275938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b58073a10 0x7f9b581eeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 >> 172.21.15.102:0/2865275938 conn(0x7f9b580c9390 msgr2=0x7f9b5806e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.586+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 shutdown_connections 2024-04-20T21:34:46.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:46.587+0000 7f9b60400640 1 -- 172.21.15.102:0/2865275938 wait complete. 2024-04-20T21:34:46.604 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:34:48.605 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:48.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.813+0000 7f6ef4e00640 1 Processor -- start 2024-04-20T21:34:48.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.814+0000 7f6ef4e00640 1 -- start start 2024-04-20T21:34:48.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.815+0000 7f6ef4e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef00c9eb0 0x7f6ef00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.815+0000 7f6ef4e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6ef0073a10 0x7f6ef0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.815+0000 7f6ef4e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef01536d0 con 0x7f6ef014fa50 2024-04-20T21:34:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.815+0000 7f6ef4e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef0159500 con 0x7f6ef0073a10 2024-04-20T21:34:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.815+0000 7f6ef4e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef00c76c0 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6eede00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6ef014fa50 0x7f6ef00c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:45682/0 (socket says 172.21.15.102:45682) 2024-04-20T21:34:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6eee800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6ef0073a10 0x7f6ef0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6eede00640 1 -- 172.21.15.102:0/3080145459 learned_addr learned my addr 172.21.15.102:0/3080145459 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:34:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6ee5400640 1 --2- 172.21.15.102:0/3080145459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef00c9eb0 0x7f6ef00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:48.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6eed400640 1 -- 172.21.15.102:0/3080145459 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887028770 0 0) 0x7f6ee0001490 con 0x7f6ef014fa50 2024-04-20T21:34:48.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6eed400640 1 -- 172.21.15.102:0/3080145459 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ed4003520 con 0x7f6ef014fa50 2024-04-20T21:34:48.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6ee5400640 1 -- 172.21.15.102:0/3080145459 >> v2:172.21.15.102:3300/0 conn(0x7f6ef0073a10 msgr2=0x7f6ef0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:48.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6ee5400640 1 --2- 172.21.15.102:0/3080145459 >> v2:172.21.15.102:3300/0 conn(0x7f6ef0073a10 0x7f6ef0158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:48.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6ee5400640 1 -- 172.21.15.102:0/3080145459 >> v1:172.21.15.193:6789/0 conn(0x7f6ef014fa50 legacy=0x7f6ef00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:48.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.816+0000 7f6ee5400640 1 -- 172.21.15.102:0/3080145459 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef00c7960 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.817+0000 7f6ee5400640 1 --2- 172.21.15.102:0/3080145459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef00c9eb0 0x7f6ef00c4830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097a0 tx=0x7f6ed802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4b40325b9c2bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:48.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.817+0000 7f6eed400640 1 -- 172.21.15.102:0/3080145459 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8034070 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6eed400640 1 -- 172.21.15.102:0/3080145459 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8030800 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6eed400640 1 -- 172.21.15.102:0/3080145459 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8030bb0 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6ef4e00640 1 -- 172.21.15.102:0/3080145459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef00c9eb0 msgr2=0x7f6ef00c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:48.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/3080145459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef00c9eb0 0x7f6ef00c4830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097a0 tx=0x7f6ed802b940 comp rx=0 tx=0).stop 2024-04-20T21:34:48.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6ef4e00640 1 -- 172.21.15.102:0/3080145459 shutdown_connections 2024-04-20T21:34:48.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/3080145459 >> v2:172.21.15.102:3300/0 conn(0x7f6ef0073a10 0x7f6ef0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:48.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/3080145459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef00c9eb0 0x7f6ef00c4830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:48.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6ef4e00640 1 -- 172.21.15.102:0/3080145459 >> 172.21.15.102:0/3080145459 conn(0x7f6ef00c9390 msgr2=0x7f6ef00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:48.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6ef4e00640 1 -- 172.21.15.102:0/3080145459 shutdown_connections 2024-04-20T21:34:48.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.818+0000 7f6ef4e00640 1 -- 172.21.15.102:0/3080145459 wait complete. 2024-04-20T21:34:48.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.820+0000 7f6ef4e00640 1 Processor -- start 2024-04-20T21:34:48.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.820+0000 7f6ef4e00640 1 -- start start 2024-04-20T21:34:48.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ef4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef0073a10 0x7f6ef0152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:48.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ef4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef00c9eb0 0x7f6ef00cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:48.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ef4e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef014fa50 0x7f6ef00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:48.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ef4e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef00c7c00 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ef4e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef00c3d90 con 0x7f6ef0073a10 2024-04-20T21:34:48.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ef4e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef00c4030 con 0x7f6ef014fa50 2024-04-20T21:34:48.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef00c9eb0 0x7f6ef00cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:48.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef00c9eb0 0x7f6ef00cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59762/0 (socket says 172.21.15.102:59762) 2024-04-20T21:34:48.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6ee5400640 1 -- 172.21.15.102:0/1906964722 learned_addr learned my addr 172.21.15.102:0/1906964722 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:48.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6eee800640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef014fa50 0x7f6ef00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:48.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.821+0000 7f6eede00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef0073a10 0x7f6ef0152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:48.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee5400640 1 -- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef014fa50 msgr2=0x7f6ef00cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:48.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee5400640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef014fa50 0x7f6ef00cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:48.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee5400640 1 -- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef0073a10 msgr2=0x7f6ef0152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:48.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee5400640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef0073a10 0x7f6ef0152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:48.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee5400640 1 -- 172.21.15.102:0/1906964722 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef0152c70 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee5400640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef00c9eb0 0x7f6ef00cb6e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80098c0 tx=0x7f6ed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:48.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee6a00640 1 -- 172.21.15.102:0/1906964722 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8034040 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee6a00640 1 -- 172.21.15.102:0/1906964722 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8030e30 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ee6a00640 1 -- 172.21.15.102:0/1906964722 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8039d10 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.822+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef01f55b0 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.823+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef0157620 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.824+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef00c56b0 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.825+0000 7f6ee6a00640 1 -- 172.21.15.102:0/1906964722 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ed8037070 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.825+0000 7f6ee6a00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6ecc06c920 0x7f6ecc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:48.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.826+0000 7f6ee6a00640 1 -- 172.21.15.102:0/1906964722 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 7423+0+0 (secure 0 0 0) 0x7f6ed80ae020 con 0x7f6ef00c9eb0 2024-04-20T21:34:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.826+0000 7f6eede00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6ecc06c920 0x7f6ecc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:48.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.826+0000 7f6eede00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6ecc06c920 0x7f6ecc06ede0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005c00 tx=0x7f6ee0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:48.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:48.830+0000 7f6ee6a00640 1 -- 172.21.15.102:0/1906964722 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6ef00c56b0 con 0x7f6ef00c9eb0 2024-04-20T21:34:49.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.072+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef006d590 con 0x7f6ef00c9eb0 2024-04-20T21:34:49.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.073+0000 7f6ee6a00640 1 -- 172.21.15.102:0/1906964722 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+6862 (secure 0 0 0) 0x7f6ef006d590 con 0x7f6ef00c9eb0 2024-04-20T21:34:49.073 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:49.073 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:33:34.164638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":26,"addr":"172.21.15.102:6835/2655303061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2655303061},{"type":"v1","addr":"172.21.15.102:6835","nonce":2655303061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5835,"qdb_cluster":[5835]},"id":1}]} 2024-04-20T21:34:49.073 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 55 2024-04-20T21:34:49.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6ecc06c920 msgr2=0x7f6ecc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:49.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6ecc06c920 0x7f6ecc06ede0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005c00 tx=0x7f6ee0009040 comp rx=0 tx=0).stop 2024-04-20T21:34:49.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef00c9eb0 msgr2=0x7f6ef00cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef00c9eb0 0x7f6ef00cb6e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80098c0 tx=0x7f6ed80046c0 comp rx=0 tx=0).stop 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 shutdown_connections 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ef014fa50 0x7f6ef00cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef00c9eb0 0x7f6ef00cb6e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6ecc06c920 0x7f6ecc06ede0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 --2- 172.21.15.102:0/1906964722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ef0073a10 0x7f6ef0152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 >> 172.21.15.102:0/1906964722 conn(0x7f6ef00c9390 msgr2=0x7f6ef014ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 shutdown_connections 2024-04-20T21:34:49.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:49.076+0000 7f6ef4e00640 1 -- 172.21.15.102:0/1906964722 wait complete. 2024-04-20T21:34:49.094 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:34:51.095 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:51.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.304+0000 7f7f81a00640 1 Processor -- start 2024-04-20T21:34:51.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.304+0000 7f7f81a00640 1 -- start start 2024-04-20T21:34:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.305+0000 7f7f81a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7f7c14fa50 0x7f7f7c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.305+0000 7f7f81a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c0c9eb0 0x7f7f7c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.305+0000 7f7f81a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7f7c073a10 0x7f7f7c0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.305+0000 7f7f81a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f7f7c159500 con 0x7f7f7c073a10 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.305+0000 7f7f81a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7f7c1536d0 con 0x7f7f7c14fa50 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.305+0000 7f7f81a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c0c76c0 con 0x7f7f7c0c9eb0 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7f7c073a10 0x7f7f7c0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7aa00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7f7c14fa50 0x7f7f7c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c0c9eb0 0x7f7f7c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7aa00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7f7c14fa50 0x7f7f7c0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37762/0 (socket says 172.21.15.102:37762) 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c0c9eb0 0x7f7f7c158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:51580/0 (socket says 172.21.15.102:51580) 2024-04-20T21:34:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7f7c073a10 0x7f7f7c0c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59770/0 (socket says 172.21.15.102:59770) 2024-04-20T21:34:51.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7aa00640 1 -- 172.21.15.102:0/1335345380 learned_addr learned my addr 172.21.15.102:0/1335345380 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:51.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7a000640 1 -- 172.21.15.102:0/1335345380 >> v2:172.21.15.193:3300/0 conn(0x7f7f7c14fa50 msgr2=0x7f7f7c0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:51.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7a000640 1 --2- 172.21.15.102:0/1335345380 >> v2:172.21.15.193:3300/0 conn(0x7f7f7c14fa50 0x7f7f7c0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7a000640 1 -- 172.21.15.102:0/1335345380 >> v2:172.21.15.102:3300/0 conn(0x7f7f7c073a10 msgr2=0x7f7f7c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:51.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7a000640 1 --2- 172.21.15.102:0/1335345380 >> v2:172.21.15.102:3300/0 conn(0x7f7f7c073a10 0x7f7f7c0c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7a000640 1 -- 172.21.15.102:0/1335345380 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c0c7960 con 0x7f7f7c0c9eb0 2024-04-20T21:34:51.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.306+0000 7f7f7aa00640 1 --2- 172.21.15.102:0/1335345380 >> v2:172.21.15.193:3300/0 conn(0x7f7f7c14fa50 0x7f7f7c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:51.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.307+0000 7f7f7a000640 1 --2- 172.21.15.102:0/1335345380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c0c9eb0 0x7f7f7c158fc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0706a0 tx=0x7f7f6802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e65dbe0a0f5bcb53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:51.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.308+0000 7f7f79600640 1 -- 172.21.15.102:0/1335345380 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68035020 con 0x7f7f7c0c9eb0 2024-04-20T21:34:51.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.308+0000 7f7f79600640 1 -- 172.21.15.102:0/1335345380 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f68031830 con 0x7f7f7c0c9eb0 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.308+0000 7f7f79600640 1 -- 172.21.15.102:0/1335345380 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68031c00 con 0x7f7f7c0c9eb0 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 -- 172.21.15.102:0/1335345380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c0c9eb0 msgr2=0x7f7f7c158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 --2- 172.21.15.102:0/1335345380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c0c9eb0 0x7f7f7c158fc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0706a0 tx=0x7f7f6802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 -- 172.21.15.102:0/1335345380 shutdown_connections 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 --2- 172.21.15.102:0/1335345380 >> v2:172.21.15.102:3300/0 conn(0x7f7f7c073a10 0x7f7f7c0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 --2- 172.21.15.102:0/1335345380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c0c9eb0 0x7f7f7c158fc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 --2- 172.21.15.102:0/1335345380 >> v2:172.21.15.193:3300/0 conn(0x7f7f7c14fa50 0x7f7f7c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 -- 172.21.15.102:0/1335345380 >> 172.21.15.102:0/1335345380 conn(0x7f7f7c0c9390 msgr2=0x7f7f7c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 -- 172.21.15.102:0/1335345380 shutdown_connections 2024-04-20T21:34:51.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.309+0000 7f7f81a00640 1 -- 172.21.15.102:0/1335345380 wait complete. 2024-04-20T21:34:51.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.311+0000 7f7f81a00640 1 Processor -- start 2024-04-20T21:34:51.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.311+0000 7f7f81a00640 1 -- start start 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.312+0000 7f7f81a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c073a10 0x7f7f7c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.312+0000 7f7f81a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f7c0c9eb0 0x7f7f7c1e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.312+0000 7f7f81a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f7c14fa50 0x7f7f7c14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.312+0000 7f7f81a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0c7c00 con 0x7f7f7c14fa50 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.312+0000 7f7f81a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0c3d90 con 0x7f7f7c0c9eb0 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.312+0000 7f7f81a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c0c4030 con 0x7f7f7c073a10 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c073a10 0x7f7f7c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f7c14fa50 0x7f7f7c14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c073a10 0x7f7f7c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:51596/0 (socket says 172.21.15.102:51596) 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 -- 172.21.15.102:0/3784873888 learned_addr learned my addr 172.21.15.102:0/3784873888 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:51.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7a000640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f7c0c9eb0 0x7f7f7c1e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 -- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f7c0c9eb0 msgr2=0x7f7f7c1e4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f7c0c9eb0 0x7f7f7c1e4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 -- 172.21.15.102:0/3784873888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f7c14fa50 msgr2=0x7f7f7c14dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f7c14fa50 0x7f7f7c14dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 -- 172.21.15.102:0/3784873888 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c1f5020 con 0x7f7f7c073a10 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7b400640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f7c14fa50 0x7f7f7c14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.313+0000 7f7f7aa00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c073a10 0x7f7f7c072900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00efc0 tx=0x7f7f6c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.314+0000 7f7f72a00640 1 -- 172.21.15.102:0/3784873888 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c014480 con 0x7f7f7c073a10 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.314+0000 7f7f72a00640 1 -- 172.21.15.102:0/3784873888 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6c019070 con 0x7f7f7c073a10 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.314+0000 7f7f72a00640 1 -- 172.21.15.102:0/3784873888 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c014820 con 0x7f7f7c073a10 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.314+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c1f4d80 con 0x7f7f7c073a10 2024-04-20T21:34:51.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.314+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c1f0d70 con 0x7f7f7c073a10 2024-04-20T21:34:51.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.315+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7c074340 con 0x7f7f7c073a10 2024-04-20T21:34:51.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.316+0000 7f7f72a00640 1 -- 172.21.15.102:0/3784873888 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f6c005d10 con 0x7f7f7c073a10 2024-04-20T21:34:51.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.317+0000 7f7f72a00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.317+0000 7f7f72a00640 1 -- 172.21.15.102:0/3784873888 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f7f6c08c700 con 0x7f7f7c073a10 2024-04-20T21:34:51.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.317+0000 7f7f7a000640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.318+0000 7f7f7a000640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7f68036000 tx=0x7f7f68008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:51.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.321+0000 7f7f72a00640 1 -- 172.21.15.102:0/3784873888 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7f6c010040 con 0x7f7f7c073a10 2024-04-20T21:34:51.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.557+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f7c0c4300 con 0x7f7f7c073a10 2024-04-20T21:34:51.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.558+0000 7f7f72a00640 1 -- 172.21.15.102:0/3784873888 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+6026 (secure 0 0 0) 0x7f7f6c059d30 con 0x7f7f7c073a10 2024-04-20T21:34:51.557 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:51.558 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:34:50.734531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:34:51.558 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 56 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.561+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7f5c06c8f0 msgr2=0x7f7f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.561+0000 7f7f81a00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7f68036000 tx=0x7f7f68008030 comp rx=0 tx=0).stop 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.561+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c073a10 msgr2=0x7f7f7c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.561+0000 7f7f81a00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c073a10 0x7f7f7c072900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00efc0 tx=0x7f7f6c00c490 comp rx=0 tx=0).stop 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.562+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 shutdown_connections 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.562+0000 7f7f81a00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f7c14fa50 0x7f7f7c14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.562+0000 7f7f81a00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.562+0000 7f7f81a00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f7c0c9eb0 0x7f7f7c1e4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.562+0000 7f7f81a00640 1 --2- 172.21.15.102:0/3784873888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f7c073a10 0x7f7f7c072900 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.562+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 >> 172.21.15.102:0/3784873888 conn(0x7f7f7c0c9390 msgr2=0x7f7f7c06c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.562+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 shutdown_connections 2024-04-20T21:34:51.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.562+0000 7f7f81a00640 1 -- 172.21.15.102:0/3784873888 wait complete. 2024-04-20T21:34:51.579 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:34:51.580 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:34:51.580 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:51.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.779+0000 7f52ce400640 1 Processor -- start 2024-04-20T21:34:51.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.779+0000 7f52ce400640 1 -- start start 2024-04-20T21:34:51.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52ce400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 0x7f52c80cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52ce400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f52c80c3700 0x7f52c806e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52ce400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c815c220 con 0x7f52c806eb70 2024-04-20T21:34:51.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52ce400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f52c80c9180 con 0x7f52c80c3700 2024-04-20T21:34:51.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52ce400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f52c80c4840 con 0x7f52c8152290 2024-04-20T21:34:51.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52bfe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f52c80c3700 0x7f52c806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52bfe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f52c80c3700 0x7f52c806e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59802/0 (socket says 172.21.15.102:59802) 2024-04-20T21:34:51.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52bfe00640 1 -- 172.21.15.102:0/1185364574 learned_addr learned my addr 172.21.15.102:0/1185364574 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:51.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.780+0000 7f52cd000640 1 --2- 172.21.15.102:0/1185364574 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 0x7f52c80cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.781+0000 7f52bfe00640 1 -- 172.21.15.102:0/1185364574 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 msgr2=0x7f52c80cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:51.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.781+0000 7f52bfe00640 1 --2- 172.21.15.102:0/1185364574 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 0x7f52c80cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.781+0000 7f52bfe00640 1 -- 172.21.15.102:0/1185364574 >> v1:172.21.15.193:6789/0 conn(0x7f52c806eb70 legacy=0x7f52c80c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.781+0000 7f52bfe00640 1 -- 172.21.15.102:0/1185364574 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c80c4560 con 0x7f52c80c3700 2024-04-20T21:34:51.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.781+0000 7f52c7e00640 1 -- 172.21.15.102:0/1185364574 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962435847 0 0) 0x7f52c815c220 con 0x7f52c806eb70 2024-04-20T21:34:51.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.782+0000 7f52bfe00640 1 --2- 172.21.15.102:0/1185364574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c806e630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009870 tx=0x7f52b002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a228c633bd4e251 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.782+0000 7f52c7e00640 1 -- 172.21.15.102:0/1185364574 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b002f840 con 0x7f52c80c3700 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.782+0000 7f52c7e00640 1 -- 172.21.15.102:0/1185364574 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b002f9e0 con 0x7f52c80c3700 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.782+0000 7f52c7e00640 1 -- 172.21.15.102:0/1185364574 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b002fdb0 con 0x7f52c80c3700 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.783+0000 7f52ce400640 1 -- 172.21.15.102:0/1185364574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 msgr2=0x7f52c806e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.783+0000 7f52ce400640 1 --2- 172.21.15.102:0/1185364574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c806e630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009870 tx=0x7f52b002ba20 comp rx=0 tx=0).stop 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.783+0000 7f52ce400640 1 -- 172.21.15.102:0/1185364574 shutdown_connections 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.783+0000 7f52ce400640 1 --2- 172.21.15.102:0/1185364574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c806e630 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.783+0000 7f52ce400640 1 --2- 172.21.15.102:0/1185364574 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 0x7f52c80cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.783+0000 7f52ce400640 1 -- 172.21.15.102:0/1185364574 >> 172.21.15.102:0/1185364574 conn(0x7f52c80cc820 msgr2=0x7f52c80ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.783+0000 7f52ce400640 1 -- 172.21.15.102:0/1185364574 shutdown_connections 2024-04-20T21:34:51.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.783+0000 7f52ce400640 1 -- 172.21.15.102:0/1185364574 wait complete. 2024-04-20T21:34:51.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.785+0000 7f52ce400640 1 Processor -- start 2024-04-20T21:34:51.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.785+0000 7f52ce400640 1 -- start start 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52ce400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c806eb70 0x7f52c813c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52ce400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c81e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52ce400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 0x7f52c80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52ce400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f52c815ba20 con 0x7f52c80c3700 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52ce400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52c815b740 con 0x7f52c806eb70 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52ce400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f52c815bd00 con 0x7f52c8152290 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c81e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c81e2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59816/0 (socket says 172.21.15.102:59816) 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52bfe00640 1 -- 172.21.15.102:0/740103756 learned_addr learned my addr 172.21.15.102:0/740103756 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52cd000640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c806eb70 0x7f52c813c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52cda00640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 0x7f52c80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52bfe00640 1 -- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 msgr2=0x7f52c80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52bfe00640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 0x7f52c80ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52bfe00640 1 -- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c806eb70 msgr2=0x7f52c813c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:51.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52bfe00640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c806eb70 0x7f52c813c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:51.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52bfe00640 1 -- 172.21.15.102:0/740103756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c81e3a00 con 0x7f52c80c3700 2024-04-20T21:34:51.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.786+0000 7f52cd000640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c806eb70 0x7f52c813c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:51.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.787+0000 7f52bfe00640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c81e2e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f52b002ba60 tx=0x7f52b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:51.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.787+0000 7f52c5600640 1 -- 172.21.15.102:0/740103756 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0047070 con 0x7f52c80c3700 2024-04-20T21:34:51.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.787+0000 7f52c5600640 1 -- 172.21.15.102:0/740103756 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0037bc0 con 0x7f52c80c3700 2024-04-20T21:34:51.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.787+0000 7f52c5600640 1 -- 172.21.15.102:0/740103756 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0042480 con 0x7f52c80c3700 2024-04-20T21:34:51.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.787+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c81e1250 con 0x7f52c80c3700 2024-04-20T21:34:51.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.787+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c80739a0 con 0x7f52c80c3700 2024-04-20T21:34:51.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.789+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c80c8660 con 0x7f52c80c3700 2024-04-20T21:34:51.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.789+0000 7f52c5600640 1 -- 172.21.15.102:0/740103756 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52b004a070 con 0x7f52c80c3700 2024-04-20T21:34:51.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.790+0000 7f52c5600640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f52a406c8f0 0x7f52a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:51.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.790+0000 7f52c5600640 1 -- 172.21.15.102:0/740103756 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f52b00ae3b0 con 0x7f52c80c3700 2024-04-20T21:34:51.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.791+0000 7f52cd000640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f52a406c8f0 0x7f52a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:51.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.791+0000 7f52cd000640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f52a406c8f0 0x7f52a406edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f52c80c6740 tx=0x7f52b8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:51.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:51.795+0000 7f52c5600640 1 -- 172.21.15.102:0/740103756 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f52c80c8660 con 0x7f52c80c3700 2024-04-20T21:34:52.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.032+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52c804fd60 con 0x7f52c80c3700 2024-04-20T21:34:52.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.033+0000 7f52c5600640 1 -- 172.21.15.102:0/740103756 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+6026 (secure 0 0 0) 0x7f52c81e3a00 con 0x7f52c80c3700 2024-04-20T21:34:52.033 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:52.033 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:34:50.734531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:34:52.033 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 56 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f52a406c8f0 msgr2=0x7f52a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f52a406c8f0 0x7f52a406edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f52c80c6740 tx=0x7f52b8008040 comp rx=0 tx=0).stop 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 msgr2=0x7f52c81e2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c81e2e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f52b002ba60 tx=0x7f52b0004730 comp rx=0 tx=0).stop 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 shutdown_connections 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f52c8152290 0x7f52c80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52c80c3700 0x7f52c81e2e40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f52a406c8f0 0x7f52a406edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 --2- 172.21.15.102:0/740103756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52c806eb70 0x7f52c813c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.036+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 >> 172.21.15.102:0/740103756 conn(0x7f52c80cc820 msgr2=0x7f52c8144b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.037+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 shutdown_connections 2024-04-20T21:34:52.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:52.037+0000 7f52ce400640 1 -- 172.21.15.102:0/740103756 wait complete. 2024-04-20T21:34:52.056 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 56, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 56, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5994": { "addr": "172.21.15.102:6833/3985712430", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 3985712430, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 3985712430, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5994, "incarnation": 56, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:34:50.734531+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5994 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:34:54.057 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:54.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.268+0000 7f121b200640 1 Processor -- start 2024-04-20T21:34:54.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.268+0000 7f121b200640 1 -- start start 2024-04-20T21:34:54.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f121b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c1516d0 0x7f121c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:54.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f121b200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f121c151ab0 0x7f121c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:54.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f121b200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f121c0c3d20 0x7f121c0c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:54.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f121b200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f121c0c4dd0 con 0x7f121c0c3d20 2024-04-20T21:34:54.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f121b200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f121c06ef60 con 0x7f121c151ab0 2024-04-20T21:34:54.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f121b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f121c06f240 con 0x7f121c1516d0 2024-04-20T21:34:54.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f1219e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c1516d0 0x7f121c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:54.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f121a800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f121c0c3d20 0x7f121c0c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:54.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f1219e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c1516d0 0x7f121c0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:51620/0 (socket says 172.21.15.102:51620) 2024-04-20T21:34:54.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f1219e00640 1 -- 172.21.15.102:0/3594083003 learned_addr learned my addr 172.21.15.102:0/3594083003 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:54.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.270+0000 7f1219400640 1 --2- 172.21.15.102:0/3594083003 >> v2:172.21.15.193:3300/0 conn(0x7f121c151ab0 0x7f121c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:54.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.271+0000 7f121a800640 1 -- 172.21.15.102:0/3594083003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c1516d0 msgr2=0x7f121c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:54.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.271+0000 7f121a800640 1 --2- 172.21.15.102:0/3594083003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c1516d0 0x7f121c0cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.271+0000 7f121a800640 1 -- 172.21.15.102:0/3594083003 >> v2:172.21.15.193:3300/0 conn(0x7f121c151ab0 msgr2=0x7f121c157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:54.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.271+0000 7f121a800640 1 --2- 172.21.15.102:0/3594083003 >> v2:172.21.15.193:3300/0 conn(0x7f121c151ab0 0x7f121c157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.271+0000 7f121a800640 1 -- 172.21.15.102:0/3594083003 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c06f520 con 0x7f121c0c3d20 2024-04-20T21:34:54.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.272+0000 7f121a800640 1 --2- 172.21.15.102:0/3594083003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c0c3d20 0x7f121c0c41b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f121c0c9280 tx=0x7f121402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4040999f214a0d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.272+0000 7f1218a00640 1 -- 172.21.15.102:0/3594083003 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214039070 con 0x7f121c0c3d20 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.272+0000 7f1218a00640 1 -- 172.21.15.102:0/3594083003 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121402cc60 con 0x7f121c0c3d20 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.272+0000 7f1218a00640 1 -- 172.21.15.102:0/3594083003 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214034660 con 0x7f121c0c3d20 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 -- 172.21.15.102:0/3594083003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c0c3d20 msgr2=0x7f121c0c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 --2- 172.21.15.102:0/3594083003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c0c3d20 0x7f121c0c41b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f121c0c9280 tx=0x7f121402beb0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 -- 172.21.15.102:0/3594083003 shutdown_connections 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 --2- 172.21.15.102:0/3594083003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c0c3d20 0x7f121c0c41b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 --2- 172.21.15.102:0/3594083003 >> v2:172.21.15.193:3300/0 conn(0x7f121c151ab0 0x7f121c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 --2- 172.21.15.102:0/3594083003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c1516d0 0x7f121c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 -- 172.21.15.102:0/3594083003 >> 172.21.15.102:0/3594083003 conn(0x7f121c0cc820 msgr2=0x7f121c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:54.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 -- 172.21.15.102:0/3594083003 shutdown_connections 2024-04-20T21:34:54.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.273+0000 7f121b200640 1 -- 172.21.15.102:0/3594083003 wait complete. 2024-04-20T21:34:54.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.275+0000 7f121b200640 1 Processor -- start 2024-04-20T21:34:54.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.275+0000 7f121b200640 1 -- start start 2024-04-20T21:34:54.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.276+0000 7f121b200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c0c3d20 0x7f121c073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:54.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c1516d0 0x7f121c1f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:54.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121c151ab0 0x7f121c1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:54.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f121c0c4a40 con 0x7f121c1516d0 2024-04-20T21:34:54.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f121c0c5300 con 0x7f121c151ab0 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121b200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f121c0c4bc0 con 0x7f121c0c3d20 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f1219e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c0c3d20 0x7f121c073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f1219e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c0c3d20 0x7f121c073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:51636/0 (socket says 172.21.15.102:51636) 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f1219e00640 1 -- 172.21.15.102:0/3028475313 learned_addr learned my addr 172.21.15.102:0/3028475313 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f1219400640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c1516d0 0x7f121c1f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121a800640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121c151ab0 0x7f121c1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121a800640 1 -- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c0c3d20 msgr2=0x7f121c073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121a800640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c0c3d20 0x7f121c073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121a800640 1 -- 172.21.15.102:0/3028475313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c1516d0 msgr2=0x7f121c1f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121a800640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c1516d0 0x7f121c1f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f121a800640 1 -- 172.21.15.102:0/3028475313 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c0caad0 con 0x7f121c151ab0 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f1219e00640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c0c3d20 0x7f121c073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:54.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.277+0000 7f1219400640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c1516d0 0x7f121c1f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:54.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.278+0000 7f121a800640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121c151ab0 0x7f121c1f46d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1214033000 tx=0x7f1214004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:54.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.278+0000 7f1212000640 1 -- 172.21.15.102:0/3028475313 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214039050 con 0x7f121c151ab0 2024-04-20T21:34:54.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.278+0000 7f1212000640 1 -- 172.21.15.102:0/3028475313 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1214045070 con 0x7f121c151ab0 2024-04-20T21:34:54.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.278+0000 7f1212000640 1 -- 172.21.15.102:0/3028475313 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214009450 con 0x7f121c151ab0 2024-04-20T21:34:54.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.278+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c1f0b70 con 0x7f121c151ab0 2024-04-20T21:34:54.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.278+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c1f0e10 con 0x7f121c151ab0 2024-04-20T21:34:54.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.279+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121c04fd60 con 0x7f121c151ab0 2024-04-20T21:34:54.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.280+0000 7f1212000640 1 -- 172.21.15.102:0/3028475313 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f121403e850 con 0x7f121c151ab0 2024-04-20T21:34:54.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.281+0000 7f1212000640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f11fc06c8f0 0x7f11fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:54.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.281+0000 7f1212000640 1 -- 172.21.15.102:0/3028475313 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f12140aeeb0 con 0x7f121c151ab0 2024-04-20T21:34:54.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.281+0000 7f1219e00640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f11fc06c8f0 0x7f11fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:54.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.282+0000 7f1219e00640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f11fc06c8f0 0x7f11fc06edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f120c0114b0 tx=0x7f120c00a350 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:54.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.286+0000 7f1212000640 1 -- 172.21.15.102:0/3028475313 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f121407c570 con 0x7f121c151ab0 2024-04-20T21:34:54.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.529+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f121c1f0fe0 con 0x7f121c151ab0 2024-04-20T21:34:54.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.530+0000 7f1212000640 1 -- 172.21.15.102:0/3028475313 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+6026 (secure 0 0 0) 0x7f1214080870 con 0x7f121c151ab0 2024-04-20T21:34:54.530 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:54.530 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:34:50.734531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:34:54.530 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 56 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f11fc06c8f0 msgr2=0x7f11fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f11fc06c8f0 0x7f11fc06edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f120c0114b0 tx=0x7f120c00a350 comp rx=0 tx=0).stop 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121c151ab0 msgr2=0x7f121c1f46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121c151ab0 0x7f121c1f46d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1214033000 tx=0x7f1214004a40 comp rx=0 tx=0).stop 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 shutdown_connections 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f11fc06c8f0 0x7f11fc06edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121c151ab0 0x7f121c1f46d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f121c1516d0 0x7f121c1f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 --2- 172.21.15.102:0/3028475313 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f121c0c3d20 0x7f121c073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.533+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 >> 172.21.15.102:0/3028475313 conn(0x7f121c0cc820 msgr2=0x7f121c14f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.534+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 shutdown_connections 2024-04-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:54.534+0000 7f121b200640 1 -- 172.21.15.102:0/3028475313 wait complete. 2024-04-20T21:34:54.552 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:34:56.553 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.740+0000 7fd921e00640 1 Processor -- start 2024-04-20T21:34:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.740+0000 7fd921e00640 1 -- start start 2024-04-20T21:34:56.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.742+0000 7fd921e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd91c073230 0x7fd91c073660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:56.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.742+0000 7fd921e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd91c15e7b0 con 0x7fd91c15be30 2024-04-20T21:34:56.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.742+0000 7fd921e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd91c155940 con 0x7fd91c155560 2024-04-20T21:34:56.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.742+0000 7fd921e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fd91c15f0c0 con 0x7fd91c073230 2024-04-20T21:34:56.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.742+0000 7fd91b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd91c073230 0x7fd91c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:56.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.742+0000 7fd91a000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd91c15be30 0x7fd91c0ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47600/0 (socket says 172.21.15.102:47600) 2024-04-20T21:34:56.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.743+0000 7fd91b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd91c073230 0x7fd91c073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59838/0 (socket says 172.21.15.102:59838) 2024-04-20T21:34:56.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.743+0000 7fd91b400640 1 -- 172.21.15.102:0/3728745455 learned_addr learned my addr 172.21.15.102:0/3728745455 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:56.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.743+0000 7fd91aa00640 1 --1- 172.21.15.102:0/3728745455 >> v1:172.21.15.193:6789/0 conn(0x7fd91c155560 0x7fd91c0c5cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:45702/0 (socket says 172.21.15.102:45702) 2024-04-20T21:34:56.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.743+0000 7fd91b400640 1 -- 172.21.15.102:0/3728745455 >> v1:172.21.15.193:6789/0 conn(0x7fd91c155560 legacy=0x7fd91c0c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:56.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.743+0000 7fd91b400640 1 -- 172.21.15.102:0/3728745455 >> v1:172.21.15.102:6789/0 conn(0x7fd91c15be30 legacy=0x7fd91c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:56.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.743+0000 7fd919600640 1 -- 172.21.15.102:0/3728745455 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386155505 0 0) 0x7fd91c155940 con 0x7fd91c155560 2024-04-20T21:34:56.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.743+0000 7fd91b400640 1 -- 172.21.15.102:0/3728745455 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c0cbea0 con 0x7fd91c073230 2024-04-20T21:34:56.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.744+0000 7fd91b400640 1 --2- 172.21.15.102:0/3728745455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c073230 0x7fd91c073660 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd914009e50 tx=0x7fd91402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b46807ab659810 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:56.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.744+0000 7fd919600640 1 -- 172.21.15.102:0/3728745455 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91402c9f0 con 0x7fd91c073230 2024-04-20T21:34:56.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.744+0000 7fd919600640 1 -- 172.21.15.102:0/3728745455 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd91402cb90 con 0x7fd91c073230 2024-04-20T21:34:56.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.744+0000 7fd919600640 1 -- 172.21.15.102:0/3728745455 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140343f0 con 0x7fd91c073230 2024-04-20T21:34:56.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.745+0000 7fd921e00640 1 -- 172.21.15.102:0/3728745455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c073230 msgr2=0x7fd91c073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.745+0000 7fd921e00640 1 --2- 172.21.15.102:0/3728745455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c073230 0x7fd91c073660 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd914009e50 tx=0x7fd91402bed0 comp rx=0 tx=0).stop 2024-04-20T21:34:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.745+0000 7fd921e00640 1 -- 172.21.15.102:0/3728745455 shutdown_connections 2024-04-20T21:34:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.745+0000 7fd921e00640 1 --2- 172.21.15.102:0/3728745455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c073230 0x7fd91c073660 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.745+0000 7fd921e00640 1 -- 172.21.15.102:0/3728745455 >> 172.21.15.102:0/3728745455 conn(0x7fd91c06cd40 msgr2=0x7fd91c06d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.745+0000 7fd921e00640 1 -- 172.21.15.102:0/3728745455 shutdown_connections 2024-04-20T21:34:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.745+0000 7fd921e00640 1 -- 172.21.15.102:0/3728745455 wait complete. 2024-04-20T21:34:56.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.747+0000 7fd921e00640 1 Processor -- start 2024-04-20T21:34:56.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.747+0000 7fd921e00640 1 -- start start 2024-04-20T21:34:56.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd921e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c073230 0x7fd91c1e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:56.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd921e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91c155560 0x7fd91c1e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd921e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c15be30 0x7fd91c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd921e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c15ed50 con 0x7fd91c15be30 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd921e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c15ea70 con 0x7fd91c073230 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd921e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd91c0cbb60 con 0x7fd91c155560 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd91b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c15be30 0x7fd91c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd91b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c15be30 0x7fd91c0b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59848/0 (socket says 172.21.15.102:59848) 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd91b400640 1 -- 172.21.15.102:0/4171407516 learned_addr learned my addr 172.21.15.102:0/4171407516 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd91a000640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91c155560 0x7fd91c1e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.748+0000 7fd91aa00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c073230 0x7fd91c1e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd91a000640 1 -- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c073230 msgr2=0x7fd91c1e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd91a000640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c073230 0x7fd91c1e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd91a000640 1 -- 172.21.15.102:0/4171407516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c15be30 msgr2=0x7fd91c0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:56.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd91a000640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c15be30 0x7fd91c0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:56.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd91a000640 1 -- 172.21.15.102:0/4171407516 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c1f0c80 con 0x7fd91c155560 2024-04-20T21:34:56.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd91b400640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c15be30 0x7fd91c0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:56.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd91a000640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91c155560 0x7fd91c1e4aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c520 tx=0x7fd90800c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:56.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd913400640 1 -- 172.21.15.102:0/4171407516 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908018070 con 0x7fd91c155560 2024-04-20T21:34:56.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd913400640 1 -- 172.21.15.102:0/4171407516 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd908007500 con 0x7fd91c155560 2024-04-20T21:34:56.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd913400640 1 -- 172.21.15.102:0/4171407516 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908004a20 con 0x7fd91c155560 2024-04-20T21:34:56.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.749+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c0d16b0 con 0x7fd91c155560 2024-04-20T21:34:56.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.750+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c1f1220 con 0x7fd91c155560 2024-04-20T21:34:56.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.751+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91c074340 con 0x7fd91c155560 2024-04-20T21:34:56.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.752+0000 7fd913400640 1 -- 172.21.15.102:0/4171407516 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd908004bc0 con 0x7fd91c155560 2024-04-20T21:34:56.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.752+0000 7fd913400640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:56.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.753+0000 7fd913400640 1 -- 172.21.15.102:0/4171407516 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7fd90808b530 con 0x7fd91c155560 2024-04-20T21:34:56.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.753+0000 7fd91aa00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:56.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.754+0000 7fd91aa00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000f60 tx=0x7fd90c007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:56.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.758+0000 7fd913400640 1 -- 172.21.15.102:0/4171407516 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd908058b60 con 0x7fd91c155560 2024-04-20T21:34:56.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:56.999+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd91c1539b0 con 0x7fd91c155560 2024-04-20T21:34:56.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.000+0000 7fd913400640 1 -- 172.21.15.102:0/4171407516 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+6026 (secure 0 0 0) 0x7fd90805ce60 con 0x7fd91c155560 2024-04-20T21:34:57.000 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:57.000 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:34:50.734531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:34:57.001 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 56 2024-04-20T21:34:57.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8fc06c8f0 msgr2=0x7fd8fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:57.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000f60 tx=0x7fd90c007d50 comp rx=0 tx=0).stop 2024-04-20T21:34:57.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91c155560 msgr2=0x7fd91c1e4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:57.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91c155560 0x7fd91c1e4aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c520 tx=0x7fd90800c9e0 comp rx=0 tx=0).stop 2024-04-20T21:34:57.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 shutdown_connections 2024-04-20T21:34:57.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd91c15be30 0x7fd91c0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:57.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91c155560 0x7fd91c1e4aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:57.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:57.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 --2- 172.21.15.102:0/4171407516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd91c073230 0x7fd91c1e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:57.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 >> 172.21.15.102:0/4171407516 conn(0x7fd91c06cd40 msgr2=0x7fd91c152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:57.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 shutdown_connections 2024-04-20T21:34:57.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:57.003+0000 7fd921e00640 1 -- 172.21.15.102:0/4171407516 wait complete. 2024-04-20T21:34:57.022 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:34:58.879 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:58.886 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:59.023 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:59.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.209+0000 7efe62600640 1 Processor -- start 2024-04-20T21:34:59.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.209+0000 7efe62600640 1 -- start start 2024-04-20T21:34:59.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.210+0000 7efe62600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c073a10 0x7efe5c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe62600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe5c1536d0 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe62600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe5c0c76c0 con 0x7efe5c14fa50 2024-04-20T21:34:59.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe62600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efe5c0c7960 con 0x7efe5c073a10 2024-04-20T21:34:59.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5aa00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efe5c0c9eb0 0x7efe5c161930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52886/0 (socket says 172.21.15.102:52886) 2024-04-20T21:34:59.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5b400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7efe5c14fa50 0x7efe5c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:44814/0 (socket says 172.21.15.102:44814) 2024-04-20T21:34:59.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5aa00640 1 -- 172.21.15.102:0/391493099 learned_addr learned my addr 172.21.15.102:0/391493099 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:34:59.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5be00640 1 --2- 172.21.15.102:0/391493099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c073a10 0x7efe5c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5a000640 1 -- 172.21.15.102:0/391493099 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1571965745 0 0) 0x7efe5c0c76c0 con 0x7efe5c14fa50 2024-04-20T21:34:59.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5a000640 1 -- 172.21.15.102:0/391493099 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe40003520 con 0x7efe5c14fa50 2024-04-20T21:34:59.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5a000640 1 -- 172.21.15.102:0/391493099 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235960462 0 0) 0x7efe48001110 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5a000640 1 -- 172.21.15.102:0/391493099 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe5c0c76c0 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5be00640 1 -- 172.21.15.102:0/391493099 >> v1:172.21.15.193:6789/0 conn(0x7efe5c14fa50 legacy=0x7efe5c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5be00640 1 -- 172.21.15.102:0/391493099 >> v1:172.21.15.102:6789/0 conn(0x7efe5c0c9eb0 legacy=0x7efe5c161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.211+0000 7efe5be00640 1 -- 172.21.15.102:0/391493099 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c0c3d90 con 0x7efe5c073a10 2024-04-20T21:34:59.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.213+0000 7efe5be00640 1 --2- 172.21.15.102:0/391493099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c073a10 0x7efe5c0c4830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0c3f60 tx=0x7efe5402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b07220b76b5053 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.213+0000 7efe5a000640 1 -- 172.21.15.102:0/391493099 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54039070 con 0x7efe5c073a10 2024-04-20T21:34:59.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.213+0000 7efe5a000640 1 -- 172.21.15.102:0/391493099 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5402cd50 con 0x7efe5c073a10 2024-04-20T21:34:59.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.213+0000 7efe5a000640 1 -- 172.21.15.102:0/391493099 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54034640 con 0x7efe5c073a10 2024-04-20T21:34:59.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.214+0000 7efe62600640 1 -- 172.21.15.102:0/391493099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c073a10 msgr2=0x7efe5c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.214+0000 7efe62600640 1 --2- 172.21.15.102:0/391493099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c073a10 0x7efe5c0c4830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0c3f60 tx=0x7efe5402bfa0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.214+0000 7efe62600640 1 -- 172.21.15.102:0/391493099 shutdown_connections 2024-04-20T21:34:59.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.214+0000 7efe62600640 1 --2- 172.21.15.102:0/391493099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c073a10 0x7efe5c0c4830 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.214+0000 7efe62600640 1 -- 172.21.15.102:0/391493099 >> 172.21.15.102:0/391493099 conn(0x7efe5c0c9390 msgr2=0x7efe5c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.214+0000 7efe62600640 1 -- 172.21.15.102:0/391493099 shutdown_connections 2024-04-20T21:34:59.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.214+0000 7efe62600640 1 -- 172.21.15.102:0/391493099 wait complete. 2024-04-20T21:34:59.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.216+0000 7efe62600640 1 Processor -- start 2024-04-20T21:34:59.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.216+0000 7efe62600640 1 -- start start 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe62600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe5c073a10 0x7efe5c1e2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe62600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c0c9eb0 0x7efe5c14c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe62600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe5c14fa50 0x7efe5c1f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe62600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c0c4030 con 0x7efe5c14fa50 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe62600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c0c7c00 con 0x7efe5c073a10 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe62600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efe5c0c4310 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe5c073a10 0x7efe5c1e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c0c9eb0 0x7efe5c14c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe5c14fa50 0x7efe5c1f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c0c9eb0 0x7efe5c14c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56620/0 (socket says 172.21.15.102:56620) 2024-04-20T21:34:59.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe5c073a10 0x7efe5c1e2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:43086/0 (socket says 172.21.15.102:43086) 2024-04-20T21:34:59.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5aa00640 1 -- 172.21.15.102:0/1349806286 learned_addr learned my addr 172.21.15.102:0/1349806286 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5aa00640 1 -- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe5c073a10 msgr2=0x7efe5c1e2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5aa00640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe5c073a10 0x7efe5c1e2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5aa00640 1 -- 172.21.15.102:0/1349806286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe5c14fa50 msgr2=0x7efe5c1f13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5aa00640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe5c14fa50 0x7efe5c1f13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.217+0000 7efe5aa00640 1 -- 172.21.15.102:0/1349806286 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c0c49c0 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.218+0000 7efe5aa00640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c0c9eb0 0x7efe5c14c610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe4800c730 tx=0x7efe4800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.218+0000 7efe53400640 1 -- 172.21.15.102:0/1349806286 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48017070 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.218+0000 7efe53400640 1 -- 172.21.15.102:0/1349806286 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe48005ab0 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.218+0000 7efe53400640 1 -- 172.21.15.102:0/1349806286 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48007e20 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.218+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c1f4d80 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.218+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c072c50 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.219+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c074340 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.220+0000 7efe53400640 1 -- 172.21.15.102:0/1349806286 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe4801a070 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.221+0000 7efe53400640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efe3c06c8f0 0x7efe3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.221+0000 7efe53400640 1 -- 172.21.15.102:0/1349806286 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7efe4808b5d0 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.222+0000 7efe5b400640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efe3c06c8f0 0x7efe3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.222+0000 7efe5b400640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efe3c06c8f0 0x7efe3c06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efe4c000f60 tx=0x7efe4c007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.228+0000 7efe53400640 1 -- 172.21.15.102:0/1349806286 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe5c1536d0 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.461+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe5c0c7320 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.462+0000 7efe53400640 1 -- 172.21.15.102:0/1349806286 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+6026 (secure 0 0 0) 0x7efe48058c00 con 0x7efe5c0c9eb0 2024-04-20T21:34:59.461 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:34:59.461 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:34:50.734531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:34:59.461 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 56 2024-04-20T21:34:59.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efe3c06c8f0 msgr2=0x7efe3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efe3c06c8f0 0x7efe3c06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efe4c000f60 tx=0x7efe4c007d50 comp rx=0 tx=0).stop 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c0c9eb0 msgr2=0x7efe5c14c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c0c9eb0 0x7efe5c14c610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe4800c730 tx=0x7efe4800cbf0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 shutdown_connections 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe5c14fa50 0x7efe5c1f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe5c0c9eb0 0x7efe5c14c610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efe3c06c8f0 0x7efe3c06edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 --2- 172.21.15.102:0/1349806286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe5c073a10 0x7efe5c1e2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 >> 172.21.15.102:0/1349806286 conn(0x7efe5c0c9390 msgr2=0x7efe5c1e7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 shutdown_connections 2024-04-20T21:34:59.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:34:59.464+0000 7efe62600640 1 -- 172.21.15.102:0/1349806286 wait complete. 2024-04-20T21:34:59.481 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:01.482 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:01.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.667+0000 7f82f9000640 1 Processor -- start 2024-04-20T21:35:01.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.668+0000 7f82f9000640 1 -- start start 2024-04-20T21:35:01.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82f9000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82f9000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f82f40c9eb0 0x7f82f4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82f9000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f41536d0 con 0x7f82f4073a10 2024-04-20T21:35:01.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82f9000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f82f4159500 con 0x7f82f40c9eb0 2024-04-20T21:35:01.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82f9000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f82f40c76c0 con 0x7f82f414fa50 2024-04-20T21:35:01.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82f2800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f82f4073a10 0x7f82f40c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52896/0 (socket says 172.21.15.102:52896) 2024-04-20T21:35:01.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82f2800640 1 -- 172.21.15.102:0/3855254989 learned_addr learned my addr 172.21.15.102:0/3855254989 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:01.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82e9400640 1 --2- 172.21.15.102:0/3855254989 >> v2:172.21.15.193:3300/0 conn(0x7f82f40c9eb0 0x7f82f4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.669+0000 7f82f1e00640 1 --2- 172.21.15.102:0/3855254989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.670+0000 7f82f1400640 1 -- 172.21.15.102:0/3855254989 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 232310256 0 0) 0x7f82ec001110 con 0x7f82f4073a10 2024-04-20T21:35:01.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.670+0000 7f82f1e00640 1 -- 172.21.15.102:0/3855254989 >> v2:172.21.15.193:3300/0 conn(0x7f82f40c9eb0 msgr2=0x7f82f4158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.670+0000 7f82f1e00640 1 --2- 172.21.15.102:0/3855254989 >> v2:172.21.15.193:3300/0 conn(0x7f82f40c9eb0 0x7f82f4158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.670+0000 7f82f1e00640 1 -- 172.21.15.102:0/3855254989 >> v1:172.21.15.102:6789/0 conn(0x7f82f4073a10 legacy=0x7f82f40c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.670+0000 7f82f1e00640 1 -- 172.21.15.102:0/3855254989 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f40c7960 con 0x7f82f414fa50 2024-04-20T21:35:01.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.670+0000 7f82e9400640 1 --2- 172.21.15.102:0/3855254989 >> v2:172.21.15.193:3300/0 conn(0x7f82f40c9eb0 0x7f82f4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:01.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.671+0000 7f82f1e00640 1 --2- 172.21.15.102:0/3855254989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f40c4830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82f40c7b30 tx=0x7f82e402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59616283ab916870 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.671+0000 7f82f1400640 1 -- 172.21.15.102:0/3855254989 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4039070 con 0x7f82f414fa50 2024-04-20T21:35:01.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.671+0000 7f82f1400640 1 -- 172.21.15.102:0/3855254989 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e402cce0 con 0x7f82f414fa50 2024-04-20T21:35:01.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.671+0000 7f82f1400640 1 -- 172.21.15.102:0/3855254989 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4034660 con 0x7f82f414fa50 2024-04-20T21:35:01.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.672+0000 7f82f9000640 1 -- 172.21.15.102:0/3855254989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 msgr2=0x7f82f40c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.672+0000 7f82f9000640 1 --2- 172.21.15.102:0/3855254989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f40c4830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82f40c7b30 tx=0x7f82e402bf30 comp rx=0 tx=0).stop 2024-04-20T21:35:01.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.672+0000 7f82f9000640 1 -- 172.21.15.102:0/3855254989 shutdown_connections 2024-04-20T21:35:01.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.672+0000 7f82f9000640 1 --2- 172.21.15.102:0/3855254989 >> v2:172.21.15.193:3300/0 conn(0x7f82f40c9eb0 0x7f82f4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.672+0000 7f82f9000640 1 --2- 172.21.15.102:0/3855254989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f40c4830 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.672+0000 7f82f9000640 1 -- 172.21.15.102:0/3855254989 >> 172.21.15.102:0/3855254989 conn(0x7f82f40c9390 msgr2=0x7f82f40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:01.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.672+0000 7f82f9000640 1 -- 172.21.15.102:0/3855254989 shutdown_connections 2024-04-20T21:35:01.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.672+0000 7f82f9000640 1 -- 172.21.15.102:0/3855254989 wait complete. 2024-04-20T21:35:01.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.675+0000 7f82f9000640 1 Processor -- start 2024-04-20T21:35:01.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.675+0000 7f82f9000640 1 -- start start 2024-04-20T21:35:01.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.676+0000 7f82f9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82f4073a10 0x7f82f414cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f9000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f40c9eb0 0x7f82f415c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f9000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f41f1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f9000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82f40c7c00 con 0x7f82f4073a10 2024-04-20T21:35:01.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f9000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f82f40c3d90 con 0x7f82f40c9eb0 2024-04-20T21:35:01.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f9000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f82f40c4030 con 0x7f82f414fa50 2024-04-20T21:35:01.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f2800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f41f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f2800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f41f1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56626/0 (socket says 172.21.15.102:56626) 2024-04-20T21:35:01.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f2800640 1 -- 172.21.15.102:0/3904376595 learned_addr learned my addr 172.21.15.102:0/3904376595 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:01.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82f1e00640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82f4073a10 0x7f82f414cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.677+0000 7f82e9400640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f40c9eb0 0x7f82f415c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.678+0000 7f82f2800640 1 -- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f40c9eb0 msgr2=0x7f82f415c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.678+0000 7f82f2800640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f40c9eb0 0x7f82f415c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.678+0000 7f82f2800640 1 -- 172.21.15.102:0/3904376595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82f4073a10 msgr2=0x7f82f414cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.678+0000 7f82f2800640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82f4073a10 0x7f82f414cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.678+0000 7f82f2800640 1 -- 172.21.15.102:0/3904376595 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f415d320 con 0x7f82f414fa50 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.678+0000 7f82f2800640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f41f1000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c440 tx=0x7f82ec00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.679+0000 7f82eaa00640 1 -- 172.21.15.102:0/3904376595 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0074e0 con 0x7f82f414fa50 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.679+0000 7f82eaa00640 1 -- 172.21.15.102:0/3904376595 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec0047e0 con 0x7f82f414fa50 2024-04-20T21:35:01.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.679+0000 7f82eaa00640 1 -- 172.21.15.102:0/3904376595 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec004bd0 con 0x7f82f414fa50 2024-04-20T21:35:01.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.679+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f41f47e0 con 0x7f82f414fa50 2024-04-20T21:35:01.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.679+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41f4da0 con 0x7f82f414fa50 2024-04-20T21:35:01.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.680+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f4074340 con 0x7f82f414fa50 2024-04-20T21:35:01.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.682+0000 7f82eaa00640 1 -- 172.21.15.102:0/3904376595 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82ec011070 con 0x7f82f414fa50 2024-04-20T21:35:01.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.683+0000 7f82eaa00640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82d006c8f0 0x7f82d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.683+0000 7f82eaa00640 1 -- 172.21.15.102:0/3904376595 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f82ec08bdb0 con 0x7f82f414fa50 2024-04-20T21:35:01.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.683+0000 7f82f1e00640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82d006c8f0 0x7f82d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.684+0000 7f82f1e00640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82d006c8f0 0x7f82d006edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f82e4036000 tx=0x7f82e4030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.688+0000 7f82eaa00640 1 -- 172.21.15.102:0/3904376595 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f82f41536d0 con 0x7f82f414fa50 2024-04-20T21:35:01.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.955+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82f40c7320 con 0x7f82f414fa50 2024-04-20T21:35:01.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.956+0000 7f82eaa00640 1 -- 172.21.15.102:0/3904376595 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+6026 (secure 0 0 0) 0x7f82ec059ab0 con 0x7f82f414fa50 2024-04-20T21:35:01.956 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:01.956 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:34:50.734531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:01.956 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 56 2024-04-20T21:35:01.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82d006c8f0 msgr2=0x7f82d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82d006c8f0 0x7f82d006edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f82e4036000 tx=0x7f82e4030000 comp rx=0 tx=0).stop 2024-04-20T21:35:01.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 msgr2=0x7f82f41f1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f41f1000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c440 tx=0x7f82ec00c900 comp rx=0 tx=0).stop 2024-04-20T21:35:01.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 shutdown_connections 2024-04-20T21:35:01.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82f414fa50 0x7f82f41f1000 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82d006c8f0 0x7f82d006edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f40c9eb0 0x7f82f415c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 --2- 172.21.15.102:0/3904376595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82f4073a10 0x7f82f414cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 >> 172.21.15.102:0/3904376595 conn(0x7f82f40c9390 msgr2=0x7f82f40cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:01.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 shutdown_connections 2024-04-20T21:35:01.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:01.960+0000 7f82f9000640 1 -- 172.21.15.102:0/3904376595 wait complete. 2024-04-20T21:35:01.986 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:03.987 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:04.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.194+0000 7ff693200640 1 Processor -- start 2024-04-20T21:35:04.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.194+0000 7ff693200640 1 -- start start 2024-04-20T21:35:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff693200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff694155530 0x7ff69415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff693200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff6940cb6a0 0x7ff6940cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff693200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69415ef60 con 0x7ff6940cc470 2024-04-20T21:35:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff693200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff6940c5ca0 con 0x7ff694155530 2024-04-20T21:35:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff693200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff6940cd5f0 con 0x7ff6940cb6a0 2024-04-20T21:35:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff691e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff694155530 0x7ff69415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff691e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff694155530 0x7ff69415e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51560/0 (socket says 172.21.15.102:51560) 2024-04-20T21:35:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff691e00640 1 -- 172.21.15.102:0/1028596968 learned_addr learned my addr 172.21.15.102:0/1028596968 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:04.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.196+0000 7ff692800640 1 --2- 172.21.15.102:0/1028596968 >> v2:172.21.15.193:3300/0 conn(0x7ff6940cb6a0 0x7ff6940cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:04.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.197+0000 7ff691e00640 1 -- 172.21.15.102:0/1028596968 >> v2:172.21.15.193:3300/0 conn(0x7ff6940cb6a0 msgr2=0x7ff6940cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:04.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.197+0000 7ff691e00640 1 --2- 172.21.15.102:0/1028596968 >> v2:172.21.15.193:3300/0 conn(0x7ff6940cb6a0 0x7ff6940cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.197+0000 7ff691e00640 1 -- 172.21.15.102:0/1028596968 >> v1:172.21.15.193:6789/0 conn(0x7ff6940cc470 legacy=0x7ff6940cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:04.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.197+0000 7ff691e00640 1 -- 172.21.15.102:0/1028596968 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69415f240 con 0x7ff694155530 2024-04-20T21:35:04.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.198+0000 7ff691e00640 1 --2- 172.21.15.102:0/1028596968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff694155530 0x7ff69415e640 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff69408e290 tx=0x7ff68402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5afd27efd8da3dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:04.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.198+0000 7ff690a00640 1 -- 172.21.15.102:0/1028596968 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684034070 con 0x7ff694155530 2024-04-20T21:35:04.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.198+0000 7ff690a00640 1 -- 172.21.15.102:0/1028596968 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff684030860 con 0x7ff694155530 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.198+0000 7ff690a00640 1 -- 172.21.15.102:0/1028596968 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684030c30 con 0x7ff694155530 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.199+0000 7ff693200640 1 -- 172.21.15.102:0/1028596968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff694155530 msgr2=0x7ff69415e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.199+0000 7ff693200640 1 --2- 172.21.15.102:0/1028596968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff694155530 0x7ff69415e640 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff69408e290 tx=0x7ff68402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.199+0000 7ff693200640 1 -- 172.21.15.102:0/1028596968 shutdown_connections 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.199+0000 7ff693200640 1 --2- 172.21.15.102:0/1028596968 >> v2:172.21.15.193:3300/0 conn(0x7ff6940cb6a0 0x7ff6940cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.199+0000 7ff693200640 1 --2- 172.21.15.102:0/1028596968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff694155530 0x7ff69415e640 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.199+0000 7ff693200640 1 -- 172.21.15.102:0/1028596968 >> 172.21.15.102:0/1028596968 conn(0x7ff694075080 msgr2=0x7ff694075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.199+0000 7ff693200640 1 -- 172.21.15.102:0/1028596968 shutdown_connections 2024-04-20T21:35:04.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.199+0000 7ff693200640 1 -- 172.21.15.102:0/1028596968 wait complete. 2024-04-20T21:35:04.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.201+0000 7ff693200640 1 Processor -- start 2024-04-20T21:35:04.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.201+0000 7ff693200640 1 -- start start 2024-04-20T21:35:04.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff693200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6940cb6a0 0x7ff694149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:04.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff693200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6940cc470 0x7ff69415e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:04.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff693200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff694155530 0x7ff6941e6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:04.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff693200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff6940cd280 con 0x7ff6940cc470 2024-04-20T21:35:04.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff693200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff6940ccfa0 con 0x7ff6940cb6a0 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff693200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff6940ccbe0 con 0x7ff694155530 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6940cc470 0x7ff69415e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6940cb6a0 0x7ff694149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6940cc470 0x7ff69415e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51562/0 (socket says 172.21.15.102:51562) 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6940cb6a0 0x7ff694149b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:43122/0 (socket says 172.21.15.102:43122) 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691400640 1 -- 172.21.15.102:0/1270462271 learned_addr learned my addr 172.21.15.102:0/1270462271 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff692800640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff694155530 0x7ff6941e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691400640 1 -- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff694155530 msgr2=0x7ff6941e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691400640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff694155530 0x7ff6941e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691400640 1 -- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6940cb6a0 msgr2=0x7ff694149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:04.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691400640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6940cb6a0 0x7ff694149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691400640 1 -- 172.21.15.102:0/1270462271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69415d860 con 0x7ff6940cc470 2024-04-20T21:35:04.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.202+0000 7ff691e00640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6940cb6a0 0x7ff694149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:04.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.203+0000 7ff691400640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6940cc470 0x7ff69415e2d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff68000c400 tx=0x7ff68000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:04.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.203+0000 7ff68a000640 1 -- 172.21.15.102:0/1270462271 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6800052d0 con 0x7ff6940cc470 2024-04-20T21:35:04.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.203+0000 7ff68a000640 1 -- 172.21.15.102:0/1270462271 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff680005490 con 0x7ff6940cc470 2024-04-20T21:35:04.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.203+0000 7ff68a000640 1 -- 172.21.15.102:0/1270462271 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680007260 con 0x7ff6940cc470 2024-04-20T21:35:04.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.203+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6941e6c80 con 0x7ff6940cc470 2024-04-20T21:35:04.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.203+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69415dad0 con 0x7ff6940cc470 2024-04-20T21:35:04.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.204+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69404fd60 con 0x7ff6940cc470 2024-04-20T21:35:04.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.205+0000 7ff68a000640 1 -- 172.21.15.102:0/1270462271 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff680007400 con 0x7ff6940cc470 2024-04-20T21:35:04.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.206+0000 7ff68a000640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff67406c8f0 0x7ff67406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:04.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.206+0000 7ff68a000640 1 -- 172.21.15.102:0/1270462271 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7ff68008b750 con 0x7ff6940cc470 2024-04-20T21:35:04.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.206+0000 7ff691e00640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff67406c8f0 0x7ff67406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:04.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.207+0000 7ff691e00640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff67406c8f0 0x7ff67406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff684036000 tx=0x7ff68403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:04.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.211+0000 7ff68a000640 1 -- 172.21.15.102:0/1270462271 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff680058e10 con 0x7ff6940cc470 2024-04-20T21:35:04.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.437+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6940cc9c0 con 0x7ff6940cc470 2024-04-20T21:35:04.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.438+0000 7ff68a000640 1 -- 172.21.15.102:0/1270462271 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+6030 (secure 0 0 0) 0x7ff68005d110 con 0x7ff6940cc470 2024-04-20T21:35:04.437 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:04.437 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:02.543557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:04.437 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 57 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.440+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff67406c8f0 msgr2=0x7ff67406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.440+0000 7ff693200640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff67406c8f0 0x7ff67406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff684036000 tx=0x7ff68403b040 comp rx=0 tx=0).stop 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6940cc470 msgr2=0x7ff69415e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6940cc470 0x7ff69415e2d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff68000c400 tx=0x7ff68000c8c0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 shutdown_connections 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff694155530 0x7ff6941e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6940cc470 0x7ff69415e2d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff67406c8f0 0x7ff67406edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 --2- 172.21.15.102:0/1270462271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6940cb6a0 0x7ff694149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 >> 172.21.15.102:0/1270462271 conn(0x7ff694075080 msgr2=0x7ff6940ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:04.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 shutdown_connections 2024-04-20T21:35:04.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:04.441+0000 7ff693200640 1 -- 172.21.15.102:0/1270462271 wait complete. 2024-04-20T21:35:04.458 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 57, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 57, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5994": { "addr": "172.21.15.102:6833/3985712430", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 3985712430, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 3985712430, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5994, "incarnation": 56, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 21 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:35:02.543557+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5994 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:35:06.459 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:06.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.667+0000 7f0c61e00640 1 Processor -- start 2024-04-20T21:35:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.667+0000 7f0c61e00640 1 -- start start 2024-04-20T21:35:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c61e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0c5c14fa50 0x7f0c5c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c61e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c073a10 0x7f0c5c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c61e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c5c1536d0 con 0x7f0c5c0c9eb0 2024-04-20T21:35:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c61e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0c5c159500 con 0x7f0c5c14fa50 2024-04-20T21:35:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c61e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0c5c0c76c0 con 0x7f0c5c073a10 2024-04-20T21:35:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c5aa00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0c5c14fa50 0x7f0c5c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c5b400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c073a10 0x7f0c5c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c5b400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c073a10 0x7f0c5c158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56642/0 (socket says 172.21.15.102:56642) 2024-04-20T21:35:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c5aa00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0c5c14fa50 0x7f0c5c0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:43124/0 (socket says 172.21.15.102:43124) 2024-04-20T21:35:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.669+0000 7f0c5b400640 1 -- 172.21.15.102:0/1346331018 learned_addr learned my addr 172.21.15.102:0/1346331018 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.670+0000 7f0c5b400640 1 -- 172.21.15.102:0/1346331018 >> v2:172.21.15.193:3300/0 conn(0x7f0c5c14fa50 msgr2=0x7f0c5c0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.670+0000 7f0c5b400640 1 --2- 172.21.15.102:0/1346331018 >> v2:172.21.15.193:3300/0 conn(0x7f0c5c14fa50 0x7f0c5c0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.670+0000 7f0c5b400640 1 -- 172.21.15.102:0/1346331018 >> v1:172.21.15.102:6789/0 conn(0x7f0c5c0c9eb0 legacy=0x7f0c5c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.670+0000 7f0c5b400640 1 -- 172.21.15.102:0/1346331018 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c0c7960 con 0x7f0c5c073a10 2024-04-20T21:35:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.671+0000 7f0c5b400640 1 --2- 172.21.15.102:0/1346331018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c073a10 0x7f0c5c158fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009e50 tx=0x7f0c5402bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63dd0aae7a5e109f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:06.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.671+0000 7f0c59600640 1 -- 172.21.15.102:0/1346331018 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5402cb90 con 0x7f0c5c073a10 2024-04-20T21:35:06.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.671+0000 7f0c59600640 1 -- 172.21.15.102:0/1346331018 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c5402cd30 con 0x7f0c5c073a10 2024-04-20T21:35:06.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.671+0000 7f0c59600640 1 -- 172.21.15.102:0/1346331018 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54034610 con 0x7f0c5c073a10 2024-04-20T21:35:06.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.672+0000 7f0c61e00640 1 -- 172.21.15.102:0/1346331018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c073a10 msgr2=0x7f0c5c158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:06.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.672+0000 7f0c61e00640 1 --2- 172.21.15.102:0/1346331018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c073a10 0x7f0c5c158fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009e50 tx=0x7f0c5402bff0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.672+0000 7f0c61e00640 1 -- 172.21.15.102:0/1346331018 shutdown_connections 2024-04-20T21:35:06.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.672+0000 7f0c61e00640 1 --2- 172.21.15.102:0/1346331018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c073a10 0x7f0c5c158fc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.672+0000 7f0c61e00640 1 --2- 172.21.15.102:0/1346331018 >> v2:172.21.15.193:3300/0 conn(0x7f0c5c14fa50 0x7f0c5c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.672+0000 7f0c61e00640 1 -- 172.21.15.102:0/1346331018 >> 172.21.15.102:0/1346331018 conn(0x7f0c5c0c9390 msgr2=0x7f0c5c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:06.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.672+0000 7f0c61e00640 1 -- 172.21.15.102:0/1346331018 shutdown_connections 2024-04-20T21:35:06.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.672+0000 7f0c61e00640 1 -- 172.21.15.102:0/1346331018 wait complete. 2024-04-20T21:35:06.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.674+0000 7f0c61e00640 1 Processor -- start 2024-04-20T21:35:06.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.674+0000 7f0c61e00640 1 -- start start 2024-04-20T21:35:06.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c61e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c5c073a10 0x7f0c5c158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:06.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c61e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c0c9eb0 0x7f0c5c0ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c61e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c5c14fa50 0x7f0c5c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c61e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c0c7c00 con 0x7f0c5c14fa50 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c61e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c0c3d90 con 0x7f0c5c073a10 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c61e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0c5c0c4030 con 0x7f0c5c0c9eb0 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c5c14fa50 0x7f0c5c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c5c14fa50 0x7f0c5c156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51578/0 (socket says 172.21.15.102:51578) 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5b400640 1 -- 172.21.15.102:0/3489732271 learned_addr learned my addr 172.21.15.102:0/3489732271 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5a000640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c0c9eb0 0x7f0c5c0ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5aa00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c5c073a10 0x7f0c5c158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5b400640 1 -- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c0c9eb0 msgr2=0x7f0c5c0ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5b400640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c0c9eb0 0x7f0c5c0ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5b400640 1 -- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c5c073a10 msgr2=0x7f0c5c158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:06.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5b400640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c5c073a10 0x7f0c5c158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.675+0000 7f0c5b400640 1 -- 172.21.15.102:0/3489732271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c1566e0 con 0x7f0c5c14fa50 2024-04-20T21:35:06.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.676+0000 7f0c5b400640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c5c14fa50 0x7f0c5c156e20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0c54036040 tx=0x7f0c54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:06.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.676+0000 7f0c53400640 1 -- 172.21.15.102:0/3489732271 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54043020 con 0x7f0c5c14fa50 2024-04-20T21:35:06.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.676+0000 7f0c53400640 1 -- 172.21.15.102:0/3489732271 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c5403e450 con 0x7f0c5c14fa50 2024-04-20T21:35:06.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.676+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5c071950 con 0x7f0c5c14fa50 2024-04-20T21:35:06.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.676+0000 7f0c53400640 1 -- 172.21.15.102:0/3489732271 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54009510 con 0x7f0c5c14fa50 2024-04-20T21:35:06.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.677+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c5c071f10 con 0x7f0c5c14fa50 2024-04-20T21:35:06.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.678+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c5c074340 con 0x7f0c5c14fa50 2024-04-20T21:35:06.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.679+0000 7f0c53400640 1 -- 172.21.15.102:0/3489732271 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c54046020 con 0x7f0c5c14fa50 2024-04-20T21:35:06.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.680+0000 7f0c53400640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c3c06c8f0 0x7f0c3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:06.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.680+0000 7f0c53400640 1 -- 172.21.15.102:0/3489732271 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f0c540ae760 con 0x7f0c5c14fa50 2024-04-20T21:35:06.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.680+0000 7f0c5aa00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c3c06c8f0 0x7f0c3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:06.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.681+0000 7f0c5aa00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c3c06c8f0 0x7f0c3c06edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009b40 tx=0x7f0c4c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:06.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.684+0000 7f0c53400640 1 -- 172.21.15.102:0/3489732271 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c5404d030 con 0x7f0c5c14fa50 2024-04-20T21:35:06.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.929+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c5c1568e0 con 0x7f0c5c14fa50 2024-04-20T21:35:06.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.930+0000 7f0c53400640 1 -- 172.21.15.102:0/3489732271 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+6030 (secure 0 0 0) 0x7f0c5407be30 con 0x7f0c5c14fa50 2024-04-20T21:35:06.929 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:06.929 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:02.543557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:06.930 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 57 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.932+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c3c06c8f0 msgr2=0x7f0c3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.932+0000 7f0c61e00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c3c06c8f0 0x7f0c3c06edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009b40 tx=0x7f0c4c009190 comp rx=0 tx=0).stop 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.932+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c5c14fa50 msgr2=0x7f0c5c156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.932+0000 7f0c61e00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c5c14fa50 0x7f0c5c156e20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0c54036040 tx=0x7f0c54004a40 comp rx=0 tx=0).stop 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.933+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 shutdown_connections 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.933+0000 7f0c61e00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c5c14fa50 0x7f0c5c156e20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.933+0000 7f0c61e00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c5c0c9eb0 0x7f0c5c0ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.933+0000 7f0c61e00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0c3c06c8f0 0x7f0c3c06edb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.933+0000 7f0c61e00640 1 --2- 172.21.15.102:0/3489732271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c5c073a10 0x7f0c5c158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:06.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.933+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 >> 172.21.15.102:0/3489732271 conn(0x7f0c5c0c9390 msgr2=0x7f0c5c14c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:06.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.933+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 shutdown_connections 2024-04-20T21:35:06.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:06.933+0000 7f0c61e00640 1 -- 172.21.15.102:0/3489732271 wait complete. 2024-04-20T21:35:06.951 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:08.952 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:09.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.163+0000 7f376e200640 1 Processor -- start 2024-04-20T21:35:09.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.163+0000 7f376e200640 1 -- start start 2024-04-20T21:35:09.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.165+0000 7f376e200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f376815ee00 0x7f376806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.165+0000 7f376e200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37680c8f20 con 0x7f376815ea20 2024-04-20T21:35:09.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.165+0000 7f376e200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376806cb80 con 0x7f376815f2b0 2024-04-20T21:35:09.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.165+0000 7f376e200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f376806ce60 con 0x7f376815ee00 2024-04-20T21:35:09.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.165+0000 7f3767400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f376815ea20 0x7f37680c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34446/0 (socket says 172.21.15.102:34446) 2024-04-20T21:35:09.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.165+0000 7f3767400640 1 -- 172.21.15.102:0/3287131665 learned_addr learned my addr 172.21.15.102:0/3287131665 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:09.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.165+0000 7f3766a00640 1 --2- 172.21.15.102:0/3287131665 >> v2:172.21.15.193:3300/0 conn(0x7f376815ee00 0x7f376806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.166+0000 7f3766000640 1 -- 172.21.15.102:0/3287131665 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164399886 0 0) 0x7f37680c8f20 con 0x7f376815ea20 2024-04-20T21:35:09.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.166+0000 7f3766000640 1 -- 172.21.15.102:0/3287131665 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f374c003520 con 0x7f376815ea20 2024-04-20T21:35:09.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.166+0000 7f3766000640 1 -- 172.21.15.102:0/3287131665 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2032233583 0 0) 0x7f376806cb80 con 0x7f376815f2b0 2024-04-20T21:35:09.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.166+0000 7f3766000640 1 -- 172.21.15.102:0/3287131665 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37680c8f20 con 0x7f376815f2b0 2024-04-20T21:35:09.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.166+0000 7f3766a00640 1 -- 172.21.15.102:0/3287131665 >> v1:172.21.15.193:6789/0 conn(0x7f376815f2b0 legacy=0x7f3768154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.166+0000 7f3766a00640 1 -- 172.21.15.102:0/3287131665 >> v1:172.21.15.102:6789/0 conn(0x7f376815ea20 legacy=0x7f37680c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.166+0000 7f3766a00640 1 -- 172.21.15.102:0/3287131665 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376806d140 con 0x7f376815ee00 2024-04-20T21:35:09.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.167+0000 7f3766a00640 1 --2- 172.21.15.102:0/3287131665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ee00 0x7f376806d940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f37500097a0 tx=0x7f375002b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cef742be5acc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:09.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.167+0000 7f3766000640 1 -- 172.21.15.102:0/3287131665 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375002f810 con 0x7f376815ee00 2024-04-20T21:35:09.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.167+0000 7f3766000640 1 -- 172.21.15.102:0/3287131665 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375002f9b0 con 0x7f376815ee00 2024-04-20T21:35:09.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.167+0000 7f3766000640 1 -- 172.21.15.102:0/3287131665 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375002fd80 con 0x7f376815ee00 2024-04-20T21:35:09.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.168+0000 7f376e200640 1 -- 172.21.15.102:0/3287131665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ee00 msgr2=0x7f376806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.168+0000 7f376e200640 1 --2- 172.21.15.102:0/3287131665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ee00 0x7f376806d940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f37500097a0 tx=0x7f375002b9f0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.168+0000 7f376e200640 1 -- 172.21.15.102:0/3287131665 shutdown_connections 2024-04-20T21:35:09.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.168+0000 7f376e200640 1 --2- 172.21.15.102:0/3287131665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ee00 0x7f376806d940 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.168+0000 7f376e200640 1 -- 172.21.15.102:0/3287131665 >> 172.21.15.102:0/3287131665 conn(0x7f37680c9390 msgr2=0x7f37680c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:09.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.168+0000 7f376e200640 1 -- 172.21.15.102:0/3287131665 shutdown_connections 2024-04-20T21:35:09.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.168+0000 7f376e200640 1 -- 172.21.15.102:0/3287131665 wait complete. 2024-04-20T21:35:09.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.170+0000 7f376e200640 1 Processor -- start 2024-04-20T21:35:09.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.171+0000 7f376e200640 1 -- start start 2024-04-20T21:35:09.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f376e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ea20 0x7f37681450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f376e200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f376815ee00 0x7f37681f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f376e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376815f2b0 0x7f376814a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f376e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37680c8c80 con 0x7f376815f2b0 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f376e200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37680c8b00 con 0x7f376815ea20 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f376e200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3768155590 con 0x7f376815ee00 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f3767e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376815f2b0 0x7f376814a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f3767e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376815f2b0 0x7f376814a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56594/0 (socket says 172.21.15.102:56594) 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f3767e00640 1 -- 172.21.15.102:0/3356527815 learned_addr learned my addr 172.21.15.102:0/3356527815 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f3766a00640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f376815ee00 0x7f37681f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.172+0000 7f3767400640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ea20 0x7f37681450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f3767e00640 1 -- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f376815ee00 msgr2=0x7f37681f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f3767e00640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f376815ee00 0x7f37681f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f3767e00640 1 -- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ea20 msgr2=0x7f37681450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f3767e00640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ea20 0x7f37681450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f3767e00640 1 -- 172.21.15.102:0/3356527815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37681f44f0 con 0x7f376815f2b0 2024-04-20T21:35:09.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f3767400640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ea20 0x7f37681450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:09.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f3767e00640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376815f2b0 0x7f376814a3c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f376000c710 tx=0x7f376000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:09.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f375f400640 1 -- 172.21.15.102:0/3356527815 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760003b70 con 0x7f376815f2b0 2024-04-20T21:35:09.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f375f400640 1 -- 172.21.15.102:0/3356527815 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3760003d10 con 0x7f376815f2b0 2024-04-20T21:35:09.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37681f47d0 con 0x7f376815f2b0 2024-04-20T21:35:09.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f375f400640 1 -- 172.21.15.102:0/3356527815 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376000aa50 con 0x7f376815f2b0 2024-04-20T21:35:09.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.173+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681f4d90 con 0x7f376815f2b0 2024-04-20T21:35:09.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.175+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3768070310 con 0x7f376815f2b0 2024-04-20T21:35:09.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.176+0000 7f375f400640 1 -- 172.21.15.102:0/3356527815 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f376000abf0 con 0x7f376815f2b0 2024-04-20T21:35:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.177+0000 7f375f400640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f374406c8f0 0x7f374406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.177+0000 7f375f400640 1 -- 172.21.15.102:0/3356527815 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f376008c060 con 0x7f376815f2b0 2024-04-20T21:35:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.177+0000 7f3767400640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f374406c8f0 0x7f374406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.178+0000 7f3767400640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f374406c8f0 0x7f374406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f37580011e0 tx=0x7f3758007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:09.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.181+0000 7f375f400640 1 -- 172.21.15.102:0/3356527815 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3768070310 con 0x7f376815f2b0 2024-04-20T21:35:09.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.421+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3768074340 con 0x7f376815f2b0 2024-04-20T21:35:09.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.422+0000 7f375f400640 1 -- 172.21.15.102:0/3356527815 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6027 (secure 0 0 0) 0x7f376000ce20 con 0x7f376815f2b0 2024-04-20T21:35:09.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:08.245010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:09.421 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 58 2024-04-20T21:35:09.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f374406c8f0 msgr2=0x7f374406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f374406c8f0 0x7f374406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f37580011e0 tx=0x7f3758007d30 comp rx=0 tx=0).stop 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376815f2b0 msgr2=0x7f376814a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376815f2b0 0x7f376814a3c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f376000c710 tx=0x7f376000cbd0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 shutdown_connections 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376815f2b0 0x7f376814a3c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f376815ee00 0x7f37681f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f374406c8f0 0x7f374406edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 --2- 172.21.15.102:0/3356527815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f376815ea20 0x7f37681450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 >> 172.21.15.102:0/3356527815 conn(0x7f37680c9390 msgr2=0x7f3768154c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 shutdown_connections 2024-04-20T21:35:09.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:09.424+0000 7f376e200640 1 -- 172.21.15.102:0/3356527815 wait complete. 2024-04-20T21:35:09.441 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:11.442 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:11.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.652+0000 7f6e58000640 1 Processor -- start 2024-04-20T21:35:11.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.652+0000 7f6e58000640 1 -- start start 2024-04-20T21:35:11.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.654+0000 7f6e58000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e500c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:11.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.654+0000 7f6e58000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e501536d0 con 0x7f6e50073a10 2024-04-20T21:35:11.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.654+0000 7f6e58000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e500c76c0 con 0x7f6e500c9eb0 2024-04-20T21:35:11.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.654+0000 7f6e58000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6e500c7960 con 0x7f6e5014fa50 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.654+0000 7f6e56000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6e50073a10 0x7f6e50163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:34448/0 (socket says 172.21.15.102:34448) 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.654+0000 7f6e56000640 1 -- 172.21.15.102:0/3644027717 learned_addr learned my addr 172.21.15.102:0/3644027717 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.654+0000 7f6e55600640 1 --2- 172.21.15.102:0/3644027717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e500c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.654+0000 7f6e4fe00640 1 -- 172.21.15.102:0/3644027717 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2623625218 0 0) 0x7f6e500c76c0 con 0x7f6e500c9eb0 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.655+0000 7f6e4fe00640 1 -- 172.21.15.102:0/3644027717 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e38003520 con 0x7f6e500c9eb0 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.655+0000 7f6e4fe00640 1 -- 172.21.15.102:0/3644027717 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365764012 0 0) 0x7f6e48001490 con 0x7f6e50073a10 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.655+0000 7f6e4fe00640 1 -- 172.21.15.102:0/3644027717 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e500c76c0 con 0x7f6e50073a10 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.655+0000 7f6e55600640 1 -- 172.21.15.102:0/3644027717 >> v1:172.21.15.193:6789/0 conn(0x7f6e500c9eb0 legacy=0x7f6e500c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.655+0000 7f6e55600640 1 -- 172.21.15.102:0/3644027717 >> v1:172.21.15.102:6789/0 conn(0x7f6e50073a10 legacy=0x7f6e50163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:11.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.655+0000 7f6e55600640 1 -- 172.21.15.102:0/3644027717 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e500c3d90 con 0x7f6e5014fa50 2024-04-20T21:35:11.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.656+0000 7f6e55600640 1 --2- 172.21.15.102:0/3644027717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e500c4830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6e500c3f60 tx=0x7f6e4402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5667988d7d03e0a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:11.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.656+0000 7f6e4fe00640 1 -- 172.21.15.102:0/3644027717 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44034070 con 0x7f6e5014fa50 2024-04-20T21:35:11.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.656+0000 7f6e4fe00640 1 -- 172.21.15.102:0/3644027717 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e44030880 con 0x7f6e5014fa50 2024-04-20T21:35:11.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.656+0000 7f6e4fe00640 1 -- 172.21.15.102:0/3644027717 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44030c30 con 0x7f6e5014fa50 2024-04-20T21:35:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.657+0000 7f6e58000640 1 -- 172.21.15.102:0/3644027717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 msgr2=0x7f6e500c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.657+0000 7f6e58000640 1 --2- 172.21.15.102:0/3644027717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e500c4830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6e500c3f60 tx=0x7f6e4402b9c0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.657+0000 7f6e58000640 1 -- 172.21.15.102:0/3644027717 shutdown_connections 2024-04-20T21:35:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.657+0000 7f6e58000640 1 --2- 172.21.15.102:0/3644027717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e500c4830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.657+0000 7f6e58000640 1 -- 172.21.15.102:0/3644027717 >> 172.21.15.102:0/3644027717 conn(0x7f6e500c9390 msgr2=0x7f6e500c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.657+0000 7f6e58000640 1 -- 172.21.15.102:0/3644027717 shutdown_connections 2024-04-20T21:35:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.657+0000 7f6e58000640 1 -- 172.21.15.102:0/3644027717 wait complete. 2024-04-20T21:35:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.658+0000 7f6e58000640 1 Processor -- start 2024-04-20T21:35:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.659+0000 7f6e58000640 1 -- start start 2024-04-20T21:35:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.659+0000 7f6e58000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e50073a10 0x7f6e500ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.659+0000 7f6e58000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e500c9eb0 0x7f6e5014e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e58000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e50149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e58000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6e500c4030 con 0x7f6e50073a10 2024-04-20T21:35:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e58000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6e500c7c00 con 0x7f6e500c9eb0 2024-04-20T21:35:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e58000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6e500c4310 con 0x7f6e5014fa50 2024-04-20T21:35:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e50073a10 0x7f6e500ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e50073a10 0x7f6e500ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56600/0 (socket says 172.21.15.102:56600) 2024-04-20T21:35:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e55600640 1 -- 172.21.15.102:0/1316282598 learned_addr learned my addr 172.21.15.102:0/1316282598 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e56000640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e50149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e54c00640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e500c9eb0 0x7f6e5014e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e55600640 1 -- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 msgr2=0x7f6e50149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e55600640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e50149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e55600640 1 -- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e500c9eb0 msgr2=0x7f6e5014e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e55600640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e500c9eb0 0x7f6e5014e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.660+0000 7f6e55600640 1 -- 172.21.15.102:0/1316282598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e50159190 con 0x7f6e50073a10 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.661+0000 7f6e55600640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e50073a10 0x7f6e500ca690 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6e44000c00 tx=0x7f6e440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.661+0000 7f6e4d600640 1 -- 172.21.15.102:0/1316282598 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44034040 con 0x7f6e50073a10 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.661+0000 7f6e4d600640 1 -- 172.21.15.102:0/1316282598 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e4403ada0 con 0x7f6e50073a10 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.661+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e50159450 con 0x7f6e50073a10 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.661+0000 7f6e4d600640 1 -- 172.21.15.102:0/1316282598 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44006e10 con 0x7f6e50073a10 2024-04-20T21:35:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.661+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e50157290 con 0x7f6e50073a10 2024-04-20T21:35:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.662+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e50074340 con 0x7f6e50073a10 2024-04-20T21:35:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.663+0000 7f6e4d600640 1 -- 172.21.15.102:0/1316282598 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e4404b060 con 0x7f6e50073a10 2024-04-20T21:35:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.664+0000 7f6e4d600640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6e3006c8f0 0x7f6e3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.664+0000 7f6e4d600640 1 -- 172.21.15.102:0/1316282598 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f6e440ae2f0 con 0x7f6e50073a10 2024-04-20T21:35:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.665+0000 7f6e54c00640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6e3006c8f0 0x7f6e3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:11.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.665+0000 7f6e54c00640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6e3006c8f0 0x7f6e3006edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6e5014eac0 tx=0x7f6e3c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:11.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.669+0000 7f6e4d600640 1 -- 172.21.15.102:0/1316282598 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e4403f070 con 0x7f6e50073a10 2024-04-20T21:35:11.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.909+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e50159860 con 0x7f6e50073a10 2024-04-20T21:35:11.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.910+0000 7f6e4d600640 1 -- 172.21.15.102:0/1316282598 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6027 (secure 0 0 0) 0x7f6e4407b930 con 0x7f6e50073a10 2024-04-20T21:35:11.909 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:08.245010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:11.910 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 58 2024-04-20T21:35:11.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6e3006c8f0 msgr2=0x7f6e3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:11.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6e3006c8f0 0x7f6e3006edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6e5014eac0 tx=0x7f6e3c009040 comp rx=0 tx=0).stop 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e50073a10 msgr2=0x7f6e500ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e50073a10 0x7f6e500ca690 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6e44000c00 tx=0x7f6e440046c0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 shutdown_connections 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e5014fa50 0x7f6e50149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6e3006c8f0 0x7f6e3006edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e500c9eb0 0x7f6e5014e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 --2- 172.21.15.102:0/1316282598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e50073a10 0x7f6e500ca690 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 >> 172.21.15.102:0/1316282598 conn(0x7f6e500c9390 msgr2=0x7f6e500c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 shutdown_connections 2024-04-20T21:35:11.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:11.912+0000 7f6e58000640 1 -- 172.21.15.102:0/1316282598 wait complete. 2024-04-20T21:35:11.930 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:13.930 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.140+0000 7f3bdba00640 1 Processor -- start 2024-04-20T21:35:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.140+0000 7f3bdba00640 1 -- start start 2024-04-20T21:35:14.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bdba00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3bd40c9eb0 0x7f3bd40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:14.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bdba00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3bd4073a10 0x7f3bd4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:14.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bdba00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bd41536d0 con 0x7f3bd414fa50 2024-04-20T21:35:14.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bdba00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd4159500 con 0x7f3bd40c9eb0 2024-04-20T21:35:14.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bdba00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd40c76c0 con 0x7f3bd4073a10 2024-04-20T21:35:14.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bd3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3bd40c9eb0 0x7f3bd40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:14.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bd3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3bd40c9eb0 0x7f3bd40c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56612/0 (socket says 172.21.15.102:56612) 2024-04-20T21:35:14.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bd3e00640 1 -- 172.21.15.102:0/3840739121 learned_addr learned my addr 172.21.15.102:0/3840739121 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:14.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bd9a00640 1 --2- 172.21.15.102:0/3840739121 >> v2:172.21.15.193:3300/0 conn(0x7f3bd4073a10 0x7f3bd4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:14.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.142+0000 7f3bd3e00640 1 -- 172.21.15.102:0/3840739121 >> v2:172.21.15.193:3300/0 conn(0x7f3bd4073a10 msgr2=0x7f3bd4158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:14.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.143+0000 7f3bd3e00640 1 --2- 172.21.15.102:0/3840739121 >> v2:172.21.15.193:3300/0 conn(0x7f3bd4073a10 0x7f3bd4158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.143+0000 7f3bd3e00640 1 -- 172.21.15.102:0/3840739121 >> v1:172.21.15.193:6789/0 conn(0x7f3bd414fa50 legacy=0x7f3bd40c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:14.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.143+0000 7f3bd3e00640 1 -- 172.21.15.102:0/3840739121 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd40c7960 con 0x7f3bd40c9eb0 2024-04-20T21:35:14.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.143+0000 7f3bd9a00640 1 --2- 172.21.15.102:0/3840739121 >> v2:172.21.15.193:3300/0 conn(0x7f3bd4073a10 0x7f3bd4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:14.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.143+0000 7f3bd3400640 1 -- 172.21.15.102:0/3840739121 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4108152684 0 0) 0x7f3bc8001110 con 0x7f3bd414fa50 2024-04-20T21:35:14.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.144+0000 7f3bd3e00640 1 --2- 172.21.15.102:0/3840739121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd40c4830 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4156b00 tx=0x7f3bc002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88eb36d4710eeeea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:14.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.144+0000 7f3bd3400640 1 -- 172.21.15.102:0/3840739121 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc002f850 con 0x7f3bd40c9eb0 2024-04-20T21:35:14.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.144+0000 7f3bd3400640 1 -- 172.21.15.102:0/3840739121 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc002f9f0 con 0x7f3bd40c9eb0 2024-04-20T21:35:14.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.144+0000 7f3bd3400640 1 -- 172.21.15.102:0/3840739121 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc002fdc0 con 0x7f3bd40c9eb0 2024-04-20T21:35:14.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.145+0000 7f3bdba00640 1 -- 172.21.15.102:0/3840739121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd40c9eb0 msgr2=0x7f3bd40c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:14.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.145+0000 7f3bdba00640 1 --2- 172.21.15.102:0/3840739121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd40c4830 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4156b00 tx=0x7f3bc002ba30 comp rx=0 tx=0).stop 2024-04-20T21:35:14.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.145+0000 7f3bdba00640 1 -- 172.21.15.102:0/3840739121 shutdown_connections 2024-04-20T21:35:14.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.145+0000 7f3bdba00640 1 --2- 172.21.15.102:0/3840739121 >> v2:172.21.15.193:3300/0 conn(0x7f3bd4073a10 0x7f3bd4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.145+0000 7f3bdba00640 1 --2- 172.21.15.102:0/3840739121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd40c4830 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.145+0000 7f3bdba00640 1 -- 172.21.15.102:0/3840739121 >> 172.21.15.102:0/3840739121 conn(0x7f3bd40c9390 msgr2=0x7f3bd40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:14.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.145+0000 7f3bdba00640 1 -- 172.21.15.102:0/3840739121 shutdown_connections 2024-04-20T21:35:14.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.145+0000 7f3bdba00640 1 -- 172.21.15.102:0/3840739121 wait complete. 2024-04-20T21:35:14.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.146+0000 7f3bdba00640 1 Processor -- start 2024-04-20T21:35:14.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.147+0000 7f3bdba00640 1 -- start start 2024-04-20T21:35:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.147+0000 7f3bdba00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bd4073a10 0x7f3bd40c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.147+0000 7f3bdba00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd41e2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.147+0000 7f3bdba00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd414fa50 0x7f3bd4156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.147+0000 7f3bdba00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40c7c00 con 0x7f3bd414fa50 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.147+0000 7f3bdba00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40c3d90 con 0x7f3bd40c9eb0 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.147+0000 7f3bdba00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd40c4030 con 0x7f3bd4073a10 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd414fa50 0x7f3bd4156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd41e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd41e2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:45070/0 (socket says 172.21.15.102:45070) 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd9000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bd4073a10 0x7f3bd40c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd414fa50 0x7f3bd4156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56618/0 (socket says 172.21.15.102:56618) 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd3e00640 1 -- 172.21.15.102:0/4129632198 learned_addr learned my addr 172.21.15.102:0/4129632198 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd9a00640 1 -- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bd4073a10 msgr2=0x7f3bd40c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd9a00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bd4073a10 0x7f3bd40c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd9a00640 1 -- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bd40c9eb0 msgr2=0x7f3bd41e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd9a00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd41e2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd9a00640 1 -- 172.21.15.102:0/4129632198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd4072e00 con 0x7f3bd414fa50 2024-04-20T21:35:14.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.148+0000 7f3bd3e00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd41e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:14.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.149+0000 7f3bd9a00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd414fa50 0x7f3bd4156e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009e50 tx=0x7f3bcc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.149+0000 7f3bd0c00640 1 -- 172.21.15.102:0/4129632198 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc018030 con 0x7f3bd414fa50 2024-04-20T21:35:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.149+0000 7f3bd0c00640 1 -- 172.21.15.102:0/4129632198 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bcc00cd90 con 0x7f3bd414fa50 2024-04-20T21:35:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.149+0000 7f3bd0c00640 1 -- 172.21.15.102:0/4129632198 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc003bb0 con 0x7f3bd414fa50 2024-04-20T21:35:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.149+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd4072840 con 0x7f3bd414fa50 2024-04-20T21:35:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.149+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd4149860 con 0x7f3bd414fa50 2024-04-20T21:35:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.150+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd4074340 con 0x7f3bd414fa50 2024-04-20T21:35:14.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.151+0000 7f3bd0c00640 1 -- 172.21.15.102:0/4129632198 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3bcc01b070 con 0x7f3bd414fa50 2024-04-20T21:35:14.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.152+0000 7f3bd0c00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3bb406c920 0x7f3bb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:14.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.152+0000 7f3bd0c00640 1 -- 172.21.15.102:0/4129632198 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f3bcc08cb20 con 0x7f3bd414fa50 2024-04-20T21:35:14.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.153+0000 7f3bd9000640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3bb406c920 0x7f3bb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:14.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.153+0000 7f3bd9000640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3bb406c920 0x7f3bb406ede0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8000f60 tx=0x7f3bc8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:14.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.157+0000 7f3bd0c00640 1 -- 172.21.15.102:0/4129632198 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3bcc010040 con 0x7f3bd414fa50 2024-04-20T21:35:14.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.400+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bd41568e0 con 0x7f3bd414fa50 2024-04-20T21:35:14.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.401+0000 7f3bd0c00640 1 -- 172.21.15.102:0/4129632198 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+6033 (secure 0 0 0) 0x7f3bcc05a150 con 0x7f3bd414fa50 2024-04-20T21:35:14.401 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:13.437415+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:clientreplay","state_seq":25,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:14.401 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 59 2024-04-20T21:35:14.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3bb406c920 msgr2=0x7f3bb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:14.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3bb406c920 0x7f3bb406ede0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8000f60 tx=0x7f3bc8009040 comp rx=0 tx=0).stop 2024-04-20T21:35:14.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd414fa50 msgr2=0x7f3bd4156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:14.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd414fa50 0x7f3bd4156e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009e50 tx=0x7f3bcc00c490 comp rx=0 tx=0).stop 2024-04-20T21:35:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 shutdown_connections 2024-04-20T21:35:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bd414fa50 0x7f3bd4156e30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3bb406c920 0x7f3bb406ede0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bd40c9eb0 0x7f3bd41e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 --2- 172.21.15.102:0/4129632198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bd4073a10 0x7f3bd40c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 >> 172.21.15.102:0/4129632198 conn(0x7f3bd40c9390 msgr2=0x7f3bd41441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 shutdown_connections 2024-04-20T21:35:14.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:14.404+0000 7f3bdba00640 1 -- 172.21.15.102:0/4129632198 wait complete. 2024-04-20T21:35:14.421 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:16.422 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:16.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.635+0000 7f9c9e200640 1 Processor -- start 2024-04-20T21:35:16.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.635+0000 7f9c9e200640 1 -- start start 2024-04-20T21:35:16.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c9e200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9c98162e40 0x7f9c9806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:16.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c9e200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c98159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:16.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c9e200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9806d2e0 con 0x7f9c98163660 2024-04-20T21:35:16.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c9e200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9c980c7290 con 0x7f9c98162e40 2024-04-20T21:35:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c9e200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9c980c7570 con 0x7f9c98163220 2024-04-20T21:35:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c97400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9c98162e40 0x7f9c9806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c98159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c97e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f9c98163660 0x7f9c980c7a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:32920/0 (socket says 172.21.15.102:32920) 2024-04-20T21:35:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c97e00640 1 -- 172.21.15.102:0/1334783013 learned_addr learned my addr 172.21.15.102:0/1334783013 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.637+0000 7f9c97400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9c98162e40 0x7f9c9806c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56630/0 (socket says 172.21.15.102:56630) 2024-04-20T21:35:16.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.638+0000 7f9c96a00640 1 -- 172.21.15.102:0/1334783013 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074906762 0 0) 0x7f9c9806d2e0 con 0x7f9c98163660 2024-04-20T21:35:16.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.638+0000 7f9c8fe00640 1 -- 172.21.15.102:0/1334783013 >> v2:172.21.15.102:3300/0 conn(0x7f9c98162e40 msgr2=0x7f9c9806c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:16.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.638+0000 7f9c8fe00640 1 --2- 172.21.15.102:0/1334783013 >> v2:172.21.15.102:3300/0 conn(0x7f9c98162e40 0x7f9c9806c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.638+0000 7f9c8fe00640 1 -- 172.21.15.102:0/1334783013 >> v1:172.21.15.193:6789/0 conn(0x7f9c98163660 legacy=0x7f9c980c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:16.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.638+0000 7f9c8fe00640 1 -- 172.21.15.102:0/1334783013 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9806d5c0 con 0x7f9c98163220 2024-04-20T21:35:16.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.639+0000 7f9c8fe00640 1 --2- 172.21.15.102:0/1334783013 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c98159440 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c800097a0 tx=0x7f9c8002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba3ce3dafe677a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:16.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.639+0000 7f9c96a00640 1 -- 172.21.15.102:0/1334783013 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80035020 con 0x7f9c98163220 2024-04-20T21:35:16.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.639+0000 7f9c96a00640 1 -- 172.21.15.102:0/1334783013 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c80031810 con 0x7f9c98163220 2024-04-20T21:35:16.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.639+0000 7f9c96a00640 1 -- 172.21.15.102:0/1334783013 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80031be0 con 0x7f9c98163220 2024-04-20T21:35:16.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.640+0000 7f9c9e200640 1 -- 172.21.15.102:0/1334783013 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 msgr2=0x7f9c98159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:16.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.640+0000 7f9c9e200640 1 --2- 172.21.15.102:0/1334783013 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c98159440 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c800097a0 tx=0x7f9c8002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.640+0000 7f9c9e200640 1 -- 172.21.15.102:0/1334783013 shutdown_connections 2024-04-20T21:35:16.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.640+0000 7f9c9e200640 1 --2- 172.21.15.102:0/1334783013 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c98159440 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.640+0000 7f9c9e200640 1 --2- 172.21.15.102:0/1334783013 >> v2:172.21.15.102:3300/0 conn(0x7f9c98162e40 0x7f9c9806c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.640+0000 7f9c9e200640 1 -- 172.21.15.102:0/1334783013 >> 172.21.15.102:0/1334783013 conn(0x7f9c980c9390 msgr2=0x7f9c980c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:16.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.640+0000 7f9c9e200640 1 -- 172.21.15.102:0/1334783013 shutdown_connections 2024-04-20T21:35:16.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.640+0000 7f9c9e200640 1 -- 172.21.15.102:0/1334783013 wait complete. 2024-04-20T21:35:16.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.642+0000 7f9c9e200640 1 Processor -- start 2024-04-20T21:35:16.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.642+0000 7f9c9e200640 1 -- start start 2024-04-20T21:35:16.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c9e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c98162e40 0x7f9c980c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:16.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c9e200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c980c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:16.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c9e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c98163660 0x7f9c980c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c9e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9c980c9080 con 0x7f9c98163660 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c9e200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9806db80 con 0x7f9c98162e40 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c9e200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9806cf20 con 0x7f9c98163220 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c97e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c98163660 0x7f9c980c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c97e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c98163660 0x7f9c980c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56640/0 (socket says 172.21.15.102:56640) 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c97e00640 1 -- 172.21.15.102:0/42622646 learned_addr learned my addr 172.21.15.102:0/42622646 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c8fe00640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c980c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.643+0000 7f9c97400640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c98162e40 0x7f9c980c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c97e00640 1 -- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 msgr2=0x7f9c980c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c97e00640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c980c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c97e00640 1 -- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c98162e40 msgr2=0x7f9c980c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c97e00640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c98162e40 0x7f9c980c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c97e00640 1 -- 172.21.15.102:0/42622646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c981f50c0 con 0x7f9c98163660 2024-04-20T21:35:16.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c8fe00640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c980c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c97e00640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c98163660 0x7f9c980c4410 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c7b0 tx=0x7f9c9000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c8f400640 1 -- 172.21.15.102:0/42622646 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90018070 con 0x7f9c98163660 2024-04-20T21:35:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c8f400640 1 -- 172.21.15.102:0/42622646 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9000a640 con 0x7f9c98163660 2024-04-20T21:35:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c8f400640 1 -- 172.21.15.102:0/42622646 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000aa30 con 0x7f9c98163660 2024-04-20T21:35:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.644+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c980c6b60 con 0x7f9c98163660 2024-04-20T21:35:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.645+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c981f5660 con 0x7f9c98163660 2024-04-20T21:35:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.646+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c981f6040 con 0x7f9c98163660 2024-04-20T21:35:16.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.647+0000 7f9c8f400640 1 -- 172.21.15.102:0/42622646 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c9000abd0 con 0x7f9c98163660 2024-04-20T21:35:16.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.648+0000 7f9c8f400640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c7406c8f0 0x7f9c7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:16.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.648+0000 7f9c8f400640 1 -- 172.21.15.102:0/42622646 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f9c9008b690 con 0x7f9c98163660 2024-04-20T21:35:16.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.648+0000 7f9c97400640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c7406c8f0 0x7f9c7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:16.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.649+0000 7f9c97400640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c7406c8f0 0x7f9c7406edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009b40 tx=0x7f9c88009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:16.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.652+0000 7f9c8f400640 1 -- 172.21.15.102:0/42622646 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9c981f6040 con 0x7f9c98163660 2024-04-20T21:35:16.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.895+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c98074340 con 0x7f9c98163660 2024-04-20T21:35:16.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.896+0000 7f9c8f400640 1 -- 172.21.15.102:0/42622646 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+6033 (secure 0 0 0) 0x7f9c90058d50 con 0x7f9c98163660 2024-04-20T21:35:16.895 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:13.437415+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:clientreplay","state_seq":25,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:16.896 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 59 2024-04-20T21:35:16.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.898+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c7406c8f0 msgr2=0x7f9c7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:16.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c7406c8f0 0x7f9c7406edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009b40 tx=0x7f9c88009190 comp rx=0 tx=0).stop 2024-04-20T21:35:16.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c98163660 msgr2=0x7f9c980c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:16.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c98163660 0x7f9c980c4410 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c7b0 tx=0x7f9c9000cc70 comp rx=0 tx=0).stop 2024-04-20T21:35:16.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 shutdown_connections 2024-04-20T21:35:16.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c98163660 0x7f9c980c4410 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c98163220 0x7f9c980c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9c7406c8f0 0x7f9c7406edb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 --2- 172.21.15.102:0/42622646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c98162e40 0x7f9c980c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:16.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 >> 172.21.15.102:0/42622646 conn(0x7f9c980c9390 msgr2=0x7f9c98156e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:16.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 shutdown_connections 2024-04-20T21:35:16.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:16.899+0000 7f9c9e200640 1 -- 172.21.15.102:0/42622646 wait complete. 2024-04-20T21:35:16.918 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 59, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 59, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5994": { "addr": "172.21.15.102:6833/3985712430", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 3985712430, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 3985712430, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5994, "incarnation": 56, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:clientreplay", "state_seq": 25 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:35:13.437415+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5994 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:35:18.919 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:19.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.126+0000 7fc7e8000640 1 Processor -- start 2024-04-20T21:35:19.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.126+0000 7fc7e8000640 1 -- start start 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.127+0000 7fc7e8000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.127+0000 7fc7e8000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc7e0073a10 0x7fc7e0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.127+0000 7fc7e8000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e01536d0 con 0x7fc7e014fa50 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.127+0000 7fc7e8000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e0159500 con 0x7fc7e0073a10 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.127+0000 7fc7e8000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00c76c0 con 0x7fc7e00c9eb0 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7e6000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc7e0073a10 0x7fc7e0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7e5600640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc7e014fa50 0x7fc7e00c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38538/0 (socket says 172.21.15.102:38538) 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7e5600640 1 -- 172.21.15.102:0/262357304 learned_addr learned my addr 172.21.15.102:0/262357304 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7e4c00640 1 --2- 172.21.15.102:0/262357304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:19.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7e6000640 1 -- 172.21.15.102:0/262357304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 msgr2=0x7fc7e00c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:19.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7e6000640 1 --2- 172.21.15.102:0/262357304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e00c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7e6000640 1 -- 172.21.15.102:0/262357304 >> v1:172.21.15.102:6789/0 conn(0x7fc7e014fa50 legacy=0x7fc7e00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:19.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7e6000640 1 -- 172.21.15.102:0/262357304 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e00c7960 con 0x7fc7e0073a10 2024-04-20T21:35:19.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.128+0000 7fc7dfe00640 1 -- 172.21.15.102:0/262357304 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 692918135 0 0) 0x7fc7d0001110 con 0x7fc7e014fa50 2024-04-20T21:35:19.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.129+0000 7fc7e4c00640 1 --2- 172.21.15.102:0/262357304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e00c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:19.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7e6000640 1 --2- 172.21.15.102:0/262357304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 0x7fc7e0158fc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009d90 tx=0x7fc7d802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcc5ba163f9e94b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7dfe00640 1 -- 172.21.15.102:0/262357304 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8039070 con 0x7fc7e0073a10 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7dfe00640 1 -- 172.21.15.102:0/262357304 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d802cce0 con 0x7fc7e0073a10 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7dfe00640 1 -- 172.21.15.102:0/262357304 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8034660 con 0x7fc7e0073a10 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7e8000640 1 -- 172.21.15.102:0/262357304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 msgr2=0x7fc7e0158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7e8000640 1 --2- 172.21.15.102:0/262357304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 0x7fc7e0158fc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009d90 tx=0x7fc7d802bf30 comp rx=0 tx=0).stop 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7e8000640 1 -- 172.21.15.102:0/262357304 shutdown_connections 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7e8000640 1 --2- 172.21.15.102:0/262357304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 0x7fc7e0158fc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7e8000640 1 --2- 172.21.15.102:0/262357304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e00c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.130+0000 7fc7e8000640 1 -- 172.21.15.102:0/262357304 >> 172.21.15.102:0/262357304 conn(0x7fc7e00c9390 msgr2=0x7fc7e00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.131+0000 7fc7e8000640 1 -- 172.21.15.102:0/262357304 shutdown_connections 2024-04-20T21:35:19.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.131+0000 7fc7e8000640 1 -- 172.21.15.102:0/262357304 wait complete. 2024-04-20T21:35:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.132+0000 7fc7e8000640 1 Processor -- start 2024-04-20T21:35:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.132+0000 7fc7e8000640 1 -- start start 2024-04-20T21:35:19.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.133+0000 7fc7e8000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 0x7fc7e01f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:19.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.133+0000 7fc7e8000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e0143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:19.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.133+0000 7fc7e8000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e014fa50 0x7fc7e0144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:19.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.133+0000 7fc7e8000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00c7c00 con 0x7fc7e0073a10 2024-04-20T21:35:19.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.133+0000 7fc7e8000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00c3d90 con 0x7fc7e014fa50 2024-04-20T21:35:19.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.133+0000 7fc7e8000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00c4030 con 0x7fc7e00c9eb0 2024-04-20T21:35:19.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.133+0000 7fc7e5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 0x7fc7e01f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e0143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e014fa50 0x7fc7e0144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e0143e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:41530/0 (socket says 172.21.15.102:41530) 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e4c00640 1 -- 172.21.15.102:0/3523302925 learned_addr learned my addr 172.21.15.102:0/3523302925 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e6000640 1 -- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 msgr2=0x7fc7e0143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e6000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e0143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e6000640 1 -- 172.21.15.102:0/3523302925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 msgr2=0x7fc7e01f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e6000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 0x7fc7e01f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e6000640 1 -- 172.21.15.102:0/3523302925 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e01f1000 con 0x7fc7e014fa50 2024-04-20T21:35:19.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.134+0000 7fc7e6000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e014fa50 0x7fc7e0144360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8033000 tx=0x7fc7d8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:19.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.135+0000 7fc7dd600640 1 -- 172.21.15.102:0/3523302925 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8039050 con 0x7fc7e014fa50 2024-04-20T21:35:19.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.135+0000 7fc7dd600640 1 -- 172.21.15.102:0/3523302925 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d8045070 con 0x7fc7e014fa50 2024-04-20T21:35:19.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.135+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e01f12a0 con 0x7fc7e014fa50 2024-04-20T21:35:19.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.135+0000 7fc7dd600640 1 -- 172.21.15.102:0/3523302925 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8009450 con 0x7fc7e014fa50 2024-04-20T21:35:19.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.135+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e00cb430 con 0x7fc7e014fa50 2024-04-20T21:35:19.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.136+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e0074340 con 0x7fc7e014fa50 2024-04-20T21:35:19.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.137+0000 7fc7dd600640 1 -- 172.21.15.102:0/3523302925 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7d803e850 con 0x7fc7e014fa50 2024-04-20T21:35:19.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.138+0000 7fc7dd600640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc7c406c8f0 0x7fc7c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:19.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.138+0000 7fc7dd600640 1 -- 172.21.15.102:0/3523302925 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7fc7d80aed30 con 0x7fc7e014fa50 2024-04-20T21:35:19.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.139+0000 7fc7e5600640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc7c406c8f0 0x7fc7c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:19.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.139+0000 7fc7e5600640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc7c406c8f0 0x7fc7c406edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0007fd0 tx=0x7fc7d0007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:19.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.143+0000 7fc7dd600640 1 -- 172.21.15.102:0/3523302925 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc7d807c510 con 0x7fc7e014fa50 2024-04-20T21:35:19.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.421+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e01568e0 con 0x7fc7e014fa50 2024-04-20T21:35:19.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.422+0000 7fc7dd600640 1 -- 172.21.15.102:0/3523302925 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+6033 (secure 0 0 0) 0x7fc7d8080810 con 0x7fc7e014fa50 2024-04-20T21:35:19.421 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:19.422 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:13.437415+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:clientreplay","state_seq":25,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:19.422 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 59 2024-04-20T21:35:19.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc7c406c8f0 msgr2=0x7fc7c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:19.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc7c406c8f0 0x7fc7c406edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0007fd0 tx=0x7fc7d0007de0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e014fa50 msgr2=0x7fc7e0144360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:19.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e014fa50 0x7fc7e0144360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8033000 tx=0x7fc7d8004a40 comp rx=0 tx=0).stop 2024-04-20T21:35:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 shutdown_connections 2024-04-20T21:35:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc7c406c8f0 0x7fc7c406edb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e014fa50 0x7fc7e0144360 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e00c9eb0 0x7fc7e0143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 --2- 172.21.15.102:0/3523302925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7e0073a10 0x7fc7e01f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 >> 172.21.15.102:0/3523302925 conn(0x7fc7e00c9390 msgr2=0x7fc7e014c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 shutdown_connections 2024-04-20T21:35:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:19.426+0000 7fc7e8000640 1 -- 172.21.15.102:0/3523302925 wait complete. 2024-04-20T21:35:19.466 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:21.467 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:21.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.675+0000 7f2271a00640 1 Processor -- start 2024-04-20T21:35:21.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.675+0000 7f2271a00640 1 -- start start 2024-04-20T21:35:21.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f2271a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c155530 0x7f226c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:21.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f2271a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f226c0cb6a0 0x7f226c0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:21.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f2271a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f226c15ef60 con 0x7f226c0cc470 2024-04-20T21:35:21.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f2271a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f226c0c5ca0 con 0x7f226c0cb6a0 2024-04-20T21:35:21.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f2271a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f226c0cd5f0 con 0x7f226c155530 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f2271000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f226c0cb6a0 0x7f226c0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f226b400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f226c0cc470 0x7f226c0cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38544/0 (socket says 172.21.15.102:38544) 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f2271000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f226c0cb6a0 0x7f226c0cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41012/0 (socket says 172.21.15.102:41012) 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f226b400640 1 -- 172.21.15.102:0/531676586 learned_addr learned my addr 172.21.15.102:0/531676586 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.677+0000 7f226be00640 1 --2- 172.21.15.102:0/531676586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c155530 0x7f226c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.678+0000 7f2271000640 1 -- 172.21.15.102:0/531676586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c155530 msgr2=0x7f226c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.678+0000 7f2271000640 1 --2- 172.21.15.102:0/531676586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c155530 0x7f226c15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.678+0000 7f2271000640 1 -- 172.21.15.102:0/531676586 >> v1:172.21.15.102:6789/0 conn(0x7f226c0cc470 legacy=0x7f226c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.678+0000 7f226aa00640 1 -- 172.21.15.102:0/531676586 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856876990 0 0) 0x7f226c15ef60 con 0x7f226c0cc470 2024-04-20T21:35:21.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.678+0000 7f2271000640 1 -- 172.21.15.102:0/531676586 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c15f240 con 0x7f226c0cb6a0 2024-04-20T21:35:21.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.679+0000 7f2271000640 1 --2- 172.21.15.102:0/531676586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c0cb6a0 0x7f226c0cbab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2264009d90 tx=0x7f226402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8322dc403ecabc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:21.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.679+0000 7f226aa00640 1 -- 172.21.15.102:0/531676586 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264039070 con 0x7f226c0cb6a0 2024-04-20T21:35:21.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.679+0000 7f226aa00640 1 -- 172.21.15.102:0/531676586 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226402cce0 con 0x7f226c0cb6a0 2024-04-20T21:35:21.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.679+0000 7f226aa00640 1 -- 172.21.15.102:0/531676586 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264034660 con 0x7f226c0cb6a0 2024-04-20T21:35:21.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.680+0000 7f2271a00640 1 -- 172.21.15.102:0/531676586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c0cb6a0 msgr2=0x7f226c0cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:21.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.680+0000 7f2271a00640 1 --2- 172.21.15.102:0/531676586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c0cb6a0 0x7f226c0cbab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2264009d90 tx=0x7f226402bf30 comp rx=0 tx=0).stop 2024-04-20T21:35:21.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.680+0000 7f2271a00640 1 -- 172.21.15.102:0/531676586 shutdown_connections 2024-04-20T21:35:21.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.680+0000 7f2271a00640 1 --2- 172.21.15.102:0/531676586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c0cb6a0 0x7f226c0cbab0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.680+0000 7f2271a00640 1 --2- 172.21.15.102:0/531676586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c155530 0x7f226c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.680+0000 7f2271a00640 1 -- 172.21.15.102:0/531676586 >> 172.21.15.102:0/531676586 conn(0x7f226c075080 msgr2=0x7f226c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:21.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.680+0000 7f2271a00640 1 -- 172.21.15.102:0/531676586 shutdown_connections 2024-04-20T21:35:21.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.680+0000 7f2271a00640 1 -- 172.21.15.102:0/531676586 wait complete. 2024-04-20T21:35:21.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.681+0000 7f2271a00640 1 Processor -- start 2024-04-20T21:35:21.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.682+0000 7f2271a00640 1 -- start start 2024-04-20T21:35:21.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.682+0000 7f2271a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226c0cb6a0 0x7f226c0c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:21.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.682+0000 7f2271a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c0cc470 0x7f226c0b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:21.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.682+0000 7f2271a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c155530 0x7f226c15c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.682+0000 7f2271a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0cd280 con 0x7f226c155530 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0ccfa0 con 0x7f226c0cb6a0 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f226c0ccbe0 con 0x7f226c0cc470 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c155530 0x7f226c15c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c155530 0x7f226c15c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41028/0 (socket says 172.21.15.102:41028) 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271000640 1 -- 172.21.15.102:0/3295213499 learned_addr learned my addr 172.21.15.102:0/3295213499 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f226b400640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c0cc470 0x7f226c0b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f226be00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226c0cb6a0 0x7f226c0c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271000640 1 -- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c0cc470 msgr2=0x7f226c0b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271000640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c0cc470 0x7f226c0b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271000640 1 -- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226c0cb6a0 msgr2=0x7f226c0c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271000640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226c0cb6a0 0x7f226c0c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.683+0000 7f2271000640 1 -- 172.21.15.102:0/3295213499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c1f1580 con 0x7f226c155530 2024-04-20T21:35:21.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.684+0000 7f2271000640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c155530 0x7f226c15c510 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2264009eb0 tx=0x7f2264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:21.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.684+0000 7f2263e00640 1 -- 172.21.15.102:0/3295213499 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264039050 con 0x7f226c155530 2024-04-20T21:35:21.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.684+0000 7f2263e00640 1 -- 172.21.15.102:0/3295213499 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2264045070 con 0x7f226c155530 2024-04-20T21:35:21.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.684+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226c1f1860 con 0x7f226c155530 2024-04-20T21:35:21.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.684+0000 7f2263e00640 1 -- 172.21.15.102:0/3295213499 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226403e830 con 0x7f226c155530 2024-04-20T21:35:21.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.684+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226c1f1e20 con 0x7f226c155530 2024-04-20T21:35:21.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.685+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226c04fd60 con 0x7f226c155530 2024-04-20T21:35:21.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.686+0000 7f2263e00640 1 -- 172.21.15.102:0/3295213499 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f226403e9d0 con 0x7f226c155530 2024-04-20T21:35:21.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.687+0000 7f2263e00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f224c06c8f0 0x7f224c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:21.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.687+0000 7f2263e00640 1 -- 172.21.15.102:0/3295213499 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f22640aef90 con 0x7f226c155530 2024-04-20T21:35:21.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.688+0000 7f226be00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f224c06c8f0 0x7f224c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:21.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.688+0000 7f226be00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f224c06c8f0 0x7f224c06edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f225c0079a0 tx=0x7f225c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:21.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.692+0000 7f2263e00640 1 -- 172.21.15.102:0/3295213499 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f226407c650 con 0x7f226c155530 2024-04-20T21:35:21.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.934+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f226c068d30 con 0x7f226c155530 2024-04-20T21:35:21.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.935+0000 7f2263e00640 1 -- 172.21.15.102:0/3295213499 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+6034 (secure 0 0 0) 0x7f2264080950 con 0x7f226c155530 2024-04-20T21:35:21.935 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:21.935 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:20.330927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5994,"qdb_cluster":[5994]},"id":1}]} 2024-04-20T21:35:21.935 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 60 2024-04-20T21:35:21.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f224c06c8f0 msgr2=0x7f224c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:21.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f224c06c8f0 0x7f224c06edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f225c0079a0 tx=0x7f225c008040 comp rx=0 tx=0).stop 2024-04-20T21:35:21.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c155530 msgr2=0x7f226c15c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:21.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c155530 0x7f226c15c510 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2264009eb0 tx=0x7f2264004a40 comp rx=0 tx=0).stop 2024-04-20T21:35:21.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 shutdown_connections 2024-04-20T21:35:21.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226c155530 0x7f226c15c510 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226c0cc470 0x7f226c0b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f224c06c8f0 0x7f224c06edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 --2- 172.21.15.102:0/3295213499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226c0cb6a0 0x7f226c0c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:21.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 >> 172.21.15.102:0/3295213499 conn(0x7f226c075080 msgr2=0x7f226c154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:21.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 shutdown_connections 2024-04-20T21:35:21.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:21.938+0000 7f2271a00640 1 -- 172.21.15.102:0/3295213499 wait complete. 2024-04-20T21:35:21.957 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5835 2024-04-20T21:35:21.957 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.c 2024-04-20T21:35:28.957 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:35:28.958 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:35:28.958 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:35:28.966 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:35:28.966 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:29.012 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:35:29.018 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:35:29.116 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:35:29.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.150+0000 7f5598200640 1 Processor -- start 2024-04-20T21:35:29.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.150+0000 7f5598200640 1 -- start start 2024-04-20T21:35:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.151+0000 7f5598200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900cd940 0x7f559015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.151+0000 7f5598200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559015f940 con 0x7f55900ce310 2024-04-20T21:35:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.151+0000 7f5598200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559015eb60 con 0x7f55900c7900 2024-04-20T21:35:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.151+0000 7f5598200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f559015ee00 con 0x7f55900cd940 2024-04-20T21:35:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.151+0000 7f5596200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f55900ce310 0x7f5590072230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57192/0 (socket says 172.21.15.102:57192) 2024-04-20T21:35:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.151+0000 7f5596200640 1 -- 172.21.15.102:0/3896687898 learned_addr learned my addr 172.21.15.102:0/3896687898 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.151+0000 7f5594e00640 1 --2- 172.21.15.102:0/3896687898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900cd940 0x7f559015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.152+0000 7f558fe00640 1 -- 172.21.15.102:0/3896687898 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1433112549 0 0) 0x7f559015f940 con 0x7f55900ce310 2024-04-20T21:35:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.152+0000 7f558fe00640 1 -- 172.21.15.102:0/3896687898 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5578003520 con 0x7f55900ce310 2024-04-20T21:35:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.152+0000 7f558fe00640 1 -- 172.21.15.102:0/3896687898 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658624601 0 0) 0x7f559015eb60 con 0x7f55900c7900 2024-04-20T21:35:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.152+0000 7f5594e00640 1 -- 172.21.15.102:0/3896687898 >> v1:172.21.15.193:6789/0 conn(0x7f55900c7900 legacy=0x7f5590074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.152+0000 7f5594e00640 1 -- 172.21.15.102:0/3896687898 >> v1:172.21.15.102:6789/0 conn(0x7f55900ce310 legacy=0x7f5590072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.152+0000 7f5594e00640 1 -- 172.21.15.102:0/3896687898 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559015f0a0 con 0x7f55900cd940 2024-04-20T21:35:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.153+0000 7f5594e00640 1 --2- 172.21.15.102:0/3896687898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900cd940 0x7f559015bee0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097d0 tx=0x7f557c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2021d45d19739e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.153+0000 7f558fe00640 1 -- 172.21.15.102:0/3896687898 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c035020 con 0x7f55900cd940 2024-04-20T21:35:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.153+0000 7f558fe00640 1 -- 172.21.15.102:0/3896687898 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c031890 con 0x7f55900cd940 2024-04-20T21:35:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.153+0000 7f558fe00640 1 -- 172.21.15.102:0/3896687898 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c031c60 con 0x7f55900cd940 2024-04-20T21:35:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.154+0000 7f5598200640 1 -- 172.21.15.102:0/3896687898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900cd940 msgr2=0x7f559015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.154+0000 7f5598200640 1 --2- 172.21.15.102:0/3896687898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900cd940 0x7f559015bee0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097d0 tx=0x7f557c02b930 comp rx=0 tx=0).stop 2024-04-20T21:35:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.154+0000 7f5598200640 1 -- 172.21.15.102:0/3896687898 shutdown_connections 2024-04-20T21:35:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.154+0000 7f5598200640 1 --2- 172.21.15.102:0/3896687898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900cd940 0x7f559015bee0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.154+0000 7f5598200640 1 -- 172.21.15.102:0/3896687898 >> 172.21.15.102:0/3896687898 conn(0x7f559006dad0 msgr2=0x7f559006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.154+0000 7f5598200640 1 -- 172.21.15.102:0/3896687898 shutdown_connections 2024-04-20T21:35:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.154+0000 7f5598200640 1 -- 172.21.15.102:0/3896687898 wait complete. 2024-04-20T21:35:29.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.156+0000 7f5598200640 1 Processor -- start 2024-04-20T21:35:29.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.156+0000 7f5598200640 1 -- start start 2024-04-20T21:35:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5598200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900c7900 0x7f55901f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5598200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55900cd940 0x7f55901e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5598200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55900ce310 0x7f5590157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5598200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55900755b0 con 0x7f55900ce310 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5598200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f559015f660 con 0x7f55900cd940 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5598200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5590075730 con 0x7f55900c7900 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5596200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55900ce310 0x7f5590157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5594e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55900cd940 0x7f55901e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5594e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55900cd940 0x7f55901e7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:50678/0 (socket says 172.21.15.102:50678) 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5594e00640 1 -- 172.21.15.102:0/1276706746 learned_addr learned my addr 172.21.15.102:0/1276706746 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5595800640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900c7900 0x7f55901f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5594e00640 1 -- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900c7900 msgr2=0x7f55901f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5594e00640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900c7900 0x7f55901f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5594e00640 1 -- 172.21.15.102:0/1276706746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55900ce310 msgr2=0x7f5590157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5594e00640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55900ce310 0x7f5590157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.157+0000 7f5594e00640 1 -- 172.21.15.102:0/1276706746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55901e83a0 con 0x7f55900cd940 2024-04-20T21:35:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.158+0000 7f5594e00640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55900cd940 0x7f55901e7b10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097d0 tx=0x7f557c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.158+0000 7f558d600640 1 -- 172.21.15.102:0/1276706746 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c035040 con 0x7f55900cd940 2024-04-20T21:35:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.158+0000 7f558d600640 1 -- 172.21.15.102:0/1276706746 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c030d20 con 0x7f55900cd940 2024-04-20T21:35:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.158+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559015e010 con 0x7f55900cd940 2024-04-20T21:35:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.158+0000 7f558d600640 1 -- 172.21.15.102:0/1276706746 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c03a650 con 0x7f55900cd940 2024-04-20T21:35:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.158+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559015e5d0 con 0x7f55900cd940 2024-04-20T21:35:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.159+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5590069550 con 0x7f55900cd940 2024-04-20T21:35:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.160+0000 7f558d600640 1 -- 172.21.15.102:0/1276706746 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f557c038020 con 0x7f55900cd940 2024-04-20T21:35:29.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.161+0000 7f558d600640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f557006c8f0 0x7f557006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:29.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.161+0000 7f558d600640 1 -- 172.21.15.102:0/1276706746 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f557c0ae560 con 0x7f55900cd940 2024-04-20T21:35:29.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.162+0000 7f5595800640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f557006c8f0 0x7f557006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.162+0000 7f5595800640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f557006c8f0 0x7f557006edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5584000f60 tx=0x7f5584009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.166+0000 7f558d600640 1 -- 172.21.15.102:0/1276706746 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5590069550 con 0x7f55900cd940 2024-04-20T21:35:29.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.411+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5590074340 con 0x7f55900cd940 2024-04-20T21:35:29.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.413+0000 7f558d600640 1 -- 172.21.15.102:0/1276706746 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+6859 (secure 0 0 0) 0x7f557c02d040 con 0x7f55900cd940 2024-04-20T21:35:29.412 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:29.412 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:20.330927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5994,"qdb_cluster":[5994]},"id":1}]} 2024-04-20T21:35:29.412 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 61 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f557006c8f0 msgr2=0x7f557006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f557006c8f0 0x7f557006edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5584000f60 tx=0x7f5584009040 comp rx=0 tx=0).stop 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55900cd940 msgr2=0x7f55901e7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55900cd940 0x7f55901e7b10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097d0 tx=0x7f557c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 shutdown_connections 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55900ce310 0x7f5590157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f557006c8f0 0x7f557006edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55900cd940 0x7f55901e7b10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 --2- 172.21.15.102:0/1276706746 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55900c7900 0x7f55901f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:29.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.415+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 >> 172.21.15.102:0/1276706746 conn(0x7f559006dad0 msgr2=0x7f5590074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:29.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.416+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 shutdown_connections 2024-04-20T21:35:29.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:29.416+0000 7f5598200640 1 -- 172.21.15.102:0/1276706746 wait complete. 2024-04-20T21:35:29.436 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:35:29.436 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 9 secs before thrashing 2024-04-20T21:35:29.809 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-04-20T21:35:29.810+0000 7f73c8c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:35:38.436 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:38.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.706+0000 7f534be00640 1 Processor -- start 2024-04-20T21:35:38.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.706+0000 7f534be00640 1 -- start start 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f534c0c75a0 0x7f534c1637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534be00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f534c153560 con 0x7f534c159380 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534be00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f534c0c7a70 con 0x7f534c0c75a0 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f534c0c7d30 con 0x7f534c153180 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:51234/0 (socket says 172.21.15.102:51234) 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f534c0c75a0 0x7f534c1637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.708+0000 7f534aa00640 1 -- 172.21.15.102:0/3152717304 learned_addr learned my addr 172.21.15.102:0/3152717304 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.709+0000 7f534aa00640 1 -- 172.21.15.102:0/3152717304 >> v2:172.21.15.102:3300/0 conn(0x7f534c0c75a0 msgr2=0x7f534c1637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.709+0000 7f534aa00640 1 --2- 172.21.15.102:0/3152717304 >> v2:172.21.15.102:3300/0 conn(0x7f534c0c75a0 0x7f534c1637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.709+0000 7f534aa00640 1 -- 172.21.15.102:0/3152717304 >> v1:172.21.15.102:6789/0 conn(0x7f534c159380 legacy=0x7f534c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:38.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.709+0000 7f534aa00640 1 -- 172.21.15.102:0/3152717304 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c15c4d0 con 0x7f534c153180 2024-04-20T21:35:38.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.710+0000 7f534aa00640 1 --2- 172.21.15.102:0/3152717304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c158e40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f533c009870 tx=0x7f533c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=382979b1f544005c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.710+0000 7f534a000640 1 -- 172.21.15.102:0/3152717304 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c035020 con 0x7f534c153180 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.710+0000 7f534a000640 1 -- 172.21.15.102:0/3152717304 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533c031640 con 0x7f534c153180 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.710+0000 7f534a000640 1 -- 172.21.15.102:0/3152717304 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c02d040 con 0x7f534c153180 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.710+0000 7f534be00640 1 -- 172.21.15.102:0/3152717304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 msgr2=0x7f534c158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.710+0000 7f534be00640 1 --2- 172.21.15.102:0/3152717304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c158e40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f533c009870 tx=0x7f533c02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.711+0000 7f534be00640 1 -- 172.21.15.102:0/3152717304 shutdown_connections 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.711+0000 7f534be00640 1 --2- 172.21.15.102:0/3152717304 >> v2:172.21.15.102:3300/0 conn(0x7f534c0c75a0 0x7f534c1637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.711+0000 7f534be00640 1 --2- 172.21.15.102:0/3152717304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c158e40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.711+0000 7f534be00640 1 -- 172.21.15.102:0/3152717304 >> 172.21.15.102:0/3152717304 conn(0x7f534c0cc820 msgr2=0x7f534c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.711+0000 7f534be00640 1 -- 172.21.15.102:0/3152717304 shutdown_connections 2024-04-20T21:35:38.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.711+0000 7f534be00640 1 -- 172.21.15.102:0/3152717304 wait complete. 2024-04-20T21:35:38.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.712+0000 7f534be00640 1 Processor -- start 2024-04-20T21:35:38.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.712+0000 7f534be00640 1 -- start start 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.713+0000 7f534be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c0c75a0 0x7f534c1e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.713+0000 7f534be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.713+0000 7f534be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c159380 0x7f534c1e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.713+0000 7f534be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f534c15d290 con 0x7f534c0c75a0 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.713+0000 7f534be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f534c15ca90 con 0x7f534c159380 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.713+0000 7f534be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f534c0c6be0 con 0x7f534c153180 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c0c75a0 0x7f534c1e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f5343e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c0c75a0 0x7f534c1e7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55726/0 (socket says 172.21.15.102:55726) 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 -- 172.21.15.102:0/4176204342 learned_addr learned my addr 172.21.15.102:0/4176204342 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:38.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534b400640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c159380 0x7f534c1e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 -- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 msgr2=0x7f534c1f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c1f1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 -- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c159380 msgr2=0x7f534c1e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c159380 0x7f534c1e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 -- 172.21.15.102:0/4176204342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c156f00 con 0x7f534c0c75a0 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.714+0000 7f534aa00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c0c75a0 0x7f534c1e7080 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f533c000c00 tx=0x7f533c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.715+0000 7f5342a00640 1 -- 172.21.15.102:0/4176204342 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c040070 con 0x7f534c0c75a0 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.715+0000 7f5342a00640 1 -- 172.21.15.102:0/4176204342 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533c0399f0 con 0x7f534c0c75a0 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.715+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c1571e0 con 0x7f534c0c75a0 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.715+0000 7f5342a00640 1 -- 172.21.15.102:0/4176204342 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c035040 con 0x7f534c0c75a0 2024-04-20T21:35:38.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.715+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1577a0 con 0x7f534c0c75a0 2024-04-20T21:35:38.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.716+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534c04fd60 con 0x7f534c0c75a0 2024-04-20T21:35:38.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.717+0000 7f5342a00640 1 -- 172.21.15.102:0/4176204342 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f533c038020 con 0x7f534c0c75a0 2024-04-20T21:35:38.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.718+0000 7f5342a00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f532806c8f0 0x7f532806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:38.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.718+0000 7f5342a00640 1 -- 172.21.15.102:0/4176204342 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f533c0adbc0 con 0x7f534c0c75a0 2024-04-20T21:35:38.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.718+0000 7f5343e00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f532806c8f0 0x7f532806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:38.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.719+0000 7f5343e00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f532806c8f0 0x7f532806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f534c1e40c0 tx=0x7f5334007e30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:38.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.723+0000 7f5342a00640 1 -- 172.21.15.102:0/4176204342 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f533c07b160 con 0x7f534c0c75a0 2024-04-20T21:35:38.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.967+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f534c0c3690 con 0x7f534c0c75a0 2024-04-20T21:35:38.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.968+0000 7f5342a00640 1 -- 172.21.15.102:0/4176204342 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+6859 (secure 0 0 0) 0x7f533c07f460 con 0x7f534c0c75a0 2024-04-20T21:35:38.967 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:38.968 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:20.330927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5994,"qdb_cluster":[5994]},"id":1}]} 2024-04-20T21:35:38.968 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 61 2024-04-20T21:35:38.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f532806c8f0 msgr2=0x7f532806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:38.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f532806c8f0 0x7f532806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f534c1e40c0 tx=0x7f5334007e30 comp rx=0 tx=0).stop 2024-04-20T21:35:38.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c0c75a0 msgr2=0x7f534c1e7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:38.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c0c75a0 0x7f534c1e7080 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f533c000c00 tx=0x7f533c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 shutdown_connections 2024-04-20T21:35:38.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f532806c8f0 0x7f532806edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c159380 0x7f534c1e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f534c153180 0x7f534c1f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 --2- 172.21.15.102:0/4176204342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f534c0c75a0 0x7f534c1e7080 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:38.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 >> 172.21.15.102:0/4176204342 conn(0x7f534c0cc820 msgr2=0x7f534c0cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:38.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 shutdown_connections 2024-04-20T21:35:38.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:38.971+0000 7f534be00640 1 -- 172.21.15.102:0/4176204342 wait complete. 2024-04-20T21:35:38.988 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-04-20T21:35:38.989 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:35:38.989 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:35:39.085 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:35:39.086 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:39.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.290+0000 7f1aab200640 1 Processor -- start 2024-04-20T21:35:39.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.290+0000 7f1aab200640 1 -- start start 2024-04-20T21:35:39.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aab200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1aac152290 0x7f1aac0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aab200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aab200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aac15c220 con 0x7f1aac06eb70 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aab200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1aac0c9180 con 0x7f1aac152290 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aab200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1aac0c4840 con 0x7f1aac0c3700 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aa9e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1aac152290 0x7f1aac0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aa9400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aa9e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1aac152290 0x7f1aac0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55738/0 (socket says 172.21.15.102:55738) 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aa9e00640 1 -- 172.21.15.102:0/4093926018 learned_addr learned my addr 172.21.15.102:0/4093926018 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aaa800640 1 --1- 172.21.15.102:0/4093926018 >> v1:172.21.15.193:6789/0 conn(0x7f1aac06eb70 0x7f1aac0c8330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:33244/0 (socket says 172.21.15.102:33244) 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aa9e00640 1 -- 172.21.15.102:0/4093926018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 msgr2=0x7f1aac06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aa9e00640 1 --2- 172.21.15.102:0/4093926018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac06e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.292+0000 7f1aa9e00640 1 -- 172.21.15.102:0/4093926018 >> v1:172.21.15.193:6789/0 conn(0x7f1aac06eb70 legacy=0x7f1aac0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:39.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.293+0000 7f1aa9e00640 1 -- 172.21.15.102:0/4093926018 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac0c4560 con 0x7f1aac152290 2024-04-20T21:35:39.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.293+0000 7f1aa9400640 1 --2- 172.21.15.102:0/4093926018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:39.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.293+0000 7f1aa8a00640 1 -- 172.21.15.102:0/4093926018 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 814485214 0 0) 0x7f1aac15c220 con 0x7f1aac06eb70 2024-04-20T21:35:39.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.294+0000 7f1aa9e00640 1 --2- 172.21.15.102:0/4093926018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac0cd310 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009870 tx=0x7f1a9c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394e1fb00accf6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:39.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.294+0000 7f1aa8a00640 1 -- 172.21.15.102:0/4093926018 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c02f830 con 0x7f1aac152290 2024-04-20T21:35:39.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.294+0000 7f1aa8a00640 1 -- 172.21.15.102:0/4093926018 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9c02f9d0 con 0x7f1aac152290 2024-04-20T21:35:39.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.294+0000 7f1aa8a00640 1 -- 172.21.15.102:0/4093926018 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c02fda0 con 0x7f1aac152290 2024-04-20T21:35:39.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.294+0000 7f1aab200640 1 -- 172.21.15.102:0/4093926018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 msgr2=0x7f1aac0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:39.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.294+0000 7f1aab200640 1 --2- 172.21.15.102:0/4093926018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac0cd310 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009870 tx=0x7f1a9c02ba10 comp rx=0 tx=0).stop 2024-04-20T21:35:39.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.295+0000 7f1aab200640 1 -- 172.21.15.102:0/4093926018 shutdown_connections 2024-04-20T21:35:39.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.295+0000 7f1aab200640 1 --2- 172.21.15.102:0/4093926018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.295+0000 7f1aab200640 1 --2- 172.21.15.102:0/4093926018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac0cd310 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.295+0000 7f1aab200640 1 -- 172.21.15.102:0/4093926018 >> 172.21.15.102:0/4093926018 conn(0x7f1aac0cc820 msgr2=0x7f1aac0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:39.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.295+0000 7f1aab200640 1 -- 172.21.15.102:0/4093926018 shutdown_connections 2024-04-20T21:35:39.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.295+0000 7f1aab200640 1 -- 172.21.15.102:0/4093926018 wait complete. 2024-04-20T21:35:39.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.296+0000 7f1aab200640 1 Processor -- start 2024-04-20T21:35:39.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.297+0000 7f1aab200640 1 -- start start 2024-04-20T21:35:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.297+0000 7f1aab200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1aac06eb70 0x7f1aac1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aab200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac1ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aab200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac15ba20 con 0x7f1aac152290 2024-04-20T21:35:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aab200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac15b740 con 0x7f1aac06eb70 2024-04-20T21:35:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aab200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1aac15bd00 con 0x7f1aac0c3700 2024-04-20T21:35:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aaa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac1ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aa9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1aac06eb70 0x7f1aac1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aaa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac1ef2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55742/0 (socket says 172.21.15.102:55742) 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aaa800640 1 -- 172.21.15.102:0/3305921302 learned_addr learned my addr 172.21.15.102:0/3305921302 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aa9400640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aaa800640 1 -- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 msgr2=0x7f1aac0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aaa800640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aaa800640 1 -- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1aac06eb70 msgr2=0x7f1aac1445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aaa800640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1aac06eb70 0x7f1aac1445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.298+0000 7f1aaa800640 1 -- 172.21.15.102:0/3305921302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac1f0e70 con 0x7f1aac152290 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.299+0000 7f1aaa800640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac1ef2d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4010ad0 tx=0x7f1aa4010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.299+0000 7f1aa2000640 1 -- 172.21.15.102:0/3305921302 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401c070 con 0x7f1aac152290 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.299+0000 7f1aa2000640 1 -- 172.21.15.102:0/3305921302 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa400ec30 con 0x7f1aac152290 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.299+0000 7f1aa2000640 1 -- 172.21.15.102:0/3305921302 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400ee60 con 0x7f1aac152290 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.299+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aac151a90 con 0x7f1aac152290 2024-04-20T21:35:39.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.299+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aac1f1410 con 0x7f1aac152290 2024-04-20T21:35:39.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.300+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aac04fd60 con 0x7f1aac152290 2024-04-20T21:35:39.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.301+0000 7f1aa2000640 1 -- 172.21.15.102:0/3305921302 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1aa40155a0 con 0x7f1aac152290 2024-04-20T21:35:39.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.302+0000 7f1aa2000640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:39.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.302+0000 7f1aa2000640 1 -- 172.21.15.102:0/3305921302 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f1aa408fcb0 con 0x7f1aac152290 2024-04-20T21:35:39.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.303+0000 7f1aa9e00640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:39.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.303+0000 7f1aa9e00640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c036000 tx=0x7f1a9c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:39.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.307+0000 7f1aa2000640 1 -- 172.21.15.102:0/3305921302 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1aa405d2e0 con 0x7f1aac152290 2024-04-20T21:35:39.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.547+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1aac06dcb0 con 0x7f1aac152290 2024-04-20T21:35:39.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.548+0000 7f1aa2000640 1 -- 172.21.15.102:0/3305921302 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+6859 (secure 0 0 0) 0x7f1aa40615e0 con 0x7f1aac152290 2024-04-20T21:35:39.547 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:39.548 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:20.330927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5994,"qdb_cluster":[5994]},"id":1}]} 2024-04-20T21:35:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 61 2024-04-20T21:35:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a8c06c8f0 msgr2=0x7f1a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c036000 tx=0x7f1a9c008030 comp rx=0 tx=0).stop 2024-04-20T21:35:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 msgr2=0x7f1aac1ef2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac1ef2d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4010ad0 tx=0x7f1aa4010f90 comp rx=0 tx=0).stop 2024-04-20T21:35:39.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 shutdown_connections 2024-04-20T21:35:39.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1aac152290 0x7f1aac1ef2d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1aac0c3700 0x7f1aac0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 --2- 172.21.15.102:0/3305921302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1aac06eb70 0x7f1aac1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:39.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.551+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 >> 172.21.15.102:0/3305921302 conn(0x7f1aac0cc820 msgr2=0x7f1aac13cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:39.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.552+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 shutdown_connections 2024-04-20T21:35:39.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:39.552+0000 7f1aab200640 1 -- 172.21.15.102:0/3305921302 wait complete. 2024-04-20T21:35:39.571 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:35:41.572 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:41.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.782+0000 7fc856a00640 1 Processor -- start 2024-04-20T21:35:41.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.782+0000 7fc856a00640 1 -- start start 2024-04-20T21:35:41.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.784+0000 7fc856a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc85014fa50 0x7fc8500c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.784+0000 7fc856a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc850158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.784+0000 7fc856a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8501536d0 con 0x7fc8500c9eb0 2024-04-20T21:35:41.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.784+0000 7fc856a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc850159500 con 0x7fc85014fa50 2024-04-20T21:35:41.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.784+0000 7fc856a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc8500c76c0 con 0x7fc850073a10 2024-04-20T21:35:41.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc854a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc850158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc84fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc85014fa50 0x7fc8500c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc854a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc850158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:51266/0 (socket says 172.21.15.102:51266) 2024-04-20T21:35:41.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc847400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fc8500c9eb0 0x7fc8500c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:33248/0 (socket says 172.21.15.102:33248) 2024-04-20T21:35:41.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc854a00640 1 -- 172.21.15.102:0/1823065798 learned_addr learned my addr 172.21.15.102:0/1823065798 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:41.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc854a00640 1 -- 172.21.15.102:0/1823065798 >> v2:172.21.15.193:3300/0 conn(0x7fc85014fa50 msgr2=0x7fc8500c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc854a00640 1 --2- 172.21.15.102:0/1823065798 >> v2:172.21.15.193:3300/0 conn(0x7fc85014fa50 0x7fc8500c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc854a00640 1 -- 172.21.15.102:0/1823065798 >> v1:172.21.15.193:6789/0 conn(0x7fc8500c9eb0 legacy=0x7fc8500c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc854a00640 1 -- 172.21.15.102:0/1823065798 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8500c7960 con 0x7fc850073a10 2024-04-20T21:35:41.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.785+0000 7fc84fe00640 1 --2- 172.21.15.102:0/1823065798 >> v2:172.21.15.193:3300/0 conn(0x7fc85014fa50 0x7fc8500c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:41.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.786+0000 7fc854a00640 1 --2- 172.21.15.102:0/1823065798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc850158fc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc848009870 tx=0x7fc84802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5656c43d5c0419 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.786+0000 7fc84f400640 1 -- 172.21.15.102:0/1823065798 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84802f830 con 0x7fc850073a10 2024-04-20T21:35:41.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.786+0000 7fc84f400640 1 -- 172.21.15.102:0/1823065798 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84802f9d0 con 0x7fc850073a10 2024-04-20T21:35:41.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.786+0000 7fc84f400640 1 -- 172.21.15.102:0/1823065798 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84802fda0 con 0x7fc850073a10 2024-04-20T21:35:41.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.787+0000 7fc856a00640 1 -- 172.21.15.102:0/1823065798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 msgr2=0x7fc850158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.787+0000 7fc856a00640 1 --2- 172.21.15.102:0/1823065798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc850158fc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc848009870 tx=0x7fc84802ba10 comp rx=0 tx=0).stop 2024-04-20T21:35:41.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.787+0000 7fc856a00640 1 -- 172.21.15.102:0/1823065798 shutdown_connections 2024-04-20T21:35:41.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.787+0000 7fc856a00640 1 --2- 172.21.15.102:0/1823065798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc850158fc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.787+0000 7fc856a00640 1 --2- 172.21.15.102:0/1823065798 >> v2:172.21.15.193:3300/0 conn(0x7fc85014fa50 0x7fc8500c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.787+0000 7fc856a00640 1 -- 172.21.15.102:0/1823065798 >> 172.21.15.102:0/1823065798 conn(0x7fc8500c9390 msgr2=0x7fc8500c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.787+0000 7fc856a00640 1 -- 172.21.15.102:0/1823065798 shutdown_connections 2024-04-20T21:35:41.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.787+0000 7fc856a00640 1 -- 172.21.15.102:0/1823065798 wait complete. 2024-04-20T21:35:41.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.789+0000 7fc856a00640 1 Processor -- start 2024-04-20T21:35:41.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.789+0000 7fc856a00640 1 -- start start 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc856a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc8501f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc856a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8500c9eb0 0x7fc8500c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc856a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85014fa50 0x7fc8500c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc856a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc8500c7c00 con 0x7fc8500c9eb0 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc856a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8500c3d90 con 0x7fc85014fa50 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc856a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc8500c4030 con 0x7fc850073a10 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc847400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8500c9eb0 0x7fc8500c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc847400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8500c9eb0 0x7fc8500c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55750/0 (socket says 172.21.15.102:55750) 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc847400640 1 -- 172.21.15.102:0/2891491242 learned_addr learned my addr 172.21.15.102:0/2891491242 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:41.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc84fe00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc8501f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.790+0000 7fc854a00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85014fa50 0x7fc8500c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc847400640 1 -- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 msgr2=0x7fc8501f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc847400640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc8501f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc847400640 1 -- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85014fa50 msgr2=0x7fc8500c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc847400640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85014fa50 0x7fc8500c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc847400640 1 -- 172.21.15.102:0/2891491242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85015d150 con 0x7fc8500c9eb0 2024-04-20T21:35:41.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc847400640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8500c9eb0 0x7fc8500c7e20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c790 tx=0x7fc83800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc84cc00640 1 -- 172.21.15.102:0/2891491242 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838017070 con 0x7fc8500c9eb0 2024-04-20T21:35:41.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc84cc00640 1 -- 172.21.15.102:0/2891491242 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc83800a640 con 0x7fc8500c9eb0 2024-04-20T21:35:41.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc84cc00640 1 -- 172.21.15.102:0/2891491242 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83800a9f0 con 0x7fc8500c9eb0 2024-04-20T21:35:41.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.791+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8501f1510 con 0x7fc8500c9eb0 2024-04-20T21:35:41.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.792+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8501f1ad0 con 0x7fc8500c9eb0 2024-04-20T21:35:41.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.793+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8500635c0 con 0x7fc8500c9eb0 2024-04-20T21:35:41.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.794+0000 7fc84cc00640 1 -- 172.21.15.102:0/2891491242 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc83801a070 con 0x7fc8500c9eb0 2024-04-20T21:35:41.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.795+0000 7fc84cc00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc82c06c8f0 0x7fc82c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.795+0000 7fc84cc00640 1 -- 172.21.15.102:0/2891491242 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7fc83808b850 con 0x7fc8500c9eb0 2024-04-20T21:35:41.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.795+0000 7fc84fe00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc82c06c8f0 0x7fc82c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.796+0000 7fc84fe00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc82c06c8f0 0x7fc82c06edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc840009560 tx=0x7fc840009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:41.800+0000 7fc84cc00640 1 -- 172.21.15.102:0/2891491242 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc8500635c0 con 0x7fc8500c9eb0 2024-04-20T21:35:42.040 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.041+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc850074340 con 0x7fc8500c9eb0 2024-04-20T21:35:42.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.042+0000 7fc84cc00640 1 -- 172.21.15.102:0/2891491242 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+6859 (secure 0 0 0) 0x7fc8501536d0 con 0x7fc8500c9eb0 2024-04-20T21:35:42.042 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:42.042 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:20.330927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5994,"qdb_cluster":[5994]},"id":1}]} 2024-04-20T21:35:42.042 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 61 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.045+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc82c06c8f0 msgr2=0x7fc82c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.045+0000 7fc856a00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc82c06c8f0 0x7fc82c06edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc840009560 tx=0x7fc840009190 comp rx=0 tx=0).stop 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.045+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8500c9eb0 msgr2=0x7fc8500c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.045+0000 7fc856a00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8500c9eb0 0x7fc8500c7e20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c790 tx=0x7fc83800cc50 comp rx=0 tx=0).stop 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.046+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 shutdown_connections 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.046+0000 7fc856a00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc82c06c8f0 0x7fc82c06edb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.046+0000 7fc856a00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85014fa50 0x7fc8500c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.046+0000 7fc856a00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8500c9eb0 0x7fc8500c7e20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.046+0000 7fc856a00640 1 --2- 172.21.15.102:0/2891491242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc850073a10 0x7fc8501f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.046+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 >> 172.21.15.102:0/2891491242 conn(0x7fc8500c9390 msgr2=0x7fc85006e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.046+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 shutdown_connections 2024-04-20T21:35:42.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:42.046+0000 7fc856a00640 1 -- 172.21.15.102:0/2891491242 wait complete. 2024-04-20T21:35:42.063 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:35:44.064 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:44.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.273+0000 7f81d3800640 1 Processor -- start 2024-04-20T21:35:44.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.273+0000 7f81d3800640 1 -- start start 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.274+0000 7f81d3800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.274+0000 7f81d3800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f81cc073a10 0x7f81cc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.274+0000 7f81d3800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81cc1536d0 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.274+0000 7f81d3800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f81cc159500 con 0x7f81cc073a10 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.274+0000 7f81d3800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc0c76c0 con 0x7f81cc14fa50 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81d1800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f81cc073a10 0x7f81cc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81d0e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81d0e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:51280/0 (socket says 172.21.15.102:51280) 2024-04-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81d0e00640 1 -- 172.21.15.102:0/3883539604 learned_addr learned my addr 172.21.15.102:0/3883539604 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81d0e00640 1 -- 172.21.15.102:0/3883539604 >> v2:172.21.15.102:3300/0 conn(0x7f81cc073a10 msgr2=0x7f81cc158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81d0e00640 1 --2- 172.21.15.102:0/3883539604 >> v2:172.21.15.102:3300/0 conn(0x7f81cc073a10 0x7f81cc158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81d0e00640 1 -- 172.21.15.102:0/3883539604 >> v1:172.21.15.193:6789/0 conn(0x7f81cc0c9eb0 legacy=0x7f81cc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81d0e00640 1 -- 172.21.15.102:0/3883539604 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc0c7960 con 0x7f81cc14fa50 2024-04-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.275+0000 7f81cbe00640 1 -- 172.21.15.102:0/3883539604 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1715818817 0 0) 0x7f81b4001110 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.276+0000 7f81d0e00640 1 --2- 172.21.15.102:0/3883539604 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc0c4830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009870 tx=0x7f81bc02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e967521256e0b0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81cbe00640 1 -- 172.21.15.102:0/3883539604 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc02f840 con 0x7f81cc14fa50 2024-04-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81cbe00640 1 -- 172.21.15.102:0/3883539604 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81bc02f9e0 con 0x7f81cc14fa50 2024-04-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81cbe00640 1 -- 172.21.15.102:0/3883539604 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc02fdb0 con 0x7f81cc14fa50 2024-04-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81d3800640 1 -- 172.21.15.102:0/3883539604 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 msgr2=0x7f81cc0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81d3800640 1 --2- 172.21.15.102:0/3883539604 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc0c4830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009870 tx=0x7f81bc02ba20 comp rx=0 tx=0).stop 2024-04-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81d3800640 1 -- 172.21.15.102:0/3883539604 shutdown_connections 2024-04-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81d3800640 1 --2- 172.21.15.102:0/3883539604 >> v2:172.21.15.102:3300/0 conn(0x7f81cc073a10 0x7f81cc158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81d3800640 1 --2- 172.21.15.102:0/3883539604 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc0c4830 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81d3800640 1 -- 172.21.15.102:0/3883539604 >> 172.21.15.102:0/3883539604 conn(0x7f81cc0c9390 msgr2=0x7f81cc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81d3800640 1 -- 172.21.15.102:0/3883539604 shutdown_connections 2024-04-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.277+0000 7f81d3800640 1 -- 172.21.15.102:0/3883539604 wait complete. 2024-04-20T21:35:44.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.279+0000 7f81d3800640 1 Processor -- start 2024-04-20T21:35:44.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.279+0000 7f81d3800640 1 -- start start 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81d3800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81cc073a10 0x7f81cc071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81d3800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc0c9eb0 0x7f81cc0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81d3800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc14a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81d3800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc0c7c00 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81d3800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc0c3d90 con 0x7f81cc073a10 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81d3800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc0c4030 con 0x7f81cc14fa50 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc0c9eb0 0x7f81cc0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc0c9eb0 0x7f81cc0726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55762/0 (socket says 172.21.15.102:55762) 2024-04-20T21:35:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81c3e00640 1 -- 172.21.15.102:0/866535244 learned_addr learned my addr 172.21.15.102:0/866535244 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.280+0000 7f81d1800640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc14a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81d0e00640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81cc073a10 0x7f81cc071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81c3e00640 1 -- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 msgr2=0x7f81cc14a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81c3e00640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc14a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81c3e00640 1 -- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81cc073a10 msgr2=0x7f81cc071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81c3e00640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81cc073a10 0x7f81cc071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81c3e00640 1 -- 172.21.15.102:0/866535244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc14a150 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81d1800640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc14a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:44.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81c3e00640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc0c9eb0 0x7f81cc0726d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c780 tx=0x7f81b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81c9600640 1 -- 172.21.15.102:0/866535244 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4017070 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.281+0000 7f81c9600640 1 -- 172.21.15.102:0/866535244 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b40052b0 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.282+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc1f43b0 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.282+0000 7f81c9600640 1 -- 172.21.15.102:0/866535244 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b400a880 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.282+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc1f4970 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.283+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81cc074340 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.284+0000 7f81c9600640 1 -- 172.21.15.102:0/866535244 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81b401a070 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.285+0000 7f81c9600640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f81a806c920 0x7f81a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.285+0000 7f81c9600640 1 -- 172.21.15.102:0/866535244 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f81b408b4f0 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.285+0000 7f81d0e00640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f81a806c920 0x7f81a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.286+0000 7f81d0e00640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f81a806c920 0x7f81a806ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f81bc036000 tx=0x7f81bc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.289+0000 7f81c9600640 1 -- 172.21.15.102:0/866535244 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f81cc1536d0 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.528+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81cc0c7320 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.529+0000 7f81c9600640 1 -- 172.21.15.102:0/866535244 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+6859 (secure 0 0 0) 0x7f81b4007d30 con 0x7f81cc0c9eb0 2024-04-20T21:35:44.528 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:44.528 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:20.330927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5994,"qdb_cluster":[5994]},"id":1}]} 2024-04-20T21:35:44.529 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 61 2024-04-20T21:35:44.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f81a806c920 msgr2=0x7f81a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f81a806c920 0x7f81a806ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f81bc036000 tx=0x7f81bc008030 comp rx=0 tx=0).stop 2024-04-20T21:35:44.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc0c9eb0 msgr2=0x7f81cc0726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc0c9eb0 0x7f81cc0726d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c780 tx=0x7f81b400cc40 comp rx=0 tx=0).stop 2024-04-20T21:35:44.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 shutdown_connections 2024-04-20T21:35:44.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81cc14fa50 0x7f81cc14a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc0c9eb0 0x7f81cc0726d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f81a806c920 0x7f81a806ede0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 --2- 172.21.15.102:0/866535244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81cc073a10 0x7f81cc071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 >> 172.21.15.102:0/866535244 conn(0x7f81cc0c9390 msgr2=0x7f81cc06f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:44.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 shutdown_connections 2024-04-20T21:35:44.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:44.532+0000 7f81d3800640 1 -- 172.21.15.102:0/866535244 wait complete. 2024-04-20T21:35:44.550 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:35:46.551 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:46.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.764+0000 7fab65600640 1 Processor -- start 2024-04-20T21:35:46.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.764+0000 7fab65600640 1 -- start start 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab65600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fab6014fa50 0x7fab600c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab65600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab600c9eb0 0x7fab60158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab65600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fab60073a10 0x7fab600c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab65600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fab60159500 con 0x7fab6014fa50 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab65600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fab601536d0 con 0x7fab60073a10 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab65600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fab600c76c0 con 0x7fab600c9eb0 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab5ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fab6014fa50 0x7fab600c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab5ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fab6014fa50 0x7fab600c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55774/0 (socket says 172.21.15.102:55774) 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab5ea00640 1 -- 172.21.15.102:0/3046549439 learned_addr learned my addr 172.21.15.102:0/3046549439 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab56000640 1 --2- 172.21.15.102:0/3046549439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab600c9eb0 0x7fab60158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:46.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.766+0000 7fab5f400640 1 --2- 172.21.15.102:0/3046549439 >> v2:172.21.15.193:3300/0 conn(0x7fab60073a10 0x7fab600c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.767+0000 7fab5ea00640 1 -- 172.21.15.102:0/3046549439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab600c9eb0 msgr2=0x7fab60158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.767+0000 7fab5ea00640 1 --2- 172.21.15.102:0/3046549439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab600c9eb0 0x7fab60158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.767+0000 7fab5ea00640 1 -- 172.21.15.102:0/3046549439 >> v2:172.21.15.193:3300/0 conn(0x7fab60073a10 msgr2=0x7fab600c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.767+0000 7fab5ea00640 1 --2- 172.21.15.102:0/3046549439 >> v2:172.21.15.193:3300/0 conn(0x7fab60073a10 0x7fab600c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.767+0000 7fab5ea00640 1 -- 172.21.15.102:0/3046549439 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab600c7960 con 0x7fab6014fa50 2024-04-20T21:35:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.768+0000 7fab5ea00640 1 --2- 172.21.15.102:0/3046549439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab600c4830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fab6006c190 tx=0x7fab5002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a4a2e1e2f17293 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.768+0000 7fab5e000640 1 -- 172.21.15.102:0/3046549439 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50034070 con 0x7fab6014fa50 2024-04-20T21:35:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.768+0000 7fab5e000640 1 -- 172.21.15.102:0/3046549439 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab50030860 con 0x7fab6014fa50 2024-04-20T21:35:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.768+0000 7fab5e000640 1 -- 172.21.15.102:0/3046549439 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50030c30 con 0x7fab6014fa50 2024-04-20T21:35:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 -- 172.21.15.102:0/3046549439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 msgr2=0x7fab600c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 --2- 172.21.15.102:0/3046549439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab600c4830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fab6006c190 tx=0x7fab5002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:35:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 -- 172.21.15.102:0/3046549439 shutdown_connections 2024-04-20T21:35:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 --2- 172.21.15.102:0/3046549439 >> v2:172.21.15.193:3300/0 conn(0x7fab60073a10 0x7fab600c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 --2- 172.21.15.102:0/3046549439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab600c9eb0 0x7fab60158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 --2- 172.21.15.102:0/3046549439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab600c4830 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 -- 172.21.15.102:0/3046549439 >> 172.21.15.102:0/3046549439 conn(0x7fab600c9390 msgr2=0x7fab600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 -- 172.21.15.102:0/3046549439 shutdown_connections 2024-04-20T21:35:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.769+0000 7fab65600640 1 -- 172.21.15.102:0/3046549439 wait complete. 2024-04-20T21:35:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.770+0000 7fab65600640 1 Processor -- start 2024-04-20T21:35:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.771+0000 7fab65600640 1 -- start start 2024-04-20T21:35:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.771+0000 7fab65600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab60073a10 0x7fab60147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.771+0000 7fab65600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab600c9eb0 0x7fab60156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.771+0000 7fab65600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab60072150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab65600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fab600c7c00 con 0x7fab6014fa50 2024-04-20T21:35:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab65600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fab600c3d90 con 0x7fab600c9eb0 2024-04-20T21:35:46.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab65600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fab600c4030 con 0x7fab60073a10 2024-04-20T21:35:46.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab60072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:46.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab60072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55790/0 (socket says 172.21.15.102:55790) 2024-04-20T21:35:46.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5f400640 1 -- 172.21.15.102:0/1847304558 learned_addr learned my addr 172.21.15.102:0/1847304558 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:46.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5ea00640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab60073a10 0x7fab60147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:46.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab56000640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab600c9eb0 0x7fab60156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:46.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5f400640 1 -- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab60073a10 msgr2=0x7fab60147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:46.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5f400640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab60073a10 0x7fab60147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5f400640 1 -- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab600c9eb0 msgr2=0x7fab60156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5f400640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab600c9eb0 0x7fab60156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.772+0000 7fab5f400640 1 -- 172.21.15.102:0/1847304558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab601590e0 con 0x7fab6014fa50 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.773+0000 7fab5f400640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab60072150 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fab5800ef00 tx=0x7fab5800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.773+0000 7fab57400640 1 -- 172.21.15.102:0/1847304558 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58018020 con 0x7fab6014fa50 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.773+0000 7fab57400640 1 -- 172.21.15.102:0/1847304558 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab58014400 con 0x7fab6014fa50 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.773+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab60143870 con 0x7fab6014fa50 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.773+0000 7fab57400640 1 -- 172.21.15.102:0/1847304558 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab580147f0 con 0x7fab6014fa50 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.773+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab60142ab0 con 0x7fab6014fa50 2024-04-20T21:35:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.774+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab601433a0 con 0x7fab6014fa50 2024-04-20T21:35:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.775+0000 7fab57400640 1 -- 172.21.15.102:0/1847304558 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab58005d10 con 0x7fab6014fa50 2024-04-20T21:35:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.776+0000 7fab57400640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fab3c06c920 0x7fab3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.776+0000 7fab57400640 1 -- 172.21.15.102:0/1847304558 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7fab5808c8f0 con 0x7fab6014fa50 2024-04-20T21:35:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.777+0000 7fab5ea00640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fab3c06c920 0x7fab3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.777+0000 7fab5ea00640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fab3c06c920 0x7fab3c06ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fab50036000 tx=0x7fab5003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:46.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:46.781+0000 7fab57400640 1 -- 172.21.15.102:0/1847304558 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fab58022070 con 0x7fab6014fa50 2024-04-20T21:35:47.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.021+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab60074340 con 0x7fab6014fa50 2024-04-20T21:35:47.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.022+0000 7fab57400640 1 -- 172.21.15.102:0/1847304558 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+6859 (secure 0 0 0) 0x7fab58059e90 con 0x7fab6014fa50 2024-04-20T21:35:47.021 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:47.022 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:20.330927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5994,"qdb_cluster":[5994]},"id":1}]} 2024-04-20T21:35:47.022 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 61 2024-04-20T21:35:47.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.024+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fab3c06c920 msgr2=0x7fab3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:47.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fab3c06c920 0x7fab3c06ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fab50036000 tx=0x7fab5003b040 comp rx=0 tx=0).stop 2024-04-20T21:35:47.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 msgr2=0x7fab60072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:47.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab60072150 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fab5800ef00 tx=0x7fab5800b480 comp rx=0 tx=0).stop 2024-04-20T21:35:47.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 shutdown_connections 2024-04-20T21:35:47.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab6014fa50 0x7fab60072150 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:47.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fab3c06c920 0x7fab3c06ede0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:47.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab600c9eb0 0x7fab60156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:47.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 --2- 172.21.15.102:0/1847304558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab60073a10 0x7fab60147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:47.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 >> 172.21.15.102:0/1847304558 conn(0x7fab600c9390 msgr2=0x7fab600c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:47.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 shutdown_connections 2024-04-20T21:35:47.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:47.025+0000 7fab65600640 1 -- 172.21.15.102:0/1847304558 wait complete. 2024-04-20T21:35:47.041 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:35:49.042 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:49.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.253+0000 7f45f1800640 1 Processor -- start 2024-04-20T21:35:49.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.253+0000 7f45f1800640 1 -- start start 2024-04-20T21:35:49.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.254+0000 7f45f1800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:49.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45f1800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f45ec0737f0 0x7f45ec159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:49.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45f1800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f45ec073c30 0x7f45ec0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:49.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45f1800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f45ec0c6a30 con 0x7f45ec073c30 2024-04-20T21:35:49.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45f1800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f45ec0c6ec0 con 0x7f45ec0737f0 2024-04-20T21:35:49.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45f1800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f45ec0c8d00 con 0x7f45ec13bc80 2024-04-20T21:35:49.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45e3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f45ec0737f0 0x7f45ec159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:49.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45eaa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:49.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45eb400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f45ec073c30 0x7f45ec0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:49.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45e3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f45ec0737f0 0x7f45ec159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:35526/0 (socket says 172.21.15.102:35526) 2024-04-20T21:35:49.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45e3e00640 1 -- 172.21.15.102:0/7948193 learned_addr learned my addr 172.21.15.102:0/7948193 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:49.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45eaa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec0c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:43234/0 (socket says 172.21.15.102:43234) 2024-04-20T21:35:49.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45eb400640 1 -- 172.21.15.102:0/7948193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 msgr2=0x7f45ec0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:49.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45eb400640 1 --2- 172.21.15.102:0/7948193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec0c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45eb400640 1 -- 172.21.15.102:0/7948193 >> v2:172.21.15.193:3300/0 conn(0x7f45ec0737f0 msgr2=0x7f45ec159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:49.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.255+0000 7f45eb400640 1 --2- 172.21.15.102:0/7948193 >> v2:172.21.15.193:3300/0 conn(0x7f45ec0737f0 0x7f45ec159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.256+0000 7f45eb400640 1 -- 172.21.15.102:0/7948193 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec0c8fc0 con 0x7f45ec073c30 2024-04-20T21:35:49.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.257+0000 7f45eb400640 1 --2- 172.21.15.102:0/7948193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec073c30 0x7f45ec0c98f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f45ec071b90 tx=0x7f45e402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c0a8e0e81e4ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.257+0000 7f45ea000640 1 -- 172.21.15.102:0/7948193 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4039070 con 0x7f45ec073c30 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.257+0000 7f45ea000640 1 -- 172.21.15.102:0/7948193 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e402cbd0 con 0x7f45ec073c30 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.257+0000 7f45ea000640 1 -- 172.21.15.102:0/7948193 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4034440 con 0x7f45ec073c30 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.257+0000 7f45f1800640 1 -- 172.21.15.102:0/7948193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec073c30 msgr2=0x7f45ec0c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.258+0000 7f45f1800640 1 --2- 172.21.15.102:0/7948193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec073c30 0x7f45ec0c98f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f45ec071b90 tx=0x7f45e402be20 comp rx=0 tx=0).stop 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.258+0000 7f45f1800640 1 -- 172.21.15.102:0/7948193 shutdown_connections 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.258+0000 7f45f1800640 1 --2- 172.21.15.102:0/7948193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec073c30 0x7f45ec0c98f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.258+0000 7f45f1800640 1 --2- 172.21.15.102:0/7948193 >> v2:172.21.15.193:3300/0 conn(0x7f45ec0737f0 0x7f45ec159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.258+0000 7f45f1800640 1 --2- 172.21.15.102:0/7948193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.258+0000 7f45f1800640 1 -- 172.21.15.102:0/7948193 >> 172.21.15.102:0/7948193 conn(0x7f45ec06ebf0 msgr2=0x7f45ec06efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.258+0000 7f45f1800640 1 -- 172.21.15.102:0/7948193 shutdown_connections 2024-04-20T21:35:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.258+0000 7f45f1800640 1 -- 172.21.15.102:0/7948193 wait complete. 2024-04-20T21:35:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.259+0000 7f45f1800640 1 Processor -- start 2024-04-20T21:35:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.260+0000 7f45f1800640 1 -- start start 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.260+0000 7f45f1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec0737f0 0x7f45ec1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.260+0000 7f45f1800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec073c30 0x7f45ec151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45f1800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec1e3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45f1800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0c66c0 con 0x7f45ec0737f0 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45f1800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0c63e0 con 0x7f45ec073c30 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45f1800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f45ec0c8a20 con 0x7f45ec13bc80 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec0737f0 0x7f45ec1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec0737f0 0x7f45ec1515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35730/0 (socket says 172.21.15.102:35730) 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eaa00640 1 -- 172.21.15.102:0/856848933 learned_addr learned my addr 172.21.15.102:0/856848933 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45e3e00640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec073c30 0x7f45ec151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eb400640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec1e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eaa00640 1 -- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 msgr2=0x7f45ec1e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eaa00640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec1e3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eaa00640 1 -- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec073c30 msgr2=0x7f45ec151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eaa00640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec073c30 0x7f45ec151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.261+0000 7f45eaa00640 1 -- 172.21.15.102:0/856848933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec06d070 con 0x7f45ec0737f0 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.262+0000 7f45eaa00640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec0737f0 0x7f45ec1515b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e8a0 tx=0x7f45dc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.262+0000 7f45e2a00640 1 -- 172.21.15.102:0/856848933 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc017070 con 0x7f45ec0737f0 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.262+0000 7f45e2a00640 1 -- 172.21.15.102:0/856848933 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45dc00c940 con 0x7f45ec0737f0 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.262+0000 7f45e2a00640 1 -- 172.21.15.102:0/856848933 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00cd30 con 0x7f45ec0737f0 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.262+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec06cd90 con 0x7f45ec0737f0 2024-04-20T21:35:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.262+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec06c6d0 con 0x7f45ec0737f0 2024-04-20T21:35:49.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.263+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ec074340 con 0x7f45ec0737f0 2024-04-20T21:35:49.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.264+0000 7f45e2a00640 1 -- 172.21.15.102:0/856848933 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45dc01a070 con 0x7f45ec0737f0 2024-04-20T21:35:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.265+0000 7f45e2a00640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f45c806c920 0x7f45c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.265+0000 7f45e2a00640 1 -- 172.21.15.102:0/856848933 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 7509+0+0 (secure 0 0 0) 0x7f45dc08c160 con 0x7f45ec0737f0 2024-04-20T21:35:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.266+0000 7f45e3e00640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f45c806c920 0x7f45c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:49.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.266+0000 7f45e3e00640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f45c806c920 0x7f45c806ede0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f45ec14f0d0 tx=0x7f45d4009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:49.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.270+0000 7f45e2a00640 1 -- 172.21.15.102:0/856848933 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f45dc059790 con 0x7f45ec0737f0 2024-04-20T21:35:49.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.510+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45ec0ca160 con 0x7f45ec0737f0 2024-04-20T21:35:49.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.512+0000 7f45e2a00640 1 -- 172.21.15.102:0/856848933 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+6859 (secure 0 0 0) 0x7f45dc05da90 con 0x7f45ec0737f0 2024-04-20T21:35:49.511 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:49.511 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6147,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:20.330927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":27,"addr":"172.21.15.102:6833/3985712430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3985712430},{"type":"v1","addr":"172.21.15.102:6833","nonce":3985712430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5994,"qdb_cluster":[5994]},"id":1}]} 2024-04-20T21:35:49.511 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 61 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.514+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f45c806c920 msgr2=0x7f45c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.514+0000 7f45f1800640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f45c806c920 0x7f45c806ede0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f45ec14f0d0 tx=0x7f45d4009240 comp rx=0 tx=0).stop 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.514+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec0737f0 msgr2=0x7f45ec1515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.514+0000 7f45f1800640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec0737f0 0x7f45ec1515b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e8a0 tx=0x7f45dc00ed60 comp rx=0 tx=0).stop 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.515+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 shutdown_connections 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.515+0000 7f45f1800640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45ec13bc80 0x7f45ec1e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.515+0000 7f45f1800640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f45c806c920 0x7f45c806ede0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.515+0000 7f45f1800640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec073c30 0x7f45ec151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.515+0000 7f45f1800640 1 --2- 172.21.15.102:0/856848933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45ec0737f0 0x7f45ec1515b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.515+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 >> 172.21.15.102:0/856848933 conn(0x7f45ec06ebf0 msgr2=0x7f45ec0ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.515+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 shutdown_connections 2024-04-20T21:35:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:49.515+0000 7f45f1800640 1 -- 172.21.15.102:0/856848933 wait complete. 2024-04-20T21:35:49.534 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:35:51.534 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:51.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.746+0000 7f30c3e00640 1 Processor -- start 2024-04-20T21:35:51.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.747+0000 7f30c3e00640 1 -- start start 2024-04-20T21:35:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.748+0000 7f30c3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f30bc14fa50 0x7f30bc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.748+0000 7f30c3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.748+0000 7f30c3e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30bc1536d0 con 0x7f30bc0c9eb0 2024-04-20T21:35:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.748+0000 7f30c3e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc159500 con 0x7f30bc14fa50 2024-04-20T21:35:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.748+0000 7f30c3e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc0c76c0 con 0x7f30bc073a10 2024-04-20T21:35:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.748+0000 7f30c0a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f30bc0c9eb0 0x7f30bc0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37628/0 (socket says 172.21.15.102:37628) 2024-04-20T21:35:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.748+0000 7f30c0a00640 1 -- 172.21.15.102:0/4114295762 learned_addr learned my addr 172.21.15.102:0/4114295762 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30c1e00640 1 --2- 172.21.15.102:0/4114295762 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30c1400640 1 --2- 172.21.15.102:0/4114295762 >> v2:172.21.15.102:3300/0 conn(0x7f30bc14fa50 0x7f30bc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30bbe00640 1 -- 172.21.15.102:0/4114295762 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1198549093 0 0) 0x7f30ac001110 con 0x7f30bc0c9eb0 2024-04-20T21:35:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30bbe00640 1 -- 172.21.15.102:0/4114295762 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30a4003520 con 0x7f30bc0c9eb0 2024-04-20T21:35:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30c1e00640 1 -- 172.21.15.102:0/4114295762 >> v2:172.21.15.102:3300/0 conn(0x7f30bc14fa50 msgr2=0x7f30bc0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30c1e00640 1 --2- 172.21.15.102:0/4114295762 >> v2:172.21.15.102:3300/0 conn(0x7f30bc14fa50 0x7f30bc0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30c1e00640 1 -- 172.21.15.102:0/4114295762 >> v1:172.21.15.102:6789/0 conn(0x7f30bc0c9eb0 legacy=0x7f30bc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30c1e00640 1 -- 172.21.15.102:0/4114295762 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0c7960 con 0x7f30bc073a10 2024-04-20T21:35:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.749+0000 7f30c1400640 1 --2- 172.21.15.102:0/4114295762 >> v2:172.21.15.102:3300/0 conn(0x7f30bc14fa50 0x7f30bc0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.751+0000 7f30c1e00640 1 --2- 172.21.15.102:0/4114295762 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc158fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009d90 tx=0x7f30b402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21687c0a1fe16b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.751+0000 7f30bbe00640 1 -- 172.21.15.102:0/4114295762 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4039070 con 0x7f30bc073a10 2024-04-20T21:35:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.751+0000 7f30bbe00640 1 -- 172.21.15.102:0/4114295762 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b402cce0 con 0x7f30bc073a10 2024-04-20T21:35:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.751+0000 7f30bbe00640 1 -- 172.21.15.102:0/4114295762 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4034660 con 0x7f30bc073a10 2024-04-20T21:35:51.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.752+0000 7f30c3e00640 1 -- 172.21.15.102:0/4114295762 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 msgr2=0x7f30bc158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:51.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.752+0000 7f30c3e00640 1 --2- 172.21.15.102:0/4114295762 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc158fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009d90 tx=0x7f30b402bf30 comp rx=0 tx=0).stop 2024-04-20T21:35:51.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.752+0000 7f30c3e00640 1 -- 172.21.15.102:0/4114295762 shutdown_connections 2024-04-20T21:35:51.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.752+0000 7f30c3e00640 1 --2- 172.21.15.102:0/4114295762 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc158fc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:51.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.752+0000 7f30c3e00640 1 --2- 172.21.15.102:0/4114295762 >> v2:172.21.15.102:3300/0 conn(0x7f30bc14fa50 0x7f30bc0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:51.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.752+0000 7f30c3e00640 1 -- 172.21.15.102:0/4114295762 >> 172.21.15.102:0/4114295762 conn(0x7f30bc0c9390 msgr2=0x7f30bc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:51.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.752+0000 7f30c3e00640 1 -- 172.21.15.102:0/4114295762 shutdown_connections 2024-04-20T21:35:51.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.752+0000 7f30c3e00640 1 -- 172.21.15.102:0/4114295762 wait complete. 2024-04-20T21:35:51.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.754+0000 7f30c3e00640 1 Processor -- start 2024-04-20T21:35:51.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.754+0000 7f30c3e00640 1 -- start start 2024-04-20T21:35:51.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc1f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:51.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30bc0c9eb0 0x7f30bc154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:51.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30bc14fa50 0x7f30bc0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:51.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0c7c00 con 0x7f30bc0c9eb0 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0c3d90 con 0x7f30bc14fa50 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c3e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc0c4030 con 0x7f30bc073a10 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c1400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc1f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c1400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc1f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:43248/0 (socket says 172.21.15.102:43248) 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30bc0c9eb0 0x7f30bc154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c1400640 1 -- 172.21.15.102:0/3928281160 learned_addr learned my addr 172.21.15.102:0/3928281160 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.756+0000 7f30c1e00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30bc14fa50 0x7f30bc0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.757+0000 7f30c1400640 1 -- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30bc14fa50 msgr2=0x7f30bc0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.757+0000 7f30c1400640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30bc14fa50 0x7f30bc0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.757+0000 7f30c1400640 1 -- 172.21.15.102:0/3928281160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30bc0c9eb0 msgr2=0x7f30bc154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.757+0000 7f30c1400640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30bc0c9eb0 0x7f30bc154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.757+0000 7f30c1400640 1 -- 172.21.15.102:0/3928281160 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0ca810 con 0x7f30bc073a10 2024-04-20T21:35:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.757+0000 7f30c0a00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30bc0c9eb0 0x7f30bc154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:51.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.757+0000 7f30c1e00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30bc14fa50 0x7f30bc0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:51.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.757+0000 7f30c1400640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc1f1060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f30b000ebe0 tx=0x7f30b000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:51.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.758+0000 7f30b9600640 1 -- 172.21.15.102:0/3928281160 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0017070 con 0x7f30bc073a10 2024-04-20T21:35:51.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.758+0000 7f30b9600640 1 -- 172.21.15.102:0/3928281160 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b000cc30 con 0x7f30bc073a10 2024-04-20T21:35:51.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.758+0000 7f30b9600640 1 -- 172.21.15.102:0/3928281160 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0013470 con 0x7f30bc073a10 2024-04-20T21:35:51.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.758+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc0caaf0 con 0x7f30bc073a10 2024-04-20T21:35:51.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.758+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc0cb0b0 con 0x7f30bc073a10 2024-04-20T21:35:51.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.759+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc074340 con 0x7f30bc073a10 2024-04-20T21:35:51.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.760+0000 7f30b9600640 1 -- 172.21.15.102:0/3928281160 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30b001a070 con 0x7f30bc073a10 2024-04-20T21:35:51.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.761+0000 7f30b9600640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30a006c970 0x7f30a006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:51.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.761+0000 7f30b9600640 1 -- 172.21.15.102:0/3928281160 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f30b008c510 con 0x7f30bc073a10 2024-04-20T21:35:51.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.762+0000 7f30c0a00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30a006c970 0x7f30a006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:51.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.762+0000 7f30c0a00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30a006c970 0x7f30a006ee30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f30bc1533a0 tx=0x7f30ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:51.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:51.766+0000 7f30b9600640 1 -- 172.21.15.102:0/3928281160 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f30b0059b70 con 0x7f30bc073a10 2024-04-20T21:35:52.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.008+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30bc1568e0 con 0x7f30bc073a10 2024-04-20T21:35:52.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.009+0000 7f30b9600640 1 -- 172.21.15.102:0/3928281160 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6023 (secure 0 0 0) 0x7f30b005de70 con 0x7f30bc073a10 2024-04-20T21:35:52.007 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:52.007 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:50.751041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:52.007 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 62 2024-04-20T21:35:52.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30a006c970 msgr2=0x7f30a006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30a006c970 0x7f30a006ee30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f30bc1533a0 tx=0x7f30ac009040 comp rx=0 tx=0).stop 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 msgr2=0x7f30bc1f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc1f1060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f30b000ebe0 tx=0x7f30b000c520 comp rx=0 tx=0).stop 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 shutdown_connections 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f30a006c970 0x7f30a006ee30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30bc14fa50 0x7f30bc0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30bc0c9eb0 0x7f30bc154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 --2- 172.21.15.102:0/3928281160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30bc073a10 0x7f30bc1f1060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.011+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 >> 172.21.15.102:0/3928281160 conn(0x7f30bc0c9390 msgr2=0x7f30bc0c5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.012+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 shutdown_connections 2024-04-20T21:35:52.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.012+0000 7f30c3e00640 1 -- 172.21.15.102:0/3928281160 wait complete. 2024-04-20T21:35:52.029 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-04-20T21:35:52.029 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:35:52.029 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:52.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.233+0000 7f312d800640 1 Processor -- start 2024-04-20T21:35:52.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.233+0000 7f312d800640 1 -- start start 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.234+0000 7f312d800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128153180 0x7f3128158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f312d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31280c75a0 0x7f31281637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f312d800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3128153560 con 0x7f3128159380 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f312d800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f31280c7a70 con 0x7f31280c75a0 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f312d800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f31280c7d30 con 0x7f3128153180 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f3127e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128153180 0x7f3128158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f3127e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128153180 0x7f3128158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:43252/0 (socket says 172.21.15.102:43252) 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f3127e00640 1 -- 172.21.15.102:0/3717244 learned_addr learned my addr 172.21.15.102:0/3717244 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:52.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f312ce00640 1 --2- 172.21.15.102:0/3717244 >> v2:172.21.15.102:3300/0 conn(0x7f31280c75a0 0x7f31281637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:52.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f3127e00640 1 -- 172.21.15.102:0/3717244 >> v2:172.21.15.102:3300/0 conn(0x7f31280c75a0 msgr2=0x7f31281637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:52.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f3127e00640 1 --2- 172.21.15.102:0/3717244 >> v2:172.21.15.102:3300/0 conn(0x7f31280c75a0 0x7f31281637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f3127e00640 1 -- 172.21.15.102:0/3717244 >> v1:172.21.15.102:6789/0 conn(0x7f3128159380 legacy=0x7f31280c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:52.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.235+0000 7f3127e00640 1 -- 172.21.15.102:0/3717244 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312815c4d0 con 0x7f3128153180 2024-04-20T21:35:52.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.237+0000 7f3127e00640 1 --2- 172.21.15.102:0/3717244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128153180 0x7f3128158e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3118009870 tx=0x7f311802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a799c38c21a6b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:52.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.237+0000 7f3127400640 1 -- 172.21.15.102:0/3717244 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118034070 con 0x7f3128153180 2024-04-20T21:35:52.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.237+0000 7f3127400640 1 -- 172.21.15.102:0/3717244 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3118030840 con 0x7f3128153180 2024-04-20T21:35:52.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.237+0000 7f3127400640 1 -- 172.21.15.102:0/3717244 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118030c10 con 0x7f3128153180 2024-04-20T21:35:52.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.238+0000 7f312d800640 1 -- 172.21.15.102:0/3717244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128153180 msgr2=0x7f3128158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:52.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.238+0000 7f312d800640 1 --2- 172.21.15.102:0/3717244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128153180 0x7f3128158e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3118009870 tx=0x7f311802b980 comp rx=0 tx=0).stop 2024-04-20T21:35:52.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.238+0000 7f312d800640 1 -- 172.21.15.102:0/3717244 shutdown_connections 2024-04-20T21:35:52.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.238+0000 7f312d800640 1 --2- 172.21.15.102:0/3717244 >> v2:172.21.15.102:3300/0 conn(0x7f31280c75a0 0x7f31281637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.238+0000 7f312d800640 1 --2- 172.21.15.102:0/3717244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128153180 0x7f3128158e40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.238+0000 7f312d800640 1 -- 172.21.15.102:0/3717244 >> 172.21.15.102:0/3717244 conn(0x7f31280cc820 msgr2=0x7f31280ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:52.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.238+0000 7f312d800640 1 -- 172.21.15.102:0/3717244 shutdown_connections 2024-04-20T21:35:52.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.238+0000 7f312d800640 1 -- 172.21.15.102:0/3717244 wait complete. 2024-04-20T21:35:52.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.240+0000 7f312d800640 1 Processor -- start 2024-04-20T21:35:52.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.240+0000 7f312d800640 1 -- start start 2024-04-20T21:35:52.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f312d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31280c75a0 0x7f31281e2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f312d800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3128153180 0x7f31281483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f312d800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128159380 0x7f3128148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f312d800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f312815d290 con 0x7f31280c75a0 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f312d800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f312815ca90 con 0x7f3128153180 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f312d800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f31280c6be0 con 0x7f3128159380 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f3127e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31280c75a0 0x7f31281e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f3127e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31280c75a0 0x7f31281e2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35776/0 (socket says 172.21.15.102:35776) 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f3127e00640 1 -- 172.21.15.102:0/1613044745 learned_addr learned my addr 172.21.15.102:0/1613044745 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f312ce00640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128159380 0x7f3128148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:52.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.241+0000 7f311f400640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3128153180 0x7f31281483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:52.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3127e00640 1 -- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128159380 msgr2=0x7f3128148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:52.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3127e00640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128159380 0x7f3128148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3127e00640 1 -- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3128153180 msgr2=0x7f31281483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:52.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3127e00640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3128153180 0x7f31281483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3127e00640 1 -- 172.21.15.102:0/1613044745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3128157840 con 0x7f31280c75a0 2024-04-20T21:35:52.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f312ce00640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128159380 0x7f3128148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:52.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3127e00640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31280c75a0 0x7f31281e2220 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f311802b9c0 tx=0x7f31180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:52.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3124c00640 1 -- 172.21.15.102:0/1613044745 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118034040 con 0x7f31280c75a0 2024-04-20T21:35:52.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3124c00640 1 -- 172.21.15.102:0/1613044745 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3118039d90 con 0x7f31280c75a0 2024-04-20T21:35:52.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f3124c00640 1 -- 172.21.15.102:0/1613044745 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118043630 con 0x7f31280c75a0 2024-04-20T21:35:52.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.242+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3128157b20 con 0x7f31280c75a0 2024-04-20T21:35:52.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.243+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31281e3f30 con 0x7f31280c75a0 2024-04-20T21:35:52.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.244+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312804fd60 con 0x7f31280c75a0 2024-04-20T21:35:52.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.245+0000 7f3124c00640 1 -- 172.21.15.102:0/1613044745 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3118037070 con 0x7f31280c75a0 2024-04-20T21:35:52.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.245+0000 7f3124c00640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f310406c8f0 0x7f310406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:52.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.245+0000 7f3124c00640 1 -- 172.21.15.102:0/1613044745 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f31180ae3c0 con 0x7f31280c75a0 2024-04-20T21:35:52.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.246+0000 7f311f400640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f310406c8f0 0x7f310406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:52.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.246+0000 7f311f400640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f310406c8f0 0x7f310406edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f31281e38d0 tx=0x7f3110009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:52.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.250+0000 7f3124c00640 1 -- 172.21.15.102:0/1613044745 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f311803e070 con 0x7f31280c75a0 2024-04-20T21:35:52.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.492+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31280c3690 con 0x7f31280c75a0 2024-04-20T21:35:52.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.493+0000 7f3124c00640 1 -- 172.21.15.102:0/1613044745 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6023 (secure 0 0 0) 0x7f311807c1a0 con 0x7f31280c75a0 2024-04-20T21:35:52.492 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:52.492 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:50.751041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:52.492 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 62 2024-04-20T21:35:52.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f310406c8f0 msgr2=0x7f310406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:52.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f310406c8f0 0x7f310406edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f31281e38d0 tx=0x7f3110009040 comp rx=0 tx=0).stop 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31280c75a0 msgr2=0x7f31281e2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31280c75a0 0x7f31281e2220 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f311802b9c0 tx=0x7f31180046c0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 shutdown_connections 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3128159380 0x7f3128148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f310406c8f0 0x7f310406edb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3128153180 0x7f31281483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 --2- 172.21.15.102:0/1613044745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31280c75a0 0x7f31281e2220 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 >> 172.21.15.102:0/1613044745 conn(0x7f31280cc820 msgr2=0x7f31280cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 shutdown_connections 2024-04-20T21:35:52.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:52.496+0000 7f312d800640 1 -- 172.21.15.102:0/1613044745 wait complete. 2024-04-20T21:35:52.514 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 62, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 62, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6147": { "addr": "172.21.15.102:6835/945751373", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 945751373, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 945751373, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6147, "incarnation": 62, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:35:50.751041+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6147 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:35:54.515 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.729+0000 7fba54800640 1 Processor -- start 2024-04-20T21:35:54.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.729+0000 7fba54800640 1 -- start start 2024-04-20T21:35:54.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.730+0000 7fba54800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba4c15ee00 0x7fba4c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:54.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.730+0000 7fba54800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba4c0c8f20 con 0x7fba4c15ea20 2024-04-20T21:35:54.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba54800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba4c06cb80 con 0x7fba4c15f2b0 2024-04-20T21:35:54.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba54800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fba4c06ce60 con 0x7fba4c15ee00 2024-04-20T21:35:54.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba51e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fba4c15ea20 0x7fba4c0c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37642/0 (socket says 172.21.15.102:37642) 2024-04-20T21:35:54.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba51400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba4c15ee00 0x7fba4c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:54.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba51e00640 1 -- 172.21.15.102:0/988480457 learned_addr learned my addr 172.21.15.102:0/988480457 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:54.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba51400640 1 -- 172.21.15.102:0/988480457 >> v1:172.21.15.193:6789/0 conn(0x7fba4c15f2b0 legacy=0x7fba4c154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:54.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba51400640 1 -- 172.21.15.102:0/988480457 >> v1:172.21.15.102:6789/0 conn(0x7fba4c15ea20 legacy=0x7fba4c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:54.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba51400640 1 -- 172.21.15.102:0/988480457 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c06d140 con 0x7fba4c15ee00 2024-04-20T21:35:54.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.731+0000 7fba50a00640 1 -- 172.21.15.102:0/988480457 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661048231 0 0) 0x7fba4c0c8f20 con 0x7fba4c15ea20 2024-04-20T21:35:54.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.733+0000 7fba51400640 1 --2- 172.21.15.102:0/988480457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ee00 0x7fba4c06d940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0097a0 tx=0x7fba3c02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b078afc8c756f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:54.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.733+0000 7fba50a00640 1 -- 172.21.15.102:0/988480457 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c02f810 con 0x7fba4c15ee00 2024-04-20T21:35:54.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.733+0000 7fba50a00640 1 -- 172.21.15.102:0/988480457 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3c02f9b0 con 0x7fba4c15ee00 2024-04-20T21:35:54.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.733+0000 7fba50a00640 1 -- 172.21.15.102:0/988480457 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c02fd80 con 0x7fba4c15ee00 2024-04-20T21:35:54.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.733+0000 7fba54800640 1 -- 172.21.15.102:0/988480457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ee00 msgr2=0x7fba4c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:54.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.733+0000 7fba54800640 1 --2- 172.21.15.102:0/988480457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ee00 0x7fba4c06d940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0097a0 tx=0x7fba3c02b9f0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.734+0000 7fba54800640 1 -- 172.21.15.102:0/988480457 shutdown_connections 2024-04-20T21:35:54.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.734+0000 7fba54800640 1 --2- 172.21.15.102:0/988480457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ee00 0x7fba4c06d940 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.734+0000 7fba54800640 1 -- 172.21.15.102:0/988480457 >> 172.21.15.102:0/988480457 conn(0x7fba4c0c9390 msgr2=0x7fba4c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:54.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.734+0000 7fba54800640 1 -- 172.21.15.102:0/988480457 shutdown_connections 2024-04-20T21:35:54.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.734+0000 7fba54800640 1 -- 172.21.15.102:0/988480457 wait complete. 2024-04-20T21:35:54.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.735+0000 7fba54800640 1 Processor -- start 2024-04-20T21:35:54.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.735+0000 7fba54800640 1 -- start start 2024-04-20T21:35:54.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.736+0000 7fba54800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ea20 0x7fba4c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:54.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.736+0000 7fba54800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba4c15ee00 0x7fba4c1f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:54.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.736+0000 7fba54800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c15f2b0 0x7fba4c1f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:54.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.736+0000 7fba54800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c0c8c80 con 0x7fba4c15ea20 2024-04-20T21:35:54.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.736+0000 7fba54800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c0c8b00 con 0x7fba4c15f2b0 2024-04-20T21:35:54.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.736+0000 7fba54800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fba4c155590 con 0x7fba4c15ee00 2024-04-20T21:35:54.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba51400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba4c15ee00 0x7fba4c1f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba52800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c15f2b0 0x7fba4c1f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ea20 0x7fba4c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba51400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba4c15ee00 0x7fba4c1f0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:43276/0 (socket says 172.21.15.102:43276) 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba51400640 1 -- 172.21.15.102:0/3042881424 learned_addr learned my addr 172.21.15.102:0/3042881424 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba52800640 1 -- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba4c15ee00 msgr2=0x7fba4c1f0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba52800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba4c15ee00 0x7fba4c1f0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba52800640 1 -- 172.21.15.102:0/3042881424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ea20 msgr2=0x7fba4c0cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba52800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ea20 0x7fba4c0cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba52800640 1 -- 172.21.15.102:0/3042881424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c0c5970 con 0x7fba4c15f2b0 2024-04-20T21:35:54.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba51400640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba4c15ee00 0x7fba4c1f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:54.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.737+0000 7fba52800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c15f2b0 0x7fba4c1f10f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fba4800c700 tx=0x7fba4800cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:54.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.738+0000 7fba46000640 1 -- 172.21.15.102:0/3042881424 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48003b70 con 0x7fba4c15f2b0 2024-04-20T21:35:54.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.738+0000 7fba46000640 1 -- 172.21.15.102:0/3042881424 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba48003d10 con 0x7fba4c15f2b0 2024-04-20T21:35:54.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.738+0000 7fba46000640 1 -- 172.21.15.102:0/3042881424 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48005310 con 0x7fba4c15f2b0 2024-04-20T21:35:54.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.738+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c0c53b0 con 0x7fba4c15f2b0 2024-04-20T21:35:54.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.738+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c15be70 con 0x7fba4c15f2b0 2024-04-20T21:35:54.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.739+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4c074340 con 0x7fba4c15f2b0 2024-04-20T21:35:54.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.740+0000 7fba46000640 1 -- 172.21.15.102:0/3042881424 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba480054b0 con 0x7fba4c15f2b0 2024-04-20T21:35:54.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.741+0000 7fba46000640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fba3006c920 0x7fba3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:54.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.741+0000 7fba46000640 1 -- 172.21.15.102:0/3042881424 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7fba4808bfa0 con 0x7fba4c15f2b0 2024-04-20T21:35:54.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.741+0000 7fba51e00640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fba3006c920 0x7fba3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.742+0000 7fba51e00640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fba3006c920 0x7fba3006ede0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fba40007f70 tx=0x7fba40007f00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:54.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.746+0000 7fba46000640 1 -- 172.21.15.102:0/3042881424 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fba48059580 con 0x7fba4c15f2b0 2024-04-20T21:35:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.990+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba4c14a620 con 0x7fba4c15f2b0 2024-04-20T21:35:54.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.991+0000 7fba46000640 1 -- 172.21.15.102:0/3042881424 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6023 (secure 0 0 0) 0x7fba4805d880 con 0x7fba4c15f2b0 2024-04-20T21:35:54.991 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:54.991 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:50.751041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:54.991 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 62 2024-04-20T21:35:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fba3006c920 msgr2=0x7fba3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fba3006c920 0x7fba3006ede0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fba40007f70 tx=0x7fba40007f00 comp rx=0 tx=0).stop 2024-04-20T21:35:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c15f2b0 msgr2=0x7fba4c1f10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c15f2b0 0x7fba4c1f10f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fba4800c700 tx=0x7fba4800cbc0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 shutdown_connections 2024-04-20T21:35:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fba3006c920 0x7fba3006ede0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba4c15f2b0 0x7fba4c1f10f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba4c15ee00 0x7fba4c1f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 --2- 172.21.15.102:0/3042881424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba4c15ea20 0x7fba4c0cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 >> 172.21.15.102:0/3042881424 conn(0x7fba4c0c9390 msgr2=0x7fba4c06e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 shutdown_connections 2024-04-20T21:35:54.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:54.994+0000 7fba54800640 1 -- 172.21.15.102:0/3042881424 wait complete. 2024-04-20T21:35:55.013 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:57.013 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:57.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.198+0000 7f0ae1600640 1 Processor -- start 2024-04-20T21:35:57.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.198+0000 7f0ae1600640 1 -- start start 2024-04-20T21:35:57.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0ae1600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc0c9eb0 0x7f0adc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:57.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0ae1600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0adc1536d0 con 0x7f0adc14fa50 2024-04-20T21:35:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0ae1600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0adc0c76c0 con 0x7f0adc073a10 2024-04-20T21:35:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0ae1600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0adc0c7960 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0adaa00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0adc14fa50 0x7f0adc0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37646/0 (socket says 172.21.15.102:37646) 2024-04-20T21:35:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0adaa00640 1 -- 172.21.15.102:0/3344205403 learned_addr learned my addr 172.21.15.102:0/3344205403 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0ada000640 1 --2- 172.21.15.102:0/3344205403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc0c9eb0 0x7f0adc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0ad9600640 1 -- 172.21.15.102:0/3344205403 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1629331382 0 0) 0x7f0acc001110 con 0x7f0adc14fa50 2024-04-20T21:35:57.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.200+0000 7f0ad9600640 1 -- 172.21.15.102:0/3344205403 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ac0003520 con 0x7f0adc14fa50 2024-04-20T21:35:57.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.201+0000 7f0ad9600640 1 -- 172.21.15.102:0/3344205403 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322703429 0 0) 0x7f0adc0c76c0 con 0x7f0adc073a10 2024-04-20T21:35:57.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.201+0000 7f0ad9600640 1 -- 172.21.15.102:0/3344205403 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0acc001110 con 0x7f0adc073a10 2024-04-20T21:35:57.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.201+0000 7f0ada000640 1 -- 172.21.15.102:0/3344205403 >> v1:172.21.15.193:6789/0 conn(0x7f0adc073a10 legacy=0x7f0adc163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:57.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.201+0000 7f0ada000640 1 -- 172.21.15.102:0/3344205403 >> v1:172.21.15.102:6789/0 conn(0x7f0adc14fa50 legacy=0x7f0adc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:57.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.201+0000 7f0ada000640 1 -- 172.21.15.102:0/3344205403 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc0c3d90 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.202+0000 7f0ada000640 1 --2- 172.21.15.102:0/3344205403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc0c9eb0 0x7f0adc0c4830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0c3f60 tx=0x7f0ac802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4de6b6139410777 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:57.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.202+0000 7f0ad9600640 1 -- 172.21.15.102:0/3344205403 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8034070 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.202+0000 7f0ad9600640 1 -- 172.21.15.102:0/3344205403 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac8030880 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.202+0000 7f0ad9600640 1 -- 172.21.15.102:0/3344205403 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8030c30 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.203+0000 7f0ae1600640 1 -- 172.21.15.102:0/3344205403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc0c9eb0 msgr2=0x7f0adc0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.203+0000 7f0ae1600640 1 --2- 172.21.15.102:0/3344205403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc0c9eb0 0x7f0adc0c4830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0c3f60 tx=0x7f0ac802b9c0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.203+0000 7f0ae1600640 1 -- 172.21.15.102:0/3344205403 shutdown_connections 2024-04-20T21:35:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.203+0000 7f0ae1600640 1 --2- 172.21.15.102:0/3344205403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc0c9eb0 0x7f0adc0c4830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.203+0000 7f0ae1600640 1 -- 172.21.15.102:0/3344205403 >> 172.21.15.102:0/3344205403 conn(0x7f0adc0c9390 msgr2=0x7f0adc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:57.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.203+0000 7f0ae1600640 1 -- 172.21.15.102:0/3344205403 shutdown_connections 2024-04-20T21:35:57.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.203+0000 7f0ae1600640 1 -- 172.21.15.102:0/3344205403 wait complete. 2024-04-20T21:35:57.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.206+0000 7f0ae1600640 1 Processor -- start 2024-04-20T21:35:57.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.206+0000 7f0ae1600640 1 -- start start 2024-04-20T21:35:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.207+0000 7f0ae1600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc073a10 0x7f0adc1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.207+0000 7f0ae1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc0c9eb0 0x7f0adc1ef630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.207+0000 7f0ae1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0adc14fa50 0x7f0adc1f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.207+0000 7f0ae1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc0c4030 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.207+0000 7f0ae1600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc0c7c00 con 0x7f0adc14fa50 2024-04-20T21:35:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.207+0000 7f0ae1600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0adc0c4310 con 0x7f0adc073a10 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc0c9eb0 0x7f0adc1ef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc0c9eb0 0x7f0adc1ef630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35794/0 (socket says 172.21.15.102:35794) 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 -- 172.21.15.102:0/3669034647 learned_addr learned my addr 172.21.15.102:0/3669034647 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0adaa00640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc073a10 0x7f0adc1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0adb400640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0adc14fa50 0x7f0adc1f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 -- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc073a10 msgr2=0x7f0adc1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc073a10 0x7f0adc1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 -- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0adc14fa50 msgr2=0x7f0adc1f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0adc14fa50 0x7f0adc1f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 -- 172.21.15.102:0/3669034647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc1eeb70 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0adaa00640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc073a10 0x7f0adc1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:57.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.208+0000 7f0ada000640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc0c9eb0 0x7f0adc1ef630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8000c00 tx=0x7f0ac80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:57.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.209+0000 7f0ad2a00640 1 -- 172.21.15.102:0/3669034647 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8034040 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.209+0000 7f0ad2a00640 1 -- 172.21.15.102:0/3669034647 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac8030680 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.209+0000 7f0ad2a00640 1 -- 172.21.15.102:0/3669034647 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8006e10 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.209+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc1f2840 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.209+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc1f2e00 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.210+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0adc074340 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.212+0000 7f0ad2a00640 1 -- 172.21.15.102:0/3669034647 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ac8037070 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.212+0000 7f0ad2a00640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:57.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.213+0000 7f0ad2a00640 1 -- 172.21.15.102:0/3669034647 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f0ac80ae3d0 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.213+0000 7f0adaa00640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.213+0000 7f0adaa00640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0acc000f60 tx=0x7f0acc007d70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:57.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.221+0000 7f0ad2a00640 1 -- 172.21.15.102:0/3669034647 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ac807ba40 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.462+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0adc1568e0 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.463+0000 7f0ad2a00640 1 -- 172.21.15.102:0/3669034647 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6023 (secure 0 0 0) 0x7f0ac807fd40 con 0x7f0adc0c9eb0 2024-04-20T21:35:57.463 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:57.463 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:50.751041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:57.463 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 62 2024-04-20T21:35:57.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 msgr2=0x7f0abc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:57.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0acc000f60 tx=0x7f0acc007d70 comp rx=0 tx=0).stop 2024-04-20T21:35:57.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc0c9eb0 msgr2=0x7f0adc1ef630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:57.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc0c9eb0 0x7f0adc1ef630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8000c00 tx=0x7f0ac80046c0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 shutdown_connections 2024-04-20T21:35:57.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0abc06c8f0 0x7f0abc06edb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0adc14fa50 0x7f0adc1f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0adc0c9eb0 0x7f0adc1ef630 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 --2- 172.21.15.102:0/3669034647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0adc073a10 0x7f0adc1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:57.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 >> 172.21.15.102:0/3669034647 conn(0x7f0adc0c9390 msgr2=0x7f0adc14d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:57.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 shutdown_connections 2024-04-20T21:35:57.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:57.466+0000 7f0ae1600640 1 -- 172.21.15.102:0/3669034647 wait complete. 2024-04-20T21:35:57.482 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:35:59.111 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:35:59.119 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:35:59.483 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:59.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.670+0000 7f009d800640 1 Processor -- start 2024-04-20T21:35:59.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.670+0000 7f009d800640 1 -- start start 2024-04-20T21:35:59.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f009d800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f00981552a0 0x7f00980c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:59.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f009d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f00980caec0 0x7f009815b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:59.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f009d800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00980c9b30 con 0x7f009815b9a0 2024-04-20T21:35:59.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f009d800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f009815bdd0 con 0x7f00980caec0 2024-04-20T21:35:59.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f009d800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0098074fd0 con 0x7f00981552a0 2024-04-20T21:35:59.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f0097400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f00980caec0 0x7f009815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:59.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f009ce00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f009815b9a0 0x7f00980c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37908/0 (socket says 172.21.15.102:37908) 2024-04-20T21:35:59.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f009ce00640 1 -- 172.21.15.102:0/44119042 learned_addr learned my addr 172.21.15.102:0/44119042 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:35:59.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.673+0000 7f0097e00640 1 --2- 172.21.15.102:0/44119042 >> v2:172.21.15.193:3300/0 conn(0x7f00981552a0 0x7f00980c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.674+0000 7f0096a00640 1 -- 172.21.15.102:0/44119042 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386595169 0 0) 0x7f00980c9b30 con 0x7f009815b9a0 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.674+0000 7f0097400640 1 -- 172.21.15.102:0/44119042 >> v2:172.21.15.193:3300/0 conn(0x7f00981552a0 msgr2=0x7f00980c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.674+0000 7f0097400640 1 --2- 172.21.15.102:0/44119042 >> v2:172.21.15.193:3300/0 conn(0x7f00981552a0 0x7f00980c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.674+0000 7f0097400640 1 -- 172.21.15.102:0/44119042 >> v1:172.21.15.102:6789/0 conn(0x7f009815b9a0 legacy=0x7f00980c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.674+0000 7f0097400640 1 -- 172.21.15.102:0/44119042 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00980752b0 con 0x7f00980caec0 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.675+0000 7f0097400640 1 --2- 172.21.15.102:0/44119042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f009815b460 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0098073f40 tx=0x7f008402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27bc1fb2c0cebf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.675+0000 7f0096a00640 1 -- 172.21.15.102:0/44119042 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084038070 con 0x7f00980caec0 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.675+0000 7f0096a00640 1 -- 172.21.15.102:0/44119042 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0084031710 con 0x7f00980caec0 2024-04-20T21:35:59.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.675+0000 7f0096a00640 1 -- 172.21.15.102:0/44119042 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084031ae0 con 0x7f00980caec0 2024-04-20T21:35:59.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.675+0000 7f009d800640 1 -- 172.21.15.102:0/44119042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 msgr2=0x7f009815b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:59.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.675+0000 7f009d800640 1 --2- 172.21.15.102:0/44119042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f009815b460 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0098073f40 tx=0x7f008402b7b0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.676+0000 7f009d800640 1 -- 172.21.15.102:0/44119042 shutdown_connections 2024-04-20T21:35:59.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.676+0000 7f009d800640 1 --2- 172.21.15.102:0/44119042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f009815b460 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.676+0000 7f009d800640 1 --2- 172.21.15.102:0/44119042 >> v2:172.21.15.193:3300/0 conn(0x7f00981552a0 0x7f00980c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.676+0000 7f009d800640 1 -- 172.21.15.102:0/44119042 >> 172.21.15.102:0/44119042 conn(0x7f00980ca3a0 msgr2=0x7f00980ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:59.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.676+0000 7f009d800640 1 -- 172.21.15.102:0/44119042 shutdown_connections 2024-04-20T21:35:59.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.676+0000 7f009d800640 1 -- 172.21.15.102:0/44119042 wait complete. 2024-04-20T21:35:59.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.677+0000 7f009d800640 1 Processor -- start 2024-04-20T21:35:59.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.677+0000 7f009d800640 1 -- start start 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.678+0000 7f009d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f00981f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.678+0000 7f009d800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00981552a0 0x7f0098145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.678+0000 7f009d800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009815b9a0 0x7f009806f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.678+0000 7f009d800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f00980729c0 con 0x7f00980caec0 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.678+0000 7f009d800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0098075870 con 0x7f009815b9a0 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.678+0000 7f009d800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0098155680 con 0x7f00981552a0 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f00981f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00981552a0 0x7f0098145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:59.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f00981f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51330/0 (socket says 172.21.15.102:51330) 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097e00640 1 -- 172.21.15.102:0/2983483973 learned_addr learned my addr 172.21.15.102:0/2983483973 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f009ce00640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009815b9a0 0x7f009806f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097e00640 1 -- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00981552a0 msgr2=0x7f0098145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097e00640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00981552a0 0x7f0098145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097e00640 1 -- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009815b9a0 msgr2=0x7f009806f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097e00640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009815b9a0 0x7f009806f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.679+0000 7f0097e00640 1 -- 172.21.15.102:0/2983483973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009814f750 con 0x7f00980caec0 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.680+0000 7f0097e00640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f00981f4ef0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f008800e9e0 tx=0x7f008800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.680+0000 7f008fe00640 1 -- 172.21.15.102:0/2983483973 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0088017070 con 0x7f00980caec0 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.680+0000 7f008fe00640 1 -- 172.21.15.102:0/2983483973 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008800c9d0 con 0x7f00980caec0 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.680+0000 7f008fe00640 1 -- 172.21.15.102:0/2983483973 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008800cdc0 con 0x7f00980caec0 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.680+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009814fa30 con 0x7f00980caec0 2024-04-20T21:35:59.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.680+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0098158c10 con 0x7f00980caec0 2024-04-20T21:35:59.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.681+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009804fd60 con 0x7f00980caec0 2024-04-20T21:35:59.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.682+0000 7f008fe00640 1 -- 172.21.15.102:0/2983483973 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f008801a070 con 0x7f00980caec0 2024-04-20T21:35:59.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.683+0000 7f008fe00640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f007806c8f0 0x7f007806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:59.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.683+0000 7f008fe00640 1 -- 172.21.15.102:0/2983483973 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f008808c640 con 0x7f00980caec0 2024-04-20T21:35:59.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.684+0000 7f0097400640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f007806c8f0 0x7f007806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:59.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.685+0000 7f0097400640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f007806c8f0 0x7f007806edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0084035000 tx=0x7f0084008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:59.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.688+0000 7f008fe00640 1 -- 172.21.15.102:0/2983483973 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0088059c20 con 0x7f00980caec0 2024-04-20T21:35:59.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.939+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00980c8830 con 0x7f00980caec0 2024-04-20T21:35:59.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.940+0000 7f008fe00640 1 -- 172.21.15.102:0/2983483973 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6023 (secure 0 0 0) 0x7f008805df20 con 0x7f00980caec0 2024-04-20T21:35:59.939 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:35:59.939 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:50.751041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:35:59.939 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 62 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f007806c8f0 msgr2=0x7f007806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f007806c8f0 0x7f007806edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0084035000 tx=0x7f0084008030 comp rx=0 tx=0).stop 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 msgr2=0x7f00981f4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f00981f4ef0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f008800e9e0 tx=0x7f008800eea0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 shutdown_connections 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f007806c8f0 0x7f007806edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009815b9a0 0x7f009806f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00981552a0 0x7f0098145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 --2- 172.21.15.102:0/2983483973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00980caec0 0x7f00981f4ef0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.943+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 >> 172.21.15.102:0/2983483973 conn(0x7f00980ca3a0 msgr2=0x7f00981540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.944+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 shutdown_connections 2024-04-20T21:35:59.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:35:59.944+0000 7f009d800640 1 -- 172.21.15.102:0/2983483973 wait complete. 2024-04-20T21:35:59.967 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:36:01.968 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:02.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.148+0000 7f6148a00640 1 Processor -- start 2024-04-20T21:36:02.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.148+0000 7f6148a00640 1 -- start start 2024-04-20T21:36:02.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.149+0000 7f6148a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6144072940 0x7f614415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:02.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6148a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614415f940 con 0x7f61440c7900 2024-04-20T21:36:02.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6148a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614415eb60 con 0x7f61440cd940 2024-04-20T21:36:02.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6148a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f614415ee00 con 0x7f6144072940 2024-04-20T21:36:02.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6141e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f61440c7900 0x7f6144074ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37924/0 (socket says 172.21.15.102:37924) 2024-04-20T21:36:02.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6141e00640 1 -- 172.21.15.102:0/95215587 learned_addr learned my addr 172.21.15.102:0/95215587 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:36:02.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6142800640 1 --2- 172.21.15.102:0/95215587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6144072940 0x7f614415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:02.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6142800640 1 -- 172.21.15.102:0/95215587 >> v1:172.21.15.193:6789/0 conn(0x7f61440cd940 legacy=0x7f6144072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:02.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6142800640 1 -- 172.21.15.102:0/95215587 >> v1:172.21.15.102:6789/0 conn(0x7f61440c7900 legacy=0x7f6144074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:02.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6140a00640 1 -- 172.21.15.102:0/95215587 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 182072780 0 0) 0x7f614415eb60 con 0x7f61440cd940 2024-04-20T21:36:02.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.150+0000 7f6142800640 1 -- 172.21.15.102:0/95215587 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614415f0a0 con 0x7f6144072940 2024-04-20T21:36:02.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.151+0000 7f6142800640 1 --2- 172.21.15.102:0/95215587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6144072940 0x7f614415bee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f613c009dc0 tx=0x7f613c02bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42475688a9e8278a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:02.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.152+0000 7f6140a00640 1 -- 172.21.15.102:0/95215587 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c039070 con 0x7f6144072940 2024-04-20T21:36:02.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.152+0000 7f6140a00640 1 -- 172.21.15.102:0/95215587 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c02cb10 con 0x7f6144072940 2024-04-20T21:36:02.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.152+0000 7f6140a00640 1 -- 172.21.15.102:0/95215587 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c034440 con 0x7f6144072940 2024-04-20T21:36:02.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.152+0000 7f6148a00640 1 -- 172.21.15.102:0/95215587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6144072940 msgr2=0x7f614415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:02.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.152+0000 7f6148a00640 1 --2- 172.21.15.102:0/95215587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6144072940 0x7f614415bee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f613c009dc0 tx=0x7f613c02bde0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.152+0000 7f6148a00640 1 -- 172.21.15.102:0/95215587 shutdown_connections 2024-04-20T21:36:02.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.152+0000 7f6148a00640 1 --2- 172.21.15.102:0/95215587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6144072940 0x7f614415bee0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.152+0000 7f6148a00640 1 -- 172.21.15.102:0/95215587 >> 172.21.15.102:0/95215587 conn(0x7f614406dad0 msgr2=0x7f614406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:02.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.153+0000 7f6148a00640 1 -- 172.21.15.102:0/95215587 shutdown_connections 2024-04-20T21:36:02.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.153+0000 7f6148a00640 1 -- 172.21.15.102:0/95215587 wait complete. 2024-04-20T21:36:02.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.154+0000 7f6148a00640 1 Processor -- start 2024-04-20T21:36:02.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.154+0000 7f6148a00640 1 -- start start 2024-04-20T21:36:02.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6148a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6144072940 0x7f61441f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:02.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6148a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61440c7900 0x7f6144155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:02.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6148a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61440cd940 0x7f61441f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:02.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6148a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f61440755b0 con 0x7f6144072940 2024-04-20T21:36:02.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6148a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f614415f660 con 0x7f61440c7900 2024-04-20T21:36:02.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6148a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6144075730 con 0x7f61440cd940 2024-04-20T21:36:02.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6141e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6144072940 0x7f61441f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:02.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6141e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6144072940 0x7f61441f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51340/0 (socket says 172.21.15.102:51340) 2024-04-20T21:36:02.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6141e00640 1 -- 172.21.15.102:0/1582847919 learned_addr learned my addr 172.21.15.102:0/1582847919 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:02.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6142800640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61440cd940 0x7f61441f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:02.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.155+0000 7f6141400640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61440c7900 0x7f6144155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:02.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f6142800640 1 -- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61440c7900 msgr2=0x7f6144155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:02.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f6142800640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61440c7900 0x7f6144155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f6142800640 1 -- 172.21.15.102:0/1582847919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6144072940 msgr2=0x7f61441f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:02.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f6142800640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6144072940 0x7f61441f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f6142800640 1 -- 172.21.15.102:0/1582847919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6144158710 con 0x7f61440cd940 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f6141e00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6144072940 0x7f61441f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f6142800640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61440cd940 0x7f61441f1720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6144075980 tx=0x7f613c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f613a000640 1 -- 172.21.15.102:0/1582847919 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c044070 con 0x7f61440cd940 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.156+0000 7f613a000640 1 -- 172.21.15.102:0/1582847919 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c003b90 con 0x7f61440cd940 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.157+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61441f0c60 con 0x7f61440cd940 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.157+0000 7f613a000640 1 -- 172.21.15.102:0/1582847919 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c039040 con 0x7f61440cd940 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.157+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6144158c60 con 0x7f61440cd940 2024-04-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.158+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6144069550 con 0x7f61440cd940 2024-04-20T21:36:02.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.159+0000 7f613a000640 1 -- 172.21.15.102:0/1582847919 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f613c02c9d0 con 0x7f61440cd940 2024-04-20T21:36:02.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.160+0000 7f613a000640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f612406c8f0 0x7f612406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:02.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.160+0000 7f613a000640 1 -- 172.21.15.102:0/1582847919 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f613c0aecd0 con 0x7f61440cd940 2024-04-20T21:36:02.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.160+0000 7f6141e00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f612406c8f0 0x7f612406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:02.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.161+0000 7f6141e00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f612406c8f0 0x7f612406edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6134000f60 tx=0x7f6134000bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:02.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.165+0000 7f613a000640 1 -- 172.21.15.102:0/1582847919 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6144069550 con 0x7f61440cd940 2024-04-20T21:36:02.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.400+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61440c8e20 con 0x7f61440cd940 2024-04-20T21:36:02.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.401+0000 7f613a000640 1 -- 172.21.15.102:0/1582847919 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6023 (secure 0 0 0) 0x7f613c0ace20 con 0x7f61440cd940 2024-04-20T21:36:02.400 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:02.400 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:35:50.751041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:36:02.401 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 62 2024-04-20T21:36:02.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f612406c8f0 msgr2=0x7f612406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:02.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f612406c8f0 0x7f612406edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6134000f60 tx=0x7f6134000bc0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61440cd940 msgr2=0x7f61441f1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:02.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61440cd940 0x7f61441f1720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6144075980 tx=0x7f613c004a40 comp rx=0 tx=0).stop 2024-04-20T21:36:02.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 shutdown_connections 2024-04-20T21:36:02.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61440cd940 0x7f61441f1720 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f612406c8f0 0x7f612406edb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61440c7900 0x7f6144155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 --2- 172.21.15.102:0/1582847919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6144072940 0x7f61441f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:02.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 >> 172.21.15.102:0/1582847919 conn(0x7f614406dad0 msgr2=0x7f61440c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:02.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 shutdown_connections 2024-04-20T21:36:02.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:02.404+0000 7f6148a00640 1 -- 172.21.15.102:0/1582847919 wait complete. 2024-04-20T21:36:02.428 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:36:04.429 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:04.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.639+0000 7f7a78e00640 1 Processor -- start 2024-04-20T21:36:04.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.639+0000 7f7a78e00640 1 -- start start 2024-04-20T21:36:04.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.640+0000 7f7a78e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:04.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.640+0000 7f7a78e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a741536d0 con 0x7f7a7414fa50 2024-04-20T21:36:04.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.640+0000 7f7a78e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a740c76c0 con 0x7f7a74073a10 2024-04-20T21:36:04.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.640+0000 7f7a78e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7a740c7960 con 0x7f7a740c9eb0 2024-04-20T21:36:04.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a71e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7a7414fa50 0x7f7a740c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37926/0 (socket says 172.21.15.102:37926) 2024-04-20T21:36:04.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a71e00640 1 -- 172.21.15.102:0/2837169771 learned_addr learned my addr 172.21.15.102:0/2837169771 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:36:04.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a69400640 1 --2- 172.21.15.102:0/2837169771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:04.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a71400640 1 -- 172.21.15.102:0/2837169771 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1424509548 0 0) 0x7f7a740c76c0 con 0x7f7a74073a10 2024-04-20T21:36:04.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a71400640 1 -- 172.21.15.102:0/2837169771 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a58003520 con 0x7f7a74073a10 2024-04-20T21:36:04.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a71400640 1 -- 172.21.15.102:0/2837169771 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342727161 0 0) 0x7f7a64001110 con 0x7f7a7414fa50 2024-04-20T21:36:04.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a71400640 1 -- 172.21.15.102:0/2837169771 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a740c76c0 con 0x7f7a7414fa50 2024-04-20T21:36:04.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a69400640 1 -- 172.21.15.102:0/2837169771 >> v1:172.21.15.193:6789/0 conn(0x7f7a74073a10 legacy=0x7f7a74163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:04.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a69400640 1 -- 172.21.15.102:0/2837169771 >> v1:172.21.15.102:6789/0 conn(0x7f7a7414fa50 legacy=0x7f7a740c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:04.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.641+0000 7f7a69400640 1 -- 172.21.15.102:0/2837169771 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a740c3d90 con 0x7f7a740c9eb0 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.642+0000 7f7a69400640 1 --2- 172.21.15.102:0/2837169771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740c4830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a740c3f60 tx=0x7f7a5c02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a68ada5bb1e2a8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.643+0000 7f7a71400640 1 -- 172.21.15.102:0/2837169771 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c039070 con 0x7f7a740c9eb0 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.643+0000 7f7a71400640 1 -- 172.21.15.102:0/2837169771 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c02cd50 con 0x7f7a740c9eb0 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.643+0000 7f7a71400640 1 -- 172.21.15.102:0/2837169771 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c034640 con 0x7f7a740c9eb0 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.643+0000 7f7a78e00640 1 -- 172.21.15.102:0/2837169771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 msgr2=0x7f7a740c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.643+0000 7f7a78e00640 1 --2- 172.21.15.102:0/2837169771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740c4830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a740c3f60 tx=0x7f7a5c02bfa0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.643+0000 7f7a78e00640 1 -- 172.21.15.102:0/2837169771 shutdown_connections 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.643+0000 7f7a78e00640 1 --2- 172.21.15.102:0/2837169771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740c4830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.643+0000 7f7a78e00640 1 -- 172.21.15.102:0/2837169771 >> 172.21.15.102:0/2837169771 conn(0x7f7a740c9390 msgr2=0x7f7a740c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:04.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.644+0000 7f7a78e00640 1 -- 172.21.15.102:0/2837169771 shutdown_connections 2024-04-20T21:36:04.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.644+0000 7f7a78e00640 1 -- 172.21.15.102:0/2837169771 wait complete. 2024-04-20T21:36:04.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.645+0000 7f7a78e00640 1 Processor -- start 2024-04-20T21:36:04.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.645+0000 7f7a78e00640 1 -- start start 2024-04-20T21:36:04.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a78e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a74073a10 0x7f7a74072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a78e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a78e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a7414fa50 0x7f7a740cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a78e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7a740c4030 con 0x7f7a7414fa50 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a78e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a740c7c00 con 0x7f7a74073a10 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a78e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7a740c4310 con 0x7f7a740c9eb0 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a74073a10 0x7f7a74072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a69400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a74073a10 0x7f7a74072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:48442/0 (socket says 172.21.15.102:48442) 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.647+0000 7f7a71e00640 1 -- 172.21.15.102:0/2090528570 learned_addr learned my addr 172.21.15.102:0/2090528570 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.646+0000 7f7a72800640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a7414fa50 0x7f7a740cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.647+0000 7f7a69400640 1 -- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a74073a10 msgr2=0x7f7a74072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:04.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.647+0000 7f7a69400640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a74073a10 0x7f7a74072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.647+0000 7f7a69400640 1 -- 172.21.15.102:0/2090528570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a7414fa50 msgr2=0x7f7a740cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.647+0000 7f7a69400640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a7414fa50 0x7f7a740cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.647+0000 7f7a69400640 1 -- 172.21.15.102:0/2090528570 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a741f5630 con 0x7f7a740c9eb0 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.647+0000 7f7a72800640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a7414fa50 0x7f7a740cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.647+0000 7f7a69400640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740cb6a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c033000 tx=0x7f7a5c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.648+0000 7f7a6aa00640 1 -- 172.21.15.102:0/2090528570 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c039050 con 0x7f7a740c9eb0 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.648+0000 7f7a6aa00640 1 -- 172.21.15.102:0/2090528570 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c045070 con 0x7f7a740c9eb0 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.648+0000 7f7a6aa00640 1 -- 172.21.15.102:0/2090528570 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c009450 con 0x7f7a740c9eb0 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.648+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a74071a50 con 0x7f7a740c9eb0 2024-04-20T21:36:04.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.648+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a74072010 con 0x7f7a740c9eb0 2024-04-20T21:36:04.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.649+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7406f2a0 con 0x7f7a740c9eb0 2024-04-20T21:36:04.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.650+0000 7f7a6aa00640 1 -- 172.21.15.102:0/2090528570 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a5c03e850 con 0x7f7a740c9eb0 2024-04-20T21:36:04.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.651+0000 7f7a6aa00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7a5006c920 0x7f7a5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:04.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.651+0000 7f7a6aa00640 1 -- 172.21.15.102:0/2090528570 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f7a5c0aeeb0 con 0x7f7a740c9eb0 2024-04-20T21:36:04.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.651+0000 7f7a71e00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7a5006c920 0x7f7a5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:04.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.652+0000 7f7a71e00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7a5006c920 0x7f7a5006ede0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a640012b0 tx=0x7f7a64007bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:04.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.655+0000 7f7a6aa00640 1 -- 172.21.15.102:0/2090528570 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7a7406f2a0 con 0x7f7a740c9eb0 2024-04-20T21:36:04.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.898+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a74074340 con 0x7f7a740c9eb0 2024-04-20T21:36:04.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.899+0000 7f7a6aa00640 1 -- 172.21.15.102:0/2090528570 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+6027 (secure 0 0 0) 0x7f7a5c07c5b0 con 0x7f7a740c9eb0 2024-04-20T21:36:04.898 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:04.899 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:02.568893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:36:04.899 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 63 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.901+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7a5006c920 msgr2=0x7f7a5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.901+0000 7f7a78e00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7a5006c920 0x7f7a5006ede0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a640012b0 tx=0x7f7a64007bd0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.901+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 msgr2=0x7f7a740cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.901+0000 7f7a78e00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740cb6a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c033000 tx=0x7f7a5c004a40 comp rx=0 tx=0).stop 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.902+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 shutdown_connections 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.902+0000 7f7a78e00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a7414fa50 0x7f7a740cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.902+0000 7f7a78e00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a740c9eb0 0x7f7a740cb6a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.902+0000 7f7a78e00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7a5006c920 0x7f7a5006ede0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.902+0000 7f7a78e00640 1 --2- 172.21.15.102:0/2090528570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a74073a10 0x7f7a74072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.902+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 >> 172.21.15.102:0/2090528570 conn(0x7f7a740c9390 msgr2=0x7f7a7414b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:04.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.902+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 shutdown_connections 2024-04-20T21:36:04.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:04.902+0000 7f7a78e00640 1 -- 172.21.15.102:0/2090528570 wait complete. 2024-04-20T21:36:04.921 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 63, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 63, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6147": { "addr": "172.21.15.102:6835/945751373", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 945751373, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 945751373, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6147, "incarnation": 62, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:36:02.568893+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6147 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:36:06.921 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:07.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.132+0000 7f9f68800640 1 Processor -- start 2024-04-20T21:36:07.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.132+0000 7f9f68800640 1 -- start start 2024-04-20T21:36:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.133+0000 7f9f68800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9f6015f2b0 0x7f9f6006d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.133+0000 7f9f68800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f600c8f20 con 0x7f9f6015ee00 2024-04-20T21:36:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f68800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f6006cb80 con 0x7f9f6015ea20 2024-04-20T21:36:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f68800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9f6006ce60 con 0x7f9f6015f2b0 2024-04-20T21:36:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f65e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f9f6015ea20 0x7f9f600c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:46808/0 (socket says 172.21.15.102:46808) 2024-04-20T21:36:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f65e00640 1 -- 172.21.15.102:0/2574429099 learned_addr learned my addr 172.21.15.102:0/2574429099 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:36:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f66800640 1 --2- 172.21.15.102:0/2574429099 >> v2:172.21.15.193:3300/0 conn(0x7f9f6015f2b0 0x7f9f6006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f64a00640 1 -- 172.21.15.102:0/2574429099 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2409503722 0 0) 0x7f9f6006cb80 con 0x7f9f6015ea20 2024-04-20T21:36:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f64a00640 1 -- 172.21.15.102:0/2574429099 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f48003520 con 0x7f9f6015ea20 2024-04-20T21:36:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f66800640 1 -- 172.21.15.102:0/2574429099 >> v1:172.21.15.193:6789/0 conn(0x7f9f6015ea20 legacy=0x7f9f600c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f66800640 1 -- 172.21.15.102:0/2574429099 >> v1:172.21.15.102:6789/0 conn(0x7f9f6015ee00 legacy=0x7f9f60154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.134+0000 7f9f66800640 1 -- 172.21.15.102:0/2574429099 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6006d140 con 0x7f9f6015f2b0 2024-04-20T21:36:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.136+0000 7f9f66800640 1 --2- 172.21.15.102:0/2574429099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015f2b0 0x7f9f6006d940 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009d90 tx=0x7f9f5c02bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d118af6112e4bc02 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.136+0000 7f9f64a00640 1 -- 172.21.15.102:0/2574429099 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c039070 con 0x7f9f6015f2b0 2024-04-20T21:36:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.136+0000 7f9f64a00640 1 -- 172.21.15.102:0/2574429099 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c02cd80 con 0x7f9f6015f2b0 2024-04-20T21:36:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.136+0000 7f9f64a00640 1 -- 172.21.15.102:0/2574429099 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c034640 con 0x7f9f6015f2b0 2024-04-20T21:36:07.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.137+0000 7f9f68800640 1 -- 172.21.15.102:0/2574429099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015f2b0 msgr2=0x7f9f6006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.137+0000 7f9f68800640 1 --2- 172.21.15.102:0/2574429099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015f2b0 0x7f9f6006d940 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009d90 tx=0x7f9f5c02bfd0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.137+0000 7f9f68800640 1 -- 172.21.15.102:0/2574429099 shutdown_connections 2024-04-20T21:36:07.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.137+0000 7f9f68800640 1 --2- 172.21.15.102:0/2574429099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015f2b0 0x7f9f6006d940 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.137+0000 7f9f68800640 1 -- 172.21.15.102:0/2574429099 >> 172.21.15.102:0/2574429099 conn(0x7f9f600c9390 msgr2=0x7f9f600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.137+0000 7f9f68800640 1 -- 172.21.15.102:0/2574429099 shutdown_connections 2024-04-20T21:36:07.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.137+0000 7f9f68800640 1 -- 172.21.15.102:0/2574429099 wait complete. 2024-04-20T21:36:07.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.139+0000 7f9f68800640 1 Processor -- start 2024-04-20T21:36:07.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.139+0000 7f9f68800640 1 -- start start 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.140+0000 7f9f68800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015ea20 0x7f9f60071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.140+0000 7f9f68800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f6015ee00 0x7f9f6014aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.140+0000 7f9f68800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f6015f2b0 0x7f9f6015ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.140+0000 7f9f68800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f600c8c80 con 0x7f9f6015ee00 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.140+0000 7f9f68800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f600c8b00 con 0x7f9f6015ea20 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.140+0000 7f9f68800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9f60155590 con 0x7f9f6015f2b0 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f6015ee00 0x7f9f6014aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015ea20 0x7f9f60071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f66800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f6015f2b0 0x7f9f6015ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015ea20 0x7f9f60071700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:48458/0 (socket says 172.21.15.102:48458) 2024-04-20T21:36:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65e00640 1 -- 172.21.15.102:0/3187046686 learned_addr learned my addr 172.21.15.102:0/3187046686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65e00640 1 -- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f6015f2b0 msgr2=0x7f9f6015ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65e00640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f6015f2b0 0x7f9f6015ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65e00640 1 -- 172.21.15.102:0/3187046686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f6015ee00 msgr2=0x7f9f6014aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65e00640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f6015ee00 0x7f9f6014aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65e00640 1 -- 172.21.15.102:0/3187046686 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f60147ff0 con 0x7f9f6015ea20 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f66800640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f6015f2b0 0x7f9f6015ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.141+0000 7f9f65400640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f6015ee00 0x7f9f6014aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.142+0000 7f9f65e00640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015ea20 0x7f9f60071700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c500 tx=0x7f9f5400c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.142+0000 7f9f5a000640 1 -- 172.21.15.102:0/3187046686 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54017070 con 0x7f9f6015ea20 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.142+0000 7f9f5a000640 1 -- 172.21.15.102:0/3187046686 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f54005ce0 con 0x7f9f6015ea20 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.142+0000 7f9f5a000640 1 -- 172.21.15.102:0/3187046686 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54007330 con 0x7f9f6015ea20 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.142+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f601482b0 con 0x7f9f6015ea20 2024-04-20T21:36:07.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.142+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f60148870 con 0x7f9f6015ea20 2024-04-20T21:36:07.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.143+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f60069340 con 0x7f9f6015ea20 2024-04-20T21:36:07.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.144+0000 7f9f5a000640 1 -- 172.21.15.102:0/3187046686 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f5401a070 con 0x7f9f6015ea20 2024-04-20T21:36:07.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.145+0000 7f9f5a000640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f4406c920 0x7f9f4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.145+0000 7f9f5a000640 1 -- 172.21.15.102:0/3187046686 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f9f5408b640 con 0x7f9f6015ea20 2024-04-20T21:36:07.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.145+0000 7f9f65400640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f4406c920 0x7f9f4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.146+0000 7f9f65400640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f4406c920 0x7f9f4406ede0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9f601eead0 tx=0x7f9f50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.149+0000 7f9f5a000640 1 -- 172.21.15.102:0/3187046686 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9f60069340 con 0x7f9f6015ea20 2024-04-20T21:36:07.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.388+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f60074340 con 0x7f9f6015ea20 2024-04-20T21:36:07.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.389+0000 7f9f5a000640 1 -- 172.21.15.102:0/3187046686 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+6027 (secure 0 0 0) 0x7f9f54058d70 con 0x7f9f6015ea20 2024-04-20T21:36:07.388 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:07.388 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:02.568893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:36:07.388 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 63 2024-04-20T21:36:07.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f4406c920 msgr2=0x7f9f4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f4406c920 0x7f9f4406ede0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9f601eead0 tx=0x7f9f50009040 comp rx=0 tx=0).stop 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015ea20 msgr2=0x7f9f60071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015ea20 0x7f9f60071700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c500 tx=0x7f9f5400c9c0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 shutdown_connections 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f6015f2b0 0x7f9f6015ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f6015ee00 0x7f9f6014aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f4406c920 0x7f9f4406ede0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 --2- 172.21.15.102:0/3187046686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f6015ea20 0x7f9f60071700 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 >> 172.21.15.102:0/3187046686 conn(0x7f9f600c9390 msgr2=0x7f9f60154c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 shutdown_connections 2024-04-20T21:36:07.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:07.392+0000 7f9f68800640 1 -- 172.21.15.102:0/3187046686 wait complete. 2024-04-20T21:36:07.408 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:36:09.410 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.619+0000 7fe76c200640 1 Processor -- start 2024-04-20T21:36:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.619+0000 7fe76c200640 1 -- start start 2024-04-20T21:36:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe76c200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe76415ea20 0x7fe76406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe76c200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe7640710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe76c200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7640c8f20 con 0x7fe76415f2b0 2024-04-20T21:36:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe76c200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fe76406cb80 con 0x7fe76415ea20 2024-04-20T21:36:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe76c200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe76406ce60 con 0x7fe76415ee00 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe769800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe76415ea20 0x7fe76406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe768e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe7640710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe76a200640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fe76415f2b0 0x7fe7640c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43216/0 (socket says 172.21.15.102:43216) 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.621+0000 7fe76a200640 1 -- 172.21.15.102:0/1172826184 learned_addr learned my addr 172.21.15.102:0/1172826184 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.622+0000 7fe763e00640 1 -- 172.21.15.102:0/1172826184 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937300750 0 0) 0x7fe7640c8f20 con 0x7fe76415f2b0 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.622+0000 7fe763e00640 1 -- 172.21.15.102:0/1172826184 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe74c003520 con 0x7fe76415f2b0 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.622+0000 7fe768e00640 1 -- 172.21.15.102:0/1172826184 >> v2:172.21.15.193:3300/0 conn(0x7fe76415ea20 msgr2=0x7fe76406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.622+0000 7fe768e00640 1 --2- 172.21.15.102:0/1172826184 >> v2:172.21.15.193:3300/0 conn(0x7fe76415ea20 0x7fe76406d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.622+0000 7fe768e00640 1 -- 172.21.15.102:0/1172826184 >> v1:172.21.15.193:6789/0 conn(0x7fe76415f2b0 legacy=0x7fe7640c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.622+0000 7fe768e00640 1 -- 172.21.15.102:0/1172826184 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76406d140 con 0x7fe76415ee00 2024-04-20T21:36:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.622+0000 7fe769800640 1 --2- 172.21.15.102:0/1172826184 >> v2:172.21.15.193:3300/0 conn(0x7fe76415ea20 0x7fe76406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.623+0000 7fe768e00640 1 --2- 172.21.15.102:0/1172826184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe7640710a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7500097a0 tx=0x7fe75002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa755521d07d7c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.623+0000 7fe763e00640 1 -- 172.21.15.102:0/1172826184 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750034070 con 0x7fe76415ee00 2024-04-20T21:36:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.623+0000 7fe763e00640 1 -- 172.21.15.102:0/1172826184 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe750030800 con 0x7fe76415ee00 2024-04-20T21:36:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.623+0000 7fe763e00640 1 -- 172.21.15.102:0/1172826184 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750030bd0 con 0x7fe76415ee00 2024-04-20T21:36:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.624+0000 7fe76c200640 1 -- 172.21.15.102:0/1172826184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 msgr2=0x7fe7640710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.624+0000 7fe76c200640 1 --2- 172.21.15.102:0/1172826184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe7640710a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7500097a0 tx=0x7fe75002b940 comp rx=0 tx=0).stop 2024-04-20T21:36:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.624+0000 7fe76c200640 1 -- 172.21.15.102:0/1172826184 shutdown_connections 2024-04-20T21:36:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.624+0000 7fe76c200640 1 --2- 172.21.15.102:0/1172826184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe7640710a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.624+0000 7fe76c200640 1 --2- 172.21.15.102:0/1172826184 >> v2:172.21.15.193:3300/0 conn(0x7fe76415ea20 0x7fe76406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.624+0000 7fe76c200640 1 -- 172.21.15.102:0/1172826184 >> 172.21.15.102:0/1172826184 conn(0x7fe7640c9390 msgr2=0x7fe7640c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.624+0000 7fe76c200640 1 -- 172.21.15.102:0/1172826184 shutdown_connections 2024-04-20T21:36:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.624+0000 7fe76c200640 1 -- 172.21.15.102:0/1172826184 wait complete. 2024-04-20T21:36:09.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.626+0000 7fe76c200640 1 Processor -- start 2024-04-20T21:36:09.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.626+0000 7fe76c200640 1 -- start start 2024-04-20T21:36:09.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe76c200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe76415ea20 0x7fe764150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:09.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe76c200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe764146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:09.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe76c200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe76415f2b0 0x7fe764147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe76c200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe7640c8c80 con 0x7fe76415f2b0 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe76c200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7640c8b00 con 0x7fe76415ea20 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe76c200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe764154e80 con 0x7fe76415ee00 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe769800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe76415ea20 0x7fe764150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe768e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe764146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe769800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe76415ea20 0x7fe764150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51376/0 (socket says 172.21.15.102:51376) 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe769800640 1 -- 172.21.15.102:0/3340615133 learned_addr learned my addr 172.21.15.102:0/3340615133 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.627+0000 7fe76a200640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe76415f2b0 0x7fe764147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe769800640 1 -- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 msgr2=0x7fe764146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe769800640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe764146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe769800640 1 -- 172.21.15.102:0/3340615133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe76415f2b0 msgr2=0x7fe764147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe769800640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe76415f2b0 0x7fe764147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe769800640 1 -- 172.21.15.102:0/3340615133 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7641f0a60 con 0x7fe76415ea20 2024-04-20T21:36:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe769800640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe76415ea20 0x7fe764150780 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe75800ec10 tx=0x7fe75800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe761600640 1 -- 172.21.15.102:0/3340615133 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758017070 con 0x7fe76415ea20 2024-04-20T21:36:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe761600640 1 -- 172.21.15.102:0/3340615133 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75800cc30 con 0x7fe76415ea20 2024-04-20T21:36:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe764147c60 con 0x7fe76415ea20 2024-04-20T21:36:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.628+0000 7fe761600640 1 -- 172.21.15.102:0/3340615133 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758013470 con 0x7fe76415ea20 2024-04-20T21:36:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.629+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641f0fc0 con 0x7fe76415ea20 2024-04-20T21:36:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.630+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764074340 con 0x7fe76415ea20 2024-04-20T21:36:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.631+0000 7fe761600640 1 -- 172.21.15.102:0/3340615133 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe75801a070 con 0x7fe76415ea20 2024-04-20T21:36:09.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.632+0000 7fe761600640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe74406c8f0 0x7fe74406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:09.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.632+0000 7fe761600640 1 -- 172.21.15.102:0/3340615133 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7fe75808c240 con 0x7fe76415ea20 2024-04-20T21:36:09.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.632+0000 7fe768e00640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe74406c8f0 0x7fe74406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:09.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.633+0000 7fe768e00640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe74406c8f0 0x7fe74406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe750036000 tx=0x7fe75003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:09.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.636+0000 7fe761600640 1 -- 172.21.15.102:0/3340615133 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7641f0a60 con 0x7fe76415ea20 2024-04-20T21:36:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.866+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe764071930 con 0x7fe76415ea20 2024-04-20T21:36:09.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.867+0000 7fe761600640 1 -- 172.21.15.102:0/3340615133 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+6024 (secure 0 0 0) 0x7fe758059820 con 0x7fe76415ea20 2024-04-20T21:36:09.867 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:07.811382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:36:09.867 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 64 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.871+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe74406c8f0 msgr2=0x7fe74406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.871+0000 7fe76c200640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe74406c8f0 0x7fe74406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe750036000 tx=0x7fe75003b040 comp rx=0 tx=0).stop 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.871+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe76415ea20 msgr2=0x7fe764150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.871+0000 7fe76c200640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe76415ea20 0x7fe764150780 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe75800ec10 tx=0x7fe75800c520 comp rx=0 tx=0).stop 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.872+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 shutdown_connections 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.872+0000 7fe76c200640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe76415f2b0 0x7fe764147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.872+0000 7fe76c200640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe76415ee00 0x7fe764146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.872+0000 7fe76c200640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe74406c8f0 0x7fe74406edb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.872+0000 7fe76c200640 1 --2- 172.21.15.102:0/3340615133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe76415ea20 0x7fe764150780 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:09.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.872+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 >> 172.21.15.102:0/3340615133 conn(0x7fe7640c9390 msgr2=0x7fe764072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:09.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.872+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 shutdown_connections 2024-04-20T21:36:09.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:09.872+0000 7fe76c200640 1 -- 172.21.15.102:0/3340615133 wait complete. 2024-04-20T21:36:09.888 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:36:11.889 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.096+0000 7faaf8000640 1 Processor -- start 2024-04-20T21:36:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.096+0000 7faaf8000640 1 -- start start 2024-04-20T21:36:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.097+0000 7faaf8000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faaf00c8db0 0x7faaf0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.097+0000 7faaf8000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faaf0071330 0x7faaf01628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:12.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.097+0000 7faaf8000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf0158e80 con 0x7faaf0152170 2024-04-20T21:36:12.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.097+0000 7faaf8000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7faaf00c6dc0 con 0x7faaf00c8db0 2024-04-20T21:36:12.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.097+0000 7faaf8000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7faaf0071f70 con 0x7faaf0071330 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf6000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faaf0071330 0x7faaf01628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf6000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faaf0071330 0x7faaf01628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51388/0 (socket says 172.21.15.102:51388) 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf4c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faaf00c8db0 0x7faaf0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf5600640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7faaf0152170 0x7faaf0158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43230/0 (socket says 172.21.15.102:43230) 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf6000640 1 -- 172.21.15.102:0/127056431 learned_addr learned my addr 172.21.15.102:0/127056431 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf6000640 1 -- 172.21.15.102:0/127056431 >> v2:172.21.15.102:3300/0 conn(0x7faaf00c8db0 msgr2=0x7faaf0070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf6000640 1 --2- 172.21.15.102:0/127056431 >> v2:172.21.15.102:3300/0 conn(0x7faaf00c8db0 0x7faaf0070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf6000640 1 -- 172.21.15.102:0/127056431 >> v1:172.21.15.193:6789/0 conn(0x7faaf0152170 legacy=0x7faaf0158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaf6000640 1 -- 172.21.15.102:0/127056431 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf00c91c0 con 0x7faaf0071330 2024-04-20T21:36:12.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.098+0000 7faaefe00640 1 -- 172.21.15.102:0/127056431 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1285166837 0 0) 0x7faaf0158e80 con 0x7faaf0152170 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.099+0000 7faaf6000640 1 --2- 172.21.15.102:0/127056431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01628a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faaf006c190 tx=0x7faae802bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf504452a06512a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaefe00640 1 -- 172.21.15.102:0/127056431 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae802cb20 con 0x7faaf0071330 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaefe00640 1 -- 172.21.15.102:0/127056431 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae802ccc0 con 0x7faaf0071330 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaefe00640 1 -- 172.21.15.102:0/127056431 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8034610 con 0x7faaf0071330 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaf8000640 1 -- 172.21.15.102:0/127056431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 msgr2=0x7faaf01628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaf8000640 1 --2- 172.21.15.102:0/127056431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01628a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faaf006c190 tx=0x7faae802bf80 comp rx=0 tx=0).stop 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaf8000640 1 -- 172.21.15.102:0/127056431 shutdown_connections 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaf8000640 1 --2- 172.21.15.102:0/127056431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01628a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaf8000640 1 --2- 172.21.15.102:0/127056431 >> v2:172.21.15.102:3300/0 conn(0x7faaf00c8db0 0x7faaf0070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.100+0000 7faaf8000640 1 -- 172.21.15.102:0/127056431 >> 172.21.15.102:0/127056431 conn(0x7faaf00c9390 msgr2=0x7faaf00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.101+0000 7faaf8000640 1 -- 172.21.15.102:0/127056431 shutdown_connections 2024-04-20T21:36:12.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.101+0000 7faaf8000640 1 -- 172.21.15.102:0/127056431 wait complete. 2024-04-20T21:36:12.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.102+0000 7faaf8000640 1 Processor -- start 2024-04-20T21:36:12.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.102+0000 7faaf8000640 1 -- start start 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.103+0000 7faaf8000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.103+0000 7faaf8000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaf00c8db0 0x7faaf0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.103+0000 7faaf8000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf0152170 0x7faaf01f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.103+0000 7faaf8000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faaf00724b0 con 0x7faaf0152170 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.103+0000 7faaf8000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0072210 con 0x7faaf0071330 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.103+0000 7faaf8000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faaf0158250 con 0x7faaf00c8db0 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01f1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51398/0 (socket says 172.21.15.102:51398) 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 -- 172.21.15.102:0/3708920494 learned_addr learned my addr 172.21.15.102:0/3708920494 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf6000640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf0152170 0x7faaf01f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf4c00640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaf00c8db0 0x7faaf0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:12.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 -- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaf00c8db0 msgr2=0x7faaf0159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaf00c8db0 0x7faaf0159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 -- 172.21.15.102:0/3708920494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf0152170 msgr2=0x7faaf01f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf0152170 0x7faaf01f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 -- 172.21.15.102:0/3708920494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf014e9c0 con 0x7faaf0071330 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.104+0000 7faaf5600640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01f1330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faae400c530 tx=0x7faae400c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.105+0000 7faaed600640 1 -- 172.21.15.102:0/3708920494 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4018070 con 0x7faaf0071330 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.105+0000 7faaed600640 1 -- 172.21.15.102:0/3708920494 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae4007500 con 0x7faaf0071330 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.105+0000 7faaed600640 1 -- 172.21.15.102:0/3708920494 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4004a20 con 0x7faaf0071330 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.105+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf00c7460 con 0x7faaf0071330 2024-04-20T21:36:12.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.105+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf00c7700 con 0x7faaf0071330 2024-04-20T21:36:12.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.106+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf0074340 con 0x7faaf0071330 2024-04-20T21:36:12.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.107+0000 7faaed600640 1 -- 172.21.15.102:0/3708920494 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faae4004bc0 con 0x7faaf0071330 2024-04-20T21:36:12.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.108+0000 7faaed600640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:12.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.108+0000 7faaed600640 1 -- 172.21.15.102:0/3708920494 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7faae408b530 con 0x7faaf0071330 2024-04-20T21:36:12.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.108+0000 7faaf4c00640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:12.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.108+0000 7faaf4c00640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faad006c8f0 0x7faad006edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faaf0146680 tx=0x7faadc009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:12.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.112+0000 7faaed600640 1 -- 172.21.15.102:0/3708920494 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faae4058b20 con 0x7faaf0071330 2024-04-20T21:36:12.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.353+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaf00c3460 con 0x7faaf0071330 2024-04-20T21:36:12.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.354+0000 7faaed600640 1 -- 172.21.15.102:0/3708920494 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+6024 (secure 0 0 0) 0x7faae405ce20 con 0x7faaf0071330 2024-04-20T21:36:12.353 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:07.811382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:36:12.354 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 64 2024-04-20T21:36:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faad006c8f0 msgr2=0x7faad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faad006c8f0 0x7faad006edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faaf0146680 tx=0x7faadc009190 comp rx=0 tx=0).stop 2024-04-20T21:36:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 msgr2=0x7faaf01f1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01f1330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faae400c530 tx=0x7faae400c9f0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 shutdown_connections 2024-04-20T21:36:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf0152170 0x7faaf01f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaf00c8db0 0x7faaf0159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 --2- 172.21.15.102:0/3708920494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaf0071330 0x7faaf01f1330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 >> 172.21.15.102:0/3708920494 conn(0x7faaf00c9390 msgr2=0x7faaf006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 shutdown_connections 2024-04-20T21:36:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:12.357+0000 7faaf8000640 1 -- 172.21.15.102:0/3708920494 wait complete. 2024-04-20T21:36:12.374 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:36:14.375 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:14.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.584+0000 7f4997200640 1 Processor -- start 2024-04-20T21:36:14.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.584+0000 7f4997200640 1 -- start start 2024-04-20T21:36:14.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.585+0000 7f4997200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f49980c3700 0x7f49980cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.585+0000 7f4997200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f49980c8330 0x7f49980c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.585+0000 7f4997200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499815c220 con 0x7f4998152290 2024-04-20T21:36:14.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.585+0000 7f4997200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f49980c9180 con 0x7f49980c8330 2024-04-20T21:36:14.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.586+0000 7f4997200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f49980c4840 con 0x7f49980c3700 2024-04-20T21:36:14.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.586+0000 7f4996800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f49980c8330 0x7f49980c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.586+0000 7f4996800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f49980c8330 0x7f49980c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43514/0 (socket says 172.21.15.102:43514) 2024-04-20T21:36:14.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.586+0000 7f4996800640 1 -- 172.21.15.102:0/2717314331 learned_addr learned my addr 172.21.15.102:0/2717314331 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:14.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.586+0000 7f498d400640 1 --2- 172.21.15.102:0/2717314331 >> v2:172.21.15.193:3300/0 conn(0x7f49980c3700 0x7f49980cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.587+0000 7f4996800640 1 -- 172.21.15.102:0/2717314331 >> v2:172.21.15.193:3300/0 conn(0x7f49980c3700 msgr2=0x7f49980cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.587+0000 7f4996800640 1 --2- 172.21.15.102:0/2717314331 >> v2:172.21.15.193:3300/0 conn(0x7f49980c3700 0x7f49980cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.587+0000 7f4996800640 1 -- 172.21.15.102:0/2717314331 >> v1:172.21.15.193:6789/0 conn(0x7f4998152290 legacy=0x7f499806e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.587+0000 7f4996800640 1 -- 172.21.15.102:0/2717314331 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49980c4560 con 0x7f49980c8330 2024-04-20T21:36:14.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.588+0000 7f4996800640 1 --2- 172.21.15.102:0/2717314331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49980c8330 0x7f49980c8740 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4998069910 tx=0x7f499002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460b0e11e7665400 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.588+0000 7f4995400640 1 -- 172.21.15.102:0/2717314331 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499002cbb0 con 0x7f49980c8330 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.588+0000 7f4995400640 1 -- 172.21.15.102:0/2717314331 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499002cd50 con 0x7f49980c8330 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.588+0000 7f4995400640 1 -- 172.21.15.102:0/2717314331 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990034610 con 0x7f49980c8330 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.589+0000 7f4997200640 1 -- 172.21.15.102:0/2717314331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49980c8330 msgr2=0x7f49980c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.589+0000 7f4997200640 1 --2- 172.21.15.102:0/2717314331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49980c8330 0x7f49980c8740 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4998069910 tx=0x7f499002c010 comp rx=0 tx=0).stop 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.589+0000 7f4997200640 1 -- 172.21.15.102:0/2717314331 shutdown_connections 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.589+0000 7f4997200640 1 --2- 172.21.15.102:0/2717314331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49980c8330 0x7f49980c8740 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.589+0000 7f4997200640 1 --2- 172.21.15.102:0/2717314331 >> v2:172.21.15.193:3300/0 conn(0x7f49980c3700 0x7f49980cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.589+0000 7f4997200640 1 -- 172.21.15.102:0/2717314331 >> 172.21.15.102:0/2717314331 conn(0x7f49980cc820 msgr2=0x7f49980ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.589+0000 7f4997200640 1 -- 172.21.15.102:0/2717314331 shutdown_connections 2024-04-20T21:36:14.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.589+0000 7f4997200640 1 -- 172.21.15.102:0/2717314331 wait complete. 2024-04-20T21:36:14.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.591+0000 7f4997200640 1 Processor -- start 2024-04-20T21:36:14.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.591+0000 7f4997200640 1 -- start start 2024-04-20T21:36:14.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4997200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49980c3700 0x7f499815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4997200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49980c8330 0x7f49981619d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4997200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4998152290 0x7f4998161f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4997200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f499815ba20 con 0x7f4998152290 2024-04-20T21:36:14.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4997200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f499815b740 con 0x7f49980c8330 2024-04-20T21:36:14.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4997200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f499815bd00 con 0x7f49980c3700 2024-04-20T21:36:14.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f498d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49980c8330 0x7f49981619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4995e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49980c3700 0x7f499815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f498d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49980c8330 0x7f49981619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51414/0 (socket says 172.21.15.102:51414) 2024-04-20T21:36:14.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4995e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49980c3700 0x7f499815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:54142/0 (socket says 172.21.15.102:54142) 2024-04-20T21:36:14.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f498d400640 1 -- 172.21.15.102:0/2296144327 learned_addr learned my addr 172.21.15.102:0/2296144327 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:14.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.592+0000 7f4996800640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4998152290 0x7f4998161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f498d400640 1 -- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49980c3700 msgr2=0x7f499815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f498d400640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49980c3700 0x7f499815a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f498d400640 1 -- 172.21.15.102:0/2296144327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4998152290 msgr2=0x7f4998161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f498d400640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4998152290 0x7f4998161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f498d400640 1 -- 172.21.15.102:0/2296144327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49981655f0 con 0x7f49980c8330 2024-04-20T21:36:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f4995e00640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49980c3700 0x7f499815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f498d400640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49980c8330 0x7f49981619d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f498000e8d0 tx=0x7f498000ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f498ea00640 1 -- 172.21.15.102:0/2296144327 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980017070 con 0x7f49980c8330 2024-04-20T21:36:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.593+0000 7f498ea00640 1 -- 172.21.15.102:0/2296144327 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498000c940 con 0x7f49980c8330 2024-04-20T21:36:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.594+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49981f5060 con 0x7f49980c8330 2024-04-20T21:36:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.594+0000 7f498ea00640 1 -- 172.21.15.102:0/2296144327 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498000cd30 con 0x7f49980c8330 2024-04-20T21:36:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.594+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49981f5620 con 0x7f49980c8330 2024-04-20T21:36:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.595+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499804fd60 con 0x7f49980c8330 2024-04-20T21:36:14.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.596+0000 7f498ea00640 1 -- 172.21.15.102:0/2296144327 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f498001a070 con 0x7f49980c8330 2024-04-20T21:36:14.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.597+0000 7f498ea00640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f497406c8f0 0x7f497406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.597+0000 7f498ea00640 1 -- 172.21.15.102:0/2296144327 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f498008c1e0 con 0x7f49980c8330 2024-04-20T21:36:14.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.597+0000 7f4995e00640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f497406c8f0 0x7f497406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.598+0000 7f4995e00640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f497406c8f0 0x7f497406edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4988001b70 tx=0x7f4988009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.602+0000 7f498ea00640 1 -- 172.21.15.102:0/2296144327 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f49800597f0 con 0x7f49980c8330 2024-04-20T21:36:14.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.853+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f499815b0c0 con 0x7f49980c8330 2024-04-20T21:36:14.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.854+0000 7f498ea00640 1 -- 172.21.15.102:0/2296144327 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+6030 (secure 0 0 0) 0x7f498005daf0 con 0x7f49980c8330 2024-04-20T21:36:14.854 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:13.204954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:36:14.854 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 65 2024-04-20T21:36:14.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.857+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f497406c8f0 msgr2=0x7f497406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.857+0000 7f4997200640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f497406c8f0 0x7f497406edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4988001b70 tx=0x7f4988009040 comp rx=0 tx=0).stop 2024-04-20T21:36:14.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.857+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49980c8330 msgr2=0x7f49981619d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.857+0000 7f4997200640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49980c8330 0x7f49981619d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f498000e8d0 tx=0x7f498000ed90 comp rx=0 tx=0).stop 2024-04-20T21:36:14.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.858+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 shutdown_connections 2024-04-20T21:36:14.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.858+0000 7f4997200640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4998152290 0x7f4998161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.858+0000 7f4997200640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f497406c8f0 0x7f497406edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.858+0000 7f4997200640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49980c8330 0x7f49981619d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.858+0000 7f4997200640 1 --2- 172.21.15.102:0/2296144327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49980c3700 0x7f499815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.858+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 >> 172.21.15.102:0/2296144327 conn(0x7f49980cc820 msgr2=0x7f499806e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:14.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.858+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 shutdown_connections 2024-04-20T21:36:14.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:14.858+0000 7f4997200640 1 -- 172.21.15.102:0/2296144327 wait complete. 2024-04-20T21:36:14.874 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:36:16.875 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:17.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.084+0000 7f5785600640 1 Processor -- start 2024-04-20T21:36:17.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.084+0000 7f5785600640 1 -- start start 2024-04-20T21:36:17.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f5785600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5780155110 0x7f578006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:17.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f5785600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f578006f3d0 0x7f578015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:17.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f5785600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578015f3c0 con 0x7f57800c4040 2024-04-20T21:36:17.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f5785600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f57800c7c30 con 0x7f5780155110 2024-04-20T21:36:17.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f5785600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f578006ea00 con 0x7f578006f3d0 2024-04-20T21:36:17.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f577f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f578006f3d0 0x7f578015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:17.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f577fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5780155110 0x7f578006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:17.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f5784c00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f57800c4040 0x7f578015eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43252/0 (socket says 172.21.15.102:43252) 2024-04-20T21:36:17.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f5784c00640 1 -- 172.21.15.102:0/3877095803 learned_addr learned my addr 172.21.15.102:0/3877095803 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:36:17.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f577f400640 1 -- 172.21.15.102:0/3877095803 >> v2:172.21.15.193:3300/0 conn(0x7f5780155110 msgr2=0x7f578006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:17.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.086+0000 7f577f400640 1 --2- 172.21.15.102:0/3877095803 >> v2:172.21.15.193:3300/0 conn(0x7f5780155110 0x7f578006ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.087+0000 7f577f400640 1 -- 172.21.15.102:0/3877095803 >> v1:172.21.15.193:6789/0 conn(0x7f57800c4040 legacy=0x7f578015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:17.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.087+0000 7f577f400640 1 -- 172.21.15.102:0/3877095803 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578006e6d0 con 0x7f578006f3d0 2024-04-20T21:36:17.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.088+0000 7f577f400640 1 --2- 172.21.15.102:0/3877095803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f578006f3d0 0x7f578015e580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57800c7db0 tx=0x7f576802b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a6f1fb3015fd10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:17.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.088+0000 7f577ea00640 1 -- 172.21.15.102:0/3877095803 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768037070 con 0x7f578006f3d0 2024-04-20T21:36:17.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.088+0000 7f577ea00640 1 -- 172.21.15.102:0/3877095803 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5768030710 con 0x7f578006f3d0 2024-04-20T21:36:17.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.088+0000 7f577ea00640 1 -- 172.21.15.102:0/3877095803 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768030ae0 con 0x7f578006f3d0 2024-04-20T21:36:17.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.088+0000 7f5785600640 1 -- 172.21.15.102:0/3877095803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f578006f3d0 msgr2=0x7f578015e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:17.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.088+0000 7f5785600640 1 --2- 172.21.15.102:0/3877095803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f578006f3d0 0x7f578015e580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57800c7db0 tx=0x7f576802b770 comp rx=0 tx=0).stop 2024-04-20T21:36:17.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.089+0000 7f5785600640 1 -- 172.21.15.102:0/3877095803 shutdown_connections 2024-04-20T21:36:17.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.089+0000 7f5785600640 1 --2- 172.21.15.102:0/3877095803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f578006f3d0 0x7f578015e580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.089+0000 7f5785600640 1 --2- 172.21.15.102:0/3877095803 >> v2:172.21.15.193:3300/0 conn(0x7f5780155110 0x7f578006ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.089+0000 7f5785600640 1 -- 172.21.15.102:0/3877095803 >> 172.21.15.102:0/3877095803 conn(0x7f57800cc820 msgr2=0x7f57800ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:17.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.089+0000 7f5785600640 1 -- 172.21.15.102:0/3877095803 shutdown_connections 2024-04-20T21:36:17.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.089+0000 7f5785600640 1 -- 172.21.15.102:0/3877095803 wait complete. 2024-04-20T21:36:17.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.090+0000 7f5785600640 1 Processor -- start 2024-04-20T21:36:17.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.091+0000 7f5785600640 1 -- start start 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.091+0000 7f5785600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f578006f3d0 0x7f57801f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.091+0000 7f5785600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57800c4040 0x7f57800723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.091+0000 7f5785600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5780155110 0x7f5780073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.091+0000 7f5785600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f57801554f0 con 0x7f5780155110 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.091+0000 7f5785600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57800c8160 con 0x7f578006f3d0 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.091+0000 7f5785600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5780155670 con 0x7f57800c4040 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f577f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57800c4040 0x7f57800723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f5784c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5780155110 0x7f5780073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f577f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57800c4040 0x7f57800723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:54162/0 (socket says 172.21.15.102:54162) 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f577f400640 1 -- 172.21.15.102:0/4046907107 learned_addr learned my addr 172.21.15.102:0/4046907107 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:17.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f577fe00640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f578006f3d0 0x7f57801f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:17.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f5784c00640 1 -- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57800c4040 msgr2=0x7f57800723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f5784c00640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57800c4040 0x7f57800723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f5784c00640 1 -- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f578006f3d0 msgr2=0x7f57801f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f5784c00640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f578006f3d0 0x7f57801f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.092+0000 7f5784c00640 1 -- 172.21.15.102:0/4046907107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57800728f0 con 0x7f5780155110 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.093+0000 7f5784c00640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5780155110 0x7f5780073780 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f577800caf0 tx=0x7f5778002060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.093+0000 7f5777e00640 1 -- 172.21.15.102:0/4046907107 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778018070 con 0x7f5780155110 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.093+0000 7f5777e00640 1 -- 172.21.15.102:0/4046907107 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577800a700 con 0x7f5780155110 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.093+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5780159890 con 0x7f5780155110 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.093+0000 7f5777e00640 1 -- 172.21.15.102:0/4046907107 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577800aad0 con 0x7f5780155110 2024-04-20T21:36:17.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.093+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5780071310 con 0x7f5780155110 2024-04-20T21:36:17.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.094+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578004fd60 con 0x7f5780155110 2024-04-20T21:36:17.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.095+0000 7f5777e00640 1 -- 172.21.15.102:0/4046907107 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f577801b070 con 0x7f5780155110 2024-04-20T21:36:17.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.096+0000 7f5777e00640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f575c06c8f0 0x7f575c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:17.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.096+0000 7f5777e00640 1 -- 172.21.15.102:0/4046907107 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f577808b910 con 0x7f5780155110 2024-04-20T21:36:17.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.096+0000 7f577fe00640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f575c06c8f0 0x7f575c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:17.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.097+0000 7f577fe00640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f575c06c8f0 0x7f575c06edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5770009aa0 tx=0x7f5770009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:17.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.100+0000 7f5777e00640 1 -- 172.21.15.102:0/4046907107 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f577800ac70 con 0x7f5780155110 2024-04-20T21:36:17.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.338+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57801589b0 con 0x7f5780155110 2024-04-20T21:36:17.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.339+0000 7f5777e00640 1 -- 172.21.15.102:0/4046907107 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+6030 (secure 0 0 0) 0x7f5778058f90 con 0x7f5780155110 2024-04-20T21:36:17.338 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:17.339 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:13.204954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:36:17.339 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 65 2024-04-20T21:36:17.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.341+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f575c06c8f0 msgr2=0x7f575c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:17.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.341+0000 7f5785600640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f575c06c8f0 0x7f575c06edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5770009aa0 tx=0x7f5770009190 comp rx=0 tx=0).stop 2024-04-20T21:36:17.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.341+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5780155110 msgr2=0x7f5780073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:17.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.341+0000 7f5785600640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5780155110 0x7f5780073780 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f577800caf0 tx=0x7f5778002060 comp rx=0 tx=0).stop 2024-04-20T21:36:17.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.342+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 shutdown_connections 2024-04-20T21:36:17.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.342+0000 7f5785600640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5780155110 0x7f5780073780 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.342+0000 7f5785600640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57800c4040 0x7f57800723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.342+0000 7f5785600640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f575c06c8f0 0x7f575c06edb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.342+0000 7f5785600640 1 --2- 172.21.15.102:0/4046907107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f578006f3d0 0x7f57801f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:17.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.342+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 >> 172.21.15.102:0/4046907107 conn(0x7f57800cc820 msgr2=0x7f5780152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:17.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.342+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 shutdown_connections 2024-04-20T21:36:17.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:17.342+0000 7f5785600640 1 -- 172.21.15.102:0/4046907107 wait complete. 2024-04-20T21:36:17.361 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 65, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 65, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6147": { "addr": "172.21.15.102:6835/945751373", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 945751373, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 945751373, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6147, "incarnation": 62, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:clientreplay", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:36:13.204954+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6147 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:36:19.362 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:19.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.535+0000 7f9f88000640 1 Processor -- start 2024-04-20T21:36:19.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.535+0000 7f9f88000640 1 -- start start 2024-04-20T21:36:19.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.537+0000 7f9f88000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80163220 0x7f9f80159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:19.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.537+0000 7f9f88000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8006d2e0 con 0x7f9f80162e40 2024-04-20T21:36:19.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.537+0000 7f9f88000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8006d5c0 con 0x7f9f80163660 2024-04-20T21:36:19.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.537+0000 7f9f88000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9f800c7290 con 0x7f9f80163220 2024-04-20T21:36:19.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.537+0000 7f9f85600640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9f80162e40 0x7f9f8006c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56114/0 (socket says 172.21.15.102:56114) 2024-04-20T21:36:19.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.537+0000 7f9f85600640 1 -- 172.21.15.102:0/2191990157 learned_addr learned my addr 172.21.15.102:0/2191990157 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:36:19.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.537+0000 7f9f84c00640 1 --2- 172.21.15.102:0/2191990157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80163220 0x7f9f80159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:19.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.538+0000 7f9f7fe00640 1 -- 172.21.15.102:0/2191990157 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 304026068 0 0) 0x7f9f8006d2e0 con 0x7f9f80162e40 2024-04-20T21:36:19.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.538+0000 7f9f7fe00640 1 -- 172.21.15.102:0/2191990157 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f68003520 con 0x7f9f80162e40 2024-04-20T21:36:19.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.538+0000 7f9f7fe00640 1 -- 172.21.15.102:0/2191990157 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301517548 0 0) 0x7f9f8006d5c0 con 0x7f9f80163660 2024-04-20T21:36:19.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.538+0000 7f9f7fe00640 1 -- 172.21.15.102:0/2191990157 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f8006d2e0 con 0x7f9f80163660 2024-04-20T21:36:19.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.538+0000 7f9f84c00640 1 -- 172.21.15.102:0/2191990157 >> v1:172.21.15.193:6789/0 conn(0x7f9f80163660 legacy=0x7f9f800c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:19.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.538+0000 7f9f84c00640 1 -- 172.21.15.102:0/2191990157 >> v1:172.21.15.102:6789/0 conn(0x7f9f80162e40 legacy=0x7f9f8006c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:19.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.538+0000 7f9f84c00640 1 -- 172.21.15.102:0/2191990157 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8006d8a0 con 0x7f9f80163220 2024-04-20T21:36:19.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.539+0000 7f9f84c00640 1 --2- 172.21.15.102:0/2191990157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80163220 0x7f9f80159440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f700097d0 tx=0x7f9f7002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39a92eb0bdccd09c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:19.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.539+0000 7f9f7fe00640 1 -- 172.21.15.102:0/2191990157 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70035020 con 0x7f9f80163220 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.539+0000 7f9f7fe00640 1 -- 172.21.15.102:0/2191990157 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f70031890 con 0x7f9f80163220 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.539+0000 7f9f7fe00640 1 -- 172.21.15.102:0/2191990157 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70031c60 con 0x7f9f80163220 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.540+0000 7f9f88000640 1 -- 172.21.15.102:0/2191990157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80163220 msgr2=0x7f9f80159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.540+0000 7f9f88000640 1 --2- 172.21.15.102:0/2191990157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80163220 0x7f9f80159440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f700097d0 tx=0x7f9f7002b930 comp rx=0 tx=0).stop 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.540+0000 7f9f88000640 1 -- 172.21.15.102:0/2191990157 shutdown_connections 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.540+0000 7f9f88000640 1 --2- 172.21.15.102:0/2191990157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80163220 0x7f9f80159440 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.540+0000 7f9f88000640 1 -- 172.21.15.102:0/2191990157 >> 172.21.15.102:0/2191990157 conn(0x7f9f800c9390 msgr2=0x7f9f800c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.540+0000 7f9f88000640 1 -- 172.21.15.102:0/2191990157 shutdown_connections 2024-04-20T21:36:19.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.540+0000 7f9f88000640 1 -- 172.21.15.102:0/2191990157 wait complete. 2024-04-20T21:36:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.542+0000 7f9f88000640 1 Processor -- start 2024-04-20T21:36:19.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.542+0000 7f9f88000640 1 -- start start 2024-04-20T21:36:19.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f88000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80162e40 0x7f9f801607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:19.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f88000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f80163220 0x7f9f8014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f88000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f80163660 0x7f9f801f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f88000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f800c9080 con 0x7f9f80163660 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f88000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f800c7570 con 0x7f9f80163220 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f88000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9f800c8110 con 0x7f9f80162e40 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f86000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f80163660 0x7f9f801f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f84c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f80163220 0x7f9f8014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f85600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80162e40 0x7f9f801607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f84c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f80163220 0x7f9f8014de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:35282/0 (socket says 172.21.15.102:35282) 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f84c00640 1 -- 172.21.15.102:0/3262279935 learned_addr learned my addr 172.21.15.102:0/3262279935 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.543+0000 7f9f84c00640 1 -- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80162e40 msgr2=0x7f9f801607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:19.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f84c00640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80162e40 0x7f9f801607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f84c00640 1 -- 172.21.15.102:0/3262279935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f80163660 msgr2=0x7f9f801f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f84c00640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f80163660 0x7f9f801f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f84c00640 1 -- 172.21.15.102:0/3262279935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f80162260 con 0x7f9f80163220 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f86000640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f80163660 0x7f9f801f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f85600640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80162e40 0x7f9f801607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f84c00640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f80163220 0x7f9f8014de60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9f700098f0 tx=0x7f9f700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f7d600640 1 -- 172.21.15.102:0/3262279935 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70035040 con 0x7f9f80163220 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f7d600640 1 -- 172.21.15.102:0/3262279935 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f70030b10 con 0x7f9f80163220 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f7d600640 1 -- 172.21.15.102:0/3262279935 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70044430 con 0x7f9f80163220 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f801f53e0 con 0x7f9f80163220 2024-04-20T21:36:19.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.544+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f801627d0 con 0x7f9f80163220 2024-04-20T21:36:19.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.545+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f80074340 con 0x7f9f80163220 2024-04-20T21:36:19.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.546+0000 7f9f7d600640 1 -- 172.21.15.102:0/3262279935 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f70038020 con 0x7f9f80163220 2024-04-20T21:36:19.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.548+0000 7f9f7d600640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6406c920 0x7f9f6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:19.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.548+0000 7f9f7d600640 1 -- 172.21.15.102:0/3262279935 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f9f700ae650 con 0x7f9f80163220 2024-04-20T21:36:19.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.548+0000 7f9f85600640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6406c920 0x7f9f6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:19.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.548+0000 7f9f85600640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6406c920 0x7f9f6406ede0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9f74000f60 tx=0x7f9f74007d70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:19.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.552+0000 7f9f7d600640 1 -- 172.21.15.102:0/3262279935 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9f7002d040 con 0x7f9f80163220 2024-04-20T21:36:19.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.792+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f801588e0 con 0x7f9f80163220 2024-04-20T21:36:19.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.793+0000 7f9f7d600640 1 -- 172.21.15.102:0/3262279935 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6031 (secure 0 0 0) 0x7f9f7007bcc0 con 0x7f9f80163220 2024-04-20T21:36:19.792 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:19.793 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:36:19.793 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 66 2024-04-20T21:36:19.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6406c920 msgr2=0x7f9f6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:19.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6406c920 0x7f9f6406ede0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9f74000f60 tx=0x7f9f74007d70 comp rx=0 tx=0).stop 2024-04-20T21:36:19.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f80163220 msgr2=0x7f9f8014de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:19.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f80163220 0x7f9f8014de60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9f700098f0 tx=0x7f9f700046c0 comp rx=0 tx=0).stop 2024-04-20T21:36:19.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 shutdown_connections 2024-04-20T21:36:19.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f80163660 0x7f9f801f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:19.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9f6406c920 0x7f9f6406ede0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:19.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f80163220 0x7f9f8014de60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:19.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 --2- 172.21.15.102:0/3262279935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f80162e40 0x7f9f801607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:19.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 >> 172.21.15.102:0/3262279935 conn(0x7f9f800c9390 msgr2=0x7f9f80156a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:19.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 shutdown_connections 2024-04-20T21:36:19.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:19.796+0000 7f9f88000640 1 -- 172.21.15.102:0/3262279935 wait complete. 2024-04-20T21:36:19.813 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=5994 2024-04-20T21:36:19.813 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 9 secs before reviving mds.a 2024-04-20T21:36:28.814 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-04-20T21:36:28.814 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:36:28.814 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:36:28.822 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:36:28.822 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:29.055 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-20T21:36:29.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.088+0000 7f5be7600640 1 Processor -- start 2024-04-20T21:36:29.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.088+0000 7f5be7600640 1 -- start start 2024-04-20T21:36:29.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be7600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5be015ee00 0x7f5be006d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:29.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be7600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015f2b0 0x7f5be00710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be7600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be00c8f20 con 0x7f5be015ea20 2024-04-20T21:36:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be7600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f5be006cb80 con 0x7f5be015ee00 2024-04-20T21:36:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be7600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5be006ce60 con 0x7f5be015f2b0 2024-04-20T21:36:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be5600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015f2b0 0x7f5be00710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5bd7e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5be015ee00 0x7f5be006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be4c00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f5be015ea20 0x7f5be00c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:60776/0 (socket says 172.21.15.102:60776) 2024-04-20T21:36:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be4c00640 1 -- 172.21.15.102:0/840864449 learned_addr learned my addr 172.21.15.102:0/840864449 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:36:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be5600640 1 -- 172.21.15.102:0/840864449 >> v2:172.21.15.193:3300/0 conn(0x7f5be015ee00 msgr2=0x7f5be006d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:29.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be5600640 1 --2- 172.21.15.102:0/840864449 >> v2:172.21.15.193:3300/0 conn(0x7f5be015ee00 0x7f5be006d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be5600640 1 -- 172.21.15.102:0/840864449 >> v1:172.21.15.193:6789/0 conn(0x7f5be015ea20 legacy=0x7f5be00c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:29.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5bdfe00640 1 -- 172.21.15.102:0/840864449 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2082008915 0 0) 0x7f5be00c8f20 con 0x7f5be015ea20 2024-04-20T21:36:29.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.090+0000 7f5be5600640 1 -- 172.21.15.102:0/840864449 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be006d140 con 0x7f5be015f2b0 2024-04-20T21:36:29.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.091+0000 7f5be5600640 1 --2- 172.21.15.102:0/840864449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015f2b0 0x7f5be00710a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009d90 tx=0x7f5bd802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708d7ad3eec1b8fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:29.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5bdfe00640 1 -- 172.21.15.102:0/840864449 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8039070 con 0x7f5be015f2b0 2024-04-20T21:36:29.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5bdfe00640 1 -- 172.21.15.102:0/840864449 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd802cce0 con 0x7f5be015f2b0 2024-04-20T21:36:29.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5bdfe00640 1 -- 172.21.15.102:0/840864449 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8034660 con 0x7f5be015f2b0 2024-04-20T21:36:29.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5be7600640 1 -- 172.21.15.102:0/840864449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015f2b0 msgr2=0x7f5be00710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:29.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5be7600640 1 --2- 172.21.15.102:0/840864449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015f2b0 0x7f5be00710a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009d90 tx=0x7f5bd802bf30 comp rx=0 tx=0).stop 2024-04-20T21:36:29.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5be7600640 1 -- 172.21.15.102:0/840864449 shutdown_connections 2024-04-20T21:36:29.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5be7600640 1 --2- 172.21.15.102:0/840864449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015f2b0 0x7f5be00710a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5be7600640 1 --2- 172.21.15.102:0/840864449 >> v2:172.21.15.193:3300/0 conn(0x7f5be015ee00 0x7f5be006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.092+0000 7f5be7600640 1 -- 172.21.15.102:0/840864449 >> 172.21.15.102:0/840864449 conn(0x7f5be00c9390 msgr2=0x7f5be00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:29.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.093+0000 7f5be7600640 1 -- 172.21.15.102:0/840864449 shutdown_connections 2024-04-20T21:36:29.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.093+0000 7f5be7600640 1 -- 172.21.15.102:0/840864449 wait complete. 2024-04-20T21:36:29.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.094+0000 7f5be7600640 1 Processor -- start 2024-04-20T21:36:29.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.094+0000 7f5be7600640 1 -- start start 2024-04-20T21:36:29.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.095+0000 7f5be7600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5be015ea20 0x7f5be0150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:29.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.095+0000 7f5be7600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015ee00 0x7f5be015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:29.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.095+0000 7f5be7600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be015f2b0 0x7f5be0146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:29.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.095+0000 7f5be7600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5be00c8c80 con 0x7f5be015ea20 2024-04-20T21:36:29.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.095+0000 7f5be7600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5be00c8b00 con 0x7f5be015f2b0 2024-04-20T21:36:29.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.095+0000 7f5be7600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5be0154e80 con 0x7f5be015ee00 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.095+0000 7f5be4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5be015ea20 0x7f5be0150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.095+0000 7f5be5600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be015f2b0 0x7f5be0146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be5600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be015f2b0 0x7f5be0146dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54328/0 (socket says 172.21.15.102:54328) 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5bd7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015ee00 0x7f5be015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5be015ea20 0x7f5be0150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53602/0 (socket says 172.21.15.102:53602) 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be5600640 1 -- 172.21.15.102:0/329399870 learned_addr learned my addr 172.21.15.102:0/329399870 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be4c00640 1 -- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015ee00 msgr2=0x7f5be015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be4c00640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015ee00 0x7f5be015ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be4c00640 1 -- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be015f2b0 msgr2=0x7f5be0146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be4c00640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be015f2b0 0x7f5be0146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be4c00640 1 -- 172.21.15.102:0/329399870 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be00c4070 con 0x7f5be015ea20 2024-04-20T21:36:29.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.096+0000 7f5be4c00640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5be015ea20 0x7f5be0150c80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c480 tx=0x7f5bd000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:29.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.097+0000 7f5bdd600640 1 -- 172.21.15.102:0/329399870 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0018070 con 0x7f5be015ea20 2024-04-20T21:36:29.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.097+0000 7f5bdd600640 1 -- 172.21.15.102:0/329399870 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd00074e0 con 0x7f5be015ea20 2024-04-20T21:36:29.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.097+0000 7f5bdd600640 1 -- 172.21.15.102:0/329399870 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0004a20 con 0x7f5be015ea20 2024-04-20T21:36:29.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.097+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be00c4310 con 0x7f5be015ea20 2024-04-20T21:36:29.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.097+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be00c48d0 con 0x7f5be015ea20 2024-04-20T21:36:29.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.098+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be0152bc0 con 0x7f5be015ea20 2024-04-20T21:36:29.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.099+0000 7f5bdd600640 1 -- 172.21.15.102:0/329399870 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bd0004bc0 con 0x7f5be015ea20 2024-04-20T21:36:29.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.100+0000 7f5bdd600640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bbc06c8f0 0x7f5bbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:29.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.100+0000 7f5bdd600640 1 -- 172.21.15.102:0/329399870 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f5bd008b500 con 0x7f5be015ea20 2024-04-20T21:36:29.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.100+0000 7f5bd7e00640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bbc06c8f0 0x7f5bbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:29.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.101+0000 7f5bd7e00640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bbc06c8f0 0x7f5bbc06edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5be00cc860 tx=0x7f5bc8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:29.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.104+0000 7f5bdd600640 1 -- 172.21.15.102:0/329399870 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5be0152bc0 con 0x7f5be015ea20 2024-04-20T21:36:29.231 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:36:29.237 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:36:29.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.354+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5be0153220 con 0x7f5be015ea20 2024-04-20T21:36:29.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.355+0000 7f5bdd600640 1 -- 172.21.15.102:0/329399870 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6031 (secure 0 0 0) 0x7f5bd0058ae0 con 0x7f5be015ea20 2024-04-20T21:36:29.354 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:29.354 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:36:29.354 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 66 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bbc06c8f0 msgr2=0x7f5bbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bbc06c8f0 0x7f5bbc06edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5be00cc860 tx=0x7f5bc8009190 comp rx=0 tx=0).stop 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5be015ea20 msgr2=0x7f5be0150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5be015ea20 0x7f5be0150c80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c480 tx=0x7f5bd000c940 comp rx=0 tx=0).stop 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 shutdown_connections 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5bbc06c8f0 0x7f5bbc06edb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be015f2b0 0x7f5be0146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5be015ee00 0x7f5be015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 --2- 172.21.15.102:0/329399870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5be015ea20 0x7f5be0150c80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 >> 172.21.15.102:0/329399870 conn(0x7f5be00c9390 msgr2=0x7f5be0151f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:29.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 shutdown_connections 2024-04-20T21:36:29.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:29.358+0000 7f5be7600640 1 -- 172.21.15.102:0/329399870 wait complete. 2024-04-20T21:36:29.377 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-04-20T21:36:29.395 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:36:29.396+0000 7f184da00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:36:31.378 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:31.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.585+0000 7f0fcf400640 1 Processor -- start 2024-04-20T21:36:31.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.585+0000 7f0fcf400640 1 -- start start 2024-04-20T21:36:31.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.587+0000 7f0fcf400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8155610 0x7f0fc8073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:31.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.587+0000 7f0fcf400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0fc8073c90 0x7f0fc815cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:31.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.587+0000 7f0fcf400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc815b600 con 0x7f0fc806c4e0 2024-04-20T21:36:31.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.587+0000 7f0fcf400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc815c870 con 0x7f0fc8073c90 2024-04-20T21:36:31.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.587+0000 7f0fcf400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc815b8e0 con 0x7f0fc8155610 2024-04-20T21:36:31.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.587+0000 7f0fcd400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0fc8073c90 0x7f0fc815cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.587+0000 7f0fcd400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0fc8073c90 0x7f0fc815cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53616/0 (socket says 172.21.15.102:53616) 2024-04-20T21:36:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.588+0000 7f0fcd400640 1 -- 172.21.15.102:0/1208928783 learned_addr learned my addr 172.21.15.102:0/1208928783 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.588+0000 7f0fcca00640 1 --2- 172.21.15.102:0/1208928783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8155610 0x7f0fc8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.588+0000 7f0fcd400640 1 -- 172.21.15.102:0/1208928783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8155610 msgr2=0x7f0fc8073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.588+0000 7f0fcd400640 1 --2- 172.21.15.102:0/1208928783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8155610 0x7f0fc8073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.588+0000 7f0fcd400640 1 -- 172.21.15.102:0/1208928783 >> v1:172.21.15.193:6789/0 conn(0x7f0fc806c4e0 legacy=0x7f0fc80ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.588+0000 7f0fcd400640 1 -- 172.21.15.102:0/1208928783 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc815bbc0 con 0x7f0fc8073c90 2024-04-20T21:36:31.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.588+0000 7f0fc7e00640 1 -- 172.21.15.102:0/1208928783 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472936009 0 0) 0x7f0fc815b600 con 0x7f0fc806c4e0 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.589+0000 7f0fcd400640 1 --2- 172.21.15.102:0/1208928783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc8073c90 0x7f0fc815cd00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009870 tx=0x7f0fc002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c60a0c72634eeee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.589+0000 7f0fc7e00640 1 -- 172.21.15.102:0/1208928783 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc002f840 con 0x7f0fc8073c90 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.589+0000 7f0fc7e00640 1 -- 172.21.15.102:0/1208928783 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc002f9e0 con 0x7f0fc8073c90 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.589+0000 7f0fc7e00640 1 -- 172.21.15.102:0/1208928783 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc002fdb0 con 0x7f0fc8073c90 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.590+0000 7f0fcf400640 1 -- 172.21.15.102:0/1208928783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc8073c90 msgr2=0x7f0fc815cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.590+0000 7f0fcf400640 1 --2- 172.21.15.102:0/1208928783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc8073c90 0x7f0fc815cd00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009870 tx=0x7f0fc002ba20 comp rx=0 tx=0).stop 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.590+0000 7f0fcf400640 1 -- 172.21.15.102:0/1208928783 shutdown_connections 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.590+0000 7f0fcf400640 1 --2- 172.21.15.102:0/1208928783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc8073c90 0x7f0fc815cd00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.590+0000 7f0fcf400640 1 --2- 172.21.15.102:0/1208928783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8155610 0x7f0fc8073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.590+0000 7f0fcf400640 1 -- 172.21.15.102:0/1208928783 >> 172.21.15.102:0/1208928783 conn(0x7f0fc806ebf0 msgr2=0x7f0fc806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.590+0000 7f0fcf400640 1 -- 172.21.15.102:0/1208928783 shutdown_connections 2024-04-20T21:36:31.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.590+0000 7f0fcf400640 1 -- 172.21.15.102:0/1208928783 wait complete. 2024-04-20T21:36:31.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.592+0000 7f0fcf400640 1 Processor -- start 2024-04-20T21:36:31.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.592+0000 7f0fcf400640 1 -- start start 2024-04-20T21:36:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc806c4e0 0x7f0fc81585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcf400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8073c90 0x7f0fc8153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fc8155610 0x7f0fc81562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcf400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc815b250 con 0x7f0fc806c4e0 2024-04-20T21:36:31.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcf400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc815c180 con 0x7f0fc8155610 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcf400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc8155b40 con 0x7f0fc8073c90 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc806c4e0 0x7f0fc81585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc806c4e0 0x7f0fc81585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53620/0 (socket says 172.21.15.102:53620) 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcca00640 1 -- 172.21.15.102:0/23815387 learned_addr learned my addr 172.21.15.102:0/23815387 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fcd400640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fc8155610 0x7f0fc81562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.593+0000 7f0fbfe00640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8073c90 0x7f0fc8153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fcca00640 1 -- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8073c90 msgr2=0x7f0fc8153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fcca00640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8073c90 0x7f0fc8153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fcca00640 1 -- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fc8155610 msgr2=0x7f0fc81562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fcca00640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fc8155610 0x7f0fc81562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fcca00640 1 -- 172.21.15.102:0/23815387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc81547b0 con 0x7f0fc806c4e0 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fbfe00640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8073c90 0x7f0fc8153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fcca00640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc806c4e0 0x7f0fc81585e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800efc0 tx=0x7f0fb800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:31.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fc5600640 1 -- 172.21.15.102:0/23815387 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8018020 con 0x7f0fc806c4e0 2024-04-20T21:36:31.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fc5600640 1 -- 172.21.15.102:0/23815387 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb8014480 con 0x7f0fc806c4e0 2024-04-20T21:36:31.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fc5600640 1 -- 172.21.15.102:0/23815387 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8014850 con 0x7f0fc806c4e0 2024-04-20T21:36:31.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc81f1280 con 0x7f0fc806c4e0 2024-04-20T21:36:31.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.594+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc81f1820 con 0x7f0fc806c4e0 2024-04-20T21:36:31.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.596+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc80ce590 con 0x7f0fc806c4e0 2024-04-20T21:36:31.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.597+0000 7f0fc5600640 1 -- 172.21.15.102:0/23815387 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fb8005d10 con 0x7f0fc806c4e0 2024-04-20T21:36:31.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.597+0000 7f0fc5600640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0fa406c8f0 0x7f0fa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:31.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.598+0000 7f0fc5600640 1 -- 172.21.15.102:0/23815387 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f0fb808cc70 con 0x7f0fc806c4e0 2024-04-20T21:36:31.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.598+0000 7f0fbfe00640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0fa406c8f0 0x7f0fa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:31.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.598+0000 7f0fbfe00640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0fa406c8f0 0x7f0fa406edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80cc450 tx=0x7f0fb0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:31.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.602+0000 7f0fc5600640 1 -- 172.21.15.102:0/23815387 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0fc80ce590 con 0x7f0fc806c4e0 2024-04-20T21:36:31.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.853+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fc806cf30 con 0x7f0fc806c4e0 2024-04-20T21:36:31.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.855+0000 7f0fc5600640 1 -- 172.21.15.102:0/23815387 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7f0fc806cf30 con 0x7f0fc806c4e0 2024-04-20T21:36:31.854 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:36:31.854 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:36:31.854 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.857+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0fa406c8f0 msgr2=0x7f0fa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.857+0000 7f0fcf400640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0fa406c8f0 0x7f0fa406edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80cc450 tx=0x7f0fb0009040 comp rx=0 tx=0).stop 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.857+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc806c4e0 msgr2=0x7f0fc81585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.857+0000 7f0fcf400640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc806c4e0 0x7f0fc81585e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800efc0 tx=0x7f0fb800c490 comp rx=0 tx=0).stop 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.857+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 shutdown_connections 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.857+0000 7f0fcf400640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0fa406c8f0 0x7f0fa406edb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.858+0000 7f0fcf400640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fc8155610 0x7f0fc81562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.858+0000 7f0fcf400640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fc8073c90 0x7f0fc8153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.858+0000 7f0fcf400640 1 --2- 172.21.15.102:0/23815387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc806c4e0 0x7f0fc81585e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.858+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 >> 172.21.15.102:0/23815387 conn(0x7f0fc806ebf0 msgr2=0x7f0fc815d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:31.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.858+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 shutdown_connections 2024-04-20T21:36:31.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:36:31.858+0000 7f0fcf400640 1 -- 172.21.15.102:0/23815387 wait complete. 2024-04-20T21:36:31.875 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-04-20T21:36:31.875 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 91 secs before thrashing 2024-04-20T21:36:59.345 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:36:59.353 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:37:11.483 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 1% (136/13275) Updating files: 1% (165/13275) Updating files: 2% (266/13275) Updating files: 2% (301/13275) Updating files: 3% (399/13275) Updating files: 3% (415/13275) Updating files: 3% (463/13275) Updating files: 3% (496/13275) Updating files: 4% (531/13275) Updating files: 4% (550/13275) Updating files: 4% (622/13275) Updating files: 5% (664/13275) Updating files: 5% (686/13275) Updating files: 5% (701/13275) Updating files: 5% (726/13275) Updating files: 5% (753/13275) Updating files: 5% (754/13275) Updating files: 6% (797/13275) Updating files: 6% (800/13275) Updating files: 6% (905/13275) Updating files: 6% (913/13275) Updating files: 7% (930/13275) Updating files: 7% (962/13275) Updating files: 7% (980/13275) Updating files: 7% (1043/13275) Updating files: 8% (1062/13275) Updating files: 8% (1077/13275) Updating files: 8% (1147/13275) Updating files: 8% (1184/13275) Updating files: 9% (1195/13275) Updating files: 9% (1295/13275) Updating files: 10% (1328/13275) Updating files: 10% (1345/13275) Updating files: 10% (1365/13275) Updating files: 10% (1400/13275) Updating files: 11% (1461/13275) Updating files: 11% (1478/13275) Updating files: 11% (1553/13275) Updating files: 11% (1592/13275) Updating files: 12% (1593/13275) Updating files: 12% (1635/13275) Updating files: 12% (1650/13275) Updating files: 12% (1676/13275) Updating files: 12% (1719/13275) Updating files: 13% (1726/13275) Updating files: 13% (1748/13275) Updating files: 13% (1786/13275) Updating files: 13% (1829/13275) Updating files: 14% (1859/13275) Updating files: 14% (1911/13275) Updating files: 14% (1963/13275) Updating files: 14% (1969/13275) Updating files: 15% (1992/13275) Updating files: 15% (1996/13275) Updating files: 15% (2007/13275) Updating files: 15% (2068/13275) Updating files: 16% (2124/13275) Updating files: 16% (2151/13275) Updating files: 16% (2206/13275) Updating files: 17% (2257/13275) Updating files: 17% (2262/13275) Updating files: 17% (2310/13275) Updating files: 17% (2363/13275) Updating files: 17% (2379/13275) Updating files: 18% (2390/13275) Updating files: 18% (2428/13275) Updating files: 18% (2435/13275) Updating files: 18% (2489/13275) Updating files: 19% (2523/13275) Updating files: 19% (2525/13275) Updating files: 19% (2563/13275) Updating files: 19% (2604/13275) Updating files: 19% (2633/13275) Updating files: 20% (2655/13275) Updating files: 20% (2748/13275) Updating files: 20% (2784/13275) Updating files: 21% (2788/13275) Updating files: 21% (2816/13275) Updating files: 21% (2861/13275) Updating files: 22% (2921/13275) Updating files: 22% (2927/13275) Updating files: 22% (2952/13275) Updating files: 22% (2982/13275) Updating files: 22% (3021/13275) Updating files: 23% (3054/13275) Updating files: 23% (3064/13275) Updating files: 23% (3115/13275) Updating files: 23% (3128/13275) Updating files: 23% (3141/13275) Updating files: 23% (3159/13275) Updating files: 24% (3186/13275) Updating files: 24% (3195/13275) Updating files: 24% (3279/13275) Updating files: 24% (3287/13275) Updating files: 24% (3313/13275) Updating files: 25% (3319/13275) Updating files: 25% (3356/13275) Updating files: 25% (3366/13275) Updating files: 25% (3425/13275) Updating files: 26% (3452/13275) Updating files: 26% (3499/13275) Updating files: 27% (3585/13275) Updating files: 27% (3595/13275) Updating files: 27% (3598/13275) Updating files: 27% (3637/13275) Updating files: 27% (3701/13275) Updating files: 28% (3717/13275) Updating files: 28% (3738/13275) Updating files: 28% (3755/13275) Updating files: 28% (3784/13275) Updating files: 28% (3793/13275) Updating files: 28% (3804/13275) Updating files: 29% (3850/13275) Updating files: 29% (3869/13275) Updating files: 29% (3908/13275) Updating files: 29% (3951/13275) Updating files: 30% (3983/13275) Updating files: 30% (4006/13275) Updating files: 30% (4040/13275) Updating files: 30% (4083/13275) Updating files: 30% (4112/13275) Updating files: 31% (4116/13275) Updating files: 31% (4144/13275) Updating files: 31% (4181/13275) Updating files: 31% (4209/13275) Updating files: 31% (4220/13275) Updating files: 32% (4248/13275) Updating files: 32% (4255/13275) Updating files: 32% (4276/13275) Updating files: 32% (4297/13275) Updating files: 32% (4352/13275) Updating files: 33% (4381/13275) Updating files: 33% (4404/13275) Updating files: 33% (4456/13275) Updating files: 33% (4486/13275) Updating files: 33% (4505/13275) Updating files: 34% (4514/13275) Updating files: 34% (4543/13275) Updating files: 34% (4573/13275) Updating files: 35% (4647/13275) Updating files: 35% (4661/13275) Updating files: 36% (4779/13275) Updating files: 36% (4795/13275) Updating files: 36% (4867/13275) Updating files: 37% (4912/13275) Updating files: 37% (4942/13275) Updating files: 37% (4983/13275) Updating files: 38% (5045/13275) Updating files: 38% (5080/13275) Updating files: 38% (5134/13275) Updating files: 39% (5178/13275) Updating files: 39% (5211/13275) Updating files: 39% (5295/13275) Updating files: 39% (5297/13275) Updating files: 40% (5310/13275) Updating files: 40% (5339/13275) Updating files: 40% (5378/13275) Updating files: 40% (5419/13275) Updating files: 40% (5438/13275) Updating files: 41% (5443/13275) Updating files: 41% (5451/13275) Updating files: 41% (5524/13275) Updating files: 42% (5576/13275) Updating files: 42% (5613/13275) Updating files: 43% (5709/13275) Updating files: 43% (5759/13275) Updating files: 44% (5841/13275) Updating files: 44% (5964/13275) Updating files: 45% (5974/13275) Updating files: 45% (6030/13275) Updating files: 45% (6049/13275) Updating files: 45% (6066/13275) Updating files: 46% (6107/13275) Updating files: 46% (6152/13275) Updating files: 46% (6168/13275) Updating files: 46% (6212/13275) Updating files: 47% (6240/13275) Updating files: 47% (6275/13275) Updating files: 47% (6333/13275) Updating files: 48% (6372/13275) Updating files: 48% (6433/13275) Updating files: 48% (6494/13275) Updating files: 49% (6505/13275) Updating files: 49% (6511/13275) Updating files: 49% (6599/13275) Updating files: 50% (6638/13275) Updating files: 50% (6671/13275) Updating files: 51% (6771/13275) Updating files: 51% (6776/13275) Updating files: 51% (6829/13275) Updating files: 51% (6852/13275) Updating files: 51% (6900/13275) Updating files: 52% (6903/13275) Updating files: 52% (6943/13275) Updating files: 53% (7036/13275) Updating files: 53% (7054/13275) Updating files: 53% (7071/13275) Updating files: 54% (7169/13275) Updating files: 54% (7189/13275) Updating files: 54% (7237/13275) Updating files: 54% (7295/13275) Updating files: 55% (7302/13275) Updating files: 55% (7391/13275) Updating files: 56% (7434/13275) Updating files: 56% (7476/13275) Updating files: 56% (7510/13275) Updating files: 56% (7536/13275) Updating files: 57% (7567/13275) Updating files: 57% (7572/13275) Updating files: 57% (7661/13275) Updating files: 57% (7670/13275) Updating files: 58% (7700/13275) Updating files: 58% (7765/13275) Updating files: 58% (7768/13275) Updating files: 59% (7833/13275) Updating files: 59% (7855/13275) Updating files: 59% (7904/13275) Updating files: 59% (7944/13275) Updating files: 60% (7965/13275) Updating files: 60% (8004/13275) Updating files: 60% (8054/13275) Updating files: 60% (8072/13275) Updating files: 61% (8098/13275) Updating files: 61% (8126/13275) Updating files: 61% (8154/13275) Updating files: 61% (8190/13275) Updating files: 62% (8231/13275) Updating files: 62% (8261/13275) Updating files: 62% (8324/13275) Updating files: 63% (8364/13275) Updating files: 63% (8370/13275) Updating files: 63% (8384/13275) Updating files: 63% (8445/13275) Updating files: 63% (8494/13275) Updating files: 64% (8496/13275) Updating files: 64% (8520/13275) Updating files: 64% (8543/13275) Updating files: 64% (8574/13275) Updating files: 64% (8625/13275) Updating files: 65% (8629/13275) Updating files: 65% (8639/13275) Updating files: 65% (8664/13275) Updating files: 65% (8710/13275) Updating files: 65% (8735/13275) Updating files: 66% (8762/13275) Updating files: 66% (8792/13275) Updating files: 66% (8820/13275) Updating files: 66% (8868/13275) Updating files: 67% (8895/13275) Updating files: 67% (8908/13275) Updating files: 68% (9027/13275) Updating files: 68% (9041/13275) Updating files: 68% (9057/13275) Updating files: 68% (9077/13275) Updating files: 68% (9110/13275) Updating files: 68% (9138/13275) Updating files: 69% (9160/13275) Updating files: 69% (9207/13275) Updating files: 69% (9277/13275) Updating files: 70% (9293/13275) Updating files: 70% (9308/13275) Updating files: 71% (9426/13275) Updating files: 71% (9431/13275) Updating files: 71% (9437/13275) Updating files: 71% (9468/13275) Updating files: 71% (9535/13275) Updating files: 72% (9558/13275) Updating files: 72% (9567/13275) Updating files: 72% (9628/13275) Updating files: 73% (9691/13275) Updating files: 73% (9711/13275) Updating files: 73% (9794/13275) Updating files: 74% (9824/13275) Updating files: 74% (9888/13275) Updating files: 74% (9910/13275) Updating files: 75% (9957/13275) Updating files: 75% (9998/13275) Updating files: 75% (10019/13275) Updating files: 75% (10037/13275) Updating files: 75% (10052/13275) Updating files: 76% (10089/13275) Updating files: 76% (10115/13275) Updating files: 76% (10206/13275) Updating files: 76% (10221/13275) Updating files: 77% (10222/13275) Updating files: 77% (10340/13275) Updating files: 78% (10355/13275) Updating files: 78% (10388/13275) Updating files: 78% (10465/13275) Updating files: 78% (10484/13275) Updating files: 79% (10488/13275) Updating files: 79% (10535/13275) Updating files: 79% (10558/13275) Updating files: 80% (10620/13275) Updating files: 80% (10638/13275) Updating files: 80% (10658/13275) Updating files: 80% (10685/13275) Updating files: 80% (10705/13275) Updating files: 80% (10714/13275) Updating files: 80% (10733/13275) Updating files: 81% (10753/13275) Updating files: 81% (10799/13275) Updating files: 81% (10859/13275) Updating files: 81% (10885/13275) Updating files: 82% (10886/13275) Updating files: 82% (10970/13275) Updating files: 82% (10997/13275) Updating files: 83% (11019/13275) Updating files: 83% (11038/13275) Updating files: 83% (11052/13275) Updating files: 83% (11104/13275) Updating files: 83% (11138/13275) Updating files: 84% (11151/13275) Updating files: 84% (11155/13275) Updating files: 85% (11284/13275) Updating files: 85% (11289/13275) Updating files: 85% (11377/13275) Updating files: 86% (11417/13275) Updating files: 86% (11428/13275) Updating files: 86% (11466/13275) Updating files: 86% (11506/13275) Updating files: 86% (11523/13275) Updating files: 87% (11550/13275) Updating files: 87% (11581/13275) Updating files: 87% (11654/13275) Updating files: 88% (11682/13275) Updating files: 88% (11733/13275) Updating files: 88% (11806/13275) Updating files: 89% (11815/13275) Updating files: 89% (11920/13275) Updating files: 90% (11948/13275) Updating files: 90% (11977/13275) Updating files: 90% (12045/13275) Updating files: 91% (12081/13275) Updating files: 91% (12089/13275) Updating files: 91% (12150/13275) Updating files: 91% (12174/13275) Updating files: 91% (12181/13275) Updating files: 91% (12201/13275) Updating files: 91% (12209/13275) Updating files: 92% (12213/13275) Updating files: 92% (12260/13275) Updating files: 92% (12293/13275) Updating files: 93% (12346/13275) Updating files: 93% (12350/13275) Updating files: 94% (12479/13275) Updating files: 94% (12533/13275) Updating files: 95% (12612/13275) Updating files: 95% (12705/13275) Updating files: 96% (12744/13275) Updating files: 96% (12815/13275) Updating files: 96% (12842/13275) Updating files: 96% (12862/13275) Updating files: 97% (12877/13275) Updating files: 97% (12926/13275) Updating files: 98% (13010/13275) Updating files: 98% (13039/13275) Updating files: 98% (13060/13275) Updating files: 98% (13112/13275) Updating files: 99% (13143/13275) Updating files: 99% (13156/13275) Updating files: 99% (13195/13275) Updating files: 100% (13275/13275) Updating files: 100% (13275/13275), done. 2024-04-20T21:37:11.632 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 204c175eea3 msg: blobhash on entity_name_t causes strange badness... should look into that, but avoid for now 2024-04-20T21:37:11.658 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.1 2024-04-20T21:37:11.707 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:37:11.707 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 2 -eq 48 ']' 2024-04-20T21:37:11.707 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.2 2024-04-20T21:37:11.707 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.2 2024-04-20T21:37:11.707 INFO:tasks.workunit.client.0.smithi102.stdout:v0.2 2024-04-20T21:37:11.707 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.2 2024-04-20T21:37:16.250 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 44% (99/222) Updating files: 45% (100/222) Updating files: 46% (103/222) Updating files: 47% (105/222) Updating files: 48% (107/222) Updating files: 49% (109/222) Updating files: 50% (111/222) Updating files: 51% (114/222) Updating files: 52% (116/222) Updating files: 53% (118/222) Updating files: 54% (120/222) Updating files: 55% (123/222) Updating files: 56% (125/222) Updating files: 57% (127/222) Updating files: 58% (129/222) Updating files: 59% (131/222) Updating files: 60% (134/222) Updating files: 61% (136/222) Updating files: 62% (138/222) Updating files: 63% (140/222) Updating files: 64% (143/222) Updating files: 65% (145/222) Updating files: 66% (147/222) Updating files: 67% (149/222) Updating files: 68% (151/222) Updating files: 69% (154/222) Updating files: 70% (156/222) Updating files: 71% (158/222) Updating files: 72% (160/222) Updating files: 73% (163/222) Updating files: 74% (165/222) Updating files: 75% (167/222) Updating files: 76% (169/222) Updating files: 77% (171/222) Updating files: 78% (174/222) Updating files: 79% (176/222) Updating files: 79% (177/222) Updating files: 80% (178/222) Updating files: 81% (180/222) Updating files: 82% (183/222) Updating files: 83% (185/222) Updating files: 84% (187/222) Updating files: 85% (189/222) Updating files: 86% (191/222) Updating files: 87% (194/222) Updating files: 88% (196/222) Updating files: 89% (198/222) Updating files: 90% (200/222) Updating files: 91% (203/222) Updating files: 92% (205/222) Updating files: 93% (207/222) Updating files: 94% (209/222) Updating files: 95% (211/222) Updating files: 96% (214/222) Updating files: 97% (216/222) Updating files: 98% (218/222) Updating files: 99% (220/222) Updating files: 100% (222/222) Updating files: 100% (222/222), done. 2024-04-20T21:37:16.318 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 2b5f76f48e9 readme 2024-04-20T21:37:16.336 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.2 2024-04-20T21:37:16.447 INFO:tasks.workunit.client.0.smithi102.stdout:v0.3 2024-04-20T21:37:16.448 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:37:16.448 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 3 -eq 48 ']' 2024-04-20T21:37:16.448 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.3 2024-04-20T21:37:16.448 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.3 2024-04-20T21:37:16.448 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.3 2024-04-20T21:37:23.940 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 1% (4/295) Updating files: 2% (6/295) Updating files: 3% (9/295) Updating files: 4% (12/295) Updating files: 5% (15/295) Updating files: 6% (18/295) Updating files: 7% (21/295) Updating files: 8% (24/295) Updating files: 9% (27/295) Updating files: 9% (28/295) Updating files: 10% (30/295) Updating files: 11% (33/295) Updating files: 12% (36/295) Updating files: 13% (39/295) Updating files: 14% (42/295) Updating files: 15% (45/295) Updating files: 16% (48/295) Updating files: 17% (51/295) Updating files: 18% (54/295) Updating files: 19% (57/295) Updating files: 20% (59/295) Updating files: 20% (61/295) Updating files: 21% (62/295) Updating files: 22% (65/295) Updating files: 23% (68/295) Updating files: 24% (71/295) Updating files: 25% (74/295) Updating files: 26% (77/295) Updating files: 27% (80/295) Updating files: 27% (81/295) Updating files: 28% (83/295) Updating files: 29% (86/295) Updating files: 30% (89/295) Updating files: 31% (92/295) Updating files: 32% (95/295) Updating files: 33% (98/295) Updating files: 34% (101/295) Updating files: 35% (104/295) Updating files: 36% (107/295) Updating files: 37% (110/295) Updating files: 38% (113/295) Updating files: 39% (116/295) Updating files: 40% (118/295) Updating files: 40% (120/295) Updating files: 41% (121/295) Updating files: 42% (124/295) Updating files: 43% (127/295) Updating files: 44% (130/295) Updating files: 45% (133/295) Updating files: 46% (136/295) Updating files: 47% (139/295) Updating files: 48% (142/295) Updating files: 49% (145/295) Updating files: 50% (148/295) Updating files: 51% (151/295) Updating files: 52% (154/295) Updating files: 53% (157/295) Updating files: 54% (160/295) Updating files: 55% (163/295) Updating files: 56% (166/295) Updating files: 57% (169/295) Updating files: 58% (172/295) Updating files: 59% (175/295) Updating files: 60% (177/295) Updating files: 61% (180/295) Updating files: 62% (183/295) Updating files: 63% (186/295) Updating files: 64% (189/295) Updating files: 65% (192/295) Updating files: 66% (195/295) Updating files: 67% (198/295) Updating files: 68% (201/295) Updating files: 69% (204/295) Updating files: 70% (207/295) Updating files: 71% (210/295) Updating files: 72% (213/295) Updating files: 73% (216/295) Updating files: 74% (219/295) Updating files: 75% (222/295) Updating files: 76% (225/295) Updating files: 77% (228/295) Updating files: 78% (231/295) Updating files: 79% (234/295) Updating files: 80% (236/295) Updating files: 81% (239/295) Updating files: 82% (242/295) Updating files: 83% (245/295) Updating files: 84% (248/295) Updating files: 85% (251/295) Updating files: 86% (254/295) Updating files: 87% (257/295) Updating files: 88% (260/295) Updating files: 89% (263/295) Updating files: 90% (266/295) Updating files: 91% (269/295) Updating files: 92% (272/295) Updating files: 93% (275/295) Updating files: 94% (278/295) Updating files: 95% (281/295) Updating files: 96% (284/295) Updating files: 97% (287/295) Updating files: 98% (290/295) Updating files: 99% (293/295) Updating files: 100% (295/295) Updating files: 100% (295/295), done. 2024-04-20T21:37:24.004 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 2e986fe9ddf include ceph.spec in configure.ac 2024-04-20T21:37:24.030 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.3 2024-04-20T21:37:24.340 INFO:tasks.workunit.client.0.smithi102.stdout:v0.4 2024-04-20T21:37:24.340 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:37:24.341 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 4 -eq 48 ']' 2024-04-20T21:37:24.341 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.4 2024-04-20T21:37:24.341 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.4 2024-04-20T21:37:24.341 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.4 2024-04-20T21:37:29.493 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:37:29.502 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:37:30.292 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 35% (87/247) Updating files: 36% (89/247) Updating files: 37% (92/247) Updating files: 38% (94/247) Updating files: 39% (97/247) Updating files: 40% (99/247) Updating files: 41% (102/247) Updating files: 42% (104/247) Updating files: 42% (106/247) Updating files: 43% (107/247) Updating files: 44% (109/247) Updating files: 45% (112/247) Updating files: 46% (114/247) Updating files: 47% (117/247) Updating files: 48% (119/247) Updating files: 49% (122/247) Updating files: 50% (124/247) Updating files: 51% (126/247) Updating files: 52% (129/247) Updating files: 53% (131/247) Updating files: 54% (134/247) Updating files: 55% (136/247) Updating files: 56% (139/247) Updating files: 57% (141/247) Updating files: 58% (144/247) Updating files: 59% (146/247) Updating files: 60% (149/247) Updating files: 61% (151/247) Updating files: 62% (154/247) Updating files: 63% (156/247) Updating files: 64% (159/247) Updating files: 65% (161/247) Updating files: 66% (164/247) Updating files: 67% (166/247) Updating files: 68% (168/247) Updating files: 69% (171/247) Updating files: 70% (173/247) Updating files: 71% (176/247) Updating files: 72% (178/247) Updating files: 73% (181/247) Updating files: 74% (183/247) Updating files: 75% (186/247) Updating files: 76% (188/247) Updating files: 77% (191/247) Updating files: 78% (193/247) Updating files: 79% (196/247) Updating files: 80% (198/247) Updating files: 81% (201/247) Updating files: 82% (203/247) Updating files: 83% (206/247) Updating files: 84% (208/247) Updating files: 85% (210/247) Updating files: 86% (213/247) Updating files: 87% (215/247) Updating files: 88% (218/247) Updating files: 89% (220/247) Updating files: 90% (223/247) Updating files: 91% (225/247) Updating files: 92% (228/247) Updating files: 93% (230/247) Updating files: 94% (233/247) Updating files: 94% (234/247) Updating files: 95% (235/247) Updating files: 96% (238/247) Updating files: 97% (240/247) Updating files: 98% (243/247) Updating files: 99% (245/247) Updating files: 100% (247/247) Updating files: 100% (247/247), done. 2024-04-20T21:37:31.878 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 13b1bf7c492 kclient: initializing kaddr (merge fix) 2024-04-20T21:37:31.900 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.4 2024-04-20T21:37:31.965 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:37:31.965 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 5 -eq 48 ']' 2024-04-20T21:37:31.965 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.5 2024-04-20T21:37:31.965 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.5 2024-04-20T21:37:31.965 INFO:tasks.workunit.client.0.smithi102.stdout:v0.5 2024-04-20T21:37:31.965 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.5 2024-04-20T21:37:36.294 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 46% (115/245) Updating files: 47% (116/245) Updating files: 48% (118/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 88% (217/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2024-04-20T21:37:36.354 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 1dd420957f9 journal: detect size of raw block devices properly 2024-04-20T21:37:36.372 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.5 2024-04-20T21:37:36.489 INFO:tasks.workunit.client.0.smithi102.stdout:v0.6 2024-04-20T21:37:36.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:37:36.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 6 -eq 48 ']' 2024-04-20T21:37:36.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.6 2024-04-20T21:37:36.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.6 2024-04-20T21:37:36.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.6 2024-04-20T21:37:45.632 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 42% (96/227) Updating files: 43% (98/227) Updating files: 44% (100/227) Updating files: 45% (103/227) Updating files: 46% (105/227) Updating files: 47% (107/227) Updating files: 48% (109/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 82% (187/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 85% (193/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2024-04-20T21:37:46.986 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at a4c752df713 v0.6 2024-04-20T21:37:46.994 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.6 2024-04-20T21:37:47.705 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:37:47.706 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 7 -eq 48 ']' 2024-04-20T21:37:47.706 INFO:tasks.workunit.client.0.smithi102.stdout:v0.7 2024-04-20T21:37:47.706 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.7 2024-04-20T21:37:47.706 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.7 2024-04-20T21:37:47.706 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.7 2024-04-20T21:37:51.075 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 63% (128/201) Updating files: 64% (129/201) Updating files: 65% (131/201) Updating files: 66% (133/201) Updating files: 67% (135/201) Updating files: 68% (137/201) Updating files: 69% (139/201) Updating files: 70% (141/201) Updating files: 71% (143/201) Updating files: 72% (145/201) Updating files: 73% (147/201) Updating files: 74% (149/201) Updating files: 75% (151/201) Updating files: 76% (153/201) Updating files: 77% (155/201) Updating files: 78% (157/201) Updating files: 79% (159/201) Updating files: 80% (161/201) Updating files: 81% (163/201) Updating files: 82% (165/201) Updating files: 83% (167/201) Updating files: 84% (169/201) Updating files: 85% (171/201) Updating files: 86% (173/201) Updating files: 87% (175/201) Updating files: 88% (177/201) Updating files: 89% (179/201) Updating files: 90% (181/201) Updating files: 91% (183/201) Updating files: 92% (185/201) Updating files: 92% (186/201) Updating files: 93% (187/201) Updating files: 94% (189/201) Updating files: 95% (191/201) Updating files: 96% (193/201) Updating files: 97% (195/201) Updating files: 98% (197/201) Updating files: 99% (199/201) Updating files: 100% (201/201) Updating files: 100% (201/201), done. 2024-04-20T21:37:51.160 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at c46e1b320c6 mkcephfs: btrfsctl -a between mkfs and mount 2024-04-20T21:37:51.172 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.7 2024-04-20T21:37:51.270 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:37:51.270 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 8 -eq 48 ']' 2024-04-20T21:37:51.271 INFO:tasks.workunit.client.0.smithi102.stdout:v0.8 2024-04-20T21:37:51.271 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.8 2024-04-20T21:37:51.271 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.8 2024-04-20T21:37:51.271 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.8 2024-04-20T21:37:59.590 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:37:59.598 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:38:02.877 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:03.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.087+0000 7f01e4600640 1 Processor -- start 2024-04-20T21:38:03.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.087+0000 7f01e4600640 1 -- start start 2024-04-20T21:38:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01e4600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc163660 0x7f01dc0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01e4600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01dc06d2e0 con 0x7f01dc162e40 2024-04-20T21:38:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01e4600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01dc06d5c0 con 0x7f01dc163220 2024-04-20T21:38:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01e4600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc0c7290 con 0x7f01dc163660 2024-04-20T21:38:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01e1c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f01dc162e40 0x7f01dc06c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55338/0 (socket says 172.21.15.102:55338) 2024-04-20T21:38:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01e1c00640 1 -- 172.21.15.102:0/2103172276 learned_addr learned my addr 172.21.15.102:0/2103172276 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01e2600640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc163660 0x7f01dc0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4200998865 0 0) 0x7f01dc06d2e0 con 0x7f01dc162e40 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.089+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01c4003520 con 0x7f01dc162e40 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2888009378 0 0) 0x7f01dc06d5c0 con 0x7f01dc163220 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01dc06d2e0 con 0x7f01dc163220 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 410070875 0 0) 0x7f01c4003520 con 0x7f01dc162e40 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc163660 msgr2=0x7f01dc0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc163660 0x7f01dc0c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 >> v1:172.21.15.193:6789/0 conn(0x7f01dc163220 legacy=0x7f01dc159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc06d8a0 con 0x7f01dc162e40 2024-04-20T21:38:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1987239767 0 0) 0x7f01d0001670 con 0x7f01dc162e40 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 >> v1:172.21.15.102:6789/0 conn(0x7f01dc162e40 legacy=0x7f01dc06c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01c4003b70 0x7f01c4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c40051c0 0x7f01c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0c7290 con 0x7f01c40051c0 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f01c400bfb0 con 0x7f01c4004800 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f01c400c1b0 con 0x7f01c4003b70 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.090+0000 7f01e1c00640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01c4003b70 0x7f01c4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.091+0000 7f01e1200640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.091+0000 7f01e1c00640 1 -- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4004800 msgr2=0x7f01c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.091+0000 7f01e1c00640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.091+0000 7f01e1c00640 1 -- 172.21.15.102:0/2103172276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c40051c0 msgr2=0x7f01c400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.091+0000 7f01e2600640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c40051c0 0x7f01c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.092+0000 7f01e1c00640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c40051c0 0x7f01c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.092+0000 7f01e1c00640 1 -- 172.21.15.102:0/2103172276 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c4003520 con 0x7f01c4003b70 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.092+0000 7f01e1200640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.092+0000 7f01e2600640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c40051c0 0x7f01c400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.093+0000 7f01e1c00640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01c4003b70 0x7f01c4003f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c210 tx=0x7f01c4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb59619364ce276 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.093+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0007ce0 con 0x7f01c4003b70 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.093+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d0030070 con 0x7f01c4003b70 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.093+0000 7f01dbe00640 1 -- 172.21.15.102:0/2103172276 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0005d90 con 0x7f01c4003b70 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.094+0000 7f01e4600640 1 -- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01c4003b70 msgr2=0x7f01c4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.094+0000 7f01e4600640 1 --2- 172.21.15.102:0/2103172276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01c4003b70 0x7f01c4003f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c210 tx=0x7f01c4005830 comp rx=0 tx=0).stop 2024-04-20T21:38:03.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.094+0000 7f01e2600640 1 -- 172.21.15.102:0/2103172276 reap_dead start 2024-04-20T21:38:03.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.094+0000 7f01e4600640 1 -- 172.21.15.102:0/2103172276 shutdown_connections 2024-04-20T21:38:03.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.094+0000 7f01e4600640 1 -- 172.21.15.102:0/2103172276 >> 172.21.15.102:0/2103172276 conn(0x7f01dc0c9390 msgr2=0x7f01dc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:03.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.094+0000 7f01e4600640 1 -- 172.21.15.102:0/2103172276 shutdown_connections 2024-04-20T21:38:03.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.094+0000 7f01e4600640 1 -- 172.21.15.102:0/2103172276 wait complete. 2024-04-20T21:38:03.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.096+0000 7f01e4600640 1 Processor -- start 2024-04-20T21:38:03.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.096+0000 7f01e4600640 1 -- start start 2024-04-20T21:38:03.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.097+0000 7f01e4600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01dc0c4140 0x7f01dc14c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.097+0000 7f01e4600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc071280 0x7f01dc161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e4600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01dc0c4520 0x7f01dc161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e4600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0c9080 con 0x7f01dc0c4520 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e4600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0c7570 con 0x7f01dc0c4140 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e4600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc159b50 con 0x7f01dc071280 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01dc0c4140 0x7f01dc14c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc071280 0x7f01dc161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01dc0c4140 0x7f01dc14c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:43692/0 (socket says 172.21.15.102:43692) 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1c00640 1 -- 172.21.15.102:0/1417697495 learned_addr learned my addr 172.21.15.102:0/1417697495 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e2600640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01dc0c4520 0x7f01dc161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1200640 1 -- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01dc0c4140 msgr2=0x7f01dc14c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1200640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01dc0c4140 0x7f01dc14c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1200640 1 -- 172.21.15.102:0/1417697495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01dc0c4520 msgr2=0x7f01dc161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1200640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01dc0c4520 0x7f01dc161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1200640 1 -- 172.21.15.102:0/1417697495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc0c67b0 con 0x7f01dc071280 2024-04-20T21:38:03.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.098+0000 7f01e1c00640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01dc0c4140 0x7f01dc14c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:03.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.099+0000 7f01e1200640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc071280 0x7f01dc161a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f01c800d7a0 tx=0x7f01c800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:03.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.099+0000 7f01d9600640 1 -- 172.21.15.102:0/1417697495 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8016070 con 0x7f01dc071280 2024-04-20T21:38:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.099+0000 7f01d9600640 1 -- 172.21.15.102:0/1417697495 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c8007220 con 0x7f01dc071280 2024-04-20T21:38:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.099+0000 7f01d9600640 1 -- 172.21.15.102:0/1417697495 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8007630 con 0x7f01dc071280 2024-04-20T21:38:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.099+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc0c6a90 con 0x7f01dc071280 2024-04-20T21:38:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.099+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc14ab90 con 0x7f01dc071280 2024-04-20T21:38:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.100+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc0c3a30 con 0x7f01dc071280 2024-04-20T21:38:03.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.102+0000 7f01d9600640 1 -- 172.21.15.102:0/1417697495 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01c80067d0 con 0x7f01dc071280 2024-04-20T21:38:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.103+0000 7f01d9600640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f01bc06c920 0x7f01bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.103+0000 7f01d9600640 1 -- 172.21.15.102:0/1417697495 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f01c808c110 con 0x7f01dc071280 2024-04-20T21:38:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.103+0000 7f01e1c00640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f01bc06c920 0x7f01bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.104+0000 7f01e1c00640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f01bc06c920 0x7f01bc06ede0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f01dc14cfd0 tx=0x7f01d0007000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:03.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.107+0000 7f01d9600640 1 -- 172.21.15.102:0/1417697495 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f01dc0c3a30 con 0x7f01dc071280 2024-04-20T21:38:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.424+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01dc074340 con 0x7f01dc071280 2024-04-20T21:38:03.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.426+0000 7f01d9600640 1 -- 172.21.15.102:0/1417697495 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7f01c8059850 con 0x7f01dc071280 2024-04-20T21:38:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:03.425 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:38:03.426 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:38:03.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f01bc06c920 msgr2=0x7f01bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f01bc06c920 0x7f01bc06ede0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f01dc14cfd0 tx=0x7f01d0007000 comp rx=0 tx=0).stop 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc071280 msgr2=0x7f01dc161a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc071280 0x7f01dc161a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f01c800d7a0 tx=0x7f01c800dc60 comp rx=0 tx=0).stop 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 shutdown_connections 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01dc0c4520 0x7f01dc161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01dc071280 0x7f01dc161a00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f01bc06c920 0x7f01bc06ede0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 --2- 172.21.15.102:0/1417697495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01dc0c4140 0x7f01dc14c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 >> 172.21.15.102:0/1417697495 conn(0x7f01dc0c9390 msgr2=0x7f01dc157a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.429+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 shutdown_connections 2024-04-20T21:38:03.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.430+0000 7f01e4600640 1 -- 172.21.15.102:0/1417697495 wait complete. 2024-04-20T21:38:03.454 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:38:03.455 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:38:03.455 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:38:03.488 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:38:03.488 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.690+0000 7f4c1e200640 1 Processor -- start 2024-04-20T21:38:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.690+0000 7f4c1e200640 1 -- start start 2024-04-20T21:38:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.691+0000 7f4c1e200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c18155530 0x7f4c1815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.692+0000 7f4c1e200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4c180cb6a0 0x7f4c180cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.692+0000 7f4c1e200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c1815ef60 con 0x7f4c180cc470 2024-04-20T21:38:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.692+0000 7f4c1e200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f4c180c5ca0 con 0x7f4c180cb6a0 2024-04-20T21:38:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.692+0000 7f4c1e200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4c180cd5f0 con 0x7f4c18155530 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.692+0000 7f4c1ce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c18155530 0x7f4c1815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.692+0000 7f4c1d800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4c180cb6a0 0x7f4c180cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.692+0000 7f4c0fe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f4c180cc470 0x7f4c180cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:60996/0 (socket says 172.21.15.102:60996) 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.692+0000 7f4c0fe00640 1 -- 172.21.15.102:0/2476593769 learned_addr learned my addr 172.21.15.102:0/2476593769 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.693+0000 7f4c17e00640 1 -- 172.21.15.102:0/2476593769 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4025731329 0 0) 0x7f4c1815ef60 con 0x7f4c180cc470 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.693+0000 7f4c1ce00640 1 -- 172.21.15.102:0/2476593769 >> v2:172.21.15.193:3300/0 conn(0x7f4c180cb6a0 msgr2=0x7f4c180cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.693+0000 7f4c1ce00640 1 --2- 172.21.15.102:0/2476593769 >> v2:172.21.15.193:3300/0 conn(0x7f4c180cb6a0 0x7f4c180cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.693+0000 7f4c1ce00640 1 -- 172.21.15.102:0/2476593769 >> v1:172.21.15.193:6789/0 conn(0x7f4c180cc470 legacy=0x7f4c180cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.693+0000 7f4c1ce00640 1 -- 172.21.15.102:0/2476593769 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1815f240 con 0x7f4c18155530 2024-04-20T21:38:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.693+0000 7f4c1d800640 1 --2- 172.21.15.102:0/2476593769 >> v2:172.21.15.193:3300/0 conn(0x7f4c180cb6a0 0x7f4c180cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:03.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.694+0000 7f4c1ce00640 1 --2- 172.21.15.102:0/2476593769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c18155530 0x7f4c1815e640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4c080097a0 tx=0x7f4c0802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef22042187f61d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:03.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.694+0000 7f4c17e00640 1 -- 172.21.15.102:0/2476593769 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08034070 con 0x7f4c18155530 2024-04-20T21:38:03.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.694+0000 7f4c17e00640 1 -- 172.21.15.102:0/2476593769 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c08030800 con 0x7f4c18155530 2024-04-20T21:38:03.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.694+0000 7f4c17e00640 1 -- 172.21.15.102:0/2476593769 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08030bd0 con 0x7f4c18155530 2024-04-20T21:38:03.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.695+0000 7f4c1e200640 1 -- 172.21.15.102:0/2476593769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c18155530 msgr2=0x7f4c1815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.695+0000 7f4c1e200640 1 --2- 172.21.15.102:0/2476593769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c18155530 0x7f4c1815e640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4c080097a0 tx=0x7f4c0802b940 comp rx=0 tx=0).stop 2024-04-20T21:38:03.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.695+0000 7f4c1e200640 1 -- 172.21.15.102:0/2476593769 shutdown_connections 2024-04-20T21:38:03.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.695+0000 7f4c1e200640 1 --2- 172.21.15.102:0/2476593769 >> v2:172.21.15.193:3300/0 conn(0x7f4c180cb6a0 0x7f4c180cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.695+0000 7f4c1e200640 1 --2- 172.21.15.102:0/2476593769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c18155530 0x7f4c1815e640 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.695+0000 7f4c1e200640 1 -- 172.21.15.102:0/2476593769 >> 172.21.15.102:0/2476593769 conn(0x7f4c18075080 msgr2=0x7f4c18075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:03.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.695+0000 7f4c1e200640 1 -- 172.21.15.102:0/2476593769 shutdown_connections 2024-04-20T21:38:03.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.695+0000 7f4c1e200640 1 -- 172.21.15.102:0/2476593769 wait complete. 2024-04-20T21:38:03.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.697+0000 7f4c1e200640 1 Processor -- start 2024-04-20T21:38:03.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.697+0000 7f4c1e200640 1 -- start start 2024-04-20T21:38:03.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.698+0000 7f4c1e200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c180cb6a0 0x7f4c18148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.698+0000 7f4c1e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c180cc470 0x7f4c18147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.698+0000 7f4c1e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c18155530 0x7f4c181f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.698+0000 7f4c1e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4c180cd280 con 0x7f4c18155530 2024-04-20T21:38:03.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.698+0000 7f4c1e200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4c180ccfa0 con 0x7f4c180cc470 2024-04-20T21:38:03.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.698+0000 7f4c1e200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4c180ccbe0 con 0x7f4c180cb6a0 2024-04-20T21:38:03.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c180cc470 0x7f4c18147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c1ce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c180cb6a0 0x7f4c18148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c1d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c18155530 0x7f4c181f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c180cc470 0x7f4c18147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:43704/0 (socket says 172.21.15.102:43704) 2024-04-20T21:38:03.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c1ce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c180cb6a0 0x7f4c18148660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:43762/0 (socket says 172.21.15.102:43762) 2024-04-20T21:38:03.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 -- 172.21.15.102:0/939259670 learned_addr learned my addr 172.21.15.102:0/939259670 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:03.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 -- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c180cb6a0 msgr2=0x7f4c18148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c180cb6a0 0x7f4c18148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 -- 172.21.15.102:0/939259670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c18155530 msgr2=0x7f4c181f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c18155530 0x7f4c181f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 -- 172.21.15.102:0/939259670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c181f5460 con 0x7f4c180cc470 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c1d800640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c18155530 0x7f4c181f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c0fe00640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c180cc470 0x7f4c18147b60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000c4a0 tx=0x7f4c0000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.699+0000 7f4c15600640 1 -- 172.21.15.102:0/939259670 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c000074e0 con 0x7f4c180cc470 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.700+0000 7f4c15600640 1 -- 172.21.15.102:0/939259670 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c000047e0 con 0x7f4c180cc470 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.700+0000 7f4c15600640 1 -- 172.21.15.102:0/939259670 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00004c10 con 0x7f4c180cc470 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.700+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c18147080 con 0x7f4c180cc470 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.700+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c18147640 con 0x7f4c180cc470 2024-04-20T21:38:03.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.701+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1804fd60 con 0x7f4c180cc470 2024-04-20T21:38:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.702+0000 7f4c15600640 1 -- 172.21.15.102:0/939259670 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c00011070 con 0x7f4c180cc470 2024-04-20T21:38:03.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.702+0000 7f4c15600640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:03.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.703+0000 7f4c15600640 1 -- 172.21.15.102:0/939259670 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f4c0008bed0 con 0x7f4c180cc470 2024-04-20T21:38:03.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.703+0000 7f4c1ce00640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:03.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.703+0000 7f4c1ce00640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4bf406c8f0 0x7f4bf406edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4c08036000 tx=0x7f4c08008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:03.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.707+0000 7f4c15600640 1 -- 172.21.15.102:0/939259670 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4c00059b10 con 0x7f4c180cc470 2024-04-20T21:38:03.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.946+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c18147810 con 0x7f4c180cc470 2024-04-20T21:38:03.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.947+0000 7f4c15600640 1 -- 172.21.15.102:0/939259670 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7f4c00059410 con 0x7f4c180cc470 2024-04-20T21:38:03.946 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:03.946 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:38:03.947 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.949+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4bf406c8f0 msgr2=0x7f4bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.949+0000 7f4c1e200640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4bf406c8f0 0x7f4bf406edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4c08036000 tx=0x7f4c08008030 comp rx=0 tx=0).stop 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.949+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c180cc470 msgr2=0x7f4c18147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.949+0000 7f4c1e200640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c180cc470 0x7f4c18147b60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000c4a0 tx=0x7f4c0000c8a0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.950+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 shutdown_connections 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.950+0000 7f4c1e200640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c18155530 0x7f4c181f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.950+0000 7f4c1e200640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.950+0000 7f4c1e200640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c180cc470 0x7f4c18147b60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.950+0000 7f4c1e200640 1 --2- 172.21.15.102:0/939259670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c180cb6a0 0x7f4c18148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.950+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 >> 172.21.15.102:0/939259670 conn(0x7f4c18075080 msgr2=0x7f4c18153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.950+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 shutdown_connections 2024-04-20T21:38:03.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:03.950+0000 7f4c1e200640 1 -- 172.21.15.102:0/939259670 wait complete. 2024-04-20T21:38:03.968 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:38:05.969 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:06.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.178+0000 7f775c400640 1 Processor -- start 2024-04-20T21:38:06.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.178+0000 7f775c400640 1 -- start start 2024-04-20T21:38:06.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f775c400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f775414fa50 0x7f77540c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f775c400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7754073a10 0x7f7754158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f775c400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77541536d0 con 0x7f77540c9eb0 2024-04-20T21:38:06.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f775c400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7754159500 con 0x7f7754073a10 2024-04-20T21:38:06.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f775c400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f77540c76c0 con 0x7f775414fa50 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f775a400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7754073a10 0x7f7754158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f7759000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f77540c9eb0 0x7f77540c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:32778/0 (socket says 172.21.15.102:32778) 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f7759000640 1 -- 172.21.15.102:0/62397638 learned_addr learned my addr 172.21.15.102:0/62397638 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.180+0000 7f7759a00640 1 --2- 172.21.15.102:0/62397638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f775414fa50 0x7f77540c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.181+0000 7f7753e00640 1 -- 172.21.15.102:0/62397638 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3281450820 0 0) 0x7f7740001110 con 0x7f77540c9eb0 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.181+0000 7f7753e00640 1 -- 172.21.15.102:0/62397638 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f773c003520 con 0x7f77540c9eb0 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.181+0000 7f775a400640 1 -- 172.21.15.102:0/62397638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f775414fa50 msgr2=0x7f77540c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.181+0000 7f775a400640 1 --2- 172.21.15.102:0/62397638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f775414fa50 0x7f77540c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.181+0000 7f775a400640 1 -- 172.21.15.102:0/62397638 >> v1:172.21.15.193:6789/0 conn(0x7f77540c9eb0 legacy=0x7f77540c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.181+0000 7f775a400640 1 -- 172.21.15.102:0/62397638 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77540c7960 con 0x7f7754073a10 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.182+0000 7f775a400640 1 --2- 172.21.15.102:0/62397638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7754073a10 0x7f7754158fc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f774c009d90 tx=0x7f774c02bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc495161f1f00bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.182+0000 7f7753e00640 1 -- 172.21.15.102:0/62397638 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c039070 con 0x7f7754073a10 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.182+0000 7f7753e00640 1 -- 172.21.15.102:0/62397638 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c02cce0 con 0x7f7754073a10 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.182+0000 7f7753e00640 1 -- 172.21.15.102:0/62397638 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c034640 con 0x7f7754073a10 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.183+0000 7f775c400640 1 -- 172.21.15.102:0/62397638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7754073a10 msgr2=0x7f7754158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.183+0000 7f775c400640 1 --2- 172.21.15.102:0/62397638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7754073a10 0x7f7754158fc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f774c009d90 tx=0x7f774c02bf30 comp rx=0 tx=0).stop 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.183+0000 7f775c400640 1 -- 172.21.15.102:0/62397638 shutdown_connections 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.183+0000 7f775c400640 1 --2- 172.21.15.102:0/62397638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7754073a10 0x7f7754158fc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.183+0000 7f775c400640 1 --2- 172.21.15.102:0/62397638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f775414fa50 0x7f77540c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.183+0000 7f775c400640 1 -- 172.21.15.102:0/62397638 >> 172.21.15.102:0/62397638 conn(0x7f77540c9390 msgr2=0x7f77540c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:06.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.183+0000 7f775c400640 1 -- 172.21.15.102:0/62397638 shutdown_connections 2024-04-20T21:38:06.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.183+0000 7f775c400640 1 -- 172.21.15.102:0/62397638 wait complete. 2024-04-20T21:38:06.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.185+0000 7f775c400640 1 Processor -- start 2024-04-20T21:38:06.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.185+0000 7f775c400640 1 -- start start 2024-04-20T21:38:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.185+0000 7f775c400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7754073a10 0x7f775414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f775c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77540c9eb0 0x7f77540cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f775c400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f775414fa50 0x7f7754158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f775c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f77540c7c00 con 0x7f77540c9eb0 2024-04-20T21:38:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f775c400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f77540c3d90 con 0x7f775414fa50 2024-04-20T21:38:06.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f775c400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f77540c4030 con 0x7f7754073a10 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77540c9eb0 0x7f77540cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77540c9eb0 0x7f77540cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53396/0 (socket says 172.21.15.102:53396) 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7754073a10 0x7f775414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759000640 1 -- 172.21.15.102:0/1212793864 learned_addr learned my addr 172.21.15.102:0/1212793864 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f775a400640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f775414fa50 0x7f7754158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759a00640 1 -- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f775414fa50 msgr2=0x7f7754158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759a00640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f775414fa50 0x7f7754158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759a00640 1 -- 172.21.15.102:0/1212793864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77540c9eb0 msgr2=0x7f77540cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759a00640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77540c9eb0 0x7f77540cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759a00640 1 -- 172.21.15.102:0/1212793864 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77540cc1d0 con 0x7f7754073a10 2024-04-20T21:38:06.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.186+0000 7f7759000640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77540c9eb0 0x7f77540cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:06.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.187+0000 7f7759a00640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7754073a10 0x7f775414eba0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f774800e850 tx=0x7f774800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.187+0000 7f7751600640 1 -- 172.21.15.102:0/1212793864 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748017070 con 0x7f7754073a10 2024-04-20T21:38:06.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.187+0000 7f7751600640 1 -- 172.21.15.102:0/1212793864 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774800c8e0 con 0x7f7754073a10 2024-04-20T21:38:06.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.187+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77540cc490 con 0x7f7754073a10 2024-04-20T21:38:06.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.187+0000 7f7751600640 1 -- 172.21.15.102:0/1212793864 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774800ccb0 con 0x7f7754073a10 2024-04-20T21:38:06.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.187+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77540712e0 con 0x7f7754073a10 2024-04-20T21:38:06.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.188+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7754069340 con 0x7f7754073a10 2024-04-20T21:38:06.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.189+0000 7f7751600640 1 -- 172.21.15.102:0/1212793864 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f774801a070 con 0x7f7754073a10 2024-04-20T21:38:06.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.190+0000 7f7751600640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f773406c8f0 0x7f773406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.190+0000 7f7751600640 1 -- 172.21.15.102:0/1212793864 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f774808c5c0 con 0x7f7754073a10 2024-04-20T21:38:06.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.191+0000 7f7759000640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f773406c8f0 0x7f773406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.191+0000 7f7759000640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f773406c8f0 0x7f773406edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f775414cf10 tx=0x7f7740007bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.195+0000 7f7751600640 1 -- 172.21.15.102:0/1212793864 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7754069340 con 0x7f7754073a10 2024-04-20T21:38:06.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.435+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77540c3690 con 0x7f7754073a10 2024-04-20T21:38:06.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.436+0000 7f7751600640 1 -- 172.21.15.102:0/1212793864 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7f77540c3690 con 0x7f7754073a10 2024-04-20T21:38:06.436 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:06.436 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:38:06.436 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:38:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.438+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f773406c8f0 msgr2=0x7f773406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.438+0000 7f775c400640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f773406c8f0 0x7f773406edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f775414cf10 tx=0x7f7740007bd0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7754073a10 msgr2=0x7f775414eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7754073a10 0x7f775414eba0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f774800e850 tx=0x7f774800ed10 comp rx=0 tx=0).stop 2024-04-20T21:38:06.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 shutdown_connections 2024-04-20T21:38:06.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f773406c8f0 0x7f773406edb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f775414fa50 0x7f7754158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77540c9eb0 0x7f77540cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 --2- 172.21.15.102:0/1212793864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7754073a10 0x7f775414eba0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 >> 172.21.15.102:0/1212793864 conn(0x7f77540c9390 msgr2=0x7f77540c2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:06.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 shutdown_connections 2024-04-20T21:38:06.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:06.439+0000 7f775c400640 1 -- 172.21.15.102:0/1212793864 wait complete. 2024-04-20T21:38:06.458 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:38:08.458 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:08.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.667+0000 7fde81000640 1 Processor -- start 2024-04-20T21:38:08.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.667+0000 7fde81000640 1 -- start start 2024-04-20T21:38:08.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde81000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fde7c155530 0x7fde7c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde81000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fde7c0cc470 0x7fde7c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde81000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c15ef60 con 0x7fde7c0cdfc0 2024-04-20T21:38:08.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde81000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c0c5ca0 con 0x7fde7c0cc470 2024-04-20T21:38:08.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde81000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c0cd5f0 con 0x7fde7c155530 2024-04-20T21:38:08.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde7aa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fde7c0cc470 0x7fde7c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde7be00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fde7c0cdfc0 0x7fde7c0cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38220/0 (socket says 172.21.15.102:38220) 2024-04-20T21:38:08.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde7be00640 1 -- 172.21.15.102:0/3939716695 learned_addr learned my addr 172.21.15.102:0/3939716695 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:08.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.669+0000 7fde7b400640 1 --2- 172.21.15.102:0/3939716695 >> v2:172.21.15.193:3300/0 conn(0x7fde7c155530 0x7fde7c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.670+0000 7fde7b400640 1 -- 172.21.15.102:0/3939716695 >> v2:172.21.15.102:3300/0 conn(0x7fde7c0cc470 msgr2=0x7fde7c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.670+0000 7fde7b400640 1 --2- 172.21.15.102:0/3939716695 >> v2:172.21.15.102:3300/0 conn(0x7fde7c0cc470 0x7fde7c0cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.670+0000 7fde7b400640 1 -- 172.21.15.102:0/3939716695 >> v1:172.21.15.102:6789/0 conn(0x7fde7c0cdfc0 legacy=0x7fde7c0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.670+0000 7fde7b400640 1 -- 172.21.15.102:0/3939716695 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c15f240 con 0x7fde7c155530 2024-04-20T21:38:08.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.671+0000 7fde7b400640 1 --2- 172.21.15.102:0/3939716695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c155530 0x7fde7c15e640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fde7c06a050 tx=0x7fde6c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7634e74f3dde9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.671+0000 7fde7a000640 1 -- 172.21.15.102:0/3939716695 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c035020 con 0x7fde7c155530 2024-04-20T21:38:08.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.671+0000 7fde7a000640 1 -- 172.21.15.102:0/3939716695 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c031830 con 0x7fde7c155530 2024-04-20T21:38:08.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.671+0000 7fde7a000640 1 -- 172.21.15.102:0/3939716695 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c031c00 con 0x7fde7c155530 2024-04-20T21:38:08.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.672+0000 7fde81000640 1 -- 172.21.15.102:0/3939716695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c155530 msgr2=0x7fde7c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.672+0000 7fde81000640 1 --2- 172.21.15.102:0/3939716695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c155530 0x7fde7c15e640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fde7c06a050 tx=0x7fde6c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.672+0000 7fde81000640 1 -- 172.21.15.102:0/3939716695 shutdown_connections 2024-04-20T21:38:08.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.672+0000 7fde81000640 1 --2- 172.21.15.102:0/3939716695 >> v2:172.21.15.102:3300/0 conn(0x7fde7c0cc470 0x7fde7c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.672+0000 7fde81000640 1 --2- 172.21.15.102:0/3939716695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c155530 0x7fde7c15e640 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.672+0000 7fde81000640 1 -- 172.21.15.102:0/3939716695 >> 172.21.15.102:0/3939716695 conn(0x7fde7c075080 msgr2=0x7fde7c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:08.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.672+0000 7fde81000640 1 -- 172.21.15.102:0/3939716695 shutdown_connections 2024-04-20T21:38:08.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.672+0000 7fde81000640 1 -- 172.21.15.102:0/3939716695 wait complete. 2024-04-20T21:38:08.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.674+0000 7fde81000640 1 Processor -- start 2024-04-20T21:38:08.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.674+0000 7fde81000640 1 -- start start 2024-04-20T21:38:08.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde81000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c0cc470 0x7fde7c1f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde81000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c0cdfc0 0x7fde7c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde81000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c155530 0x7fde7c0ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde81000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0cd280 con 0x7fde7c155530 2024-04-20T21:38:08.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde81000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0ccfa0 con 0x7fde7c0cdfc0 2024-04-20T21:38:08.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde81000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0ccbe0 con 0x7fde7c0cc470 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c155530 0x7fde7c0ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7b400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c0cc470 0x7fde7c1f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c0cdfc0 0x7fde7c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7b400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c0cc470 0x7fde7c1f1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:32918/0 (socket says 172.21.15.102:32918) 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7b400640 1 -- 172.21.15.102:0/319963300 learned_addr learned my addr 172.21.15.102:0/319963300 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c0cdfc0 0x7fde7c1512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:55876/0 (socket says 172.21.15.102:55876) 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7be00640 1 -- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c0cc470 msgr2=0x7fde7c1f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7be00640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c0cc470 0x7fde7c1f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7be00640 1 -- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c0cdfc0 msgr2=0x7fde7c1512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.675+0000 7fde7be00640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c0cdfc0 0x7fde7c1512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde7be00640 1 -- 172.21.15.102:0/319963300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c14a010 con 0x7fde7c155530 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde7b400640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c0cc470 0x7fde7c1f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:08.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde7aa00640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c0cdfc0 0x7fde7c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:08.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde7be00640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c155530 0x7fde7c0ceb70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde7400c3c0 tx=0x7fde7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde73400640 1 -- 172.21.15.102:0/319963300 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74017070 con 0x7fde7c155530 2024-04-20T21:38:08.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde73400640 1 -- 172.21.15.102:0/319963300 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde74005ce0 con 0x7fde7c155530 2024-04-20T21:38:08.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c14a2f0 con 0x7fde7c155530 2024-04-20T21:38:08.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde73400640 1 -- 172.21.15.102:0/319963300 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740072d0 con 0x7fde7c155530 2024-04-20T21:38:08.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.676+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c0b5ca0 con 0x7fde7c155530 2024-04-20T21:38:08.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.678+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c04fd60 con 0x7fde7c155530 2024-04-20T21:38:08.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.678+0000 7fde73400640 1 -- 172.21.15.102:0/319963300 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde7401a070 con 0x7fde7c155530 2024-04-20T21:38:08.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.679+0000 7fde73400640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5806c8f0 0x7fde5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.679+0000 7fde73400640 1 -- 172.21.15.102:0/319963300 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7fde7408b660 con 0x7fde7c155530 2024-04-20T21:38:08.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.680+0000 7fde7b400640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5806c8f0 0x7fde5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.681+0000 7fde7b400640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5806c8f0 0x7fde5806edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009560 tx=0x7fde6c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.684+0000 7fde73400640 1 -- 172.21.15.102:0/319963300 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fde74058bb0 con 0x7fde7c155530 2024-04-20T21:38:08.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.924+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde7c0cc9c0 con 0x7fde7c155530 2024-04-20T21:38:08.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.925+0000 7fde73400640 1 -- 172.21.15.102:0/319963300 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7fde7405ceb0 con 0x7fde7c155530 2024-04-20T21:38:08.924 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:38:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5806c8f0 msgr2=0x7fde5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5806c8f0 0x7fde5806edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009560 tx=0x7fde6c03c040 comp rx=0 tx=0).stop 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c155530 msgr2=0x7fde7c0ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c155530 0x7fde7c0ceb70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde7400c3c0 tx=0x7fde7400c880 comp rx=0 tx=0).stop 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 shutdown_connections 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c155530 0x7fde7c0ceb70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fde5806c8f0 0x7fde5806edb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde7c0cdfc0 0x7fde7c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 --2- 172.21.15.102:0/319963300 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde7c0cc470 0x7fde7c1f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 >> 172.21.15.102:0/319963300 conn(0x7fde7c075080 msgr2=0x7fde7c153230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 shutdown_connections 2024-04-20T21:38:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:08.928+0000 7fde81000640 1 -- 172.21.15.102:0/319963300 wait complete. 2024-04-20T21:38:08.947 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:38:10.948 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:11.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.157+0000 7f4060200640 1 Processor -- start 2024-04-20T21:38:11.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.158+0000 7f4060200640 1 -- start start 2024-04-20T21:38:11.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f4060200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4058162e40 0x7f405806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:11.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f4060200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 0x7f40580c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:11.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f4060200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f405806d2e0 con 0x7f4058163220 2024-04-20T21:38:11.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f4060200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f40580c7290 con 0x7f4058162e40 2024-04-20T21:38:11.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f4060200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f40580c7570 con 0x7f4058163660 2024-04-20T21:38:11.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f405d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4058162e40 0x7f405806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:11.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f405d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4058162e40 0x7f405806c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45146/0 (socket says 172.21.15.102:45146) 2024-04-20T21:38:11.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f405d800640 1 -- 172.21.15.102:0/207029785 learned_addr learned my addr 172.21.15.102:0/207029785 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:11.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.159+0000 7f405e200640 1 --2- 172.21.15.102:0/207029785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 0x7f40580c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:11.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.160+0000 7f405d800640 1 -- 172.21.15.102:0/207029785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 msgr2=0x7f40580c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:11.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.160+0000 7f405d800640 1 --2- 172.21.15.102:0/207029785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 0x7f40580c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.160+0000 7f405d800640 1 -- 172.21.15.102:0/207029785 >> v1:172.21.15.193:6789/0 conn(0x7f4058163220 legacy=0x7f4058159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:11.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.160+0000 7f405d800640 1 -- 172.21.15.102:0/207029785 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405806d5c0 con 0x7f4058162e40 2024-04-20T21:38:11.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.161+0000 7f405d800640 1 --2- 172.21.15.102:0/207029785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f405806c9e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f40580c7410 tx=0x7f404c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66658bdb0fed2f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:11.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.161+0000 7f4057e00640 1 -- 172.21.15.102:0/207029785 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c034070 con 0x7f4058162e40 2024-04-20T21:38:11.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.161+0000 7f4057e00640 1 -- 172.21.15.102:0/207029785 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c030840 con 0x7f4058162e40 2024-04-20T21:38:11.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.161+0000 7f4057e00640 1 -- 172.21.15.102:0/207029785 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c030c10 con 0x7f4058162e40 2024-04-20T21:38:11.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.162+0000 7f4060200640 1 -- 172.21.15.102:0/207029785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 msgr2=0x7f405806c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:11.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.162+0000 7f4060200640 1 --2- 172.21.15.102:0/207029785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f405806c9e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f40580c7410 tx=0x7f404c02b980 comp rx=0 tx=0).stop 2024-04-20T21:38:11.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.162+0000 7f4060200640 1 -- 172.21.15.102:0/207029785 shutdown_connections 2024-04-20T21:38:11.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.162+0000 7f4060200640 1 --2- 172.21.15.102:0/207029785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 0x7f40580c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.162+0000 7f4060200640 1 --2- 172.21.15.102:0/207029785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f405806c9e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.162+0000 7f4060200640 1 -- 172.21.15.102:0/207029785 >> 172.21.15.102:0/207029785 conn(0x7f40580c9390 msgr2=0x7f40580c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:11.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.162+0000 7f4060200640 1 -- 172.21.15.102:0/207029785 shutdown_connections 2024-04-20T21:38:11.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.162+0000 7f4060200640 1 -- 172.21.15.102:0/207029785 wait complete. 2024-04-20T21:38:11.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.164+0000 7f4060200640 1 Processor -- start 2024-04-20T21:38:11.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.164+0000 7f4060200640 1 -- start start 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f4060200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f40580c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f4060200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4058163220 0x7f4058153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f4060200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 0x7f4058161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f4060200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f40580c9080 con 0x7f4058162e40 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f4060200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f405806db80 con 0x7f4058163220 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f4060200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f405806cf20 con 0x7f4058163660 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f405d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f40580c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f405d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f40580c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45156/0 (socket says 172.21.15.102:45156) 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f405d800640 1 -- 172.21.15.102:0/1285152059 learned_addr learned my addr 172.21.15.102:0/1285152059 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f405ce00640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4058163220 0x7f4058153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:11.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.165+0000 7f405e200640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 0x7f4058161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:11.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f405d800640 1 -- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 msgr2=0x7f4058161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:11.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f405d800640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 0x7f4058161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f405d800640 1 -- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4058163220 msgr2=0x7f4058153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:11.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f405d800640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4058163220 0x7f4058153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f405d800640 1 -- 172.21.15.102:0/1285152059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40581f88b0 con 0x7f4058162e40 2024-04-20T21:38:11.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f405d800640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f40580c62f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f404c000c00 tx=0x7f404c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f4055600640 1 -- 172.21.15.102:0/1285152059 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c048070 con 0x7f4058162e40 2024-04-20T21:38:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f4055600640 1 -- 172.21.15.102:0/1285152059 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c030640 con 0x7f4058162e40 2024-04-20T21:38:11.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.166+0000 7f4055600640 1 -- 172.21.15.102:0/1285152059 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c034040 con 0x7f4058162e40 2024-04-20T21:38:11.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.167+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40581f8b50 con 0x7f4058162e40 2024-04-20T21:38:11.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.167+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40581f9110 con 0x7f4058162e40 2024-04-20T21:38:11.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.168+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40581561a0 con 0x7f4058162e40 2024-04-20T21:38:11.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.169+0000 7f4055600640 1 -- 172.21.15.102:0/1285152059 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f404c04b070 con 0x7f4058162e40 2024-04-20T21:38:11.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.170+0000 7f4055600640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:11.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.170+0000 7f4055600640 1 -- 172.21.15.102:0/1285152059 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f404c0ae2f0 con 0x7f4058162e40 2024-04-20T21:38:11.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.170+0000 7f405ce00640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:11.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.171+0000 7f405ce00640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f403c06c8f0 0x7f403c06edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4058162550 tx=0x7f4048009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:11.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.174+0000 7f4055600640 1 -- 172.21.15.102:0/1285152059 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f40581561a0 con 0x7f4058162e40 2024-04-20T21:38:11.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.415+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4058074340 con 0x7f4058162e40 2024-04-20T21:38:11.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.416+0000 7f4055600640 1 -- 172.21.15.102:0/1285152059 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7f404c07b960 con 0x7f4058162e40 2024-04-20T21:38:11.416 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:38:11.416 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:38:11.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f403c06c8f0 msgr2=0x7f403c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:11.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f403c06c8f0 0x7f403c06edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4058162550 tx=0x7f4048009040 comp rx=0 tx=0).stop 2024-04-20T21:38:11.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 msgr2=0x7f40580c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:11.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f40580c62f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f404c000c00 tx=0x7f404c004730 comp rx=0 tx=0).stop 2024-04-20T21:38:11.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 shutdown_connections 2024-04-20T21:38:11.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4058163660 0x7f4058161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4058163220 0x7f4058153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 --2- 172.21.15.102:0/1285152059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4058162e40 0x7f40580c62f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:11.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 >> 172.21.15.102:0/1285152059 conn(0x7f40580c9390 msgr2=0x7f4058155a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:11.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 shutdown_connections 2024-04-20T21:38:11.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:11.419+0000 7f4060200640 1 -- 172.21.15.102:0/1285152059 wait complete. 2024-04-20T21:38:11.436 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:38:13.437 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:13.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.649+0000 7f9d2fe00640 1 Processor -- start 2024-04-20T21:38:13.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.649+0000 7f9d2fe00640 1 -- start start 2024-04-20T21:38:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.650+0000 7f9d2fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d28152210 0x7f9d280c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d2fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d280cd8d0 0x7f9d280cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d2fe00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d28158c90 con 0x7f9d28072460 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d2fe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9d280c9210 con 0x7f9d28152210 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d2fe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2815c3e0 con 0x7f9d280cd8d0 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d2d400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d28152210 0x7f9d280c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d2d400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9d28152210 0x7f9d280c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45162/0 (socket says 172.21.15.102:45162) 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d2d400640 1 -- 172.21.15.102:0/1120625985 learned_addr learned my addr 172.21.15.102:0/1120625985 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d2de00640 1 --2- 172.21.15.102:0/1120625985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d280cd8d0 0x7f9d280cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d27e00640 1 -- 172.21.15.102:0/1120625985 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939281710 0 0) 0x7f9d28158c90 con 0x7f9d28072460 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.651+0000 7f9d27e00640 1 -- 172.21.15.102:0/1120625985 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d10005730 con 0x7f9d28072460 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.652+0000 7f9d2d400640 1 -- 172.21.15.102:0/1120625985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d280cd8d0 msgr2=0x7f9d280cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.652+0000 7f9d2d400640 1 --2- 172.21.15.102:0/1120625985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d280cd8d0 0x7f9d280cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.652+0000 7f9d2d400640 1 -- 172.21.15.102:0/1120625985 >> v1:172.21.15.193:6789/0 conn(0x7f9d28072460 legacy=0x7f9d2815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:13.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.652+0000 7f9d2d400640 1 -- 172.21.15.102:0/1120625985 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d280c8b20 con 0x7f9d28152210 2024-04-20T21:38:13.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.653+0000 7f9d2d400640 1 --2- 172.21.15.102:0/1120625985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d28152210 0x7f9d280c95f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009870 tx=0x7f9d1c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984fd3d9607df126 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:13.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.653+0000 7f9d27e00640 1 -- 172.21.15.102:0/1120625985 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c02f830 con 0x7f9d28152210 2024-04-20T21:38:13.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.653+0000 7f9d27e00640 1 -- 172.21.15.102:0/1120625985 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1c02f9d0 con 0x7f9d28152210 2024-04-20T21:38:13.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.653+0000 7f9d27e00640 1 -- 172.21.15.102:0/1120625985 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c02fd80 con 0x7f9d28152210 2024-04-20T21:38:13.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.654+0000 7f9d2fe00640 1 -- 172.21.15.102:0/1120625985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d28152210 msgr2=0x7f9d280c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:13.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.654+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/1120625985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d28152210 0x7f9d280c95f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009870 tx=0x7f9d1c02ba10 comp rx=0 tx=0).stop 2024-04-20T21:38:13.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.654+0000 7f9d2fe00640 1 -- 172.21.15.102:0/1120625985 shutdown_connections 2024-04-20T21:38:13.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.654+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/1120625985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d280cd8d0 0x7f9d280cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.654+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/1120625985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d28152210 0x7f9d280c95f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.654+0000 7f9d2fe00640 1 -- 172.21.15.102:0/1120625985 >> 172.21.15.102:0/1120625985 conn(0x7f9d2806b940 msgr2=0x7f9d2806bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:13.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.654+0000 7f9d2fe00640 1 -- 172.21.15.102:0/1120625985 shutdown_connections 2024-04-20T21:38:13.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.654+0000 7f9d2fe00640 1 -- 172.21.15.102:0/1120625985 wait complete. 2024-04-20T21:38:13.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.656+0000 7f9d2fe00640 1 Processor -- start 2024-04-20T21:38:13.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.656+0000 7f9d2fe00640 1 -- start start 2024-04-20T21:38:13.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d28072460 0x7f9d2814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:13.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280cd8d0 0x7f9d2814f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:13.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d28152210 0x7f9d2814f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:13.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2815b4d0 con 0x7f9d280cd8d0 2024-04-20T21:38:13.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d280c88d0 con 0x7f9d28152210 2024-04-20T21:38:13.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2fe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2815b7b0 con 0x7f9d28072460 2024-04-20T21:38:13.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280cd8d0 0x7f9d2814f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:13.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280cd8d0 0x7f9d2814f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45170/0 (socket says 172.21.15.102:45170) 2024-04-20T21:38:13.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d28072460 0x7f9d2814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:13.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2ca00640 1 -- 172.21.15.102:0/2505714465 learned_addr learned my addr 172.21.15.102:0/2505714465 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:13.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.657+0000 7f9d2de00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d28152210 0x7f9d2814f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:13.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d2ca00640 1 -- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d28072460 msgr2=0x7f9d2814dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:13.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d2ca00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d28072460 0x7f9d2814dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d2ca00640 1 -- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d28152210 msgr2=0x7f9d2814f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:13.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d2ca00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d28152210 0x7f9d2814f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d2ca00640 1 -- 172.21.15.102:0/2505714465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d281512d0 con 0x7f9d280cd8d0 2024-04-20T21:38:13.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d2ca00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280cd8d0 0x7f9d2814f450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400c500 tx=0x7f9d1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:13.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d25600640 1 -- 172.21.15.102:0/2505714465 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d140052d0 con 0x7f9d280cd8d0 2024-04-20T21:38:13.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d25600640 1 -- 172.21.15.102:0/2505714465 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d14005490 con 0x7f9d280cd8d0 2024-04-20T21:38:13.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d25600640 1 -- 172.21.15.102:0/2505714465 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14012870 con 0x7f9d280cd8d0 2024-04-20T21:38:13.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.658+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d281f4230 con 0x7f9d280cd8d0 2024-04-20T21:38:13.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.659+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d281f47f0 con 0x7f9d280cd8d0 2024-04-20T21:38:13.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.660+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d28071f20 con 0x7f9d280cd8d0 2024-04-20T21:38:13.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.661+0000 7f9d25600640 1 -- 172.21.15.102:0/2505714465 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d14012a10 con 0x7f9d280cd8d0 2024-04-20T21:38:13.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.662+0000 7f9d25600640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d0806c8f0 0x7f9d0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:13.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.662+0000 7f9d25600640 1 -- 172.21.15.102:0/2505714465 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f9d1408bec0 con 0x7f9d280cd8d0 2024-04-20T21:38:13.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.662+0000 7f9d2d400640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d0806c8f0 0x7f9d0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:13.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.663+0000 7f9d2d400640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d0806c8f0 0x7f9d0806edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c036000 tx=0x7f9d1c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:13.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.666+0000 7f9d25600640 1 -- 172.21.15.102:0/2505714465 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d28071f20 con 0x7f9d280cd8d0 2024-04-20T21:38:13.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.918+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d28068c30 con 0x7f9d280cd8d0 2024-04-20T21:38:13.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.919+0000 7f9d25600640 1 -- 172.21.15.102:0/2505714465 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7f9d140594b0 con 0x7f9d280cd8d0 2024-04-20T21:38:13.919 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:38:13.919 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:38:13.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d0806c8f0 msgr2=0x7f9d0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:13.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d0806c8f0 0x7f9d0806edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c036000 tx=0x7f9d1c008030 comp rx=0 tx=0).stop 2024-04-20T21:38:13.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280cd8d0 msgr2=0x7f9d2814f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:13.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280cd8d0 0x7f9d2814f450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400c500 tx=0x7f9d1400c9c0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 shutdown_connections 2024-04-20T21:38:13.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9d0806c8f0 0x7f9d0806edb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d28152210 0x7f9d2814f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d280cd8d0 0x7f9d2814f450 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 --2- 172.21.15.102:0/2505714465 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d28072460 0x7f9d2814dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:13.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 >> 172.21.15.102:0/2505714465 conn(0x7f9d2806b940 msgr2=0x7f9d280c6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:13.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 shutdown_connections 2024-04-20T21:38:13.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:13.922+0000 7f9d2fe00640 1 -- 172.21.15.102:0/2505714465 wait complete. 2024-04-20T21:38:13.939 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:38:15.941 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:16.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.151+0000 7f2876800640 1 Processor -- start 2024-04-20T21:38:16.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.151+0000 7f2876800640 1 -- start start 2024-04-20T21:38:16.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.152+0000 7f2876800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2870163220 0x7f2870159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.152+0000 7f2876800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870163660 0x7f2870159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.152+0000 7f2876800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287006d2e0 con 0x7f2870162e40 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.152+0000 7f2876800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f28700c7290 con 0x7f2870163220 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.152+0000 7f2876800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f28700c7570 con 0x7f2870163660 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286f400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2870162e40 0x7f287006c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38226/0 (socket says 172.21.15.102:38226) 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286f400640 1 -- 172.21.15.102:0/639586374 learned_addr learned my addr 172.21.15.102:0/639586374 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286ea00640 1 --2- 172.21.15.102:0/639586374 >> v2:172.21.15.102:3300/0 conn(0x7f2870163220 0x7f2870159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286fe00640 1 --2- 172.21.15.102:0/639586374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870163660 0x7f2870159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286e000640 1 -- 172.21.15.102:0/639586374 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2555819103 0 0) 0x7f287006d2e0 con 0x7f2870162e40 2024-04-20T21:38:16.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286e000640 1 -- 172.21.15.102:0/639586374 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2854003520 con 0x7f2870162e40 2024-04-20T21:38:16.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286ea00640 1 -- 172.21.15.102:0/639586374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870163660 msgr2=0x7f2870159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286ea00640 1 --2- 172.21.15.102:0/639586374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870163660 0x7f2870159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286ea00640 1 -- 172.21.15.102:0/639586374 >> v1:172.21.15.102:6789/0 conn(0x7f2870162e40 legacy=0x7f287006c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.153+0000 7f286ea00640 1 -- 172.21.15.102:0/639586374 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287006d5c0 con 0x7f2870163220 2024-04-20T21:38:16.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.154+0000 7f286ea00640 1 --2- 172.21.15.102:0/639586374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163220 0x7f2870159440 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f28700c7410 tx=0x7f285c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d5bb8a46c0cbf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.155+0000 7f286e000640 1 -- 172.21.15.102:0/639586374 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c035020 con 0x7f2870163220 2024-04-20T21:38:16.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.155+0000 7f286e000640 1 -- 172.21.15.102:0/639586374 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c031800 con 0x7f2870163220 2024-04-20T21:38:16.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.155+0000 7f286e000640 1 -- 172.21.15.102:0/639586374 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c031bd0 con 0x7f2870163220 2024-04-20T21:38:16.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.155+0000 7f2876800640 1 -- 172.21.15.102:0/639586374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163220 msgr2=0x7f2870159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.155+0000 7f2876800640 1 --2- 172.21.15.102:0/639586374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163220 0x7f2870159440 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f28700c7410 tx=0x7f285c02b8a0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.156+0000 7f2876800640 1 -- 172.21.15.102:0/639586374 shutdown_connections 2024-04-20T21:38:16.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.156+0000 7f2876800640 1 --2- 172.21.15.102:0/639586374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870163660 0x7f2870159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.156+0000 7f2876800640 1 --2- 172.21.15.102:0/639586374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163220 0x7f2870159440 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.156+0000 7f2876800640 1 -- 172.21.15.102:0/639586374 >> 172.21.15.102:0/639586374 conn(0x7f28700c9390 msgr2=0x7f28700c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.156+0000 7f2876800640 1 -- 172.21.15.102:0/639586374 shutdown_connections 2024-04-20T21:38:16.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.156+0000 7f2876800640 1 -- 172.21.15.102:0/639586374 wait complete. 2024-04-20T21:38:16.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.157+0000 7f2876800640 1 Processor -- start 2024-04-20T21:38:16.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.158+0000 7f2876800640 1 -- start start 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.158+0000 7f2876800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870162e40 0x7f28701f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.158+0000 7f2876800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2870163220 0x7f28701f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.158+0000 7f2876800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163660 0x7f28700cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.158+0000 7f2876800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28700c9080 con 0x7f2870163660 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.158+0000 7f2876800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f287006db80 con 0x7f2870163220 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.158+0000 7f2876800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f28700c7a00 con 0x7f2870162e40 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870162e40 0x7f28701f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2870163220 0x7f28701f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870162e40 0x7f28701f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:32980/0 (socket says 172.21.15.102:32980) 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286f400640 1 -- 172.21.15.102:0/1474488865 learned_addr learned my addr 172.21.15.102:0/1474488865 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:16.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286fe00640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163660 0x7f28700cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286ea00640 1 -- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870162e40 msgr2=0x7f28701f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286ea00640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870162e40 0x7f28701f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286ea00640 1 -- 172.21.15.102:0/1474488865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163660 msgr2=0x7f28700cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286ea00640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163660 0x7f28700cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286ea00640 1 -- 172.21.15.102:0/1474488865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28701f8ae0 con 0x7f2870163220 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286fe00640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163660 0x7f28700cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.159+0000 7f286f400640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870162e40 0x7f28701f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.160+0000 7f286ea00640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2870163220 0x7f28701f85a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2870163b10 tx=0x7f285c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.160+0000 7f2867400640 1 -- 172.21.15.102:0/1474488865 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c035040 con 0x7f2870163220 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.160+0000 7f2867400640 1 -- 172.21.15.102:0/1474488865 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c031e50 con 0x7f2870163220 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.160+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28701f5030 con 0x7f2870163220 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.160+0000 7f2867400640 1 -- 172.21.15.102:0/1474488865 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c046480 con 0x7f2870163220 2024-04-20T21:38:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.160+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28701f8d70 con 0x7f2870163220 2024-04-20T21:38:16.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.162+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2870074340 con 0x7f2870163220 2024-04-20T21:38:16.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.162+0000 7f2867400640 1 -- 172.21.15.102:0/1474488865 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f285c038020 con 0x7f2870163220 2024-04-20T21:38:16.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.163+0000 7f2867400640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f285006c8f0 0x7f285006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.163+0000 7f2867400640 1 -- 172.21.15.102:0/1474488865 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f285c0ae0f0 con 0x7f2870163220 2024-04-20T21:38:16.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.163+0000 7f286f400640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f285006c8f0 0x7f285006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.164+0000 7f286f400640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f285006c8f0 0x7f285006edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2860005880 tx=0x7f2860009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.168+0000 7f2867400640 1 -- 172.21.15.102:0/1474488865 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f285c07b6d0 con 0x7f2870163220 2024-04-20T21:38:16.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.411+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2870071d50 con 0x7f2870163220 2024-04-20T21:38:16.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.412+0000 7f2867400640 1 -- 172.21.15.102:0/1474488865 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7f285c07f9d0 con 0x7f2870163220 2024-04-20T21:38:16.412 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:16.412 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:38:16.412 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:38:16.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.415+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f285006c8f0 msgr2=0x7f285006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.415+0000 7f2876800640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f285006c8f0 0x7f285006edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2860005880 tx=0x7f2860009040 comp rx=0 tx=0).stop 2024-04-20T21:38:16.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.415+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2870163220 msgr2=0x7f28701f85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.415+0000 7f2876800640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2870163220 0x7f28701f85a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2870163b10 tx=0x7f285c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.416+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 shutdown_connections 2024-04-20T21:38:16.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.416+0000 7f2876800640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2870163660 0x7f28700cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.416+0000 7f2876800640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f285006c8f0 0x7f285006edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.416+0000 7f2876800640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2870163220 0x7f28701f85a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.416+0000 7f2876800640 1 --2- 172.21.15.102:0/1474488865 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2870162e40 0x7f28701f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.416+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 >> 172.21.15.102:0/1474488865 conn(0x7f28700c9390 msgr2=0x7f287014e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.416+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 shutdown_connections 2024-04-20T21:38:16.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:16.416+0000 7f2876800640 1 -- 172.21.15.102:0/1474488865 wait complete. 2024-04-20T21:38:16.434 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:38:18.435 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:18.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.658+0000 7f7167c00640 1 Processor -- start 2024-04-20T21:38:18.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.658+0000 7f7167c00640 1 -- start start 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7167c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f716014fa50 0x7f71600c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7167c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7160073a10 0x7f7160158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7167c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71601536d0 con 0x7f71600c9eb0 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7167c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f7160159500 con 0x7f716014fa50 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7167c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f71600c76c0 con 0x7f7160073a10 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7165200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f716014fa50 0x7f71600c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7165200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f716014fa50 0x7f71600c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34868/0 (socket says 172.21.15.102:34868) 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7165200640 1 -- 172.21.15.102:0/2891909451 learned_addr learned my addr 172.21.15.102:0/2891909451 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:18.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f7165c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7160073a10 0x7f7160158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.660+0000 7f715fe00640 1 --1- 172.21.15.102:0/2891909451 >> v1:172.21.15.193:6789/0 conn(0x7f71600c9eb0 0x7f71600c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55362/0 (socket says 172.21.15.102:55362) 2024-04-20T21:38:18.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.661+0000 7f7165c00640 1 -- 172.21.15.102:0/2891909451 >> v2:172.21.15.102:3300/0 conn(0x7f716014fa50 msgr2=0x7f71600c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.661+0000 7f7165c00640 1 --2- 172.21.15.102:0/2891909451 >> v2:172.21.15.102:3300/0 conn(0x7f716014fa50 0x7f71600c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.661+0000 7f7165c00640 1 -- 172.21.15.102:0/2891909451 >> v1:172.21.15.193:6789/0 conn(0x7f71600c9eb0 legacy=0x7f71600c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.661+0000 7f7165c00640 1 -- 172.21.15.102:0/2891909451 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71600c7960 con 0x7f7160073a10 2024-04-20T21:38:18.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.661+0000 7f715f400640 1 -- 172.21.15.102:0/2891909451 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2010492354 0 0) 0x7f714c001110 con 0x7f71600c9eb0 2024-04-20T21:38:18.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.662+0000 7f7165c00640 1 --2- 172.21.15.102:0/2891909451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f7160158fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7160156b00 tx=0x7f715802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3cacf87c0d66185 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.662+0000 7f715f400640 1 -- 172.21.15.102:0/2891909451 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158039070 con 0x7f7160073a10 2024-04-20T21:38:18.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.662+0000 7f715f400640 1 -- 172.21.15.102:0/2891909451 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715802cd00 con 0x7f7160073a10 2024-04-20T21:38:18.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.662+0000 7f715f400640 1 -- 172.21.15.102:0/2891909451 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158034660 con 0x7f7160073a10 2024-04-20T21:38:18.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.663+0000 7f7167c00640 1 -- 172.21.15.102:0/2891909451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 msgr2=0x7f7160158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.663+0000 7f7167c00640 1 --2- 172.21.15.102:0/2891909451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f7160158fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7160156b00 tx=0x7f715802bf50 comp rx=0 tx=0).stop 2024-04-20T21:38:18.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.663+0000 7f7167c00640 1 -- 172.21.15.102:0/2891909451 shutdown_connections 2024-04-20T21:38:18.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.663+0000 7f7167c00640 1 --2- 172.21.15.102:0/2891909451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f7160158fc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.663+0000 7f7167c00640 1 --2- 172.21.15.102:0/2891909451 >> v2:172.21.15.102:3300/0 conn(0x7f716014fa50 0x7f71600c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.663+0000 7f7167c00640 1 -- 172.21.15.102:0/2891909451 >> 172.21.15.102:0/2891909451 conn(0x7f71600c9390 msgr2=0x7f71600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.663+0000 7f7167c00640 1 -- 172.21.15.102:0/2891909451 shutdown_connections 2024-04-20T21:38:18.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.663+0000 7f7167c00640 1 -- 172.21.15.102:0/2891909451 wait complete. 2024-04-20T21:38:18.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.664+0000 7f7167c00640 1 Processor -- start 2024-04-20T21:38:18.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.665+0000 7f7167c00640 1 -- start start 2024-04-20T21:38:18.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.665+0000 7f7167c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f716014a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.665+0000 7f7167c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71600c9eb0 0x7f71601e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.665+0000 7f7167c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f716014fa50 0x7f716015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.665+0000 7f7167c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71600c7c00 con 0x7f71600c9eb0 2024-04-20T21:38:18.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.665+0000 7f7167c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71600c3d90 con 0x7f7160073a10 2024-04-20T21:38:18.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.665+0000 7f7167c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f71600c4030 con 0x7f716014fa50 2024-04-20T21:38:18.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f716014a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f715fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71600c9eb0 0x7f71601e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f716014fa50 0x7f716015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f716014a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:50898/0 (socket says 172.21.15.102:50898) 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 -- 172.21.15.102:0/2512680669 learned_addr learned my addr 172.21.15.102:0/2512680669 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 -- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f716014fa50 msgr2=0x7f716015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f716014fa50 0x7f716015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 -- 172.21.15.102:0/2512680669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71600c9eb0 msgr2=0x7f71601e3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71600c9eb0 0x7f71601e3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 -- 172.21.15.102:0/2512680669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716015d030 con 0x7f7160073a10 2024-04-20T21:38:18.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.666+0000 7f7165200640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f716014a9a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f715400e840 tx=0x7f715400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.667+0000 7f715cc00640 1 -- 172.21.15.102:0/2512680669 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154017070 con 0x7f7160073a10 2024-04-20T21:38:18.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.667+0000 7f715cc00640 1 -- 172.21.15.102:0/2512680669 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715400c8e0 con 0x7f7160073a10 2024-04-20T21:38:18.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.667+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71601e4640 con 0x7f7160073a10 2024-04-20T21:38:18.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.667+0000 7f715cc00640 1 -- 172.21.15.102:0/2512680669 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715400ccd0 con 0x7f7160073a10 2024-04-20T21:38:18.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.667+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601473d0 con 0x7f7160073a10 2024-04-20T21:38:18.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.668+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7160074340 con 0x7f7160073a10 2024-04-20T21:38:18.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.669+0000 7f715cc00640 1 -- 172.21.15.102:0/2512680669 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f715401a070 con 0x7f7160073a10 2024-04-20T21:38:18.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.670+0000 7f715cc00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f714006c920 0x7f714006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.670+0000 7f715cc00640 1 -- 172.21.15.102:0/2512680669 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7595+0+0 (secure 0 0 0) 0x7f715408c540 con 0x7f7160073a10 2024-04-20T21:38:18.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.670+0000 7f715fe00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f714006c920 0x7f714006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.671+0000 7f715fe00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f714006c920 0x7f714006ede0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f714c000f60 tx=0x7f714c007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.675+0000 7f715cc00640 1 -- 172.21.15.102:0/2512680669 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7154059a90 con 0x7f7160073a10 2024-04-20T21:38:18.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.920+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f716015d3c0 con 0x7f7160073a10 2024-04-20T21:38:18.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.921+0000 7f715cc00640 1 -- 172.21.15.102:0/2512680669 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+6856 (secure 0 0 0) 0x7f715405dd90 con 0x7f7160073a10 2024-04-20T21:38:18.920 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:18.920 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:36:19.777686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.102:6835/945751373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":945751373},{"type":"v1","addr":"172.21.15.102:6835","nonce":945751373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6147,"qdb_cluster":[6147]},"id":1}]} 2024-04-20T21:38:18.920 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 67 2024-04-20T21:38:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f714006c920 msgr2=0x7f714006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f714006c920 0x7f714006ede0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f714c000f60 tx=0x7f714c007b50 comp rx=0 tx=0).stop 2024-04-20T21:38:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 msgr2=0x7f716014a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f716014a9a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f715400e840 tx=0x7f715400ed00 comp rx=0 tx=0).stop 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 shutdown_connections 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f716014fa50 0x7f716015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71600c9eb0 0x7f71601e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f714006c920 0x7f714006ede0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 --2- 172.21.15.102:0/2512680669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7160073a10 0x7f716014a9a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 >> 172.21.15.102:0/2512680669 conn(0x7f71600c9390 msgr2=0x7f716014bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 shutdown_connections 2024-04-20T21:38:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:18.924+0000 7f7167c00640 1 -- 172.21.15.102:0/2512680669 wait complete. 2024-04-20T21:38:18.941 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:38:20.942 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:21.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.147+0000 7f073ce00640 1 Processor -- start 2024-04-20T21:38:21.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.147+0000 7f073ce00640 1 -- start start 2024-04-20T21:38:21.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.149+0000 7f073ce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07380cb4c0 0x7f07380cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.149+0000 7f073ce00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f073815b750 0x7f073815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.149+0000 7f073ce00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07380cb8e0 con 0x7f07381555e0 2024-04-20T21:38:21.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.149+0000 7f073ce00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f07380cc840 con 0x7f073815b750 2024-04-20T21:38:21.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.149+0000 7f073ce00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0738073a90 con 0x7f07380cb4c0 2024-04-20T21:38:21.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.150+0000 7f0735400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07380cb4c0 0x7f07380cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.149+0000 7f0735e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f07381555e0 0x7f07380cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42122/0 (socket says 172.21.15.102:42122) 2024-04-20T21:38:21.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.150+0000 7f0735e00640 1 -- 172.21.15.102:0/1543397840 learned_addr learned my addr 172.21.15.102:0/1543397840 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:21.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.150+0000 7f0736800640 1 --2- 172.21.15.102:0/1543397840 >> v2:172.21.15.193:3300/0 conn(0x7f073815b750 0x7f073815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.150+0000 7f0736800640 1 -- 172.21.15.102:0/1543397840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07380cb4c0 msgr2=0x7f07380cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:21.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.150+0000 7f0736800640 1 --2- 172.21.15.102:0/1543397840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07380cb4c0 0x7f07380cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.150+0000 7f0736800640 1 -- 172.21.15.102:0/1543397840 >> v1:172.21.15.102:6789/0 conn(0x7f07381555e0 legacy=0x7f07380cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.150+0000 7f0736800640 1 -- 172.21.15.102:0/1543397840 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073815c100 con 0x7f073815b750 2024-04-20T21:38:21.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.150+0000 7f0735400640 1 --2- 172.21.15.102:0/1543397840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07380cb4c0 0x7f07380cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:21.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.151+0000 7f0736800640 1 --2- 172.21.15.102:0/1543397840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f073815b750 0x7f073815bbc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f07300097a0 tx=0x7f073002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf64e889162cb7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:21.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.151+0000 7f0734a00640 1 -- 172.21.15.102:0/1543397840 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730034070 con 0x7f073815b750 2024-04-20T21:38:21.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f0734a00640 1 -- 172.21.15.102:0/1543397840 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0730030800 con 0x7f073815b750 2024-04-20T21:38:21.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f0734a00640 1 -- 172.21.15.102:0/1543397840 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730030bd0 con 0x7f073815b750 2024-04-20T21:38:21.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f073ce00640 1 -- 172.21.15.102:0/1543397840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f073815b750 msgr2=0x7f073815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f073ce00640 1 --2- 172.21.15.102:0/1543397840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f073815b750 0x7f073815bbc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f07300097a0 tx=0x7f073002b940 comp rx=0 tx=0).stop 2024-04-20T21:38:21.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f073ce00640 1 -- 172.21.15.102:0/1543397840 shutdown_connections 2024-04-20T21:38:21.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f073ce00640 1 --2- 172.21.15.102:0/1543397840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f073815b750 0x7f073815bbc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f073ce00640 1 --2- 172.21.15.102:0/1543397840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07380cb4c0 0x7f07380cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f073ce00640 1 -- 172.21.15.102:0/1543397840 >> 172.21.15.102:0/1543397840 conn(0x7f07380754d0 msgr2=0x7f0738072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:21.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f073ce00640 1 -- 172.21.15.102:0/1543397840 shutdown_connections 2024-04-20T21:38:21.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.152+0000 7f073ce00640 1 -- 172.21.15.102:0/1543397840 wait complete. 2024-04-20T21:38:21.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.154+0000 7f073ce00640 1 Processor -- start 2024-04-20T21:38:21.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.154+0000 7f073ce00640 1 -- start start 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f073ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07380cb4c0 0x7f0738142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f073ce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07381555e0 0x7f07381f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f073ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f073815b750 0x7f07380cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f073ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07381559c0 con 0x7f073815b750 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f073ce00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f073815e840 con 0x7f07380cb4c0 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f073ce00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f073815ea90 con 0x7f07381555e0 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f0736800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f073815b750 0x7f07380cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f0736800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f073815b750 0x7f07380cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34904/0 (socket says 172.21.15.102:34904) 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f0735e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07380cb4c0 0x7f0738142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f0736800640 1 -- 172.21.15.102:0/3414163639 learned_addr learned my addr 172.21.15.102:0/3414163639 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.155+0000 7f0735400640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07381555e0 0x7f07381f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.156+0000 7f0736800640 1 -- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07381555e0 msgr2=0x7f07381f0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.156+0000 7f0736800640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07381555e0 0x7f07381f0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.156+0000 7f0736800640 1 -- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07380cb4c0 msgr2=0x7f0738142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.156+0000 7f0736800640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07380cb4c0 0x7f0738142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.156+0000 7f0736800640 1 -- 172.21.15.102:0/3414163639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073815a370 con 0x7f073815b750 2024-04-20T21:38:21.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.156+0000 7f0736800640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f073815b750 0x7f07380cd4c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f07300098c0 tx=0x7f07300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:21.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.156+0000 7f072e000640 1 -- 172.21.15.102:0/3414163639 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730034040 con 0x7f073815b750 2024-04-20T21:38:21.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.157+0000 7f072e000640 1 -- 172.21.15.102:0/3414163639 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073003ad20 con 0x7f073815b750 2024-04-20T21:38:21.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.157+0000 7f072e000640 1 -- 172.21.15.102:0/3414163639 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730006e10 con 0x7f073815b750 2024-04-20T21:38:21.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.157+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07381f5160 con 0x7f073815b750 2024-04-20T21:38:21.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.157+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073815c950 con 0x7f073815b750 2024-04-20T21:38:21.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.158+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0738068c30 con 0x7f073815b750 2024-04-20T21:38:21.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.159+0000 7f072e000640 1 -- 172.21.15.102:0/3414163639 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0730037070 con 0x7f073815b750 2024-04-20T21:38:21.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.160+0000 7f072e000640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f071406c8f0 0x7f071406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.160+0000 7f072e000640 1 -- 172.21.15.102:0/3414163639 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f07300ae230 con 0x7f073815b750 2024-04-20T21:38:21.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.160+0000 7f0735e00640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f071406c8f0 0x7f071406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.161+0000 7f0735e00640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f071406c8f0 0x7f071406edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f07280012e0 tx=0x7f0728001a30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:21.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.165+0000 7f072e000640 1 -- 172.21.15.102:0/3414163639 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f073815a370 con 0x7f073815b750 2024-04-20T21:38:21.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.406+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f073815cd70 con 0x7f073815b750 2024-04-20T21:38:21.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.407+0000 7f072e000640 1 -- 172.21.15.102:0/3414163639 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+6023 (secure 0 0 0) 0x7f073003f070 con 0x7f073815b750 2024-04-20T21:38:21.405 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:21.405 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:20.801907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:21.405 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 68 2024-04-20T21:38:21.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.409+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f071406c8f0 msgr2=0x7f071406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.409+0000 7f073ce00640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f071406c8f0 0x7f071406edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f07280012e0 tx=0x7f0728001a30 comp rx=0 tx=0).stop 2024-04-20T21:38:21.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.409+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f073815b750 msgr2=0x7f07380cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.409+0000 7f073ce00640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f073815b750 0x7f07380cd4c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f07300098c0 tx=0x7f07300046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.410+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 shutdown_connections 2024-04-20T21:38:21.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.410+0000 7f073ce00640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f073815b750 0x7f07380cd4c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.410+0000 7f073ce00640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07381555e0 0x7f07381f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.410+0000 7f073ce00640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f071406c8f0 0x7f071406edb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.410+0000 7f073ce00640 1 --2- 172.21.15.102:0/3414163639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07380cb4c0 0x7f0738142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.410+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 >> 172.21.15.102:0/3414163639 conn(0x7f07380754d0 msgr2=0x7f0738072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:21.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.410+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 shutdown_connections 2024-04-20T21:38:21.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.410+0000 7f073ce00640 1 -- 172.21.15.102:0/3414163639 wait complete. 2024-04-20T21:38:21.427 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:38:21.427 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:38:21.427 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:21.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.628+0000 7f35ef800640 1 Processor -- start 2024-04-20T21:38:21.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.628+0000 7f35ef800640 1 -- start start 2024-04-20T21:38:21.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.629+0000 7f35ef800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8152170 0x7f35e80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.629+0000 7f35ef800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e8071f70 con 0x7f35e8158770 2024-04-20T21:38:21.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.629+0000 7f35ef800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e8072210 con 0x7f35e8158b50 2024-04-20T21:38:21.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.629+0000 7f35ef800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f35e80c6dc0 con 0x7f35e8152170 2024-04-20T21:38:21.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35dfe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f35e8158770 0x7f35e8070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42126/0 (socket says 172.21.15.102:42126) 2024-04-20T21:38:21.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35dfe00640 1 -- 172.21.15.102:0/4087024105 learned_addr learned my addr 172.21.15.102:0/4087024105 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:21.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35ece00640 1 --2- 172.21.15.102:0/4087024105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8152170 0x7f35e80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35e7e00640 1 -- 172.21.15.102:0/4087024105 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3168192003 0 0) 0x7f35e8071f70 con 0x7f35e8158770 2024-04-20T21:38:21.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35e7e00640 1 -- 172.21.15.102:0/4087024105 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35cc003520 con 0x7f35e8158770 2024-04-20T21:38:21.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35e7e00640 1 -- 172.21.15.102:0/4087024105 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756844097 0 0) 0x7f35e8072210 con 0x7f35e8158b50 2024-04-20T21:38:21.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35e7e00640 1 -- 172.21.15.102:0/4087024105 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35e8071f70 con 0x7f35e8158b50 2024-04-20T21:38:21.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35ece00640 1 -- 172.21.15.102:0/4087024105 >> v1:172.21.15.193:6789/0 conn(0x7f35e8158b50 legacy=0x7f35e81628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35ece00640 1 -- 172.21.15.102:0/4087024105 >> v1:172.21.15.102:6789/0 conn(0x7f35e8158770 legacy=0x7f35e8070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.630+0000 7f35ece00640 1 -- 172.21.15.102:0/4087024105 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e80724b0 con 0x7f35e8152170 2024-04-20T21:38:21.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.632+0000 7f35ece00640 1 --2- 172.21.15.102:0/4087024105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8152170 0x7f35e80c8db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35e80723e0 tx=0x7f35d802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155bc2ada1256636 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:21.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.632+0000 7f35e7e00640 1 -- 172.21.15.102:0/4087024105 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8034070 con 0x7f35e8152170 2024-04-20T21:38:21.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.632+0000 7f35e7e00640 1 -- 172.21.15.102:0/4087024105 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d8030880 con 0x7f35e8152170 2024-04-20T21:38:21.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.632+0000 7f35e7e00640 1 -- 172.21.15.102:0/4087024105 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8030c30 con 0x7f35e8152170 2024-04-20T21:38:21.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.632+0000 7f35ef800640 1 -- 172.21.15.102:0/4087024105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8152170 msgr2=0x7f35e80c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.632+0000 7f35ef800640 1 --2- 172.21.15.102:0/4087024105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8152170 0x7f35e80c8db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35e80723e0 tx=0x7f35d802b9c0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.633+0000 7f35ef800640 1 -- 172.21.15.102:0/4087024105 shutdown_connections 2024-04-20T21:38:21.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.633+0000 7f35ef800640 1 --2- 172.21.15.102:0/4087024105 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8152170 0x7f35e80c8db0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.633+0000 7f35ef800640 1 -- 172.21.15.102:0/4087024105 >> 172.21.15.102:0/4087024105 conn(0x7f35e80c9390 msgr2=0x7f35e80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:21.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.633+0000 7f35ef800640 1 -- 172.21.15.102:0/4087024105 shutdown_connections 2024-04-20T21:38:21.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.633+0000 7f35ef800640 1 -- 172.21.15.102:0/4087024105 wait complete. 2024-04-20T21:38:21.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.634+0000 7f35ef800640 1 Processor -- start 2024-04-20T21:38:21.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.635+0000 7f35ef800640 1 -- start start 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.635+0000 7f35ef800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8152170 0x7f35e815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ef800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8158770 0x7f35e8144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ef800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8158b50 0x7f35e81448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ef800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f35e80c9f20 con 0x7f35e8152170 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ef800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8158250 con 0x7f35e8158770 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ef800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f35e8157f20 con 0x7f35e8158b50 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8152170 0x7f35e815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8152170 0x7f35e815a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34914/0 (socket says 172.21.15.102:34914) 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 -- 172.21.15.102:0/1358390209 learned_addr learned my addr 172.21.15.102:0/1358390209 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35dfe00640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8158770 0x7f35e8144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ed800640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8158b50 0x7f35e81448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 -- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8158b50 msgr2=0x7f35e81448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8158b50 0x7f35e81448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 -- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8158770 msgr2=0x7f35e8144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8158770 0x7f35e8144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 -- 172.21.15.102:0/1358390209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e815afb0 con 0x7f35e8152170 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.636+0000 7f35ece00640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8152170 0x7f35e815a5b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f35d8000c00 tx=0x7f35d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.637+0000 7f35e5600640 1 -- 172.21.15.102:0/1358390209 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8047070 con 0x7f35e8152170 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.637+0000 7f35e5600640 1 -- 172.21.15.102:0/1358390209 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d8030680 con 0x7f35e8152170 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.637+0000 7f35e5600640 1 -- 172.21.15.102:0/1358390209 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8034050 con 0x7f35e8152170 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.637+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e814c240 con 0x7f35e8152170 2024-04-20T21:38:21.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.637+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81497c0 con 0x7f35e8152170 2024-04-20T21:38:21.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.638+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e8074340 con 0x7f35e8152170 2024-04-20T21:38:21.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.639+0000 7f35e5600640 1 -- 172.21.15.102:0/1358390209 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35d804a070 con 0x7f35e8152170 2024-04-20T21:38:21.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.640+0000 7f35e5600640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f35c406c8f0 0x7f35c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:21.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.640+0000 7f35e5600640 1 -- 172.21.15.102:0/1358390209 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f35d80ae120 con 0x7f35e8152170 2024-04-20T21:38:21.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.641+0000 7f35dfe00640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f35c406c8f0 0x7f35c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:21.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.641+0000 7f35dfe00640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f35c406c8f0 0x7f35c406edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35e814b760 tx=0x7f35d0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:21.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.645+0000 7f35e5600640 1 -- 172.21.15.102:0/1358390209 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f35d807b730 con 0x7f35e8152170 2024-04-20T21:38:21.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.886+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e80c3460 con 0x7f35e8152170 2024-04-20T21:38:21.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.887+0000 7f35e5600640 1 -- 172.21.15.102:0/1358390209 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+6023 (secure 0 0 0) 0x7f35d807fa30 con 0x7f35e8152170 2024-04-20T21:38:21.886 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:21.886 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:20.801907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:21.887 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 68 2024-04-20T21:38:21.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f35c406c8f0 msgr2=0x7f35c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f35c406c8f0 0x7f35c406edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35e814b760 tx=0x7f35d0009040 comp rx=0 tx=0).stop 2024-04-20T21:38:21.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8152170 msgr2=0x7f35e815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:21.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8152170 0x7f35e815a5b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f35d8000c00 tx=0x7f35d80046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 shutdown_connections 2024-04-20T21:38:21.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35e8158b50 0x7f35e81448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f35c406c8f0 0x7f35c406edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8158770 0x7f35e8144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 --2- 172.21.15.102:0/1358390209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8152170 0x7f35e815a5b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:21.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 >> 172.21.15.102:0/1358390209 conn(0x7f35e80c9390 msgr2=0x7f35e81567d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:21.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 shutdown_connections 2024-04-20T21:38:21.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:21.891+0000 7f35ef800640 1 -- 172.21.15.102:0/1358390209 wait complete. 2024-04-20T21:38:21.911 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 68, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 68, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6421": { "addr": "172.21.15.102:6833/327932375", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 327932375, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 327932375, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6421, "incarnation": 68, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:38:20.801907+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6421 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:38:23.911 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:24.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.170+0000 7ff2ba000640 1 Processor -- start 2024-04-20T21:38:24.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.170+0000 7ff2ba000640 1 -- start start 2024-04-20T21:38:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.171+0000 7ff2ba000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff2b4151c90 0x7ff2b40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.171+0000 7ff2ba000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b4071a70 0x7ff2b406d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.171+0000 7ff2ba000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b4071040 con 0x7ff2b40c7290 2024-04-20T21:38:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.171+0000 7ff2ba000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b40c9200 con 0x7ff2b4151c90 2024-04-20T21:38:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.171+0000 7ff2ba000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b4070df0 con 0x7ff2b4071a70 2024-04-20T21:38:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b3400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff2b4151c90 0x7ff2b40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b2a00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff2b40c7290 0x7ff2b4158240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55366/0 (socket says 172.21.15.102:55366) 2024-04-20T21:38:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b2a00640 1 -- 172.21.15.102:0/572868973 learned_addr learned my addr 172.21.15.102:0/572868973 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:24.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b3e00640 1 --2- 172.21.15.102:0/572868973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b4071a70 0x7ff2b406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:24.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b3400640 1 -- 172.21.15.102:0/572868973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b4071a70 msgr2=0x7ff2b406d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:24.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b3400640 1 --2- 172.21.15.102:0/572868973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b4071a70 0x7ff2b406d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b3400640 1 -- 172.21.15.102:0/572868973 >> v1:172.21.15.193:6789/0 conn(0x7ff2b40c7290 legacy=0x7ff2b4158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:24.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b3400640 1 -- 172.21.15.102:0/572868973 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b4071320 con 0x7ff2b4151c90 2024-04-20T21:38:24.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.172+0000 7ff2b2000640 1 -- 172.21.15.102:0/572868973 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630023044 0 0) 0x7ff2b4071040 con 0x7ff2b40c7290 2024-04-20T21:38:24.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.173+0000 7ff2b3400640 1 --2- 172.21.15.102:0/572868973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4151c90 0x7ff2b40c8cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40097a0 tx=0x7ff2a402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa209da9de24747 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:24.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2b2000640 1 -- 172.21.15.102:0/572868973 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4034070 con 0x7ff2b4151c90 2024-04-20T21:38:24.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2b2000640 1 -- 172.21.15.102:0/572868973 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a4030610 con 0x7ff2b4151c90 2024-04-20T21:38:24.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2b2000640 1 -- 172.21.15.102:0/572868973 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40309e0 con 0x7ff2b4151c90 2024-04-20T21:38:24.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2ba000640 1 -- 172.21.15.102:0/572868973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4151c90 msgr2=0x7ff2b40c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:24.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2ba000640 1 --2- 172.21.15.102:0/572868973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4151c90 0x7ff2b40c8cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40097a0 tx=0x7ff2a402b960 comp rx=0 tx=0).stop 2024-04-20T21:38:24.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2ba000640 1 -- 172.21.15.102:0/572868973 shutdown_connections 2024-04-20T21:38:24.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2ba000640 1 --2- 172.21.15.102:0/572868973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b4071a70 0x7ff2b406d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2ba000640 1 --2- 172.21.15.102:0/572868973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4151c90 0x7ff2b40c8cc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2ba000640 1 -- 172.21.15.102:0/572868973 >> 172.21.15.102:0/572868973 conn(0x7ff2b40c9390 msgr2=0x7ff2b40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:24.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2ba000640 1 -- 172.21.15.102:0/572868973 shutdown_connections 2024-04-20T21:38:24.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.174+0000 7ff2ba000640 1 -- 172.21.15.102:0/572868973 wait complete. 2024-04-20T21:38:24.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.176+0000 7ff2ba000640 1 Processor -- start 2024-04-20T21:38:24.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.176+0000 7ff2ba000640 1 -- start start 2024-04-20T21:38:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2ba000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4071a70 0x7ff2b414da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2ba000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b40c7290 0x7ff2b414cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2ba000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2b4151c90 0x7ff2b4154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2ba000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4157a40 con 0x7ff2b4151c90 2024-04-20T21:38:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2ba000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4157db0 con 0x7ff2b4071a70 2024-04-20T21:38:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2ba000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b40ca1f0 con 0x7ff2b40c7290 2024-04-20T21:38:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2b3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2b4151c90 0x7ff2b4154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2b3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2b4151c90 0x7ff2b4154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34920/0 (socket says 172.21.15.102:34920) 2024-04-20T21:38:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2b3e00640 1 -- 172.21.15.102:0/876156464 learned_addr learned my addr 172.21.15.102:0/876156464 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.177+0000 7ff2b3400640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4071a70 0x7ff2b414da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.178+0000 7ff2b2a00640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b40c7290 0x7ff2b414cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.178+0000 7ff2b3e00640 1 -- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b40c7290 msgr2=0x7ff2b414cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.178+0000 7ff2b3e00640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b40c7290 0x7ff2b414cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.178+0000 7ff2b3e00640 1 -- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4071a70 msgr2=0x7ff2b414da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.178+0000 7ff2b3e00640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4071a70 0x7ff2b414da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.178+0000 7ff2b3e00640 1 -- 172.21.15.102:0/876156464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b41f1750 con 0x7ff2b4151c90 2024-04-20T21:38:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.178+0000 7ff2b3400640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4071a70 0x7ff2b414da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.178+0000 7ff2b3e00640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2b4151c90 0x7ff2b4154650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00efc0 tx=0x7ff2ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.179+0000 7ff2ab400640 1 -- 172.21.15.102:0/876156464 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac018020 con 0x7ff2b4151c90 2024-04-20T21:38:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.179+0000 7ff2ab400640 1 -- 172.21.15.102:0/876156464 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ac014480 con 0x7ff2b4151c90 2024-04-20T21:38:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.179+0000 7ff2ab400640 1 -- 172.21.15.102:0/876156464 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac014870 con 0x7ff2b4151c90 2024-04-20T21:38:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.179+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b41f1a30 con 0x7ff2b4151c90 2024-04-20T21:38:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.179+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41f1ff0 con 0x7ff2b4151c90 2024-04-20T21:38:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.180+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b4068c30 con 0x7ff2b4151c90 2024-04-20T21:38:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.181+0000 7ff2ab400640 1 -- 172.21.15.102:0/876156464 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2ac005d10 con 0x7ff2b4151c90 2024-04-20T21:38:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.182+0000 7ff2ab400640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff29406c920 0x7ff29406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.182+0000 7ff2ab400640 1 -- 172.21.15.102:0/876156464 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7ff2ac08cff0 con 0x7ff2b4151c90 2024-04-20T21:38:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.182+0000 7ff2b3400640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff29406c920 0x7ff29406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:24.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.183+0000 7ff2b3400640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff29406c920 0x7ff29406ede0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4036000 tx=0x7ff2a4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:24.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.187+0000 7ff2ab400640 1 -- 172.21.15.102:0/876156464 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff2b4068c30 con 0x7ff2b4151c90 2024-04-20T21:38:24.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.431+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2b4071040 con 0x7ff2b4151c90 2024-04-20T21:38:24.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.431+0000 7ff2ab400640 1 -- 172.21.15.102:0/876156464 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+6023 (secure 0 0 0) 0x7ff2b4071040 con 0x7ff2b4151c90 2024-04-20T21:38:24.431 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:24.431 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:20.801907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:24.431 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 68 2024-04-20T21:38:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff29406c920 msgr2=0x7ff29406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff29406c920 0x7ff29406ede0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4036000 tx=0x7ff2a4008030 comp rx=0 tx=0).stop 2024-04-20T21:38:24.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2b4151c90 msgr2=0x7ff2b4154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2b4151c90 0x7ff2b4154650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00efc0 tx=0x7ff2ac00c490 comp rx=0 tx=0).stop 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 shutdown_connections 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2b4151c90 0x7ff2b4154650 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2b40c7290 0x7ff2b414cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff29406c920 0x7ff29406ede0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 --2- 172.21.15.102:0/876156464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4071a70 0x7ff2b414da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 >> 172.21.15.102:0/876156464 conn(0x7ff2b40c9390 msgr2=0x7ff2b4071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.434+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 shutdown_connections 2024-04-20T21:38:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:24.435+0000 7ff2ba000640 1 -- 172.21.15.102:0/876156464 wait complete. 2024-04-20T21:38:24.452 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:26.453 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:26.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.669+0000 7effea000640 1 Processor -- start 2024-04-20T21:38:26.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.669+0000 7effea000640 1 -- start start 2024-04-20T21:38:26.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.671+0000 7effea000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7effe40c7c10 0x7effe40c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:26.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.671+0000 7effea000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe4158380 con 0x7effe4157d10 2024-04-20T21:38:26.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effea000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe4158660 con 0x7effe4151fa0 2024-04-20T21:38:26.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effea000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7effe415c2b0 con 0x7effe40c7c10 2024-04-20T21:38:26.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effe9600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7effe40c7c10 0x7effe40c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:26.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effe3e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7effe4157d10 0x7effe40cd210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42140/0 (socket says 172.21.15.102:42140) 2024-04-20T21:38:26.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effe3e00640 1 -- 172.21.15.102:0/346254660 learned_addr learned my addr 172.21.15.102:0/346254660 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:26.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effe9600640 1 -- 172.21.15.102:0/346254660 >> v1:172.21.15.193:6789/0 conn(0x7effe4151fa0 legacy=0x7effe415bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:26.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effe9600640 1 -- 172.21.15.102:0/346254660 >> v1:172.21.15.102:6789/0 conn(0x7effe4157d10 legacy=0x7effe40cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:26.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effe9600640 1 -- 172.21.15.102:0/346254660 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe4158940 con 0x7effe40c7c10 2024-04-20T21:38:26.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.672+0000 7effe3400640 1 -- 172.21.15.102:0/346254660 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405309917 0 0) 0x7effe4158380 con 0x7effe4157d10 2024-04-20T21:38:26.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.674+0000 7effe9600640 1 --2- 172.21.15.102:0/346254660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe40c7c10 0x7effe40c8060 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7effdc009d90 tx=0x7effdc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9426bd4f26148958 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:26.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.674+0000 7effe3400640 1 -- 172.21.15.102:0/346254660 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc039070 con 0x7effe40c7c10 2024-04-20T21:38:26.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.674+0000 7effe3400640 1 -- 172.21.15.102:0/346254660 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effdc02cd00 con 0x7effe40c7c10 2024-04-20T21:38:26.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.674+0000 7effe3400640 1 -- 172.21.15.102:0/346254660 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc034660 con 0x7effe40c7c10 2024-04-20T21:38:26.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.675+0000 7effea000640 1 -- 172.21.15.102:0/346254660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe40c7c10 msgr2=0x7effe40c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:26.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.675+0000 7effea000640 1 --2- 172.21.15.102:0/346254660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe40c7c10 0x7effe40c8060 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7effdc009d90 tx=0x7effdc02bf50 comp rx=0 tx=0).stop 2024-04-20T21:38:26.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.675+0000 7effea000640 1 -- 172.21.15.102:0/346254660 shutdown_connections 2024-04-20T21:38:26.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.675+0000 7effea000640 1 --2- 172.21.15.102:0/346254660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe40c7c10 0x7effe40c8060 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:26.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.675+0000 7effea000640 1 -- 172.21.15.102:0/346254660 >> 172.21.15.102:0/346254660 conn(0x7effe4074ea0 msgr2=0x7effe4075280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:26.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.675+0000 7effea000640 1 -- 172.21.15.102:0/346254660 shutdown_connections 2024-04-20T21:38:26.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.675+0000 7effea000640 1 -- 172.21.15.102:0/346254660 wait complete. 2024-04-20T21:38:26.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.677+0000 7effea000640 1 Processor -- start 2024-04-20T21:38:26.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.677+0000 7effea000640 1 -- start start 2024-04-20T21:38:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.678+0000 7effea000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7effe40c7c10 0x7effe41f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.678+0000 7effea000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe4151fa0 0x7effe41596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.678+0000 7effea000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effe4157d10 0x7effe4144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.678+0000 7effea000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7effe4158c20 con 0x7effe4151fa0 2024-04-20T21:38:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.678+0000 7effea000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7effe415b680 con 0x7effe4157d10 2024-04-20T21:38:26.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.678+0000 7effea000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7effe40c25e0 con 0x7effe40c7c10 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.678+0000 7effe3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe4151fa0 0x7effe41596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.678+0000 7effe8c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7effe40c7c10 0x7effe41f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effe4157d10 0x7effe4144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe8c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7effe40c7c10 0x7effe41f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:58332/0 (socket says 172.21.15.102:58332) 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effe4157d10 0x7effe4144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:50958/0 (socket says 172.21.15.102:50958) 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe8c00640 1 -- 172.21.15.102:0/2379331356 learned_addr learned my addr 172.21.15.102:0/2379331356 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe3e00640 1 -- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7effe40c7c10 msgr2=0x7effe41f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe3e00640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7effe40c7c10 0x7effe41f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe3e00640 1 -- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effe4157d10 msgr2=0x7effe4144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe3e00640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effe4157d10 0x7effe4144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe3e00640 1 -- 172.21.15.102:0/2379331356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe41f0ab0 con 0x7effe4151fa0 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.679+0000 7effe3e00640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe4151fa0 0x7effe41596a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7effd000c410 tx=0x7effd000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.680+0000 7effe0c00640 1 -- 172.21.15.102:0/2379331356 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd00074e0 con 0x7effe4151fa0 2024-04-20T21:38:26.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.680+0000 7effe0c00640 1 -- 172.21.15.102:0/2379331356 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd00047e0 con 0x7effe4151fa0 2024-04-20T21:38:26.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.680+0000 7effe0c00640 1 -- 172.21.15.102:0/2379331356 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0004bb0 con 0x7effe4151fa0 2024-04-20T21:38:26.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.680+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe4158ed0 con 0x7effe4151fa0 2024-04-20T21:38:26.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.680+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe415ad20 con 0x7effe4151fa0 2024-04-20T21:38:26.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.681+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe404fd60 con 0x7effe4151fa0 2024-04-20T21:38:26.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.683+0000 7effe0c00640 1 -- 172.21.15.102:0/2379331356 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effd0011070 con 0x7effe4151fa0 2024-04-20T21:38:26.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.684+0000 7effe0c00640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7effc406c8f0 0x7effc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:26.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.684+0000 7effe0c00640 1 -- 172.21.15.102:0/2379331356 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7effd008bd40 con 0x7effe4151fa0 2024-04-20T21:38:26.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.684+0000 7effe8c00640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7effc406c8f0 0x7effc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:26.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.685+0000 7effe8c00640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7effc406c8f0 0x7effc406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7effd8007fd0 tx=0x7effd8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:26.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.692+0000 7effe0c00640 1 -- 172.21.15.102:0/2379331356 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7effd0059360 con 0x7effe4151fa0 2024-04-20T21:38:26.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.942+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effe4146440 con 0x7effe4151fa0 2024-04-20T21:38:26.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.944+0000 7effe0c00640 1 -- 172.21.15.102:0/2379331356 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+6023 (secure 0 0 0) 0x7effd005d660 con 0x7effe4151fa0 2024-04-20T21:38:26.943 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:26.943 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:20.801907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:26.943 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 68 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7effc406c8f0 msgr2=0x7effc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7effc406c8f0 0x7effc406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7effd8007fd0 tx=0x7effd8009040 comp rx=0 tx=0).stop 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe4151fa0 msgr2=0x7effe41596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe4151fa0 0x7effe41596a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7effd000c410 tx=0x7effd000c8d0 comp rx=0 tx=0).stop 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 shutdown_connections 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7effc406c8f0 0x7effc406edb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effe4157d10 0x7effe4144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effe4151fa0 0x7effe41596a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:26.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 --2- 172.21.15.102:0/2379331356 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7effe40c7c10 0x7effe41f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:26.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 >> 172.21.15.102:0/2379331356 conn(0x7effe4074ea0 msgr2=0x7effe40721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:26.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 shutdown_connections 2024-04-20T21:38:26.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:26.947+0000 7effea000640 1 -- 172.21.15.102:0/2379331356 wait complete. 2024-04-20T21:38:26.964 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:28.966 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.157+0000 7f03eac00640 1 Processor -- start 2024-04-20T21:38:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.157+0000 7f03eac00640 1 -- start start 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.158+0000 7f03eac00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f03e4153180 0x7f03e4158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.158+0000 7f03eac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e40c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.158+0000 7f03eac00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e4153560 con 0x7f03e40c75a0 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.158+0000 7f03eac00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f03e40c7a70 con 0x7f03e4153180 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.158+0000 7f03eac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f03e40c7d30 con 0x7f03e4159380 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.158+0000 7f03e9800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f03e4153180 0x7f03e4158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03ea200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f03e40c75a0 0x7f03e4070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47924/0 (socket says 172.21.15.102:47924) 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03ea200640 1 -- 172.21.15.102:0/396095269 learned_addr learned my addr 172.21.15.102:0/396095269 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03e8e00640 1 --2- 172.21.15.102:0/396095269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e40c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03e9800640 1 -- 172.21.15.102:0/396095269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 msgr2=0x7f03e40c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03e9800640 1 --2- 172.21.15.102:0/396095269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e40c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03e9800640 1 -- 172.21.15.102:0/396095269 >> v1:172.21.15.102:6789/0 conn(0x7f03e40c75a0 legacy=0x7f03e4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03e9800640 1 -- 172.21.15.102:0/396095269 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e415c4d0 con 0x7f03e4153180 2024-04-20T21:38:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03e3e00640 1 -- 172.21.15.102:0/396095269 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2172354514 0 0) 0x7f03e4153560 con 0x7f03e40c75a0 2024-04-20T21:38:29.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.159+0000 7f03e8e00640 1 --2- 172.21.15.102:0/396095269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e40c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:29.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.160+0000 7f03e9800640 1 --2- 172.21.15.102:0/396095269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e4153180 0x7f03e4158e40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f03d80097a0 tx=0x7f03d802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ebfcf76cac8fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:29.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03e3e00640 1 -- 172.21.15.102:0/396095269 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8035020 con 0x7f03e4153180 2024-04-20T21:38:29.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03e3e00640 1 -- 172.21.15.102:0/396095269 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d8031810 con 0x7f03e4153180 2024-04-20T21:38:29.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03e3e00640 1 -- 172.21.15.102:0/396095269 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8031be0 con 0x7f03e4153180 2024-04-20T21:38:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03eac00640 1 -- 172.21.15.102:0/396095269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e4153180 msgr2=0x7f03e4158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03eac00640 1 --2- 172.21.15.102:0/396095269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e4153180 0x7f03e4158e40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f03d80097a0 tx=0x7f03d802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03eac00640 1 -- 172.21.15.102:0/396095269 shutdown_connections 2024-04-20T21:38:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03eac00640 1 --2- 172.21.15.102:0/396095269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e40c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03eac00640 1 --2- 172.21.15.102:0/396095269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e4153180 0x7f03e4158e40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03eac00640 1 -- 172.21.15.102:0/396095269 >> 172.21.15.102:0/396095269 conn(0x7f03e40cc820 msgr2=0x7f03e40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03eac00640 1 -- 172.21.15.102:0/396095269 shutdown_connections 2024-04-20T21:38:29.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.161+0000 7f03eac00640 1 -- 172.21.15.102:0/396095269 wait complete. 2024-04-20T21:38:29.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.163+0000 7f03eac00640 1 Processor -- start 2024-04-20T21:38:29.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.163+0000 7f03eac00640 1 -- start start 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03eac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e40c75a0 0x7f03e40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03eac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03e4153180 0x7f03e415b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03eac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e41f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03eac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f03e415d290 con 0x7f03e40c75a0 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03eac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03e415ca90 con 0x7f03e4153180 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03eac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f03e40c6be0 con 0x7f03e4159380 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03e9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e40c75a0 0x7f03e40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03e9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e40c75a0 0x7f03e40caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58054/0 (socket says 172.21.15.102:58054) 2024-04-20T21:38:29.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03e9800640 1 -- 172.21.15.102:0/44144979 learned_addr learned my addr 172.21.15.102:0/44144979 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03e8e00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03e4153180 0x7f03e415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.164+0000 7f03ea200640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e41f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e9800640 1 -- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 msgr2=0x7f03e41f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e9800640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e41f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e9800640 1 -- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03e4153180 msgr2=0x7f03e415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e9800640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03e4153180 0x7f03e415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e9800640 1 -- 172.21.15.102:0/44144979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e41f5540 con 0x7f03e40c75a0 2024-04-20T21:38:29.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e9800640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e40c75a0 0x7f03e40caa90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f03d80098c0 tx=0x7f03d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:29.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e1600640 1 -- 172.21.15.102:0/44144979 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8035040 con 0x7f03e40c75a0 2024-04-20T21:38:29.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e1600640 1 -- 172.21.15.102:0/44144979 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d8039cf0 con 0x7f03e40c75a0 2024-04-20T21:38:29.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03e1600640 1 -- 172.21.15.102:0/44144979 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8039ea0 con 0x7f03e40c75a0 2024-04-20T21:38:29.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.165+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e40cb1c0 con 0x7f03e40c75a0 2024-04-20T21:38:29.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.166+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e41f5800 con 0x7f03e40c75a0 2024-04-20T21:38:29.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.167+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e41f1700 con 0x7f03e40c75a0 2024-04-20T21:38:29.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.168+0000 7f03e1600640 1 -- 172.21.15.102:0/44144979 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03d8038020 con 0x7f03e40c75a0 2024-04-20T21:38:29.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.169+0000 7f03e1600640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f03c406c8f0 0x7f03c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:29.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.169+0000 7f03e1600640 1 -- 172.21.15.102:0/44144979 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f03d80adf20 con 0x7f03e40c75a0 2024-04-20T21:38:29.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.169+0000 7f03e8e00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f03c406c8f0 0x7f03c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:29.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.170+0000 7f03e8e00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f03c406c8f0 0x7f03c406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009560 tx=0x7f03d0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:29.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.174+0000 7f03e1600640 1 -- 172.21.15.102:0/44144979 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f03e41f1700 con 0x7f03e40c75a0 2024-04-20T21:38:29.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.411+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e404fd60 con 0x7f03e40c75a0 2024-04-20T21:38:29.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.412+0000 7f03e1600640 1 -- 172.21.15.102:0/44144979 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+6023 (secure 0 0 0) 0x7f03e404fd60 con 0x7f03e40c75a0 2024-04-20T21:38:29.411 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:29.411 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:20.801907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:29.412 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 68 2024-04-20T21:38:29.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f03c406c8f0 msgr2=0x7f03c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:29.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f03c406c8f0 0x7f03c406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009560 tx=0x7f03d0009190 comp rx=0 tx=0).stop 2024-04-20T21:38:29.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e40c75a0 msgr2=0x7f03e40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:29.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e40c75a0 0x7f03e40caa90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f03d80098c0 tx=0x7f03d80046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 shutdown_connections 2024-04-20T21:38:29.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03e4159380 0x7f03e41f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f03c406c8f0 0x7f03c406edb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03e4153180 0x7f03e415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 --2- 172.21.15.102:0/44144979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03e40c75a0 0x7f03e40caa90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 >> 172.21.15.102:0/44144979 conn(0x7f03e40cc820 msgr2=0x7f03e41520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 shutdown_connections 2024-04-20T21:38:29.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:29.415+0000 7f03eac00640 1 -- 172.21.15.102:0/44144979 wait complete. 2024-04-20T21:38:29.438 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:29.726 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:38:29.732 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:38:31.439 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.624+0000 7f6ed6c00640 1 Processor -- start 2024-04-20T21:38:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.624+0000 7f6ed6c00640 1 -- start start 2024-04-20T21:38:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.626+0000 7f6ed6c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0155560 0x7f6ed015be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.626+0000 7f6ed6c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6ed00c5cc0 0x7f6ed00ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.626+0000 7f6ed6c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6ed00c60c0 0x7f6ed0073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.626+0000 7f6ed6c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed015f0c0 con 0x7f6ed00c60c0 2024-04-20T21:38:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.626+0000 7f6ed6c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed015e7b0 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.626+0000 7f6ed6c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed0155940 con 0x7f6ed0155560 2024-04-20T21:38:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.626+0000 7f6ed4c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6ed00c60c0 0x7f6ed0073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ed4c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6ed00c60c0 0x7f6ed0073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58056/0 (socket says 172.21.15.102:58056) 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ed4c00640 1 -- 172.21.15.102:0/437539012 learned_addr learned my addr 172.21.15.102:0/437539012 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ecf400640 1 --2- 172.21.15.102:0/437539012 >> v2:172.21.15.193:3300/0 conn(0x7f6ed00c5cc0 0x7f6ed00ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ecfe00640 1 --2- 172.21.15.102:0/437539012 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0155560 0x7f6ed015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ed4c00640 1 -- 172.21.15.102:0/437539012 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0155560 msgr2=0x7f6ed015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ed4c00640 1 --2- 172.21.15.102:0/437539012 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0155560 0x7f6ed015be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ed4c00640 1 -- 172.21.15.102:0/437539012 >> v2:172.21.15.193:3300/0 conn(0x7f6ed00c5cc0 msgr2=0x7f6ed00ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ed4c00640 1 --2- 172.21.15.102:0/437539012 >> v2:172.21.15.193:3300/0 conn(0x7f6ed00c5cc0 0x7f6ed00ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ed4c00640 1 -- 172.21.15.102:0/437539012 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed00cbea0 con 0x7f6ed00c60c0 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ecfe00640 1 --2- 172.21.15.102:0/437539012 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0155560 0x7f6ed015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.627+0000 7f6ecf400640 1 --2- 172.21.15.102:0/437539012 >> v2:172.21.15.193:3300/0 conn(0x7f6ed00c5cc0 0x7f6ed00ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.628+0000 7f6ed4c00640 1 --2- 172.21.15.102:0/437539012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c60c0 0x7f6ed0073230 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00d1b20 tx=0x7f6ec802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd3af9f2612e919 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.628+0000 7f6ecea00640 1 -- 172.21.15.102:0/437539012 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec802cb00 con 0x7f6ed00c60c0 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.628+0000 7f6ecea00640 1 -- 172.21.15.102:0/437539012 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec802cca0 con 0x7f6ed00c60c0 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.628+0000 7f6ecea00640 1 -- 172.21.15.102:0/437539012 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8034610 con 0x7f6ed00c60c0 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 -- 172.21.15.102:0/437539012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c60c0 msgr2=0x7f6ed0073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/437539012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c60c0 0x7f6ed0073230 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00d1b20 tx=0x7f6ec802bf60 comp rx=0 tx=0).stop 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 -- 172.21.15.102:0/437539012 shutdown_connections 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/437539012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c60c0 0x7f6ed0073230 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/437539012 >> v2:172.21.15.193:3300/0 conn(0x7f6ed00c5cc0 0x7f6ed00ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/437539012 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0155560 0x7f6ed015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 -- 172.21.15.102:0/437539012 >> 172.21.15.102:0/437539012 conn(0x7f6ed006cd40 msgr2=0x7f6ed006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 -- 172.21.15.102:0/437539012 shutdown_connections 2024-04-20T21:38:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.629+0000 7f6ed6c00640 1 -- 172.21.15.102:0/437539012 wait complete. 2024-04-20T21:38:31.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.631+0000 7f6ed6c00640 1 Processor -- start 2024-04-20T21:38:31.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.631+0000 7f6ed6c00640 1 -- start start 2024-04-20T21:38:31.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ed6c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c5cc0 0x7f6ed014abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ed6c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed0155560 0x7f6ed01569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ed6c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0156f10 0x7f6ed01f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ed6c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed015ed50 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ed6c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed015ea70 con 0x7f6ed0155560 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ed6c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed00cbb60 con 0x7f6ed0156f10 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c5cc0 0x7f6ed014abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c5cc0 0x7f6ed014abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58064/0 (socket says 172.21.15.102:58064) 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ecfe00640 1 -- 172.21.15.102:0/1339913552 learned_addr learned my addr 172.21.15.102:0/1339913552 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ecf400640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed0155560 0x7f6ed01569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.632+0000 7f6ed4c00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0156f10 0x7f6ed01f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ecfe00640 1 -- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0156f10 msgr2=0x7f6ed01f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ecfe00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0156f10 0x7f6ed01f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ecfe00640 1 -- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed0155560 msgr2=0x7f6ed01569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:31.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ecfe00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed0155560 0x7f6ed01569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ecfe00640 1 -- 172.21.15.102:0/1339913552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed01587b0 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ecf400640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed0155560 0x7f6ed01569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:31.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ecfe00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c5cc0 0x7f6ed014abf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000d7d0 tx=0x7f6ec000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:31.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ec7e00640 1 -- 172.21.15.102:0/1339913552 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0004030 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ec7e00640 1 -- 172.21.15.102:0/1339913552 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec00041d0 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ec7e00640 1 -- 172.21.15.102:0/1339913552 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000b6c0 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.633+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed01e5690 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.634+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed0155c10 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.635+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed0074340 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.636+0000 7f6ec7e00640 1 -- 172.21.15.102:0/1339913552 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ec000b860 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.637+0000 7f6ec7e00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6eb006c8f0 0x7f6eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:31.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.637+0000 7f6ec7e00640 1 -- 172.21.15.102:0/1339913552 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f6ec008c4c0 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.638+0000 7f6ecf400640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6eb006c8f0 0x7f6eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:31.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.638+0000 7f6ecf400640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6eb006c8f0 0x7f6eb006edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00c76a0 tx=0x7f6ebc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:31.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.644+0000 7f6ec7e00640 1 -- 172.21.15.102:0/1339913552 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6ec00599b0 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.923+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ed00cc480 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.924+0000 7f6ec7e00640 1 -- 172.21.15.102:0/1339913552 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+6023 (secure 0 0 0) 0x7f6ec005dcb0 con 0x7f6ed00c5cc0 2024-04-20T21:38:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:31.923 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:20.801907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:31.923 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 68 2024-04-20T21:38:31.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6eb006c8f0 msgr2=0x7f6eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6eb006c8f0 0x7f6eb006edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00c76a0 tx=0x7f6ebc008040 comp rx=0 tx=0).stop 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c5cc0 msgr2=0x7f6ed014abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c5cc0 0x7f6ed014abf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000d7d0 tx=0x7f6ec000dc90 comp rx=0 tx=0).stop 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 shutdown_connections 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed0156f10 0x7f6ed01f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6eb006c8f0 0x7f6eb006edb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed0155560 0x7f6ed01569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 --2- 172.21.15.102:0/1339913552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ed00c5cc0 0x7f6ed014abf0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.926+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 >> 172.21.15.102:0/1339913552 conn(0x7f6ed006cd40 msgr2=0x7f6ed00c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.927+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 shutdown_connections 2024-04-20T21:38:31.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:31.927+0000 7f6ed6c00640 1 -- 172.21.15.102:0/1339913552 wait complete. 2024-04-20T21:38:31.943 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:33.944 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:34.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.153+0000 7f8365600640 1 Processor -- start 2024-04-20T21:38:34.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.154+0000 7f8365600640 1 -- start start 2024-04-20T21:38:34.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.155+0000 7f8365600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f83600c8620 0x7f836015c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:34.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.155+0000 7f8365600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8360155bd0 con 0x7f83600c8040 2024-04-20T21:38:34.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.155+0000 7f8365600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836015c050 con 0x7f83601557f0 2024-04-20T21:38:34.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.155+0000 7f8365600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f83600c94b0 con 0x7f83600c8620 2024-04-20T21:38:34.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.155+0000 7f8356000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f83600c8620 0x7f836015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:34.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.155+0000 7f835ea00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f83601557f0 0x7f836006c380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38064/0 (socket says 172.21.15.102:38064) 2024-04-20T21:38:34.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.155+0000 7f835ea00640 1 -- 172.21.15.102:0/1784268371 learned_addr learned my addr 172.21.15.102:0/1784268371 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:34.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.156+0000 7f8356000640 1 -- 172.21.15.102:0/1784268371 >> v1:172.21.15.193:6789/0 conn(0x7f83601557f0 legacy=0x7f836006c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:34.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.156+0000 7f8356000640 1 -- 172.21.15.102:0/1784268371 >> v1:172.21.15.102:6789/0 conn(0x7f83600c8040 legacy=0x7f8360165d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:34.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.156+0000 7f835e000640 1 -- 172.21.15.102:0/1784268371 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464400763 0 0) 0x7f836015c050 con 0x7f83601557f0 2024-04-20T21:38:34.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.156+0000 7f8356000640 1 -- 172.21.15.102:0/1784268371 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836015c330 con 0x7f83600c8620 2024-04-20T21:38:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.157+0000 7f8356000640 1 --2- 172.21.15.102:0/1784268371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83600c8620 0x7f836015c910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f83480097a0 tx=0x7f834802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d385f962b36b92a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.157+0000 7f835e000640 1 -- 172.21.15.102:0/1784268371 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037070 con 0x7f83600c8620 2024-04-20T21:38:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.157+0000 7f835e000640 1 -- 172.21.15.102:0/1784268371 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348030710 con 0x7f83600c8620 2024-04-20T21:38:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.157+0000 7f835e000640 1 -- 172.21.15.102:0/1784268371 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348030ae0 con 0x7f83600c8620 2024-04-20T21:38:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.158+0000 7f8365600640 1 -- 172.21.15.102:0/1784268371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83600c8620 msgr2=0x7f836015c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.158+0000 7f8365600640 1 --2- 172.21.15.102:0/1784268371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83600c8620 0x7f836015c910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f83480097a0 tx=0x7f834802b790 comp rx=0 tx=0).stop 2024-04-20T21:38:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.158+0000 7f8365600640 1 -- 172.21.15.102:0/1784268371 shutdown_connections 2024-04-20T21:38:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.158+0000 7f8365600640 1 --2- 172.21.15.102:0/1784268371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83600c8620 0x7f836015c910 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.158+0000 7f8365600640 1 -- 172.21.15.102:0/1784268371 >> 172.21.15.102:0/1784268371 conn(0x7f83600d0a90 msgr2=0x7f83600d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.158+0000 7f8365600640 1 -- 172.21.15.102:0/1784268371 shutdown_connections 2024-04-20T21:38:34.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.158+0000 7f8365600640 1 -- 172.21.15.102:0/1784268371 wait complete. 2024-04-20T21:38:34.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.160+0000 7f8365600640 1 Processor -- start 2024-04-20T21:38:34.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.160+0000 7f8365600640 1 -- start start 2024-04-20T21:38:34.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8365600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83600c8040 0x7f83601e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:34.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8365600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83600c8620 0x7f83601f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:34.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8365600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83601557f0 0x7f83601f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:34.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8365600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f83600c9140 con 0x7f83600c8620 2024-04-20T21:38:34.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8365600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83600c8e60 con 0x7f83601557f0 2024-04-20T21:38:34.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8365600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f836006ca90 con 0x7f83600c8040 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8356000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83600c8620 0x7f83601f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8356000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83600c8620 0x7f83601f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58076/0 (socket says 172.21.15.102:58076) 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8356000640 1 -- 172.21.15.102:0/3283487239 learned_addr learned my addr 172.21.15.102:0/3283487239 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f835f400640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83601557f0 0x7f83601f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f835ea00640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83600c8040 0x7f83601e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8356000640 1 -- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83600c8040 msgr2=0x7f83601e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8356000640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83600c8040 0x7f83601e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8356000640 1 -- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83601557f0 msgr2=0x7f83601f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8356000640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83601557f0 0x7f83601f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:34.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.161+0000 7f8356000640 1 -- 172.21.15.102:0/3283487239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8360157ff0 con 0x7f83600c8620 2024-04-20T21:38:34.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.162+0000 7f8356000640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83600c8620 0x7f83601f49b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f83480098c0 tx=0x7f83480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:34.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.162+0000 7f8357400640 1 -- 172.21.15.102:0/3283487239 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348041020 con 0x7f83600c8620 2024-04-20T21:38:34.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.162+0000 7f8357400640 1 -- 172.21.15.102:0/3283487239 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348032a40 con 0x7f83600c8620 2024-04-20T21:38:34.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.162+0000 7f8357400640 1 -- 172.21.15.102:0/3283487239 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348037040 con 0x7f83600c8620 2024-04-20T21:38:34.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.162+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83600ced50 con 0x7f83600c8620 2024-04-20T21:38:34.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.162+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83600d01c0 con 0x7f83600c8620 2024-04-20T21:38:34.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.163+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8360074340 con 0x7f83600c8620 2024-04-20T21:38:34.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.164+0000 7f8357400640 1 -- 172.21.15.102:0/3283487239 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8348044020 con 0x7f83600c8620 2024-04-20T21:38:34.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.166+0000 7f8357400640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833c06c920 0x7f833c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:34.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.166+0000 7f8357400640 1 -- 172.21.15.102:0/3283487239 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f83480adcf0 con 0x7f83600c8620 2024-04-20T21:38:34.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.166+0000 7f835ea00640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833c06c920 0x7f833c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:34.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.166+0000 7f835ea00640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833c06c920 0x7f833c06ede0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8350000f60 tx=0x7f8350007e40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:34.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.170+0000 7f8357400640 1 -- 172.21.15.102:0/3283487239 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f83480b2070 con 0x7f83600c8620 2024-04-20T21:38:34.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.403+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83601590f0 con 0x7f83600c8620 2024-04-20T21:38:34.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.405+0000 7f8357400640 1 -- 172.21.15.102:0/3283487239 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+6027 (secure 0 0 0) 0x7f834804b030 con 0x7f83600c8620 2024-04-20T21:38:34.404 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:34.404 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:33.106901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:34.404 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 69 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.407+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833c06c920 msgr2=0x7f833c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.407+0000 7f8365600640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833c06c920 0x7f833c06ede0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8350000f60 tx=0x7f8350007e40 comp rx=0 tx=0).stop 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.407+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83600c8620 msgr2=0x7f83601f49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.407+0000 7f8365600640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83600c8620 0x7f83601f49b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f83480098c0 tx=0x7f83480046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.408+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 shutdown_connections 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.408+0000 7f8365600640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f833c06c920 0x7f833c06ede0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.408+0000 7f8365600640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83601557f0 0x7f83601f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.408+0000 7f8365600640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83600c8620 0x7f83601f49b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.408+0000 7f8365600640 1 --2- 172.21.15.102:0/3283487239 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83600c8040 0x7f83601e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:34.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.408+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 >> 172.21.15.102:0/3283487239 conn(0x7f83600d0a90 msgr2=0x7f836015e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:34.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.408+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 shutdown_connections 2024-04-20T21:38:34.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:34.408+0000 7f8365600640 1 -- 172.21.15.102:0/3283487239 wait complete. 2024-04-20T21:38:34.424 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 69, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 69, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6421": { "addr": "172.21.15.102:6833/327932375", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 327932375, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 327932375, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6421, "incarnation": 68, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 33 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:38:33.106901+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6421 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:38:36.426 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:36.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.634+0000 7f1f93400640 1 Processor -- start 2024-04-20T21:38:36.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.634+0000 7f1f93400640 1 -- start start 2024-04-20T21:38:36.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.635+0000 7f1f93400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c071a70 0x7f1f8c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:36.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.635+0000 7f1f93400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c071040 con 0x7f1f8c151c90 2024-04-20T21:38:36.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.635+0000 7f1f93400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c071320 con 0x7f1f8c0c7290 2024-04-20T21:38:36.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.635+0000 7f1f93400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c0c9200 con 0x7f1f8c071a70 2024-04-20T21:38:36.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f90a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1f8c151c90 0x7f1f8c158240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47942/0 (socket says 172.21.15.102:47942) 2024-04-20T21:38:36.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f90a00640 1 -- 172.21.15.102:0/3395522475 learned_addr learned my addr 172.21.15.102:0/3395522475 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:36.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f91400640 1 --2- 172.21.15.102:0/3395522475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c071a70 0x7f1f8c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:36.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f8b400640 1 -- 172.21.15.102:0/3395522475 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781090407 0 0) 0x7f1f8c071320 con 0x7f1f8c0c7290 2024-04-20T21:38:36.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f8b400640 1 -- 172.21.15.102:0/3395522475 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f70005730 con 0x7f1f8c0c7290 2024-04-20T21:38:36.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f8b400640 1 -- 172.21.15.102:0/3395522475 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 681876591 0 0) 0x7f1f8c071040 con 0x7f1f8c151c90 2024-04-20T21:38:36.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f8b400640 1 -- 172.21.15.102:0/3395522475 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f8c071320 con 0x7f1f8c151c90 2024-04-20T21:38:36.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f91400640 1 -- 172.21.15.102:0/3395522475 >> v1:172.21.15.193:6789/0 conn(0x7f1f8c0c7290 legacy=0x7f1f8c06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:36.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f91400640 1 -- 172.21.15.102:0/3395522475 >> v1:172.21.15.102:6789/0 conn(0x7f1f8c151c90 legacy=0x7f1f8c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:36.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.636+0000 7f1f91400640 1 -- 172.21.15.102:0/3395522475 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c071600 con 0x7f1f8c071a70 2024-04-20T21:38:36.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.638+0000 7f1f91400640 1 --2- 172.21.15.102:0/3395522475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c071a70 0x7f1f8c0c8cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009dc0 tx=0x7f1f8402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49c3ed82957862 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:36.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.638+0000 7f1f8b400640 1 -- 172.21.15.102:0/3395522475 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84039070 con 0x7f1f8c071a70 2024-04-20T21:38:36.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.638+0000 7f1f8b400640 1 -- 172.21.15.102:0/3395522475 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8402cd50 con 0x7f1f8c071a70 2024-04-20T21:38:36.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.638+0000 7f1f8b400640 1 -- 172.21.15.102:0/3395522475 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84034660 con 0x7f1f8c071a70 2024-04-20T21:38:36.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.639+0000 7f1f93400640 1 -- 172.21.15.102:0/3395522475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c071a70 msgr2=0x7f1f8c0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:36.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.639+0000 7f1f93400640 1 --2- 172.21.15.102:0/3395522475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c071a70 0x7f1f8c0c8cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009dc0 tx=0x7f1f8402bfa0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.639+0000 7f1f93400640 1 -- 172.21.15.102:0/3395522475 shutdown_connections 2024-04-20T21:38:36.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.639+0000 7f1f93400640 1 --2- 172.21.15.102:0/3395522475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c071a70 0x7f1f8c0c8cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.639+0000 7f1f93400640 1 -- 172.21.15.102:0/3395522475 >> 172.21.15.102:0/3395522475 conn(0x7f1f8c0c9390 msgr2=0x7f1f8c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:36.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.639+0000 7f1f93400640 1 -- 172.21.15.102:0/3395522475 shutdown_connections 2024-04-20T21:38:36.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.639+0000 7f1f93400640 1 -- 172.21.15.102:0/3395522475 wait complete. 2024-04-20T21:38:36.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.640+0000 7f1f93400640 1 Processor -- start 2024-04-20T21:38:36.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.641+0000 7f1f93400640 1 -- start start 2024-04-20T21:38:36.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.641+0000 7f1f93400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f8c071a70 0x7f1f8c14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:36.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.641+0000 7f1f93400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c0c7290 0x7f1f8c1f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:36.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.641+0000 7f1f93400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f8c151c90 0x7f1f8c1f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:36.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.641+0000 7f1f93400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c157db0 con 0x7f1f8c071a70 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.641+0000 7f1f93400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c070df0 con 0x7f1f8c151c90 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.641+0000 7f1f93400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c0ca1f0 con 0x7f1f8c0c7290 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f90a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f8c071a70 0x7f1f8c14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f90a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f8c071a70 0x7f1f8c14f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58078/0 (socket says 172.21.15.102:58078) 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f90a00640 1 -- 172.21.15.102:0/3598297384 learned_addr learned my addr 172.21.15.102:0/3598297384 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f91400640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f8c151c90 0x7f1f8c1f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f8be00640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c0c7290 0x7f1f8c1f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f90a00640 1 -- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c0c7290 msgr2=0x7f1f8c1f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f90a00640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c0c7290 0x7f1f8c1f05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f90a00640 1 -- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f8c151c90 msgr2=0x7f1f8c1f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f90a00640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f8c151c90 0x7f1f8c1f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f90a00640 1 -- 172.21.15.102:0/3598297384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c150630 con 0x7f1f8c071a70 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.642+0000 7f1f91400640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f8c151c90 0x7f1f8c1f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:36.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.643+0000 7f1f90a00640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f8c071a70 0x7f1f8c14f6b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c3f0 tx=0x7f1f7c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:36.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.643+0000 7f1f88c00640 1 -- 172.21.15.102:0/3598297384 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c017070 con 0x7f1f8c071a70 2024-04-20T21:38:36.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.643+0000 7f1f88c00640 1 -- 172.21.15.102:0/3598297384 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7c005ce0 con 0x7f1f8c071a70 2024-04-20T21:38:36.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.643+0000 7f1f88c00640 1 -- 172.21.15.102:0/3598297384 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c007310 con 0x7f1f8c071a70 2024-04-20T21:38:36.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.643+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c1f1130 con 0x7f1f8c071a70 2024-04-20T21:38:36.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.643+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c150bd0 con 0x7f1f8c071a70 2024-04-20T21:38:36.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.644+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8c1511e0 con 0x7f1f8c071a70 2024-04-20T21:38:36.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.645+0000 7f1f88c00640 1 -- 172.21.15.102:0/3598297384 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f7c01a070 con 0x7f1f8c071a70 2024-04-20T21:38:36.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.647+0000 7f1f88c00640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:36.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.647+0000 7f1f88c00640 1 -- 172.21.15.102:0/3598297384 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f1f7c08b7b0 con 0x7f1f8c071a70 2024-04-20T21:38:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.647+0000 7f1f8be00640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:36.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.648+0000 7f1f8be00640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f78007fd0 tx=0x7f1f78009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:36.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.652+0000 7f1f88c00640 1 -- 172.21.15.102:0/3598297384 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1f7c058ca0 con 0x7f1f8c071a70 2024-04-20T21:38:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.894+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f8c074340 con 0x7f1f8c071a70 2024-04-20T21:38:36.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.895+0000 7f1f88c00640 1 -- 172.21.15.102:0/3598297384 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+6027 (secure 0 0 0) 0x7f1f7c05cfa0 con 0x7f1f8c071a70 2024-04-20T21:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:33.106901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:36.894 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 69 2024-04-20T21:38:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f6c06c8f0 msgr2=0x7f1f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f78007fd0 tx=0x7f1f78009040 comp rx=0 tx=0).stop 2024-04-20T21:38:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f8c071a70 msgr2=0x7f1f8c14f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f8c071a70 0x7f1f8c14f6b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c3f0 tx=0x7f1f7c00c8b0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 shutdown_connections 2024-04-20T21:38:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f8c151c90 0x7f1f8c1f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f8c0c7290 0x7f1f8c1f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 --2- 172.21.15.102:0/3598297384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f8c071a70 0x7f1f8c14f6b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 >> 172.21.15.102:0/3598297384 conn(0x7f1f8c0c9390 msgr2=0x7f1f8c0c5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 shutdown_connections 2024-04-20T21:38:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:36.898+0000 7f1f93400640 1 -- 172.21.15.102:0/3598297384 wait complete. 2024-04-20T21:38:36.915 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:38.916 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.118+0000 7f288e000640 1 Processor -- start 2024-04-20T21:38:39.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.118+0000 7f288e000640 1 -- start start 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.119+0000 7f288e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f28880c7290 0x7f28880c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f288e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2888071040 con 0x7f2888071a70 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f288e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2888071320 con 0x7f2888151c90 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f288e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f28880c9200 con 0x7f28880c7290 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f2887e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2888071a70 0x7f288806d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:54388/0 (socket says 172.21.15.102:54388) 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f2887e00640 1 -- 172.21.15.102:0/3801979164 learned_addr learned my addr 172.21.15.102:0/3801979164 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f2886a00640 1 --2- 172.21.15.102:0/3801979164 >> v2:172.21.15.193:3300/0 conn(0x7f28880c7290 0x7f28880c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f2886000640 1 -- 172.21.15.102:0/3801979164 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1750978804 0 0) 0x7f2888071320 con 0x7f2888151c90 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f2886000640 1 -- 172.21.15.102:0/3801979164 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f286c003520 con 0x7f2888151c90 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f2886a00640 1 -- 172.21.15.102:0/3801979164 >> v1:172.21.15.193:6789/0 conn(0x7f2888151c90 legacy=0x7f2888158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.120+0000 7f2886a00640 1 -- 172.21.15.102:0/3801979164 >> v1:172.21.15.102:6789/0 conn(0x7f2888071a70 legacy=0x7f288806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.121+0000 7f2886000640 1 -- 172.21.15.102:0/3801979164 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136881695 0 0) 0x7f2888071040 con 0x7f2888071a70 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.121+0000 7f2886a00640 1 -- 172.21.15.102:0/3801979164 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2888071600 con 0x7f28880c7290 2024-04-20T21:38:39.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.122+0000 7f2886a00640 1 --2- 172.21.15.102:0/3801979164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f28880c8cc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2878009d90 tx=0x7f287802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b263b6e499ec61 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:39.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.122+0000 7f2886000640 1 -- 172.21.15.102:0/3801979164 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878039070 con 0x7f28880c7290 2024-04-20T21:38:39.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.122+0000 7f2886000640 1 -- 172.21.15.102:0/3801979164 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f287802cd80 con 0x7f28880c7290 2024-04-20T21:38:39.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.122+0000 7f2886000640 1 -- 172.21.15.102:0/3801979164 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878034660 con 0x7f28880c7290 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.122+0000 7f288e000640 1 -- 172.21.15.102:0/3801979164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 msgr2=0x7f28880c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.122+0000 7f288e000640 1 --2- 172.21.15.102:0/3801979164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f28880c8cc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2878009d90 tx=0x7f287802bfd0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.123+0000 7f288e000640 1 -- 172.21.15.102:0/3801979164 shutdown_connections 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.123+0000 7f288e000640 1 --2- 172.21.15.102:0/3801979164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f28880c8cc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.123+0000 7f288e000640 1 -- 172.21.15.102:0/3801979164 >> 172.21.15.102:0/3801979164 conn(0x7f28880c9390 msgr2=0x7f28880c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.123+0000 7f288e000640 1 -- 172.21.15.102:0/3801979164 shutdown_connections 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.123+0000 7f288e000640 1 -- 172.21.15.102:0/3801979164 wait complete. 2024-04-20T21:38:39.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.124+0000 7f288e000640 1 Processor -- start 2024-04-20T21:38:39.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.124+0000 7f288e000640 1 -- start start 2024-04-20T21:38:39.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.125+0000 7f288e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2888071a70 0x7f288814dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.125+0000 7f288e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f288806e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.125+0000 7f288e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2888151c90 0x7f288815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.125+0000 7f288e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2888157db0 con 0x7f2888071a70 2024-04-20T21:38:39.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.125+0000 7f288e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2888070df0 con 0x7f28880c7290 2024-04-20T21:38:39.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.125+0000 7f288e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f28880ca1f0 con 0x7f2888151c90 2024-04-20T21:38:39.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2887400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2888071a70 0x7f288814dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2886a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f288806e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2886a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f288806e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37472/0 (socket says 172.21.15.102:37472) 2024-04-20T21:38:39.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2887e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2888151c90 0x7f288815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2886a00640 1 -- 172.21.15.102:0/4136498275 learned_addr learned my addr 172.21.15.102:0/4136498275 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2887400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2888071a70 0x7f288814dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53496/0 (socket says 172.21.15.102:53496) 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2886a00640 1 -- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2888151c90 msgr2=0x7f288815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2886a00640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2888151c90 0x7f288815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2886a00640 1 -- 172.21.15.102:0/4136498275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2888071a70 msgr2=0x7f288814dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2886a00640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2888071a70 0x7f288814dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2886a00640 1 -- 172.21.15.102:0/4136498275 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288814c750 con 0x7f28880c7290 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.126+0000 7f2887400640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2888071a70 0x7f288814dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.127+0000 7f2886a00640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f288806e7b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2878009d90 tx=0x7f2878004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.127+0000 7f287f400640 1 -- 172.21.15.102:0/4136498275 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878039040 con 0x7f28880c7290 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.127+0000 7f287f400640 1 -- 172.21.15.102:0/4136498275 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2878003990 con 0x7f28880c7290 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.127+0000 7f287f400640 1 -- 172.21.15.102:0/4136498275 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878003d80 con 0x7f28880c7290 2024-04-20T21:38:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.127+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288814ca30 con 0x7f28880c7290 2024-04-20T21:38:39.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.127+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288814cff0 con 0x7f28880c7290 2024-04-20T21:38:39.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.128+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2888074340 con 0x7f28880c7290 2024-04-20T21:38:39.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.129+0000 7f287f400640 1 -- 172.21.15.102:0/4136498275 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2878004030 con 0x7f28880c7290 2024-04-20T21:38:39.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.130+0000 7f287f400640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f286406c920 0x7f286406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.130+0000 7f287f400640 1 -- 172.21.15.102:0/4136498275 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f28780af290 con 0x7f28880c7290 2024-04-20T21:38:39.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.130+0000 7f2887400640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f286406c920 0x7f286406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.131+0000 7f2887400640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f286406c920 0x7f286406ede0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2870007fd0 tx=0x7f2870009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:39.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.135+0000 7f287f400640 1 -- 172.21.15.102:0/4136498275 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f287807c810 con 0x7f28880c7290 2024-04-20T21:38:39.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.381+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28880c8aa0 con 0x7f28880c7290 2024-04-20T21:38:39.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.382+0000 7f287f400640 1 -- 172.21.15.102:0/4136498275 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+6024 (secure 0 0 0) 0x7f2878080b10 con 0x7f28880c7290 2024-04-20T21:38:39.381 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:39.381 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:38.884708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:39.381 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 70 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.384+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f286406c920 msgr2=0x7f286406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.384+0000 7f288e000640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f286406c920 0x7f286406ede0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2870007fd0 tx=0x7f2870009040 comp rx=0 tx=0).stop 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.384+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 msgr2=0x7f288806e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f288806e7b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2878009d90 tx=0x7f2878004a40 comp rx=0 tx=0).stop 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 shutdown_connections 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2888151c90 0x7f288815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f286406c920 0x7f286406ede0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28880c7290 0x7f288806e7b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 --2- 172.21.15.102:0/4136498275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2888071a70 0x7f288814dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 >> 172.21.15.102:0/4136498275 conn(0x7f28880c9390 msgr2=0x7f28880725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:39.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 shutdown_connections 2024-04-20T21:38:39.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:39.385+0000 7f288e000640 1 -- 172.21.15.102:0/4136498275 wait complete. 2024-04-20T21:38:39.403 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:41.404 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:41.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.609+0000 7efdb0a00640 1 Processor -- start 2024-04-20T21:38:41.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.609+0000 7efdb0a00640 1 -- start start 2024-04-20T21:38:41.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.610+0000 7efdb0a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7efdac151c90 0x7efdac0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:41.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.610+0000 7efdb0a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdac071040 con 0x7efdac0c7290 2024-04-20T21:38:41.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.610+0000 7efdb0a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdac071320 con 0x7efdac071a70 2024-04-20T21:38:41.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.610+0000 7efdb0a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7efdac0c9200 con 0x7efdac151c90 2024-04-20T21:38:41.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda3e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efdac0c7290 0x7efdac158240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:54402/0 (socket says 172.21.15.102:54402) 2024-04-20T21:38:41.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda3e00640 1 -- 172.21.15.102:0/3250627846 learned_addr learned my addr 172.21.15.102:0/3250627846 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:41.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda9e00640 1 --2- 172.21.15.102:0/3250627846 >> v2:172.21.15.193:3300/0 conn(0x7efdac151c90 0x7efdac0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:41.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda9400640 1 -- 172.21.15.102:0/3250627846 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472373528 0 0) 0x7efdac071320 con 0x7efdac071a70 2024-04-20T21:38:41.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda9400640 1 -- 172.21.15.102:0/3250627846 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd90003440 con 0x7efdac071a70 2024-04-20T21:38:41.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda9400640 1 -- 172.21.15.102:0/3250627846 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733692735 0 0) 0x7efdac071040 con 0x7efdac0c7290 2024-04-20T21:38:41.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda9e00640 1 -- 172.21.15.102:0/3250627846 >> v1:172.21.15.193:6789/0 conn(0x7efdac071a70 legacy=0x7efdac06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:41.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda9e00640 1 -- 172.21.15.102:0/3250627846 >> v1:172.21.15.102:6789/0 conn(0x7efdac0c7290 legacy=0x7efdac158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:41.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.611+0000 7efda9e00640 1 -- 172.21.15.102:0/3250627846 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac071600 con 0x7efdac151c90 2024-04-20T21:38:41.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.612+0000 7efda9e00640 1 --2- 172.21.15.102:0/3250627846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 0x7efdac0c8cc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0097a0 tx=0x7efd9c02b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d75b21b4e29acab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:41.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efda9400640 1 -- 172.21.15.102:0/3250627846 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c02f810 con 0x7efdac151c90 2024-04-20T21:38:41.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efda9400640 1 -- 172.21.15.102:0/3250627846 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9c02f9b0 con 0x7efdac151c90 2024-04-20T21:38:41.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efda9400640 1 -- 172.21.15.102:0/3250627846 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c02fd60 con 0x7efdac151c90 2024-04-20T21:38:41.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efdb0a00640 1 -- 172.21.15.102:0/3250627846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 msgr2=0x7efdac0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:41.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efdb0a00640 1 --2- 172.21.15.102:0/3250627846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 0x7efdac0c8cc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0097a0 tx=0x7efd9c02b9f0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efdb0a00640 1 -- 172.21.15.102:0/3250627846 shutdown_connections 2024-04-20T21:38:41.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efdb0a00640 1 --2- 172.21.15.102:0/3250627846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 0x7efdac0c8cc0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efdb0a00640 1 -- 172.21.15.102:0/3250627846 >> 172.21.15.102:0/3250627846 conn(0x7efdac0c9390 msgr2=0x7efdac0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:41.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.613+0000 7efdb0a00640 1 -- 172.21.15.102:0/3250627846 shutdown_connections 2024-04-20T21:38:41.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.614+0000 7efdb0a00640 1 -- 172.21.15.102:0/3250627846 wait complete. 2024-04-20T21:38:41.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.615+0000 7efdb0a00640 1 Processor -- start 2024-04-20T21:38:41.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.615+0000 7efdb0a00640 1 -- start start 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efdb0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdac071a70 0x7efdac0c2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efdb0a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdac0c7290 0x7efdac1f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efdb0a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 0x7efdac1f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efdb0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efdac157db0 con 0x7efdac071a70 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efdb0a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efdac070df0 con 0x7efdac151c90 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efdb0a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efdac0ca1f0 con 0x7efdac0c7290 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efda9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdac071a70 0x7efdac0c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efda9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdac071a70 0x7efdac0c2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53500/0 (socket says 172.21.15.102:53500) 2024-04-20T21:38:41.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.616+0000 7efda9e00640 1 -- 172.21.15.102:0/1387619943 learned_addr learned my addr 172.21.15.102:0/1387619943 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:41.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efda3e00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdac0c7290 0x7efdac1f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:41.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efdaa800640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 0x7efdac1f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:41.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efda9e00640 1 -- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdac0c7290 msgr2=0x7efdac1f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:41.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efda9e00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdac0c7290 0x7efdac1f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efda9e00640 1 -- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 msgr2=0x7efdac1f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:41.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efda9e00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 0x7efdac1f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efda9e00640 1 -- 172.21.15.102:0/1387619943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac143fc0 con 0x7efdac071a70 2024-04-20T21:38:41.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efda9e00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdac071a70 0x7efdac0c2780 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:41.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.617+0000 7efda2000640 1 -- 172.21.15.102:0/1387619943 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c048070 con 0x7efdac071a70 2024-04-20T21:38:41.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.618+0000 7efda2000640 1 -- 172.21.15.102:0/1387619943 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9c03d040 con 0x7efdac071a70 2024-04-20T21:38:41.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.618+0000 7efda2000640 1 -- 172.21.15.102:0/1387619943 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c037a50 con 0x7efdac071a70 2024-04-20T21:38:41.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.618+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac156020 con 0x7efdac071a70 2024-04-20T21:38:41.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.618+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac158be0 con 0x7efdac071a70 2024-04-20T21:38:41.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.619+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdac0c6370 con 0x7efdac071a70 2024-04-20T21:38:41.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.620+0000 7efda2000640 1 -- 172.21.15.102:0/1387619943 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd9c04b070 con 0x7efdac071a70 2024-04-20T21:38:41.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.621+0000 7efda2000640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd8806c920 0x7efd8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:41.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.621+0000 7efda2000640 1 -- 172.21.15.102:0/1387619943 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7efd9c0ae1a0 con 0x7efdac071a70 2024-04-20T21:38:41.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.621+0000 7efda3e00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd8806c920 0x7efd8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:41.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.622+0000 7efda3e00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd8806c920 0x7efd8806ede0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efdac1e27d0 tx=0x7efd94007f00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:41.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.626+0000 7efda2000640 1 -- 172.21.15.102:0/1387619943 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efdac0c6370 con 0x7efdac071a70 2024-04-20T21:38:41.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.867+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdac0c68b0 con 0x7efdac071a70 2024-04-20T21:38:41.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.868+0000 7efda2000640 1 -- 172.21.15.102:0/1387619943 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+6024 (secure 0 0 0) 0x7efdac0c68b0 con 0x7efdac071a70 2024-04-20T21:38:41.867 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:41.867 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:38.884708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:41.867 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 70 2024-04-20T21:38:41.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.870+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd8806c920 msgr2=0x7efd8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.870+0000 7efdb0a00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd8806c920 0x7efd8806ede0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efdac1e27d0 tx=0x7efd94007f00 comp rx=0 tx=0).stop 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.870+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdac071a70 msgr2=0x7efdac0c2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.870+0000 7efdb0a00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdac071a70 0x7efdac0c2780 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.870+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 shutdown_connections 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.870+0000 7efdb0a00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd8806c920 0x7efd8806ede0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.871+0000 7efdb0a00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdac151c90 0x7efdac1f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.871+0000 7efdb0a00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdac0c7290 0x7efdac1f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.871+0000 7efdb0a00640 1 --2- 172.21.15.102:0/1387619943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdac071a70 0x7efdac0c2780 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.871+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 >> 172.21.15.102:0/1387619943 conn(0x7efdac0c9390 msgr2=0x7efdac0c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.871+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 shutdown_connections 2024-04-20T21:38:41.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:41.871+0000 7efdb0a00640 1 -- 172.21.15.102:0/1387619943 wait complete. 2024-04-20T21:38:41.889 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:43.890 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:44.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.097+0000 7f1dd3000640 1 Processor -- start 2024-04-20T21:38:44.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.097+0000 7f1dd3000640 1 -- start start 2024-04-20T21:38:44.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dd3000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc0737f0 0x7f1dcc159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dd3000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1dcc073c30 0x7f1dcc0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dd3000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dcc0c6ec0 con 0x7f1dcc13bc80 2024-04-20T21:38:44.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dd3000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1dcc0c6a30 con 0x7f1dcc073c30 2024-04-20T21:38:44.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dd3000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1dcc0c8d00 con 0x7f1dcc0737f0 2024-04-20T21:38:44.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dcb400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc0737f0 0x7f1dcc159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dcbe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f1dcc13bc80 0x7f1dcc0c84e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55256/0 (socket says 172.21.15.102:55256) 2024-04-20T21:38:44.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dcbe00640 1 -- 172.21.15.102:0/3620848378 learned_addr learned my addr 172.21.15.102:0/3620848378 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:44.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.098+0000 7f1dd1000640 1 --2- 172.21.15.102:0/3620848378 >> v2:172.21.15.193:3300/0 conn(0x7f1dcc073c30 0x7f1dcc0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.099+0000 7f1dcb400640 1 -- 172.21.15.102:0/3620848378 >> v2:172.21.15.193:3300/0 conn(0x7f1dcc073c30 msgr2=0x7f1dcc0c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.099+0000 7f1dcb400640 1 --2- 172.21.15.102:0/3620848378 >> v2:172.21.15.193:3300/0 conn(0x7f1dcc073c30 0x7f1dcc0c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.099+0000 7f1dcb400640 1 -- 172.21.15.102:0/3620848378 >> v1:172.21.15.193:6789/0 conn(0x7f1dcc13bc80 legacy=0x7f1dcc0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.099+0000 7f1dcaa00640 1 -- 172.21.15.102:0/3620848378 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702901241 0 0) 0x7f1dcc0c6ec0 con 0x7f1dcc13bc80 2024-04-20T21:38:44.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.099+0000 7f1dcb400640 1 -- 172.21.15.102:0/3620848378 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dcc0c8fc0 con 0x7f1dcc0737f0 2024-04-20T21:38:44.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.100+0000 7f1dcb400640 1 --2- 172.21.15.102:0/3620848378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc0737f0 0x7f1dcc159020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1db80097a0 tx=0x7f1db802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31466a5e46c9d0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.100+0000 7f1dcaa00640 1 -- 172.21.15.102:0/3620848378 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8035020 con 0x7f1dcc0737f0 2024-04-20T21:38:44.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.100+0000 7f1dcaa00640 1 -- 172.21.15.102:0/3620848378 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8031810 con 0x7f1dcc0737f0 2024-04-20T21:38:44.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.100+0000 7f1dcaa00640 1 -- 172.21.15.102:0/3620848378 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8031be0 con 0x7f1dcc0737f0 2024-04-20T21:38:44.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.101+0000 7f1dd3000640 1 -- 172.21.15.102:0/3620848378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc0737f0 msgr2=0x7f1dcc159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.101+0000 7f1dd3000640 1 --2- 172.21.15.102:0/3620848378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc0737f0 0x7f1dcc159020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1db80097a0 tx=0x7f1db802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.101+0000 7f1dd3000640 1 -- 172.21.15.102:0/3620848378 shutdown_connections 2024-04-20T21:38:44.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.101+0000 7f1dd3000640 1 --2- 172.21.15.102:0/3620848378 >> v2:172.21.15.193:3300/0 conn(0x7f1dcc073c30 0x7f1dcc0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.101+0000 7f1dd3000640 1 --2- 172.21.15.102:0/3620848378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc0737f0 0x7f1dcc159020 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.101+0000 7f1dd3000640 1 -- 172.21.15.102:0/3620848378 >> 172.21.15.102:0/3620848378 conn(0x7f1dcc06ebf0 msgr2=0x7f1dcc06efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.101+0000 7f1dd3000640 1 -- 172.21.15.102:0/3620848378 shutdown_connections 2024-04-20T21:38:44.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.101+0000 7f1dd3000640 1 -- 172.21.15.102:0/3620848378 wait complete. 2024-04-20T21:38:44.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.103+0000 7f1dd3000640 1 Processor -- start 2024-04-20T21:38:44.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.103+0000 7f1dd3000640 1 -- start start 2024-04-20T21:38:44.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dd3000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dcc0737f0 0x7f1dcc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dd3000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dcc073c30 0x7f1dcc0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dd3000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc13bc80 0x7f1dcc1f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dd3000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc0c66c0 con 0x7f1dcc073c30 2024-04-20T21:38:44.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dd3000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc0c63e0 con 0x7f1dcc0737f0 2024-04-20T21:38:44.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dd3000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1dcc159560 con 0x7f1dcc13bc80 2024-04-20T21:38:44.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dcb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dcc073c30 0x7f1dcc0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dcb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dcc073c30 0x7f1dcc0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53506/0 (socket says 172.21.15.102:53506) 2024-04-20T21:38:44.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dcb400640 1 -- 172.21.15.102:0/2941750588 learned_addr learned my addr 172.21.15.102:0/2941750588 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:44.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dcbe00640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dcc0737f0 0x7f1dcc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.104+0000 7f1dd1000640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc13bc80 0x7f1dcc1f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.105+0000 7f1dcb400640 1 -- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc13bc80 msgr2=0x7f1dcc1f0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.105+0000 7f1dcb400640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc13bc80 0x7f1dcc1f0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.105+0000 7f1dcb400640 1 -- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dcc0737f0 msgr2=0x7f1dcc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.105+0000 7f1dcb400640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dcc0737f0 0x7f1dcc06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.105+0000 7f1dcb400640 1 -- 172.21.15.102:0/2941750588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dcc06c9a0 con 0x7f1dcc073c30 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.105+0000 7f1dcb400640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dcc073c30 0x7f1dcc0cb280 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c00 tx=0x7f1db80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.105+0000 7f1dc3e00640 1 -- 172.21.15.102:0/2941750588 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8035050 con 0x7f1dcc073c30 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.106+0000 7f1dc3e00640 1 -- 172.21.15.102:0/2941750588 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db803bcf0 con 0x7f1dcc073c30 2024-04-20T21:38:44.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.106+0000 7f1dc3e00640 1 -- 172.21.15.102:0/2941750588 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8006e10 con 0x7f1dcc073c30 2024-04-20T21:38:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.106+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dcc14efa0 con 0x7f1dcc073c30 2024-04-20T21:38:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.106+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dcc06cc60 con 0x7f1dcc073c30 2024-04-20T21:38:44.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.107+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dcc13ab30 con 0x7f1dcc073c30 2024-04-20T21:38:44.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.108+0000 7f1dc3e00640 1 -- 172.21.15.102:0/2941750588 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db8038020 con 0x7f1dcc073c30 2024-04-20T21:38:44.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.109+0000 7f1dc3e00640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1dac06c920 0x7f1dac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.109+0000 7f1dc3e00640 1 -- 172.21.15.102:0/2941750588 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f1db80ae1a0 con 0x7f1dcc073c30 2024-04-20T21:38:44.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.109+0000 7f1dcbe00640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1dac06c920 0x7f1dac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.110+0000 7f1dcbe00640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1dac06c920 0x7f1dac06ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc007fd0 tx=0x7f1dbc007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.113+0000 7f1dc3e00640 1 -- 172.21.15.102:0/2941750588 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1dcc13ab30 con 0x7f1dcc073c30 2024-04-20T21:38:44.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.360+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dcc074340 con 0x7f1dcc073c30 2024-04-20T21:38:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.361+0000 7f1dc3e00640 1 -- 172.21.15.102:0/2941750588 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+6024 (secure 0 0 0) 0x7f1db8040070 con 0x7f1dcc073c30 2024-04-20T21:38:44.361 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:44.361 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:38.884708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 70 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1dac06c920 msgr2=0x7f1dac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1dac06c920 0x7f1dac06ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc007fd0 tx=0x7f1dbc007de0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dcc073c30 msgr2=0x7f1dcc0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dcc073c30 0x7f1dcc0cb280 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c00 tx=0x7f1db80046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 shutdown_connections 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1dcc13bc80 0x7f1dcc1f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dcc073c30 0x7f1dcc0cb280 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1dac06c920 0x7f1dac06ede0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 --2- 172.21.15.102:0/2941750588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dcc0737f0 0x7f1dcc06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.364+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 >> 172.21.15.102:0/2941750588 conn(0x7f1dcc06ebf0 msgr2=0x7f1dcc151cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.365+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 shutdown_connections 2024-04-20T21:38:44.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:44.365+0000 7f1dd3000640 1 -- 172.21.15.102:0/2941750588 wait complete. 2024-04-20T21:38:44.380 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:46.382 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:46.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.596+0000 7f9cfce00640 1 Processor -- start 2024-04-20T21:38:46.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.596+0000 7f9cfce00640 1 -- start start 2024-04-20T21:38:46.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.597+0000 7f9cfce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf806b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.597+0000 7f9cfce00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9cf80b41e0 0x7f9cf8070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.597+0000 7f9cfce00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cf80cb440 con 0x7f9cf8155560 2024-04-20T21:38:46.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.597+0000 7f9cfce00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf80ca140 con 0x7f9cf80b41e0 2024-04-20T21:38:46.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.597+0000 7f9cfce00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf80c9de0 con 0x7f9cf80cc390 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf7400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9cf8155560 0x7f9cf8073890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:54418/0 (socket says 172.21.15.102:54418) 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf7400640 1 -- 172.21.15.102:0/1744919712 learned_addr learned my addr 172.21.15.102:0/1744919712 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf7e00640 1 --2- 172.21.15.102:0/1744919712 >> v2:172.21.15.193:3300/0 conn(0x7f9cf80b41e0 0x7f9cf8070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf6a00640 1 --2- 172.21.15.102:0/1744919712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf6a00640 1 -- 172.21.15.102:0/1744919712 >> v2:172.21.15.193:3300/0 conn(0x7f9cf80b41e0 msgr2=0x7f9cf8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf6a00640 1 --2- 172.21.15.102:0/1744919712 >> v2:172.21.15.193:3300/0 conn(0x7f9cf80b41e0 0x7f9cf8070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf6a00640 1 -- 172.21.15.102:0/1744919712 >> v1:172.21.15.102:6789/0 conn(0x7f9cf8155560 legacy=0x7f9cf8073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf6a00640 1 -- 172.21.15.102:0/1744919712 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf80c96a0 con 0x7f9cf80cc390 2024-04-20T21:38:46.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.598+0000 7f9cf7e00640 1 --2- 172.21.15.102:0/1744919712 >> v2:172.21.15.193:3300/0 conn(0x7f9cf80b41e0 0x7f9cf8070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:46.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.599+0000 7f9cf6a00640 1 --2- 172.21.15.102:0/1744919712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf806b940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009d90 tx=0x7f9ce802bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44ddbded6071f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.600+0000 7f9cf6000640 1 -- 172.21.15.102:0/1744919712 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8039070 con 0x7f9cf80cc390 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.600+0000 7f9cf6000640 1 -- 172.21.15.102:0/1744919712 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce802ccf0 con 0x7f9cf80cc390 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.600+0000 7f9cf6000640 1 -- 172.21.15.102:0/1744919712 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8034660 con 0x7f9cf80cc390 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.600+0000 7f9cfce00640 1 -- 172.21.15.102:0/1744919712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 msgr2=0x7f9cf806b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.600+0000 7f9cfce00640 1 --2- 172.21.15.102:0/1744919712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf806b940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009d90 tx=0x7f9ce802bf40 comp rx=0 tx=0).stop 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.600+0000 7f9cfce00640 1 -- 172.21.15.102:0/1744919712 shutdown_connections 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.600+0000 7f9cfce00640 1 --2- 172.21.15.102:0/1744919712 >> v2:172.21.15.193:3300/0 conn(0x7f9cf80b41e0 0x7f9cf8070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.600+0000 7f9cfce00640 1 --2- 172.21.15.102:0/1744919712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf806b940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.601+0000 7f9cfce00640 1 -- 172.21.15.102:0/1744919712 >> 172.21.15.102:0/1744919712 conn(0x7f9cf8075080 msgr2=0x7f9cf8075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.601+0000 7f9cfce00640 1 -- 172.21.15.102:0/1744919712 shutdown_connections 2024-04-20T21:38:46.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.601+0000 7f9cfce00640 1 -- 172.21.15.102:0/1744919712 wait complete. 2024-04-20T21:38:46.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.603+0000 7f9cfce00640 1 Processor -- start 2024-04-20T21:38:46.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.603+0000 7f9cfce00640 1 -- start start 2024-04-20T21:38:46.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cfce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf80b41e0 0x7f9cf81f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cfce00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cfce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cf8155560 0x7f9cf81494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cfce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf80ca3e0 con 0x7f9cf8155560 2024-04-20T21:38:46.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cfce00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf80c9980 con 0x7f9cf80b41e0 2024-04-20T21:38:46.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cfce00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf806d1e0 con 0x7f9cf80cc390 2024-04-20T21:38:46.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cf7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf80b41e0 0x7f9cf81f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cf8155560 0x7f9cf81494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cf7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf80b41e0 0x7f9cf81f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37516/0 (socket says 172.21.15.102:37516) 2024-04-20T21:38:46.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.604+0000 7f9cf7400640 1 -- 172.21.15.102:0/3605654994 learned_addr learned my addr 172.21.15.102:0/3605654994 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:46.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.605+0000 7f9cf6a00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.605+0000 7f9cf7400640 1 -- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 msgr2=0x7f9cf814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.605+0000 7f9cf7400640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf814c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.605+0000 7f9cf7400640 1 -- 172.21.15.102:0/3605654994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cf8155560 msgr2=0x7f9cf81494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.605+0000 7f9cf7400640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cf8155560 0x7f9cf81494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.605+0000 7f9cf7400640 1 -- 172.21.15.102:0/3605654994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf80c8060 con 0x7f9cf80b41e0 2024-04-20T21:38:46.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.605+0000 7f9cf7400640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf80b41e0 0x7f9cf81f4300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000c3c0 tx=0x7f9ce000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.606+0000 7f9cef400640 1 -- 172.21.15.102:0/3605654994 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0017070 con 0x7f9cf80b41e0 2024-04-20T21:38:46.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.606+0000 7f9cef400640 1 -- 172.21.15.102:0/3605654994 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce0005ce0 con 0x7f9cf80b41e0 2024-04-20T21:38:46.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.606+0000 7f9cef400640 1 -- 172.21.15.102:0/3605654994 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00072f0 con 0x7f9cf80b41e0 2024-04-20T21:38:46.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.606+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf80c8c90 con 0x7f9cf80b41e0 2024-04-20T21:38:46.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.606+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf80c7ca0 con 0x7f9cf80b41e0 2024-04-20T21:38:46.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.607+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf804fd60 con 0x7f9cf80b41e0 2024-04-20T21:38:46.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.608+0000 7f9cef400640 1 -- 172.21.15.102:0/3605654994 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ce001a070 con 0x7f9cf80b41e0 2024-04-20T21:38:46.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.609+0000 7f9cef400640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9cd406c8f0 0x7f9cd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.609+0000 7f9cef400640 1 -- 172.21.15.102:0/3605654994 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f9ce008b690 con 0x7f9cf80b41e0 2024-04-20T21:38:46.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.609+0000 7f9cf6a00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9cd406c8f0 0x7f9cd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.610+0000 7f9cf6a00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9cd406c8f0 0x7f9cd406edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8036000 tx=0x7f9ce8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.614+0000 7f9cef400640 1 -- 172.21.15.102:0/3605654994 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ce0058ca0 con 0x7f9cf80b41e0 2024-04-20T21:38:46.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.856+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cf80b3db0 con 0x7f9cf80b41e0 2024-04-20T21:38:46.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.857+0000 7f9cef400640 1 -- 172.21.15.102:0/3605654994 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6030 (secure 0 0 0) 0x7f9ce005cfa0 con 0x7f9cf80b41e0 2024-04-20T21:38:46.856 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:46.856 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:44.104311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:46.856 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 71 2024-04-20T21:38:46.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9cd406c8f0 msgr2=0x7f9cd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9cd406c8f0 0x7f9cd406edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8036000 tx=0x7f9ce8030000 comp rx=0 tx=0).stop 2024-04-20T21:38:46.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf80b41e0 msgr2=0x7f9cf81f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf80b41e0 0x7f9cf81f4300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000c3c0 tx=0x7f9ce000c880 comp rx=0 tx=0).stop 2024-04-20T21:38:46.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 shutdown_connections 2024-04-20T21:38:46.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9cf8155560 0x7f9cf81494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cf80cc390 0x7f9cf814c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9cd406c8f0 0x7f9cd406edb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 --2- 172.21.15.102:0/3605654994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cf80b41e0 0x7f9cf81f4300 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 >> 172.21.15.102:0/3605654994 conn(0x7f9cf8075080 msgr2=0x7f9cf814b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 shutdown_connections 2024-04-20T21:38:46.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:46.860+0000 7f9cfce00640 1 -- 172.21.15.102:0/3605654994 wait complete. 2024-04-20T21:38:46.879 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 71, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 71, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6421": { "addr": "172.21.15.102:6833/327932375", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 327932375, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 327932375, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6421, "incarnation": 68, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:clientreplay", "state_seq": 37 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:38:44.104311+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6421 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:38:48.880 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:49.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.086+0000 7f2fdf200640 1 Processor -- start 2024-04-20T21:38:49.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.086+0000 7f2fdf200640 1 -- start start 2024-04-20T21:38:49.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe00c3700 0x7f2fe00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdf200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe015c220 con 0x7f2fe0152290 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdf200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe00c4560 con 0x7f2fe00c8330 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdf200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe00c9180 con 0x7f2fe00c3700 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdde00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2fe0152290 0x7f2fe006e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38052/0 (socket says 172.21.15.102:38052) 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdde00640 1 -- 172.21.15.102:0/1353515911 learned_addr learned my addr 172.21.15.102:0/1353515911 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdd400640 1 --2- 172.21.15.102:0/1353515911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe00c3700 0x7f2fe00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdca00640 1 -- 172.21.15.102:0/1353515911 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3618288952 0 0) 0x7f2fe015c220 con 0x7f2fe0152290 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdca00640 1 -- 172.21.15.102:0/1353515911 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fc4003520 con 0x7f2fe0152290 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.088+0000 7f2fdca00640 1 -- 172.21.15.102:0/1353515911 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099857504 0 0) 0x7f2fe00c4560 con 0x7f2fe00c8330 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.089+0000 7f2fdca00640 1 -- 172.21.15.102:0/1353515911 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fe015c220 con 0x7f2fe00c8330 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.089+0000 7f2fdd400640 1 -- 172.21.15.102:0/1353515911 >> v1:172.21.15.193:6789/0 conn(0x7f2fe00c8330 legacy=0x7f2fe0162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.089+0000 7f2fdd400640 1 -- 172.21.15.102:0/1353515911 >> v1:172.21.15.102:6789/0 conn(0x7f2fe0152290 legacy=0x7f2fe006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.089+0000 7f2fdd400640 1 -- 172.21.15.102:0/1353515911 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe015b460 con 0x7f2fe00c3700 2024-04-20T21:38:49.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.090+0000 7f2fdd400640 1 --2- 172.21.15.102:0/1353515911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe00c3700 0x7f2fe00cd310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009dc0 tx=0x7f2fc802bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248bdb47a1a3a1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.091+0000 7f2fdca00640 1 -- 172.21.15.102:0/1353515911 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8039070 con 0x7f2fe00c3700 2024-04-20T21:38:49.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.091+0000 7f2fdca00640 1 -- 172.21.15.102:0/1353515911 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc802cd60 con 0x7f2fe00c3700 2024-04-20T21:38:49.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.091+0000 7f2fdca00640 1 -- 172.21.15.102:0/1353515911 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8034660 con 0x7f2fe00c3700 2024-04-20T21:38:49.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.092+0000 7f2fdf200640 1 -- 172.21.15.102:0/1353515911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe00c3700 msgr2=0x7f2fe00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.092+0000 7f2fdf200640 1 --2- 172.21.15.102:0/1353515911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe00c3700 0x7f2fe00cd310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009dc0 tx=0x7f2fc802bfb0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.092+0000 7f2fdf200640 1 -- 172.21.15.102:0/1353515911 shutdown_connections 2024-04-20T21:38:49.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.092+0000 7f2fdf200640 1 --2- 172.21.15.102:0/1353515911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe00c3700 0x7f2fe00cd310 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.092+0000 7f2fdf200640 1 -- 172.21.15.102:0/1353515911 >> 172.21.15.102:0/1353515911 conn(0x7f2fe00cc820 msgr2=0x7f2fe00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.092+0000 7f2fdf200640 1 -- 172.21.15.102:0/1353515911 shutdown_connections 2024-04-20T21:38:49.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.092+0000 7f2fdf200640 1 -- 172.21.15.102:0/1353515911 wait complete. 2024-04-20T21:38:49.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.094+0000 7f2fdf200640 1 Processor -- start 2024-04-20T21:38:49.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.094+0000 7f2fdf200640 1 -- start start 2024-04-20T21:38:49.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe00c3700 0x7f2fe01e28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00c8330 0x7f2fe00cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe0152290 0x7f2fe013e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdf200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe015ba20 con 0x7f2fe00c8330 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdf200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe00c4840 con 0x7f2fe00c3700 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdf200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe015bd00 con 0x7f2fe0152290 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00c8330 0x7f2fe00cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00c8330 0x7f2fe00cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57560/0 (socket says 172.21.15.102:57560) 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdd400640 1 -- 172.21.15.102:0/535222319 learned_addr learned my addr 172.21.15.102:0/535222319 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdde00640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe00c3700 0x7f2fe01e28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fde800640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe0152290 0x7f2fe013e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdd400640 1 -- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe0152290 msgr2=0x7f2fe013e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdd400640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe0152290 0x7f2fe013e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdd400640 1 -- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe00c3700 msgr2=0x7f2fe01e28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdd400640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe00c3700 0x7f2fe01e28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.095+0000 7f2fdd400640 1 -- 172.21.15.102:0/535222319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe01f4770 con 0x7f2fe00c8330 2024-04-20T21:38:49.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.096+0000 7f2fdd400640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00c8330 0x7f2fe00cce70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009ee0 tx=0x7f2fc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.096+0000 7f2fd6000640 1 -- 172.21.15.102:0/535222319 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8044020 con 0x7f2fe00c8330 2024-04-20T21:38:49.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.096+0000 7f2fd6000640 1 -- 172.21.15.102:0/535222319 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc803f470 con 0x7f2fe00c8330 2024-04-20T21:38:49.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.096+0000 7f2fd6000640 1 -- 172.21.15.102:0/535222319 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8039040 con 0x7f2fe00c8330 2024-04-20T21:38:49.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.096+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe01f4a50 con 0x7f2fe00c8330 2024-04-20T21:38:49.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.096+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe01f5010 con 0x7f2fe00c8330 2024-04-20T21:38:49.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.097+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe004fd60 con 0x7f2fe00c8330 2024-04-20T21:38:49.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.098+0000 7f2fd6000640 1 -- 172.21.15.102:0/535222319 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fc8047020 con 0x7f2fe00c8330 2024-04-20T21:38:49.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.099+0000 7f2fd6000640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.099+0000 7f2fd6000640 1 -- 172.21.15.102:0/535222319 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f2fc80aeea0 con 0x7f2fe00c8330 2024-04-20T21:38:49.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.100+0000 7f2fdde00640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.100+0000 7f2fdde00640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0000f60 tx=0x7f2fd0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.104+0000 7f2fd6000640 1 -- 172.21.15.102:0/535222319 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2fc804e020 con 0x7f2fe00c8330 2024-04-20T21:38:49.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.342+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fe006dcb0 con 0x7f2fe00c8330 2024-04-20T21:38:49.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.343+0000 7f2fd6000640 1 -- 172.21.15.102:0/535222319 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6030 (secure 0 0 0) 0x7f2fc807c670 con 0x7f2fe00c8330 2024-04-20T21:38:49.343 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:49.343 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:44.104311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:49.343 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 71 2024-04-20T21:38:49.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2fbc06c8f0 msgr2=0x7f2fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0000f60 tx=0x7f2fd0009040 comp rx=0 tx=0).stop 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00c8330 msgr2=0x7f2fe00cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00c8330 0x7f2fe00cce70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009ee0 tx=0x7f2fc8004a40 comp rx=0 tx=0).stop 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 shutdown_connections 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2fe0152290 0x7f2fe013e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00c8330 0x7f2fe00cce70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 --2- 172.21.15.102:0/535222319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe00c3700 0x7f2fe01e28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 >> 172.21.15.102:0/535222319 conn(0x7f2fe00cc820 msgr2=0x7f2fe01467b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 shutdown_connections 2024-04-20T21:38:49.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:49.346+0000 7f2fdf200640 1 -- 172.21.15.102:0/535222319 wait complete. 2024-04-20T21:38:49.365 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:51.366 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:51.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.587+0000 7f4109a00640 1 Processor -- start 2024-04-20T21:38:51.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.587+0000 7f4109a00640 1 -- start start 2024-04-20T21:38:51.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.588+0000 7f4109a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f410406cc50 0x7f410406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.588+0000 7f4109a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f41040c8ea0 0x7f4104151080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.588+0000 7f4109a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4104157590 con 0x7f410406d940 2024-04-20T21:38:51.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.588+0000 7f4109a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f4104071e10 con 0x7f41040c8ea0 2024-04-20T21:38:51.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.588+0000 7f4109a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f41040720f0 con 0x7f410406cc50 2024-04-20T21:38:51.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.589+0000 7f4102a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f410406cc50 0x7f410406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.589+0000 7f4102000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f410406d940 0x7f41040c7290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:42586/0 (socket says 172.21.15.102:42586) 2024-04-20T21:38:51.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.589+0000 7f4103400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f41040c8ea0 0x7f4104151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.589+0000 7f4102000640 1 -- 172.21.15.102:0/1622951718 learned_addr learned my addr 172.21.15.102:0/1622951718 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:38:51.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.589+0000 7f4102a00640 1 -- 172.21.15.102:0/1622951718 >> v2:172.21.15.102:3300/0 conn(0x7f41040c8ea0 msgr2=0x7f4104151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.589+0000 7f4102a00640 1 --2- 172.21.15.102:0/1622951718 >> v2:172.21.15.102:3300/0 conn(0x7f41040c8ea0 0x7f4104151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.589+0000 7f4102a00640 1 -- 172.21.15.102:0/1622951718 >> v1:172.21.15.193:6789/0 conn(0x7f410406d940 legacy=0x7f41040c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.589+0000 7f4102a00640 1 -- 172.21.15.102:0/1622951718 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040723d0 con 0x7f410406cc50 2024-04-20T21:38:51.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.590+0000 7f4102a00640 1 --2- 172.21.15.102:0/1622951718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f410406cc50 0x7f410406d030 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4104070a20 tx=0x7f40f402b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5d2ae953e53afa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4101600640 1 -- 172.21.15.102:0/1622951718 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4038070 con 0x7f410406cc50 2024-04-20T21:38:51.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4101600640 1 -- 172.21.15.102:0/1622951718 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f4033640 con 0x7f410406cc50 2024-04-20T21:38:51.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4101600640 1 -- 172.21.15.102:0/1622951718 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4033a10 con 0x7f410406cc50 2024-04-20T21:38:51.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4109a00640 1 -- 172.21.15.102:0/1622951718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f410406cc50 msgr2=0x7f410406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4109a00640 1 --2- 172.21.15.102:0/1622951718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f410406cc50 0x7f410406d030 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4104070a20 tx=0x7f40f402b7a0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4109a00640 1 -- 172.21.15.102:0/1622951718 shutdown_connections 2024-04-20T21:38:51.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4109a00640 1 --2- 172.21.15.102:0/1622951718 >> v2:172.21.15.102:3300/0 conn(0x7f41040c8ea0 0x7f4104151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4109a00640 1 --2- 172.21.15.102:0/1622951718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f410406cc50 0x7f410406d030 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4109a00640 1 -- 172.21.15.102:0/1622951718 >> 172.21.15.102:0/1622951718 conn(0x7f41040c9390 msgr2=0x7f41040c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.591+0000 7f4109a00640 1 -- 172.21.15.102:0/1622951718 shutdown_connections 2024-04-20T21:38:51.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.592+0000 7f4109a00640 1 -- 172.21.15.102:0/1622951718 wait complete. 2024-04-20T21:38:51.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.593+0000 7f4109a00640 1 Processor -- start 2024-04-20T21:38:51.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.593+0000 7f4109a00640 1 -- start start 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4109a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f410406cc50 0x7f410415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4109a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f410406d940 0x7f41041f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4109a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41040c8ea0 0x7f410414c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4109a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41040c79a0 con 0x7f410406cc50 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4109a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4104072990 con 0x7f41040c8ea0 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4109a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4104157260 con 0x7f410406d940 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4102a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f410406cc50 0x7f410415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4102a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f410406cc50 0x7f410415aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57578/0 (socket says 172.21.15.102:57578) 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.594+0000 7f4102a00640 1 -- 172.21.15.102:0/542756684 learned_addr learned my addr 172.21.15.102:0/542756684 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.595+0000 7f4103400640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41040c8ea0 0x7f410414c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.595+0000 7f4102000640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f410406d940 0x7f41041f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.595+0000 7f4102a00640 1 -- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f410406d940 msgr2=0x7f41041f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.595+0000 7f4102a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f410406d940 0x7f41041f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.595+0000 7f4102a00640 1 -- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41040c8ea0 msgr2=0x7f410414c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.595+0000 7f4102a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41040c8ea0 0x7f410414c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.595+0000 7f4102a00640 1 -- 172.21.15.102:0/542756684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41041f1170 con 0x7f410406cc50 2024-04-20T21:38:51.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.595+0000 7f4102a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f410406cc50 0x7f410415aac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009890 tx=0x7f40f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.596+0000 7f40faa00640 1 -- 172.21.15.102:0/542756684 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4042020 con 0x7f410406cc50 2024-04-20T21:38:51.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.596+0000 7f40faa00640 1 -- 172.21.15.102:0/542756684 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f40308e0 con 0x7f410406cc50 2024-04-20T21:38:51.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.596+0000 7f40faa00640 1 -- 172.21.15.102:0/542756684 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4038040 con 0x7f410406cc50 2024-04-20T21:38:51.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.596+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41041543c0 con 0x7f410406cc50 2024-04-20T21:38:51.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.596+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41041f1710 con 0x7f410406cc50 2024-04-20T21:38:51.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.597+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4104074340 con 0x7f410406cc50 2024-04-20T21:38:51.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.598+0000 7f40faa00640 1 -- 172.21.15.102:0/542756684 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40f4045020 con 0x7f410406cc50 2024-04-20T21:38:51.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.599+0000 7f40faa00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f40e406c940 0x7f40e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.599+0000 7f40faa00640 1 -- 172.21.15.102:0/542756684 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f40f40ad9b0 con 0x7f410406cc50 2024-04-20T21:38:51.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.599+0000 7f4102000640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f40e406c940 0x7f40e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.600+0000 7f4102000640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f40e406c940 0x7f40e406ee00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f41040c64a0 tx=0x7f40f0007e20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.603+0000 7f40faa00640 1 -- 172.21.15.102:0/542756684 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f40f404c030 con 0x7f410406cc50 2024-04-20T21:38:51.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.846+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4104071800 con 0x7f410406cc50 2024-04-20T21:38:51.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.848+0000 7f40faa00640 1 -- 172.21.15.102:0/542756684 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6030 (secure 0 0 0) 0x7f40f407afb0 con 0x7f410406cc50 2024-04-20T21:38:51.847 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:51.847 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:44.104311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:38:51.847 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 71 2024-04-20T21:38:51.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f40e406c940 msgr2=0x7f40e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f40e406c940 0x7f40e406ee00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f41040c64a0 tx=0x7f40f0007e20 comp rx=0 tx=0).stop 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f410406cc50 msgr2=0x7f410415aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f410406cc50 0x7f410415aac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009890 tx=0x7f40f40046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 shutdown_connections 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f40e406c940 0x7f40e406ee00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41040c8ea0 0x7f410414c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f410406d940 0x7f41041f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 --2- 172.21.15.102:0/542756684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f410406cc50 0x7f410415aac0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.850+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 >> 172.21.15.102:0/542756684 conn(0x7f41040c9390 msgr2=0x7f4104155b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.851+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 shutdown_connections 2024-04-20T21:38:51.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:51.851+0000 7f4109a00640 1 -- 172.21.15.102:0/542756684 wait complete. 2024-04-20T21:38:51.868 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:38:53.868 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:54.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.076+0000 7f7c66400640 1 Processor -- start 2024-04-20T21:38:54.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.076+0000 7f7c66400640 1 -- start start 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.077+0000 7f7c66400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7c60151c90 0x7f7c600c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.077+0000 7f7c66400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7c600c7290 0x7f7c60158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.077+0000 7f7c66400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c60158780 con 0x7f7c60071a70 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.077+0000 7f7c66400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f7c600c9200 con 0x7f7c600c7290 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.077+0000 7f7c66400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7c60070df0 con 0x7f7c60151c90 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7c600c7290 0x7f7c60158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7c600c7290 0x7f7c60158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57582/0 (socket says 172.21.15.102:57582) 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5ea00640 1 -- 172.21.15.102:0/1470895361 learned_addr learned my addr 172.21.15.102:0/1470895361 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:54.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5f400640 1 --2- 172.21.15.102:0/1470895361 >> v2:172.21.15.193:3300/0 conn(0x7f7c60151c90 0x7f7c600c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:54.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5ea00640 1 -- 172.21.15.102:0/1470895361 >> v2:172.21.15.193:3300/0 conn(0x7f7c60151c90 msgr2=0x7f7c600c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:54.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5ea00640 1 --2- 172.21.15.102:0/1470895361 >> v2:172.21.15.193:3300/0 conn(0x7f7c60151c90 0x7f7c600c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5ea00640 1 -- 172.21.15.102:0/1470895361 >> v1:172.21.15.193:6789/0 conn(0x7f7c60071a70 legacy=0x7f7c6006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:54.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5ea00640 1 -- 172.21.15.102:0/1470895361 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c60071040 con 0x7f7c600c7290 2024-04-20T21:38:54.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.078+0000 7f7c5e000640 1 -- 172.21.15.102:0/1470895361 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845023364 0 0) 0x7f7c60158780 con 0x7f7c60071a70 2024-04-20T21:38:54.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.079+0000 7f7c5ea00640 1 --2- 172.21.15.102:0/1470895361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c600c7290 0x7f7c60158240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7c600ca0f0 tx=0x7f7c4c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5585d108b2a765ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:54.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c5e000640 1 -- 172.21.15.102:0/1470895361 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c02f850 con 0x7f7c600c7290 2024-04-20T21:38:54.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c5e000640 1 -- 172.21.15.102:0/1470895361 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4c02f9f0 con 0x7f7c600c7290 2024-04-20T21:38:54.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c5e000640 1 -- 172.21.15.102:0/1470895361 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c02fdc0 con 0x7f7c600c7290 2024-04-20T21:38:54.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c66400640 1 -- 172.21.15.102:0/1470895361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c600c7290 msgr2=0x7f7c60158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:54.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c66400640 1 --2- 172.21.15.102:0/1470895361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c600c7290 0x7f7c60158240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7c600ca0f0 tx=0x7f7c4c02ba30 comp rx=0 tx=0).stop 2024-04-20T21:38:54.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c66400640 1 -- 172.21.15.102:0/1470895361 shutdown_connections 2024-04-20T21:38:54.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c66400640 1 --2- 172.21.15.102:0/1470895361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c600c7290 0x7f7c60158240 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c66400640 1 --2- 172.21.15.102:0/1470895361 >> v2:172.21.15.193:3300/0 conn(0x7f7c60151c90 0x7f7c600c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c66400640 1 -- 172.21.15.102:0/1470895361 >> 172.21.15.102:0/1470895361 conn(0x7f7c600c9390 msgr2=0x7f7c600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:54.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.080+0000 7f7c66400640 1 -- 172.21.15.102:0/1470895361 shutdown_connections 2024-04-20T21:38:54.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.081+0000 7f7c66400640 1 -- 172.21.15.102:0/1470895361 wait complete. 2024-04-20T21:38:54.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.082+0000 7f7c66400640 1 Processor -- start 2024-04-20T21:38:54.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.082+0000 7f7c66400640 1 -- start start 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c66400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c60071a70 0x7f7c601573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c66400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c600c7290 0x7f7c6014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c66400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c60151c90 0x7f7c601f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c66400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60071600 con 0x7f7c60071a70 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c66400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60157db0 con 0x7f7c600c7290 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c66400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7c60157a40 con 0x7f7c60151c90 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c60071a70 0x7f7c601573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c60071a70 0x7f7c601573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57590/0 (socket says 172.21.15.102:57590) 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c5f400640 1 -- 172.21.15.102:0/3877801263 learned_addr learned my addr 172.21.15.102:0/3877801263 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c5ea00640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c600c7290 0x7f7c6014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.083+0000 7f7c5fe00640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c60151c90 0x7f7c601f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c5f400640 1 -- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c60151c90 msgr2=0x7f7c601f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:54.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c5f400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c60151c90 0x7f7c601f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c5f400640 1 -- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c600c7290 msgr2=0x7f7c6014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:54.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c5f400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c600c7290 0x7f7c6014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c5f400640 1 -- 172.21.15.102:0/3877801263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c601e3630 con 0x7f7c60071a70 2024-04-20T21:38:54.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c5f400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c60071a70 0x7f7c601573b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e6c0 tx=0x7f7c5000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:54.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c57400640 1 -- 172.21.15.102:0/3877801263 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50016070 con 0x7f7c60071a70 2024-04-20T21:38:54.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c57400640 1 -- 172.21.15.102:0/3877801263 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5000ca30 con 0x7f7c60071a70 2024-04-20T21:38:54.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c57400640 1 -- 172.21.15.102:0/3877801263 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000b040 con 0x7f7c60071a70 2024-04-20T21:38:54.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.084+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c601e3910 con 0x7f7c60071a70 2024-04-20T21:38:54.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.085+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6015b030 con 0x7f7c60071a70 2024-04-20T21:38:54.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.086+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c60069340 con 0x7f7c60071a70 2024-04-20T21:38:54.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.086+0000 7f7c57400640 1 -- 172.21.15.102:0/3877801263 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c50019070 con 0x7f7c60071a70 2024-04-20T21:38:54.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.087+0000 7f7c57400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c4006c8f0 0x7f7c4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:54.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.087+0000 7f7c57400640 1 -- 172.21.15.102:0/3877801263 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f7c5008c000 con 0x7f7c60071a70 2024-04-20T21:38:54.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.088+0000 7f7c5ea00640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c4006c8f0 0x7f7c4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:54.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.088+0000 7f7c5ea00640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c4006c8f0 0x7f7c4006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c036000 tx=0x7f7c4c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:54.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.092+0000 7f7c57400640 1 -- 172.21.15.102:0/3877801263 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c60069340 con 0x7f7c60071a70 2024-04-20T21:38:54.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.334+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c60074340 con 0x7f7c60071a70 2024-04-20T21:38:54.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.335+0000 7f7c57400640 1 -- 172.21.15.102:0/3877801263 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+6031 (secure 0 0 0) 0x7f7c500596b0 con 0x7f7c60071a70 2024-04-20T21:38:54.335 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:38:54.335 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:38:54.335 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 72 2024-04-20T21:38:54.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c4006c8f0 msgr2=0x7f7c4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c4006c8f0 0x7f7c4006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c036000 tx=0x7f7c4c008030 comp rx=0 tx=0).stop 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c60071a70 msgr2=0x7f7c601573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c60071a70 0x7f7c601573b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e6c0 tx=0x7f7c5000eb80 comp rx=0 tx=0).stop 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 shutdown_connections 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c60151c90 0x7f7c601f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7c4006c8f0 0x7f7c4006edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c600c7290 0x7f7c6014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 --2- 172.21.15.102:0/3877801263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c60071a70 0x7f7c601573b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.338+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 >> 172.21.15.102:0/3877801263 conn(0x7f7c600c9390 msgr2=0x7f7c6014b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.339+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 shutdown_connections 2024-04-20T21:38:54.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:38:54.339+0000 7f7c66400640 1 -- 172.21.15.102:0/3877801263 wait complete. 2024-04-20T21:38:54.357 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=6147 2024-04-20T21:38:54.357 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.c 2024-04-20T21:38:57.688 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 52% (117/225) Updating files: 53% (120/225) Updating files: 54% (122/225) Updating files: 55% (124/225) Updating files: 56% (126/225) Updating files: 57% (129/225) Updating files: 58% (131/225) Updating files: 59% (133/225) Updating files: 60% (135/225) Updating files: 61% (138/225) Updating files: 62% (140/225) Updating files: 63% (142/225) Updating files: 64% (144/225) Updating files: 65% (147/225) Updating files: 66% (149/225) Updating files: 67% (151/225) Updating files: 68% (153/225) Updating files: 69% (156/225) Updating files: 70% (158/225) Updating files: 71% (160/225) Updating files: 72% (162/225) Updating files: 73% (165/225) Updating files: 74% (167/225) Updating files: 75% (169/225) Updating files: 76% (171/225) Updating files: 76% (173/225) Updating files: 77% (174/225) Updating files: 78% (176/225) Updating files: 78% (177/225) Updating files: 79% (178/225) Updating files: 80% (180/225) Updating files: 81% (183/225) Updating files: 82% (185/225) Updating files: 83% (187/225) Updating files: 84% (189/225) Updating files: 85% (192/225) Updating files: 85% (193/225) Updating files: 86% (194/225) Updating files: 87% (196/225) Updating files: 88% (198/225) Updating files: 89% (201/225) Updating files: 90% (203/225) Updating files: 91% (205/225) Updating files: 91% (206/225) Updating files: 92% (207/225) Updating files: 93% (210/225) Updating files: 94% (212/225) Updating files: 95% (214/225) Updating files: 96% (216/225) Updating files: 97% (219/225) Updating files: 98% (221/225) Updating files: 99% (223/225) Updating files: 100% (225/225) Updating files: 100% (225/225), done. 2024-04-20T21:38:57.836 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 284161c00ab kclient: kill dput/d_drop debug cruft 2024-04-20T21:38:57.852 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.8 2024-04-20T21:38:57.895 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:38:57.895 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 9 -eq 48 ']' 2024-04-20T21:38:57.895 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.9 2024-04-20T21:38:57.896 INFO:tasks.workunit.client.0.smithi102.stdout:v0.9 2024-04-20T21:38:57.896 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.9 2024-04-20T21:38:57.896 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.9 2024-04-20T21:39:00.089 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:39:00.098 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:39:00.099 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-04-20T21:39:00.095+0000 7fbd9e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:39:02.357 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:39:02.358 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:39:02.358 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:39:02.365 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:39:02.365 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:02.510 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:39:02.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.549+0000 7fe2ee400640 1 Processor -- start 2024-04-20T21:39:02.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.549+0000 7fe2ee400640 1 -- start start 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.550+0000 7fe2ee400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe2e80cb980 0x7fe2e815e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.550+0000 7fe2ee400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e80cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.550+0000 7fe2ee400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2e80ccdd0 con 0x7fe2e80cbd60 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.550+0000 7fe2ee400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e815f7e0 con 0x7fe2e80cb980 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.550+0000 7fe2ee400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e815ef40 con 0x7fe2e80cdb30 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.550+0000 7fe2e7400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe2e80cb980 0x7fe2e815e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.551+0000 7fe2e7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e80cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.551+0000 7fe2e7400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe2e80cb980 0x7fe2e815e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37722/0 (socket says 172.21.15.102:37722) 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.551+0000 7fe2e7400640 1 -- 172.21.15.102:0/2695668915 learned_addr learned my addr 172.21.15.102:0/2695668915 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.551+0000 7fe2e6a00640 1 --1- 172.21.15.102:0/2695668915 >> v1:172.21.15.102:6789/0 conn(0x7fe2e80cbd60 0x7fe2e80cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:54758/0 (socket says 172.21.15.102:54758) 2024-04-20T21:39:02.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.551+0000 7fe2e7e00640 1 -- 172.21.15.102:0/2695668915 >> v2:172.21.15.102:3300/0 conn(0x7fe2e80cb980 msgr2=0x7fe2e815e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.551+0000 7fe2e7e00640 1 --2- 172.21.15.102:0/2695668915 >> v2:172.21.15.102:3300/0 conn(0x7fe2e80cb980 0x7fe2e815e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.551+0000 7fe2e7e00640 1 -- 172.21.15.102:0/2695668915 >> v1:172.21.15.102:6789/0 conn(0x7fe2e80cbd60 legacy=0x7fe2e80cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.551+0000 7fe2e7e00640 1 -- 172.21.15.102:0/2695668915 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e815f220 con 0x7fe2e80cdb30 2024-04-20T21:39:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.552+0000 7fe2e7e00640 1 --2- 172.21.15.102:0/2695668915 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e80cdfc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009d90 tx=0x7fe2e002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf101846773261 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:02.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.552+0000 7fe2e6000640 1 -- 172.21.15.102:0/2695668915 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0039070 con 0x7fe2e80cdb30 2024-04-20T21:39:02.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.552+0000 7fe2e6000640 1 -- 172.21.15.102:0/2695668915 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e002cc60 con 0x7fe2e80cdb30 2024-04-20T21:39:02.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.552+0000 7fe2e6000640 1 -- 172.21.15.102:0/2695668915 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0034660 con 0x7fe2e80cdb30 2024-04-20T21:39:02.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.553+0000 7fe2ee400640 1 -- 172.21.15.102:0/2695668915 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 msgr2=0x7fe2e80cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.553+0000 7fe2ee400640 1 --2- 172.21.15.102:0/2695668915 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e80cdfc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009d90 tx=0x7fe2e002beb0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.553+0000 7fe2ee400640 1 -- 172.21.15.102:0/2695668915 shutdown_connections 2024-04-20T21:39:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.553+0000 7fe2ee400640 1 --2- 172.21.15.102:0/2695668915 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e80cdfc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.553+0000 7fe2ee400640 1 --2- 172.21.15.102:0/2695668915 >> v2:172.21.15.102:3300/0 conn(0x7fe2e80cb980 0x7fe2e815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.553+0000 7fe2ee400640 1 -- 172.21.15.102:0/2695668915 >> 172.21.15.102:0/2695668915 conn(0x7fe2e8075080 msgr2=0x7fe2e8075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.553+0000 7fe2ee400640 1 -- 172.21.15.102:0/2695668915 shutdown_connections 2024-04-20T21:39:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.553+0000 7fe2ee400640 1 -- 172.21.15.102:0/2695668915 wait complete. 2024-04-20T21:39:02.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.555+0000 7fe2ee400640 1 Processor -- start 2024-04-20T21:39:02.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.555+0000 7fe2ee400640 1 -- start start 2024-04-20T21:39:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.556+0000 7fe2ee400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e80cb980 0x7fe2e8156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.556+0000 7fe2ee400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2e80cbd60 0x7fe2e81e60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.556+0000 7fe2ee400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e81e5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.556+0000 7fe2ee400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e80cd610 con 0x7fe2e80cbd60 2024-04-20T21:39:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.556+0000 7fe2ee400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e80cd330 con 0x7fe2e80cb980 2024-04-20T21:39:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.556+0000 7fe2ee400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e80cc1d0 con 0x7fe2e80cdb30 2024-04-20T21:39:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e80cb980 0x7fe2e8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e80cb980 0x7fe2e8156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:42918/0 (socket says 172.21.15.102:42918) 2024-04-20T21:39:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7400640 1 -- 172.21.15.102:0/1584954809 learned_addr learned my addr 172.21.15.102:0/1584954809 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7e00640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e81e5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e6a00640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2e80cbd60 0x7fe2e81e60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7400640 1 -- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 msgr2=0x7fe2e81e5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e81e5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7400640 1 -- 172.21.15.102:0/1584954809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2e80cbd60 msgr2=0x7fe2e81e60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2e80cbd60 0x7fe2e81e60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.557+0000 7fe2e7400640 1 -- 172.21.15.102:0/1584954809 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e81e5250 con 0x7fe2e80cb980 2024-04-20T21:39:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.558+0000 7fe2e7400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e80cb980 0x7fe2e8156910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e8b0 tx=0x7fe2d800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.559+0000 7fe2df400640 1 -- 172.21.15.102:0/1584954809 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8017070 con 0x7fe2e80cb980 2024-04-20T21:39:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.559+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e81e5530 con 0x7fe2e80cb980 2024-04-20T21:39:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.559+0000 7fe2df400640 1 -- 172.21.15.102:0/1584954809 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d800c940 con 0x7fe2e80cb980 2024-04-20T21:39:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.559+0000 7fe2df400640 1 -- 172.21.15.102:0/1584954809 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8017070 con 0x7fe2e80cb980 2024-04-20T21:39:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.559+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e8156110 con 0x7fe2e80cb980 2024-04-20T21:39:02.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.564+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e806c840 con 0x7fe2e80cb980 2024-04-20T21:39:02.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.563+0000 7fe2df400640 1 -- 172.21.15.102:0/1584954809 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2d801a070 con 0x7fe2e80cb980 2024-04-20T21:39:02.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.568+0000 7fe2df400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2c806caf0 0x7fe2c806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:02.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.568+0000 7fe2df400640 1 -- 172.21.15.102:0/1584954809 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7fe2d808b900 con 0x7fe2e80cb980 2024-04-20T21:39:02.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.569+0000 7fe2e6a00640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2c806caf0 0x7fe2c806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:02.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.571+0000 7fe2e6a00640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2c806caf0 0x7fe2c806efb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8153c10 tx=0x7fe2d4007cc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.576+0000 7fe2df400640 1 -- 172.21.15.102:0/1584954809 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe2e806c840 con 0x7fe2e80cb980 2024-04-20T21:39:02.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.829+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2e8074340 con 0x7fe2e80cb980 2024-04-20T21:39:02.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.831+0000 7fe2df400640 1 -- 172.21.15.102:0/1584954809 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+6031 (secure 0 0 0) 0x7fe2d8058e80 con 0x7fe2e80cb980 2024-04-20T21:39:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:02.830 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:39:02.830 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 72 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.833+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2c806caf0 msgr2=0x7fe2c806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.833+0000 7fe2ee400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2c806caf0 0x7fe2c806efb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8153c10 tx=0x7fe2d4007cc0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e80cb980 msgr2=0x7fe2e8156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e80cb980 0x7fe2e8156910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e8b0 tx=0x7fe2d800ed70 comp rx=0 tx=0).stop 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 shutdown_connections 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2e80cdb30 0x7fe2e81e5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2e80cbd60 0x7fe2e81e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2c806caf0 0x7fe2c806efb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 --2- 172.21.15.102:0/1584954809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e80cb980 0x7fe2e8156910 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:02.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 >> 172.21.15.102:0/1584954809 conn(0x7fe2e8075080 msgr2=0x7fe2e814eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:02.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 shutdown_connections 2024-04-20T21:39:02.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:02.834+0000 7fe2ee400640 1 -- 172.21.15.102:0/1584954809 wait complete. 2024-04-20T21:39:02.851 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-20T21:39:04.852 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:05.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.070+0000 7f8311400640 1 Processor -- start 2024-04-20T21:39:05.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.070+0000 7f8311400640 1 -- start start 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.071+0000 7f8311400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c14fa50 0x7f830c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.071+0000 7f8311400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f830c0c9eb0 0x7f830c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.071+0000 7f8311400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f830c073a10 0x7f830c0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.071+0000 7f8311400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f830c159500 con 0x7f830c0c9eb0 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.071+0000 7f8311400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f830c1536d0 con 0x7f830c073a10 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.071+0000 7f8311400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f830c0c76c0 con 0x7f830c14fa50 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f8303e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f830c0c9eb0 0x7f830c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f8303e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f830c0c9eb0 0x7f830c158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37734/0 (socket says 172.21.15.102:37734) 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f8303e00640 1 -- 172.21.15.102:0/1446855994 learned_addr learned my addr 172.21.15.102:0/1446855994 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f830b200640 1 --2- 172.21.15.102:0/1446855994 >> v2:172.21.15.193:3300/0 conn(0x7f830c073a10 0x7f830c0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:05.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f830a800640 1 --2- 172.21.15.102:0/1446855994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c14fa50 0x7f830c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:05.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f8303e00640 1 -- 172.21.15.102:0/1446855994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c14fa50 msgr2=0x7f830c0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:05.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f8303e00640 1 --2- 172.21.15.102:0/1446855994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c14fa50 0x7f830c0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f8303e00640 1 -- 172.21.15.102:0/1446855994 >> v2:172.21.15.193:3300/0 conn(0x7f830c073a10 msgr2=0x7f830c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:05.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f8303e00640 1 --2- 172.21.15.102:0/1446855994 >> v2:172.21.15.193:3300/0 conn(0x7f830c073a10 0x7f830c0c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.072+0000 7f8303e00640 1 -- 172.21.15.102:0/1446855994 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c0c7960 con 0x7f830c0c9eb0 2024-04-20T21:39:05.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.073+0000 7f8303e00640 1 --2- 172.21.15.102:0/1446855994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c0c9eb0 0x7f830c158fc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f830c06c190 tx=0x7f82f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cf327f4cbd4248 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:05.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8309e00640 1 -- 172.21.15.102:0/1446855994 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4034070 con 0x7f830c0c9eb0 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8309e00640 1 -- 172.21.15.102:0/1446855994 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f4030860 con 0x7f830c0c9eb0 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8309e00640 1 -- 172.21.15.102:0/1446855994 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4030c30 con 0x7f830c0c9eb0 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8311400640 1 -- 172.21.15.102:0/1446855994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c0c9eb0 msgr2=0x7f830c158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8311400640 1 --2- 172.21.15.102:0/1446855994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c0c9eb0 0x7f830c158fc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f830c06c190 tx=0x7f82f402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8311400640 1 -- 172.21.15.102:0/1446855994 shutdown_connections 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8311400640 1 --2- 172.21.15.102:0/1446855994 >> v2:172.21.15.193:3300/0 conn(0x7f830c073a10 0x7f830c0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8311400640 1 --2- 172.21.15.102:0/1446855994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c0c9eb0 0x7f830c158fc0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8311400640 1 --2- 172.21.15.102:0/1446855994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c14fa50 0x7f830c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.074+0000 7f8311400640 1 -- 172.21.15.102:0/1446855994 >> 172.21.15.102:0/1446855994 conn(0x7f830c0c9390 msgr2=0x7f830c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.075+0000 7f8311400640 1 -- 172.21.15.102:0/1446855994 shutdown_connections 2024-04-20T21:39:05.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.075+0000 7f8311400640 1 -- 172.21.15.102:0/1446855994 wait complete. 2024-04-20T21:39:05.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.076+0000 7f8311400640 1 Processor -- start 2024-04-20T21:39:05.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.076+0000 7f8311400640 1 -- start start 2024-04-20T21:39:05.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.077+0000 7f8311400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c073a10 0x7f830c0717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:05.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.077+0000 7f8311400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c0c9eb0 0x7f830c14c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:05.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.077+0000 7f8311400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c14fa50 0x7f830c148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:05.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.077+0000 7f8311400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f830c0c7c00 con 0x7f830c14fa50 2024-04-20T21:39:05.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.077+0000 7f8311400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f830c0c3d90 con 0x7f830c073a10 2024-04-20T21:39:05.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.077+0000 7f8311400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f830c0c4030 con 0x7f830c0c9eb0 2024-04-20T21:39:05.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c14fa50 0x7f830c148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:05.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c14fa50 0x7f830c148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37738/0 (socket says 172.21.15.102:37738) 2024-04-20T21:39:05.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 -- 172.21.15.102:0/3283385924 learned_addr learned my addr 172.21.15.102:0/3283385924 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:05.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830a800640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c073a10 0x7f830c0717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:05.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f8303e00640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c0c9eb0 0x7f830c14c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:05.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 -- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c0c9eb0 msgr2=0x7f830c14c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:05.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c0c9eb0 0x7f830c14c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 -- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c073a10 msgr2=0x7f830c0717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:05.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c073a10 0x7f830c0717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 -- 172.21.15.102:0/3283385924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c1f15a0 con 0x7f830c14fa50 2024-04-20T21:39:05.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.078+0000 7f830b200640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c14fa50 0x7f830c148890 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f830400ef00 tx=0x7f830400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:05.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.079+0000 7f8302a00640 1 -- 172.21.15.102:0/3283385924 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304018020 con 0x7f830c14fa50 2024-04-20T21:39:05.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.079+0000 7f8302a00640 1 -- 172.21.15.102:0/3283385924 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830400bd20 con 0x7f830c14fa50 2024-04-20T21:39:05.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.079+0000 7f8302a00640 1 -- 172.21.15.102:0/3283385924 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304014640 con 0x7f830c14fa50 2024-04-20T21:39:05.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.079+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c1f1880 con 0x7f830c14fa50 2024-04-20T21:39:05.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.079+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1f1e40 con 0x7f830c14fa50 2024-04-20T21:39:05.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.080+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c14e8b0 con 0x7f830c14fa50 2024-04-20T21:39:05.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.081+0000 7f8302a00640 1 -- 172.21.15.102:0/3283385924 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8304005d10 con 0x7f830c14fa50 2024-04-20T21:39:05.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.082+0000 7f8302a00640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82e806c920 0x7f82e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:05.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.082+0000 7f8302a00640 1 -- 172.21.15.102:0/3283385924 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f830408c830 con 0x7f830c14fa50 2024-04-20T21:39:05.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.082+0000 7f830a800640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82e806c920 0x7f82e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:05.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.083+0000 7f830a800640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82e806c920 0x7f82e806ede0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009b40 tx=0x7f82fc009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:05.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.086+0000 7f8302a00640 1 -- 172.21.15.102:0/3283385924 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f830c14e8b0 con 0x7f830c14fa50 2024-04-20T21:39:05.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.325+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f830c14f590 con 0x7f830c14fa50 2024-04-20T21:39:05.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.326+0000 7f8302a00640 1 -- 172.21.15.102:0/3283385924 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+6859 (secure 0 0 0) 0x7f8304022070 con 0x7f830c14fa50 2024-04-20T21:39:05.325 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:05.325 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:39:05.326 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 73 2024-04-20T21:39:05.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82e806c920 msgr2=0x7f82e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:05.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82e806c920 0x7f82e806ede0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009b40 tx=0x7f82fc009190 comp rx=0 tx=0).stop 2024-04-20T21:39:05.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c14fa50 msgr2=0x7f830c148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c14fa50 0x7f830c148890 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f830400ef00 tx=0x7f830400b480 comp rx=0 tx=0).stop 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 shutdown_connections 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f830c14fa50 0x7f830c148890 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f830c0c9eb0 0x7f830c14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f82e806c920 0x7f82e806ede0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 --2- 172.21.15.102:0/3283385924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c073a10 0x7f830c0717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 >> 172.21.15.102:0/3283385924 conn(0x7f830c0c9390 msgr2=0x7f830c06ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 shutdown_connections 2024-04-20T21:39:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:05.329+0000 7f8311400640 1 -- 172.21.15.102:0/3283385924 wait complete. 2024-04-20T21:39:05.347 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:39:05.347 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 12 secs before thrashing 2024-04-20T21:39:07.582 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 38% (75/193) Updating files: 39% (76/193) Updating files: 40% (78/193) Updating files: 41% (80/193) Updating files: 42% (82/193) Updating files: 43% (83/193) Updating files: 44% (85/193) Updating files: 45% (87/193) Updating files: 46% (89/193) Updating files: 47% (91/193) Updating files: 48% (93/193) Updating files: 49% (95/193) Updating files: 50% (97/193) Updating files: 50% (98/193) Updating files: 51% (99/193) Updating files: 52% (101/193) Updating files: 53% (103/193) Updating files: 54% (105/193) Updating files: 55% (107/193) Updating files: 56% (109/193) Updating files: 57% (111/193) Updating files: 58% (112/193) Updating files: 59% (114/193) Updating files: 60% (116/193) Updating files: 61% (118/193) Updating files: 62% (120/193) Updating files: 62% (121/193) Updating files: 63% (122/193) Updating files: 64% (124/193) Updating files: 65% (126/193) Updating files: 66% (128/193) Updating files: 67% (130/193) Updating files: 68% (132/193) Updating files: 69% (134/193) Updating files: 70% (136/193) Updating files: 71% (138/193) Updating files: 72% (139/193) Updating files: 73% (141/193) Updating files: 74% (143/193) Updating files: 75% (145/193) Updating files: 76% (147/193) Updating files: 77% (149/193) Updating files: 78% (151/193) Updating files: 79% (153/193) Updating files: 80% (155/193) Updating files: 81% (157/193) Updating files: 82% (159/193) Updating files: 83% (161/193) Updating files: 84% (163/193) Updating files: 85% (165/193) Updating files: 86% (166/193) Updating files: 87% (168/193) Updating files: 88% (170/193) Updating files: 89% (172/193) Updating files: 90% (174/193) Updating files: 91% (176/193) Updating files: 92% (178/193) Updating files: 93% (180/193) Updating files: 94% (182/193) Updating files: 95% (184/193) Updating files: 96% (186/193) Updating files: 97% (188/193) Updating files: 98% (190/193) Updating files: 99% (192/193) Updating files: 100% (193/193) Updating files: 100% (193/193), done. 2024-04-20T21:39:07.600 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 41481853924 kclient: strip out kernel version compatibility cruft 2024-04-20T21:39:07.606 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.9 2024-04-20T21:39:07.696 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:39:07.696 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 10 -eq 48 ']' 2024-04-20T21:39:07.696 INFO:tasks.workunit.client.0.smithi102.stdout:v0.10 2024-04-20T21:39:07.696 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.10 2024-04-20T21:39:07.696 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.10 2024-04-20T21:39:07.697 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.10 2024-04-20T21:39:13.114 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 17% (29/164) Updating files: 18% (30/164) Updating files: 19% (32/164) Updating files: 20% (33/164) Updating files: 21% (35/164) Updating files: 22% (37/164) Updating files: 23% (38/164) Updating files: 24% (40/164) Updating files: 25% (41/164) Updating files: 26% (43/164) Updating files: 27% (45/164) Updating files: 28% (46/164) Updating files: 29% (48/164) Updating files: 30% (50/164) Updating files: 31% (51/164) Updating files: 32% (53/164) Updating files: 33% (55/164) Updating files: 34% (56/164) Updating files: 35% (58/164) Updating files: 36% (60/164) Updating files: 37% (61/164) Updating files: 38% (63/164) Updating files: 39% (64/164) Updating files: 39% (65/164) Updating files: 40% (66/164) Updating files: 41% (68/164) Updating files: 42% (69/164) Updating files: 43% (71/164) Updating files: 44% (73/164) Updating files: 45% (74/164) Updating files: 46% (76/164) Updating files: 47% (78/164) Updating files: 48% (79/164) Updating files: 49% (81/164) Updating files: 50% (82/164) Updating files: 51% (84/164) Updating files: 52% (86/164) Updating files: 53% (87/164) Updating files: 54% (89/164) Updating files: 55% (91/164) Updating files: 56% (92/164) Updating files: 57% (94/164) Updating files: 58% (96/164) Updating files: 59% (97/164) Updating files: 60% (99/164) Updating files: 61% (101/164) Updating files: 62% (102/164) Updating files: 63% (104/164) Updating files: 64% (105/164) Updating files: 65% (107/164) Updating files: 66% (109/164) Updating files: 67% (110/164) Updating files: 68% (112/164) Updating files: 69% (114/164) Updating files: 70% (115/164) Updating files: 71% (117/164) Updating files: 72% (119/164) Updating files: 73% (120/164) Updating files: 74% (122/164) Updating files: 75% (123/164) Updating files: 76% (125/164) Updating files: 77% (127/164) Updating files: 78% (128/164) Updating files: 79% (130/164) Updating files: 80% (132/164) Updating files: 81% (133/164) Updating files: 82% (135/164) Updating files: 83% (137/164) Updating files: 84% (138/164) Updating files: 85% (140/164) Updating files: 86% (142/164) Updating files: 87% (143/164) Updating files: 88% (145/164) Updating files: 89% (146/164) Updating files: 90% (148/164) Updating files: 91% (150/164) Updating files: 92% (151/164) Updating files: 93% (153/164) Updating files: 94% (155/164) Updating files: 95% (156/164) Updating files: 96% (158/164) Updating files: 97% (160/164) Updating files: 98% (161/164) Updating files: 99% (163/164) Updating files: 100% (164/164) Updating files: 100% (164/164), done. 2024-04-20T21:39:13.138 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 007313fe844 v0.10 2024-04-20T21:39:13.147 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.10 2024-04-20T21:39:13.259 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:39:13.259 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 11 -eq 48 ']' 2024-04-20T21:39:13.259 INFO:tasks.workunit.client.0.smithi102.stdout:v0.11 2024-04-20T21:39:13.260 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.11 2024-04-20T21:39:13.260 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.11 2024-04-20T21:39:13.260 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.11 2024-04-20T21:39:13.872 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 9fa6f5e7d09 v0.11 2024-04-20T21:39:13.882 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.11 2024-04-20T21:39:14.090 INFO:tasks.workunit.client.0.smithi102.stdout:v0.12 2024-04-20T21:39:14.090 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:39:14.090 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 12 -eq 48 ']' 2024-04-20T21:39:14.090 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.12 2024-04-20T21:39:14.090 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.12 2024-04-20T21:39:14.090 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.12 2024-04-20T21:39:15.156 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at f524150822a v0.12 2024-04-20T21:39:15.166 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.12 2024-04-20T21:39:15.846 INFO:tasks.workunit.client.0.smithi102.stdout:v0.13 2024-04-20T21:39:15.846 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:39:15.846 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 13 -eq 48 ']' 2024-04-20T21:39:15.846 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.13 2024-04-20T21:39:15.846 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.13 2024-04-20T21:39:15.846 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.13 2024-04-20T21:39:17.348 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:17.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.589+0000 7f4508c00640 1 Processor -- start 2024-04-20T21:39:17.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.589+0000 7f4508c00640 1 -- start start 2024-04-20T21:39:17.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.591+0000 7f4508c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc038640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.591+0000 7f4508c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f44fc037170 0x7f44fc038b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.591+0000 7f4508c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0376d0 con 0x7f44fc0378f0 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.591+0000 7f4508c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f44fc0502c0 con 0x7f44fc037170 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.591+0000 7f4508c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc0505a0 con 0x7f44fc0d92b0 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.591+0000 7f4501e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc038640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.592+0000 7f4501e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc038640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50838/0 (socket says 172.21.15.102:50838) 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.592+0000 7f4501e00640 1 -- 172.21.15.102:0/2454910548 learned_addr learned my addr 172.21.15.102:0/2454910548 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.592+0000 7f4501400640 1 --2- 172.21.15.102:0/2454910548 >> v2:172.21.15.193:3300/0 conn(0x7f44fc037170 0x7f44fc038b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.592+0000 7f4501e00640 1 -- 172.21.15.102:0/2454910548 >> v2:172.21.15.193:3300/0 conn(0x7f44fc037170 msgr2=0x7f44fc038b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.592+0000 7f4501e00640 1 --2- 172.21.15.102:0/2454910548 >> v2:172.21.15.193:3300/0 conn(0x7f44fc037170 0x7f44fc038b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.592+0000 7f4501e00640 1 -- 172.21.15.102:0/2454910548 >> v1:172.21.15.193:6789/0 conn(0x7f44fc0378f0 legacy=0x7f44fc0ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.592+0000 7f4501e00640 1 -- 172.21.15.102:0/2454910548 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc050880 con 0x7f44fc0d92b0 2024-04-20T21:39:17.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.594+0000 7f4501e00640 1 --2- 172.21.15.102:0/2454910548 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc038640 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009850 tx=0x7f44f402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61958b813b9f02a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.594+0000 7f4500a00640 1 -- 172.21.15.102:0/2454910548 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4004230 con 0x7f44fc0d92b0 2024-04-20T21:39:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.594+0000 7f4500a00640 1 -- 172.21.15.102:0/2454910548 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f40043d0 con 0x7f44fc0d92b0 2024-04-20T21:39:17.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.594+0000 7f4500a00640 1 -- 172.21.15.102:0/2454910548 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4031830 con 0x7f44fc0d92b0 2024-04-20T21:39:17.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.595+0000 7f4508c00640 1 -- 172.21.15.102:0/2454910548 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 msgr2=0x7f44fc038640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.595+0000 7f4508c00640 1 --2- 172.21.15.102:0/2454910548 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc038640 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009850 tx=0x7f44f402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.597+0000 7f4508c00640 1 -- 172.21.15.102:0/2454910548 shutdown_connections 2024-04-20T21:39:17.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.597+0000 7f4508c00640 1 --2- 172.21.15.102:0/2454910548 >> v2:172.21.15.193:3300/0 conn(0x7f44fc037170 0x7f44fc038b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.597+0000 7f4508c00640 1 --2- 172.21.15.102:0/2454910548 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc038640 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.597+0000 7f4508c00640 1 -- 172.21.15.102:0/2454910548 >> 172.21.15.102:0/2454910548 conn(0x7f44fc054600 msgr2=0x7f44fc054a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.597+0000 7f4508c00640 1 -- 172.21.15.102:0/2454910548 shutdown_connections 2024-04-20T21:39:17.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.597+0000 7f4508c00640 1 -- 172.21.15.102:0/2454910548 wait complete. 2024-04-20T21:39:17.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.600+0000 7f4508c00640 1 Processor -- start 2024-04-20T21:39:17.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.600+0000 7f4508c00640 1 -- start start 2024-04-20T21:39:17.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4508c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44fc037170 0x7f44fc0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4508c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44fc0378f0 0x7f44fc0db040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4508c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc0db580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4508c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc0df4a0 con 0x7f44fc037170 2024-04-20T21:39:17.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4508c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc038120 con 0x7f44fc0378f0 2024-04-20T21:39:17.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4508c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc0df620 con 0x7f44fc0d92b0 2024-04-20T21:39:17.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4502800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc0db580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4502800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc0db580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50848/0 (socket says 172.21.15.102:50848) 2024-04-20T21:39:17.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.602+0000 7f4502800640 1 -- 172.21.15.102:0/627684413 learned_addr learned my addr 172.21.15.102:0/627684413 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:17.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.603+0000 7f4501e00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44fc037170 0x7f44fc0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.603+0000 7f4501e00640 1 -- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 msgr2=0x7f44fc0db580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.603+0000 7f4501e00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc0db580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.603+0000 7f4501e00640 1 -- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44fc0378f0 msgr2=0x7f44fc0db040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T21:39:17.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.603+0000 7f4501e00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44fc0378f0 0x7f44fc0db040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.603+0000 7f4501e00640 1 -- 172.21.15.102:0/627684413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc0cf190 con 0x7f44fc037170 2024-04-20T21:39:17.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.604+0000 7f4501e00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44fc037170 0x7f44fc0cec50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f44f4004000 tx=0x7f44f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.604+0000 7f44fa000640 1 -- 172.21.15.102:0/627684413 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f403f070 con 0x7f44fc037170 2024-04-20T21:39:17.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.604+0000 7f44fa000640 1 -- 172.21.15.102:0/627684413 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f4031a40 con 0x7f44fc037170 2024-04-20T21:39:17.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.604+0000 7f44fa000640 1 -- 172.21.15.102:0/627684413 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4031e30 con 0x7f44fc037170 2024-04-20T21:39:17.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.604+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc0dc110 con 0x7f44fc037170 2024-04-20T21:39:17.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.605+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc0dc6d0 con 0x7f44fc037170 2024-04-20T21:39:17.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.607+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc005380 con 0x7f44fc037170 2024-04-20T21:39:17.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.610+0000 7f44fa000640 1 -- 172.21.15.102:0/627684413 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44f4003610 con 0x7f44fc037170 2024-04-20T21:39:17.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.613+0000 7f44fa000640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f44e406c8f0 0x7f44e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.614+0000 7f44fa000640 1 -- 172.21.15.102:0/627684413 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f44f40ae720 con 0x7f44fc037170 2024-04-20T21:39:17.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.615+0000 7f4501400640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f44e406c8f0 0x7f44e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.617+0000 7f4501400640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f44e406c8f0 0x7f44e406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0defd0 tx=0x7f44f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.622+0000 7f44fa000640 1 -- 172.21.15.102:0/627684413 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f44f404b030 con 0x7f44fc037170 2024-04-20T21:39:17.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.906+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44fc039630 con 0x7f44fc037170 2024-04-20T21:39:17.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.907+0000 7f44fa000640 1 -- 172.21.15.102:0/627684413 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+6859 (secure 0 0 0) 0x7f44fc0cf190 con 0x7f44fc037170 2024-04-20T21:39:17.906 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:17.907 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:39:17.907 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 73 2024-04-20T21:39:17.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f44e406c8f0 msgr2=0x7f44e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f44e406c8f0 0x7f44e406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0defd0 tx=0x7f44f0008040 comp rx=0 tx=0).stop 2024-04-20T21:39:17.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44fc037170 msgr2=0x7f44fc0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44fc037170 0x7f44fc0cec50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f44f4004000 tx=0x7f44f40046c0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 shutdown_connections 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44fc0d92b0 0x7f44fc0db580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f44e406c8f0 0x7f44e406edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44fc0378f0 0x7f44fc0db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 --2- 172.21.15.102:0/627684413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44fc037170 0x7f44fc0cec50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 >> 172.21.15.102:0/627684413 conn(0x7f44fc054600 msgr2=0x7f44fc0398d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 shutdown_connections 2024-04-20T21:39:17.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:17.910+0000 7f4508c00640 1 -- 172.21.15.102:0/627684413 wait complete. 2024-04-20T21:39:17.932 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-04-20T21:39:17.932 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:39:17.932 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:39:17.958 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:39:17.958 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.198+0000 7f2749400640 1 Processor -- start 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.198+0000 7f2749400640 1 -- start start 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2749400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f274415ee00 0x7f274406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2749400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27440c8f20 con 0x7f274415ea20 2024-04-20T21:39:18.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2749400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274406cb80 con 0x7f274415f2b0 2024-04-20T21:39:18.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2749400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f274406ce60 con 0x7f274415ee00 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2741e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f274415ee00 0x7f274406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2742800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f274415ea20 0x7f27440c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55444/0 (socket says 172.21.15.102:55444) 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2741e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f274415ee00 0x7f274406d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47598/0 (socket says 172.21.15.102:47598) 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2742800640 1 -- 172.21.15.102:0/2363284804 learned_addr learned my addr 172.21.15.102:0/2363284804 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.200+0000 7f2743200640 1 --1- 172.21.15.102:0/2363284804 >> v1:172.21.15.193:6789/0 conn(0x7f274415f2b0 0x7f2744154e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:42868/0 (socket says 172.21.15.102:42868) 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.201+0000 7f2741e00640 1 -- 172.21.15.102:0/2363284804 >> v1:172.21.15.193:6789/0 conn(0x7f274415f2b0 legacy=0x7f2744154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.201+0000 7f2741400640 1 -- 172.21.15.102:0/2363284804 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118709721 0 0) 0x7f274406cb80 con 0x7f274415f2b0 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.201+0000 7f2741e00640 1 -- 172.21.15.102:0/2363284804 >> v1:172.21.15.102:6789/0 conn(0x7f274415ea20 legacy=0x7f27440c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.201+0000 7f2741e00640 1 -- 172.21.15.102:0/2363284804 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274406d140 con 0x7f274415ee00 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.202+0000 7f2741e00640 1 --2- 172.21.15.102:0/2363284804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 0x7f274406d940 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27300097a0 tx=0x7f273002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ef88c3cc3b92cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.202+0000 7f2741400640 1 -- 172.21.15.102:0/2363284804 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730038070 con 0x7f274415ee00 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.202+0000 7f2741400640 1 -- 172.21.15.102:0/2363284804 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2730031700 con 0x7f274415ee00 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.202+0000 7f2741400640 1 -- 172.21.15.102:0/2363284804 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730031ad0 con 0x7f274415ee00 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.203+0000 7f2749400640 1 -- 172.21.15.102:0/2363284804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 msgr2=0x7f274406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.203+0000 7f2749400640 1 --2- 172.21.15.102:0/2363284804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 0x7f274406d940 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27300097a0 tx=0x7f273002b820 comp rx=0 tx=0).stop 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.203+0000 7f2749400640 1 -- 172.21.15.102:0/2363284804 shutdown_connections 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.203+0000 7f2749400640 1 --2- 172.21.15.102:0/2363284804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 0x7f274406d940 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.203+0000 7f2749400640 1 -- 172.21.15.102:0/2363284804 >> 172.21.15.102:0/2363284804 conn(0x7f27440c9390 msgr2=0x7f27440c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.203+0000 7f2749400640 1 -- 172.21.15.102:0/2363284804 shutdown_connections 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.203+0000 7f2749400640 1 -- 172.21.15.102:0/2363284804 wait complete. 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.205+0000 7f2749400640 1 Processor -- start 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.205+0000 7f2749400640 1 -- start start 2024-04-20T21:39:18.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2749400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f274415ea20 0x7f27440c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2749400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 0x7f2744149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2749400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f274415f2b0 0x7f2744147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2749400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f27440c8c80 con 0x7f274415ee00 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2749400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27440c8b00 con 0x7f274415ea20 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2749400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2744155590 con 0x7f274415f2b0 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2741e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 0x7f2744149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2742800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f274415ea20 0x7f27440c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2743200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f274415f2b0 0x7f2744147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2742800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f274415ea20 0x7f27440c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:42216/0 (socket says 172.21.15.102:42216) 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2742800640 1 -- 172.21.15.102:0/2596292128 learned_addr learned my addr 172.21.15.102:0/2596292128 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2742800640 1 -- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f274415f2b0 msgr2=0x7f2744147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2742800640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f274415f2b0 0x7f2744147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2742800640 1 -- 172.21.15.102:0/2596292128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 msgr2=0x7f2744149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2742800640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 0x7f2744149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.206+0000 7f2742800640 1 -- 172.21.15.102:0/2596292128 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27441f49b0 con 0x7f274415ea20 2024-04-20T21:39:18.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.207+0000 7f2742800640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f274415ea20 0x7f27440c65d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f273400c7b0 tx=0x7f273400cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.207+0000 7f273aa00640 1 -- 172.21.15.102:0/2596292128 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734018070 con 0x7f274415ea20 2024-04-20T21:39:18.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.207+0000 7f273aa00640 1 -- 172.21.15.102:0/2596292128 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27340072d0 con 0x7f274415ea20 2024-04-20T21:39:18.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.207+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27441f4c90 con 0x7f274415ea20 2024-04-20T21:39:18.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.207+0000 7f273aa00640 1 -- 172.21.15.102:0/2596292128 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340047e0 con 0x7f274415ea20 2024-04-20T21:39:18.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.207+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27440c41b0 con 0x7f274415ea20 2024-04-20T21:39:18.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.208+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2744074340 con 0x7f274415ea20 2024-04-20T21:39:18.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.209+0000 7f273aa00640 1 -- 172.21.15.102:0/2596292128 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f273401b070 con 0x7f274415ea20 2024-04-20T21:39:18.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.210+0000 7f273aa00640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f272406c8f0 0x7f272406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.210+0000 7f273aa00640 1 -- 172.21.15.102:0/2596292128 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f273408b740 con 0x7f274415ea20 2024-04-20T21:39:18.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.210+0000 7f2741e00640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f272406c8f0 0x7f272406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.211+0000 7f2741e00640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f272406c8f0 0x7f272406edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2730035000 tx=0x7f2730008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.215+0000 7f273aa00640 1 -- 172.21.15.102:0/2596292128 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2734058d50 con 0x7f274415ea20 2024-04-20T21:39:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.454+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27440c74b0 con 0x7f274415ea20 2024-04-20T21:39:18.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.455+0000 7f273aa00640 1 -- 172.21.15.102:0/2596292128 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+6859 (secure 0 0 0) 0x7f273405d050 con 0x7f274415ea20 2024-04-20T21:39:18.454 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:18.454 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:39:18.454 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 73 2024-04-20T21:39:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f272406c8f0 msgr2=0x7f272406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f272406c8f0 0x7f272406edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2730035000 tx=0x7f2730008030 comp rx=0 tx=0).stop 2024-04-20T21:39:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f274415ea20 msgr2=0x7f27440c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f274415ea20 0x7f27440c65d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f273400c7b0 tx=0x7f273400cc70 comp rx=0 tx=0).stop 2024-04-20T21:39:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 shutdown_connections 2024-04-20T21:39:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f274415f2b0 0x7f2744147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f274415ee00 0x7f2744149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f272406c8f0 0x7f272406edb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 --2- 172.21.15.102:0/2596292128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f274415ea20 0x7f27440c65d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 >> 172.21.15.102:0/2596292128 conn(0x7f27440c9390 msgr2=0x7f2744153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 shutdown_connections 2024-04-20T21:39:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:18.458+0000 7f2749400640 1 -- 172.21.15.102:0/2596292128 wait complete. 2024-04-20T21:39:18.476 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:39:20.477 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:20.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.706+0000 7fbb30e00640 1 Processor -- start 2024-04-20T21:39:20.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.706+0000 7fbb30e00640 1 -- start start 2024-04-20T21:39:20.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.707+0000 7fbb30e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 0x7fbb2c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.707+0000 7fbb30e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbb2c158b50 0x7fbb2c1628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.707+0000 7fbb30e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2c071f70 con 0x7fbb2c158770 2024-04-20T21:39:20.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.707+0000 7fbb30e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2c0c6dc0 con 0x7fbb2c158b50 2024-04-20T21:39:20.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.707+0000 7fbb30e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c072210 con 0x7fbb2c152170 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb2a800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbb2c158b50 0x7fbb2c1628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb29e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 0x7fbb2c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb29400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fbb2c158770 0x7fbb2c070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:42870/0 (socket says 172.21.15.102:42870) 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb29400640 1 -- 172.21.15.102:0/3392228389 learned_addr learned my addr 172.21.15.102:0/3392228389 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb28a00640 1 -- 172.21.15.102:0/3392228389 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508865805 0 0) 0x7fbb2c071f70 con 0x7fbb2c158770 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb28a00640 1 -- 172.21.15.102:0/3392228389 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb10003520 con 0x7fbb2c158770 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb2a800640 1 -- 172.21.15.102:0/3392228389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 msgr2=0x7fbb2c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb2a800640 1 --2- 172.21.15.102:0/3392228389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 0x7fbb2c0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb2a800640 1 -- 172.21.15.102:0/3392228389 >> v1:172.21.15.193:6789/0 conn(0x7fbb2c158770 legacy=0x7fbb2c070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.708+0000 7fbb2a800640 1 -- 172.21.15.102:0/3392228389 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c0724b0 con 0x7fbb2c158b50 2024-04-20T21:39:20.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.709+0000 7fbb2a800640 1 --2- 172.21.15.102:0/3392228389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 0x7fbb2c1628a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400bfa0 tx=0x7fbb2402e140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cac1c037c1615702 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb28a00640 1 -- 172.21.15.102:0/3392228389 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2402eae0 con 0x7fbb2c158b50 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb28a00640 1 -- 172.21.15.102:0/3392228389 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2402ec80 con 0x7fbb2c158b50 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb28a00640 1 -- 172.21.15.102:0/3392228389 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24009740 con 0x7fbb2c158b50 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb30e00640 1 -- 172.21.15.102:0/3392228389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 msgr2=0x7fbb2c1628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb30e00640 1 --2- 172.21.15.102:0/3392228389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 0x7fbb2c1628a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400bfa0 tx=0x7fbb2402e140 comp rx=0 tx=0).stop 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb30e00640 1 -- 172.21.15.102:0/3392228389 shutdown_connections 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb30e00640 1 --2- 172.21.15.102:0/3392228389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 0x7fbb2c1628a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb30e00640 1 --2- 172.21.15.102:0/3392228389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 0x7fbb2c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb30e00640 1 -- 172.21.15.102:0/3392228389 >> 172.21.15.102:0/3392228389 conn(0x7fbb2c0c9390 msgr2=0x7fbb2c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb30e00640 1 -- 172.21.15.102:0/3392228389 shutdown_connections 2024-04-20T21:39:20.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.710+0000 7fbb30e00640 1 -- 172.21.15.102:0/3392228389 wait complete. 2024-04-20T21:39:20.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.712+0000 7fbb30e00640 1 Processor -- start 2024-04-20T21:39:20.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.712+0000 7fbb30e00640 1 -- start start 2024-04-20T21:39:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.713+0000 7fbb30e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 0x7fbb2c0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.713+0000 7fbb30e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb2c158770 0x7fbb2c1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.713+0000 7fbb30e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 0x7fbb2c0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.713+0000 7fbb30e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c0c9f20 con 0x7fbb2c158770 2024-04-20T21:39:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.713+0000 7fbb30e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c158250 con 0x7fbb2c158b50 2024-04-20T21:39:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.713+0000 7fbb30e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c157f20 con 0x7fbb2c152170 2024-04-20T21:39:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb2c158770 0x7fbb2c1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb2c158770 0x7fbb2c1f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47610/0 (socket says 172.21.15.102:47610) 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 -- 172.21.15.102:0/539037726 learned_addr learned my addr 172.21.15.102:0/539037726 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 0x7fbb2c0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb2a800640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 0x7fbb2c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 -- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 msgr2=0x7fbb2c0c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 0x7fbb2c0c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 -- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 msgr2=0x7fbb2c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 0x7fbb2c0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 -- 172.21.15.102:0/539037726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c0cbaf0 con 0x7fbb2c158770 2024-04-20T21:39:20.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.714+0000 7fbb29400640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb2c158770 0x7fbb2c1f1fb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c460 tx=0x7fbb1800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.715+0000 7fbb22000640 1 -- 172.21.15.102:0/539037726 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18017070 con 0x7fbb2c158770 2024-04-20T21:39:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.715+0000 7fbb22000640 1 -- 172.21.15.102:0/539037726 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb18005ce0 con 0x7fbb2c158770 2024-04-20T21:39:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.715+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c159c20 con 0x7fbb2c158770 2024-04-20T21:39:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.715+0000 7fbb22000640 1 -- 172.21.15.102:0/539037726 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18007290 con 0x7fbb2c158770 2024-04-20T21:39:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.715+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c15a1e0 con 0x7fbb2c158770 2024-04-20T21:39:20.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.716+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb2c15a6d0 con 0x7fbb2c158770 2024-04-20T21:39:20.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.717+0000 7fbb22000640 1 -- 172.21.15.102:0/539037726 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb1801a070 con 0x7fbb2c158770 2024-04-20T21:39:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.718+0000 7fbb22000640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbb0c06c920 0x7fbb0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.718+0000 7fbb22000640 1 -- 172.21.15.102:0/539037726 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7fbb1808b5c0 con 0x7fbb2c158770 2024-04-20T21:39:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.718+0000 7fbb29e00640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbb0c06c920 0x7fbb0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.719+0000 7fbb29e00640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbb0c06c920 0x7fbb0c06ede0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009b40 tx=0x7fbb1c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.722+0000 7fbb22000640 1 -- 172.21.15.102:0/539037726 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbb18058bd0 con 0x7fbb2c158770 2024-04-20T21:39:20.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.962+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb2c074340 con 0x7fbb2c158770 2024-04-20T21:39:20.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.964+0000 7fbb22000640 1 -- 172.21.15.102:0/539037726 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+6859 (secure 0 0 0) 0x7fbb1805ced0 con 0x7fbb2c158770 2024-04-20T21:39:20.963 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:20.963 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:39:20.963 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 73 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbb0c06c920 msgr2=0x7fbb0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbb0c06c920 0x7fbb0c06ede0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009b40 tx=0x7fbb1c009190 comp rx=0 tx=0).stop 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb2c158770 msgr2=0x7fbb2c1f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb2c158770 0x7fbb2c1f1fb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c460 tx=0x7fbb1800c920 comp rx=0 tx=0).stop 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 shutdown_connections 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbb0c06c920 0x7fbb0c06ede0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb2c158b50 0x7fbb2c0c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb2c158770 0x7fbb2c1f1fb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 --2- 172.21.15.102:0/539037726 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb2c152170 0x7fbb2c0c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.966+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 >> 172.21.15.102:0/539037726 conn(0x7fbb2c0c9390 msgr2=0x7fbb2c1567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.967+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 shutdown_connections 2024-04-20T21:39:20.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:20.967+0000 7fbb30e00640 1 -- 172.21.15.102:0/539037726 wait complete. 2024-04-20T21:39:20.983 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:39:22.984 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:23.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.195+0000 7f3873e00640 1 Processor -- start 2024-04-20T21:39:23.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.195+0000 7f3873e00640 1 -- start start 2024-04-20T21:39:23.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.196+0000 7f3873e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3874152290 0x7f38740cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.196+0000 7f3873e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38740c3700 0x7f387406e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.196+0000 7f3873e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f387406eb70 0x7f38740c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.196+0000 7f3873e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f38740c9180 con 0x7f387406eb70 2024-04-20T21:39:23.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.196+0000 7f3873e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f38740c4840 con 0x7f3874152290 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.196+0000 7f3873e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f387415c220 con 0x7f38740c3700 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3873400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f387406eb70 0x7f38740c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3873400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f387406eb70 0x7f38740c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47626/0 (socket says 172.21.15.102:47626) 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3873400640 1 -- 172.21.15.102:0/2023487551 learned_addr learned my addr 172.21.15.102:0/2023487551 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3872a00640 1 --2- 172.21.15.102:0/2023487551 >> v2:172.21.15.193:3300/0 conn(0x7f3874152290 0x7f38740cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f386a000640 1 --2- 172.21.15.102:0/2023487551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38740c3700 0x7f387406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3873400640 1 -- 172.21.15.102:0/2023487551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38740c3700 msgr2=0x7f387406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3873400640 1 --2- 172.21.15.102:0/2023487551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38740c3700 0x7f387406e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3873400640 1 -- 172.21.15.102:0/2023487551 >> v2:172.21.15.193:3300/0 conn(0x7f3874152290 msgr2=0x7f38740cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3873400640 1 --2- 172.21.15.102:0/2023487551 >> v2:172.21.15.193:3300/0 conn(0x7f3874152290 0x7f38740cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.197+0000 7f3873400640 1 -- 172.21.15.102:0/2023487551 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38740c4560 con 0x7f387406eb70 2024-04-20T21:39:23.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.198+0000 7f3873400640 1 --2- 172.21.15.102:0/2023487551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 0x7f38740c8330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38740c3eb0 tx=0x7f386c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41cac8d7c99b377 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3872000640 1 -- 172.21.15.102:0/2023487551 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c02cb00 con 0x7f387406eb70 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3872000640 1 -- 172.21.15.102:0/2023487551 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386c02cca0 con 0x7f387406eb70 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3872000640 1 -- 172.21.15.102:0/2023487551 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c034610 con 0x7f387406eb70 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3873e00640 1 -- 172.21.15.102:0/2023487551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 msgr2=0x7f38740c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3873e00640 1 --2- 172.21.15.102:0/2023487551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 0x7f38740c8330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38740c3eb0 tx=0x7f386c02bf60 comp rx=0 tx=0).stop 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3873e00640 1 -- 172.21.15.102:0/2023487551 shutdown_connections 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3873e00640 1 --2- 172.21.15.102:0/2023487551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 0x7f38740c8330 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3873e00640 1 --2- 172.21.15.102:0/2023487551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38740c3700 0x7f387406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3873e00640 1 --2- 172.21.15.102:0/2023487551 >> v2:172.21.15.193:3300/0 conn(0x7f3874152290 0x7f38740cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.199+0000 7f3873e00640 1 -- 172.21.15.102:0/2023487551 >> 172.21.15.102:0/2023487551 conn(0x7f38740cc820 msgr2=0x7f38740ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.200+0000 7f3873e00640 1 -- 172.21.15.102:0/2023487551 shutdown_connections 2024-04-20T21:39:23.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.200+0000 7f3873e00640 1 -- 172.21.15.102:0/2023487551 wait complete. 2024-04-20T21:39:23.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.201+0000 7f3873e00640 1 Processor -- start 2024-04-20T21:39:23.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.201+0000 7f3873e00640 1 -- start start 2024-04-20T21:39:23.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.202+0000 7f3873e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 0x7f38740cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.202+0000 7f3873e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38740c3700 0x7f38740c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.202+0000 7f3873e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3874152290 0x7f3874147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.202+0000 7f3873e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f387415ba20 con 0x7f387406eb70 2024-04-20T21:39:23.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.202+0000 7f3873e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f387415b740 con 0x7f38740c3700 2024-04-20T21:39:23.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.202+0000 7f3873e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f387415bd00 con 0x7f3874152290 2024-04-20T21:39:23.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3872a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 0x7f38740cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3873400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3874152290 0x7f3874147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f386a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38740c3700 0x7f38740c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3872a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 0x7f38740cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47636/0 (socket says 172.21.15.102:47636) 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3873400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3874152290 0x7f3874147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:36966/0 (socket says 172.21.15.102:36966) 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3872a00640 1 -- 172.21.15.102:0/3429346485 learned_addr learned my addr 172.21.15.102:0/3429346485 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f386a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38740c3700 0x7f38740c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:42266/0 (socket says 172.21.15.102:42266) 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3873400640 1 -- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38740c3700 msgr2=0x7f38740c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3873400640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38740c3700 0x7f38740c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3873400640 1 -- 172.21.15.102:0/3429346485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 msgr2=0x7f38740cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3873400640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 0x7f38740cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f3873400640 1 -- 172.21.15.102:0/3429346485 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38741ed8a0 con 0x7f3874152290 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.203+0000 7f386a000640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38740c3700 0x7f38740c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.204+0000 7f3873400640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3874152290 0x7f3874147610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f386c032000 tx=0x7f386c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.204+0000 7f386b400640 1 -- 172.21.15.102:0/3429346485 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c043020 con 0x7f3874152290 2024-04-20T21:39:23.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.204+0000 7f386b400640 1 -- 172.21.15.102:0/3429346485 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386c02ce50 con 0x7f3874152290 2024-04-20T21:39:23.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.204+0000 7f386b400640 1 -- 172.21.15.102:0/3429346485 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c009510 con 0x7f3874152290 2024-04-20T21:39:23.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.204+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38740cbef0 con 0x7f3874152290 2024-04-20T21:39:23.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.204+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f387415b1b0 con 0x7f3874152290 2024-04-20T21:39:23.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.205+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387404fd60 con 0x7f3874152290 2024-04-20T21:39:23.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.206+0000 7f386b400640 1 -- 172.21.15.102:0/3429346485 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f386c046020 con 0x7f3874152290 2024-04-20T21:39:23.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.207+0000 7f386b400640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f385006c8f0 0x7f385006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.207+0000 7f386b400640 1 -- 172.21.15.102:0/3429346485 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f386c0ae820 con 0x7f3874152290 2024-04-20T21:39:23.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.207+0000 7f3872a00640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f385006c8f0 0x7f385006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.208+0000 7f3872a00640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f385006c8f0 0x7f385006edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3864009c00 tx=0x7f3864009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.212+0000 7f386b400640 1 -- 172.21.15.102:0/3429346485 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f386c04d030 con 0x7f3874152290 2024-04-20T21:39:23.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.451+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f387406dcb0 con 0x7f3874152290 2024-04-20T21:39:23.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.452+0000 7f386b400640 1 -- 172.21.15.102:0/3429346485 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+6859 (secure 0 0 0) 0x7f386c07be60 con 0x7f3874152290 2024-04-20T21:39:23.452 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:23.452 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:39:23.452 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 73 2024-04-20T21:39:23.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f385006c8f0 msgr2=0x7f385006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f385006c8f0 0x7f385006edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3864009c00 tx=0x7f3864009190 comp rx=0 tx=0).stop 2024-04-20T21:39:23.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3874152290 msgr2=0x7f3874147610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3874152290 0x7f3874147610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f386c032000 tx=0x7f386c004a40 comp rx=0 tx=0).stop 2024-04-20T21:39:23.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 shutdown_connections 2024-04-20T21:39:23.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3874152290 0x7f3874147610 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f385006c8f0 0x7f385006edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38740c3700 0x7f38740c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 --2- 172.21.15.102:0/3429346485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f387406eb70 0x7f38740cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 >> 172.21.15.102:0/3429346485 conn(0x7f38740cc820 msgr2=0x7f38740c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:23.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 shutdown_connections 2024-04-20T21:39:23.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:23.456+0000 7f3873e00640 1 -- 172.21.15.102:0/3429346485 wait complete. 2024-04-20T21:39:23.474 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:39:25.475 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:25.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.687+0000 7ff2a0200640 1 Processor -- start 2024-04-20T21:39:25.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.687+0000 7ff2a0200640 1 -- start start 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.688+0000 7ff2a0200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff29814fa50 0x7ff2980c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.688+0000 7ff2a0200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff298158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.688+0000 7ff2a0200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2981536d0 con 0x7ff2980c9eb0 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.688+0000 7ff2a0200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff298159500 con 0x7ff29814fa50 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.688+0000 7ff2a0200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff2980c76c0 con 0x7ff298073a10 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.689+0000 7ff29ce00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff2980c9eb0 0x7ff2980c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55450/0 (socket says 172.21.15.102:55450) 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.689+0000 7ff29d800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff29814fa50 0x7ff2980c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.689+0000 7ff29ce00640 1 -- 172.21.15.102:0/2555688990 learned_addr learned my addr 172.21.15.102:0/2555688990 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.689+0000 7ff29e200640 1 --2- 172.21.15.102:0/2555688990 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff298158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:25.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.689+0000 7ff29d800640 1 -- 172.21.15.102:0/2555688990 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 msgr2=0x7ff298158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:25.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.689+0000 7ff29d800640 1 --2- 172.21.15.102:0/2555688990 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff298158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.689+0000 7ff29d800640 1 -- 172.21.15.102:0/2555688990 >> v1:172.21.15.102:6789/0 conn(0x7ff2980c9eb0 legacy=0x7ff2980c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:25.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.689+0000 7ff29d800640 1 -- 172.21.15.102:0/2555688990 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2980c7960 con 0x7ff29814fa50 2024-04-20T21:39:25.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.690+0000 7ff29d800640 1 --2- 172.21.15.102:0/2555688990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29814fa50 0x7ff2980c4830 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff2880097a0 tx=0x7ff28802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a294f8ed3a89fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff297e00640 1 -- 172.21.15.102:0/2555688990 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288034070 con 0x7ff29814fa50 2024-04-20T21:39:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff297e00640 1 -- 172.21.15.102:0/2555688990 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff288030800 con 0x7ff29814fa50 2024-04-20T21:39:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff297e00640 1 -- 172.21.15.102:0/2555688990 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288030bd0 con 0x7ff29814fa50 2024-04-20T21:39:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff2a0200640 1 -- 172.21.15.102:0/2555688990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29814fa50 msgr2=0x7ff2980c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff2a0200640 1 --2- 172.21.15.102:0/2555688990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29814fa50 0x7ff2980c4830 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff2880097a0 tx=0x7ff28802b940 comp rx=0 tx=0).stop 2024-04-20T21:39:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff2a0200640 1 -- 172.21.15.102:0/2555688990 shutdown_connections 2024-04-20T21:39:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff2a0200640 1 --2- 172.21.15.102:0/2555688990 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff298158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff2a0200640 1 --2- 172.21.15.102:0/2555688990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29814fa50 0x7ff2980c4830 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff2a0200640 1 -- 172.21.15.102:0/2555688990 >> 172.21.15.102:0/2555688990 conn(0x7ff2980c9390 msgr2=0x7ff2980c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff2a0200640 1 -- 172.21.15.102:0/2555688990 shutdown_connections 2024-04-20T21:39:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.691+0000 7ff2a0200640 1 -- 172.21.15.102:0/2555688990 wait complete. 2024-04-20T21:39:25.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.693+0000 7ff2a0200640 1 Processor -- start 2024-04-20T21:39:25.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.694+0000 7ff2a0200640 1 -- start start 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff2a0200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff2981efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff2a0200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2980c9eb0 0x7ff2981556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff2a0200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff29814fa50 0x7ff298155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff2a0200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff2980c7c00 con 0x7ff29814fa50 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff2a0200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2980c3d90 con 0x7ff2980c9eb0 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff2a0200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff2980c4030 con 0x7ff298073a10 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29d800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff2981efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff29814fa50 0x7ff298155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:25.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2980c9eb0 0x7ff2981556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:25.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29d800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff2981efb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:36982/0 (socket says 172.21.15.102:36982) 2024-04-20T21:39:25.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29d800640 1 -- 172.21.15.102:0/3040557629 learned_addr learned my addr 172.21.15.102:0/3040557629 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29ce00640 1 -- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 msgr2=0x7ff2981efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29ce00640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff2981efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29ce00640 1 -- 172.21.15.102:0/3040557629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff29814fa50 msgr2=0x7ff298155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29ce00640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff29814fa50 0x7ff298155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.695+0000 7ff29ce00640 1 -- 172.21.15.102:0/3040557629 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2981f2710 con 0x7ff2980c9eb0 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.696+0000 7ff29ce00640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2980c9eb0 0x7ff2981556e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00ca50 tx=0x7ff28c00cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.696+0000 7ff295600640 1 -- 172.21.15.102:0/3040557629 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c017070 con 0x7ff2980c9eb0 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.696+0000 7ff295600640 1 -- 172.21.15.102:0/3040557629 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff28c00abb0 con 0x7ff2980c9eb0 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.696+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2981f29b0 con 0x7ff2980c9eb0 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.696+0000 7ff295600640 1 -- 172.21.15.102:0/3040557629 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c012400 con 0x7ff2980c9eb0 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.696+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2981f2f70 con 0x7ff2980c9eb0 2024-04-20T21:39:25.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.697+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2981f32a0 con 0x7ff2980c9eb0 2024-04-20T21:39:25.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.698+0000 7ff295600640 1 -- 172.21.15.102:0/3040557629 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff28c01a070 con 0x7ff2980c9eb0 2024-04-20T21:39:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.699+0000 7ff295600640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.699+0000 7ff295600640 1 -- 172.21.15.102:0/3040557629 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7ff28c08bfc0 con 0x7ff2980c9eb0 2024-04-20T21:39:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.699+0000 7ff29d800640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:25.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.700+0000 7ff29d800640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff27c06c8f0 0x7ff27c06edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff288009560 tx=0x7ff28803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:25.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.704+0000 7ff295600640 1 -- 172.21.15.102:0/3040557629 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff2981536d0 con 0x7ff2980c9eb0 2024-04-20T21:39:25.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.950+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff298074340 con 0x7ff2980c9eb0 2024-04-20T21:39:25.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.951+0000 7ff295600640 1 -- 172.21.15.102:0/3040557629 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+6859 (secure 0 0 0) 0x7ff28c0595c0 con 0x7ff2980c9eb0 2024-04-20T21:39:25.950 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:25.950 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:39:25.951 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 73 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.953+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff27c06c8f0 msgr2=0x7ff27c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.953+0000 7ff2a0200640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff27c06c8f0 0x7ff27c06edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff288009560 tx=0x7ff28803b040 comp rx=0 tx=0).stop 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.953+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2980c9eb0 msgr2=0x7ff2981556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.953+0000 7ff2a0200640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2980c9eb0 0x7ff2981556e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00ca50 tx=0x7ff28c00cf10 comp rx=0 tx=0).stop 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.954+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 shutdown_connections 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.954+0000 7ff2a0200640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff29814fa50 0x7ff298155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.954+0000 7ff2a0200640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.954+0000 7ff2a0200640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2980c9eb0 0x7ff2981556e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.954+0000 7ff2a0200640 1 --2- 172.21.15.102:0/3040557629 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff298073a10 0x7ff2981efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.954+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 >> 172.21.15.102:0/3040557629 conn(0x7ff2980c9390 msgr2=0x7ff29806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.954+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 shutdown_connections 2024-04-20T21:39:25.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:25.954+0000 7ff2a0200640 1 -- 172.21.15.102:0/3040557629 wait complete. 2024-04-20T21:39:25.971 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:39:27.972 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:28.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.178+0000 7fe31a000640 1 Processor -- start 2024-04-20T21:39:28.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.179+0000 7fe31a000640 1 -- start start 2024-04-20T21:39:28.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe31a000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe3140cb4c0 0x7fe3140cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe31a000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe31415b750 0x7fe31415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe31a000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31415b250 con 0x7fe3141555e0 2024-04-20T21:39:28.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe31a000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe3140cc840 con 0x7fe3140cb4c0 2024-04-20T21:39:28.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe31a000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fe314073a90 con 0x7fe31415b750 2024-04-20T21:39:28.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe313400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe3141555e0 0x7fe3140cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57872/0 (socket says 172.21.15.102:57872) 2024-04-20T21:39:28.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe313400640 1 -- 172.21.15.102:0/4277243699 learned_addr learned my addr 172.21.15.102:0/4277243699 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:28.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe312a00640 1 --2- 172.21.15.102:0/4277243699 >> v2:172.21.15.102:3300/0 conn(0x7fe3140cb4c0 0x7fe3140cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.180+0000 7fe313e00640 1 --2- 172.21.15.102:0/4277243699 >> v2:172.21.15.193:3300/0 conn(0x7fe31415b750 0x7fe31415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.181+0000 7fe312000640 1 -- 172.21.15.102:0/4277243699 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991471334 0 0) 0x7fe31415b250 con 0x7fe3141555e0 2024-04-20T21:39:28.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.181+0000 7fe312000640 1 -- 172.21.15.102:0/4277243699 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2f4003520 con 0x7fe3141555e0 2024-04-20T21:39:28.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.181+0000 7fe313e00640 1 -- 172.21.15.102:0/4277243699 >> v2:172.21.15.102:3300/0 conn(0x7fe3140cb4c0 msgr2=0x7fe3140cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.181+0000 7fe313e00640 1 --2- 172.21.15.102:0/4277243699 >> v2:172.21.15.102:3300/0 conn(0x7fe3140cb4c0 0x7fe3140cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.181+0000 7fe313e00640 1 -- 172.21.15.102:0/4277243699 >> v1:172.21.15.102:6789/0 conn(0x7fe3141555e0 legacy=0x7fe3140cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.181+0000 7fe313e00640 1 -- 172.21.15.102:0/4277243699 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31415ea90 con 0x7fe31415b750 2024-04-20T21:39:28.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.181+0000 7fe312a00640 1 --2- 172.21.15.102:0/4277243699 >> v2:172.21.15.102:3300/0 conn(0x7fe3140cb4c0 0x7fe3140cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:28.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.182+0000 7fe313e00640 1 --2- 172.21.15.102:0/4277243699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31415b750 0x7fe31415bb60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe3140b3b80 tx=0x7fe30c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=987ef9330e3b89b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.182+0000 7fe312000640 1 -- 172.21.15.102:0/4277243699 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c039070 con 0x7fe31415b750 2024-04-20T21:39:28.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe312000640 1 -- 172.21.15.102:0/4277243699 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30c02cd10 con 0x7fe31415b750 2024-04-20T21:39:28.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe312000640 1 -- 172.21.15.102:0/4277243699 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c034640 con 0x7fe31415b750 2024-04-20T21:39:28.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe31a000640 1 -- 172.21.15.102:0/4277243699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31415b750 msgr2=0x7fe31415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe31a000640 1 --2- 172.21.15.102:0/4277243699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31415b750 0x7fe31415bb60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe3140b3b80 tx=0x7fe30c02bf60 comp rx=0 tx=0).stop 2024-04-20T21:39:28.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe31a000640 1 -- 172.21.15.102:0/4277243699 shutdown_connections 2024-04-20T21:39:28.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe31a000640 1 --2- 172.21.15.102:0/4277243699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31415b750 0x7fe31415bb60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe31a000640 1 --2- 172.21.15.102:0/4277243699 >> v2:172.21.15.102:3300/0 conn(0x7fe3140cb4c0 0x7fe3140cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe31a000640 1 -- 172.21.15.102:0/4277243699 >> 172.21.15.102:0/4277243699 conn(0x7fe3140754d0 msgr2=0x7fe314072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe31a000640 1 -- 172.21.15.102:0/4277243699 shutdown_connections 2024-04-20T21:39:28.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.183+0000 7fe31a000640 1 -- 172.21.15.102:0/4277243699 wait complete. 2024-04-20T21:39:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.185+0000 7fe31a000640 1 Processor -- start 2024-04-20T21:39:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.185+0000 7fe31a000640 1 -- start start 2024-04-20T21:39:28.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe31a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3140cb4c0 0x7fe3140c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe31a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141555e0 0x7fe314149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe31a000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe31415b750 0x7fe3140c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe31a000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141559c0 con 0x7fe3140cb4c0 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe31a000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe31415e840 con 0x7fe3141555e0 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe31a000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe3140cb910 con 0x7fe31415b750 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe313400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3140cb4c0 0x7fe3140c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe313400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3140cb4c0 0x7fe3140c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39528/0 (socket says 172.21.15.102:39528) 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe313400640 1 -- 172.21.15.102:0/2673033341 learned_addr learned my addr 172.21.15.102:0/2673033341 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe313e00640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe31415b750 0x7fe3140c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.186+0000 7fe312a00640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141555e0 0x7fe314149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe313400640 1 -- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe31415b750 msgr2=0x7fe3140c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe313400640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe31415b750 0x7fe3140c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe313400640 1 -- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141555e0 msgr2=0x7fe314149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe313400640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141555e0 0x7fe314149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe313400640 1 -- 172.21.15.102:0/2673033341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3140ca570 con 0x7fe3140cb4c0 2024-04-20T21:39:28.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe312a00640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141555e0 0x7fe314149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe313e00640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe31415b750 0x7fe3140c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe313400640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3140cb4c0 0x7fe3140c8780 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c460 tx=0x7fe30400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe30b400640 1 -- 172.21.15.102:0/2673033341 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304017070 con 0x7fe3140cb4c0 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe30b400640 1 -- 172.21.15.102:0/2673033341 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe304005ce0 con 0x7fe3140cb4c0 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3140ca810 con 0x7fe3140cb4c0 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe30b400640 1 -- 172.21.15.102:0/2673033341 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304007290 con 0x7fe3140cb4c0 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.187+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3140c7f80 con 0x7fe3140cb4c0 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.188+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe314068c30 con 0x7fe3140cb4c0 2024-04-20T21:39:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.189+0000 7fe30b400640 1 -- 172.21.15.102:0/2673033341 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe30401a070 con 0x7fe3140cb4c0 2024-04-20T21:39:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.190+0000 7fe30b400640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.190+0000 7fe30b400640 1 -- 172.21.15.102:0/2673033341 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7fe30408b6c0 con 0x7fe3140cb4c0 2024-04-20T21:39:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.191+0000 7fe312a00640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.191+0000 7fe312a00640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2f006c8f0 0x7fe2f006edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe3140c6c70 tx=0x7fe2fc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.196+0000 7fe30b400640 1 -- 172.21.15.102:0/2673033341 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe304059320 con 0x7fe3140cb4c0 2024-04-20T21:39:28.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.436+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3140d0860 con 0x7fe3140cb4c0 2024-04-20T21:39:28.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.438+0000 7fe30b400640 1 -- 172.21.15.102:0/2673033341 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+6859 (secure 0 0 0) 0x7fe304058c20 con 0x7fe3140cb4c0 2024-04-20T21:39:28.439 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:28.439 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:38:52.125321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6421":{"gid":6421,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":39,"addr":"172.21.15.102:6833/327932375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":327932375},{"type":"v1","addr":"172.21.15.102:6833","nonce":327932375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6421,"qdb_cluster":[6421]},"id":1}]} 2024-04-20T21:39:28.439 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 73 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2f006c8f0 msgr2=0x7fe2f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2f006c8f0 0x7fe2f006edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe3140c6c70 tx=0x7fe2fc008040 comp rx=0 tx=0).stop 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3140cb4c0 msgr2=0x7fe3140c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3140cb4c0 0x7fe3140c8780 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c460 tx=0x7fe30400c920 comp rx=0 tx=0).stop 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 shutdown_connections 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe31415b750 0x7fe3140c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141555e0 0x7fe314149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 --2- 172.21.15.102:0/2673033341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3140cb4c0 0x7fe3140c8780 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 >> 172.21.15.102:0/2673033341 conn(0x7fe3140754d0 msgr2=0x7fe3140c6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 shutdown_connections 2024-04-20T21:39:28.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:28.440+0000 7fe31a000640 1 -- 172.21.15.102:0/2673033341 wait complete. 2024-04-20T21:39:28.458 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T21:39:30.214 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:39:30.222 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:39:30.459 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:30.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.666+0000 7f3a15000640 1 Processor -- start 2024-04-20T21:39:30.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.666+0000 7f3a15000640 1 -- start start 2024-04-20T21:39:30.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a15000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3a10155610 0x7f3a1015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:30.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a15000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3a1015bd60 0x7f3a1015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:30.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a15000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a1015f6f0 con 0x7f3a10073950 2024-04-20T21:39:30.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a15000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3a1015b8d0 con 0x7f3a10155610 2024-04-20T21:39:30.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a15000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f3a1015c180 con 0x7f3a1015bd60 2024-04-20T21:39:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a0d400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3a1015bd60 0x7f3a1015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a0de00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3a10155610 0x7f3a1015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a0d400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3a1015bd60 0x7f3a1015f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54330/0 (socket says 172.21.15.102:54330) 2024-04-20T21:39:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a0de00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3a10155610 0x7f3a1015c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39544/0 (socket says 172.21.15.102:39544) 2024-04-20T21:39:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a0d400640 1 -- 172.21.15.102:0/4017438231 learned_addr learned my addr 172.21.15.102:0/4017438231 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:30.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.668+0000 7f3a0e800640 1 --1- 172.21.15.102:0/4017438231 >> v1:172.21.15.102:6789/0 conn(0x7f3a10073950 0x7f3a100ceb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57886/0 (socket says 172.21.15.102:57886) 2024-04-20T21:39:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.669+0000 7f3a0d400640 1 -- 172.21.15.102:0/4017438231 >> v2:172.21.15.102:3300/0 conn(0x7f3a10155610 msgr2=0x7f3a1015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.669+0000 7f3a0d400640 1 --2- 172.21.15.102:0/4017438231 >> v2:172.21.15.102:3300/0 conn(0x7f3a10155610 0x7f3a1015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.669+0000 7f3a0d400640 1 -- 172.21.15.102:0/4017438231 >> v1:172.21.15.102:6789/0 conn(0x7f3a10073950 legacy=0x7f3a100ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.669+0000 7f3a0d400640 1 -- 172.21.15.102:0/4017438231 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1015b2c0 con 0x7f3a1015bd60 2024-04-20T21:39:30.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.670+0000 7f3a0d400640 1 --2- 172.21.15.102:0/4017438231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a1015bd60 0x7f3a1015f1b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3a1006fc20 tx=0x7f39f802c010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59564dfcfcdc645e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.670+0000 7f3a0ca00640 1 -- 172.21.15.102:0/4017438231 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f802cbb0 con 0x7f3a1015bd60 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.670+0000 7f3a0ca00640 1 -- 172.21.15.102:0/4017438231 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f802cd50 con 0x7f3a1015bd60 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.670+0000 7f3a0ca00640 1 -- 172.21.15.102:0/4017438231 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8034610 con 0x7f3a1015bd60 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.671+0000 7f3a15000640 1 -- 172.21.15.102:0/4017438231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a1015bd60 msgr2=0x7f3a1015f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.671+0000 7f3a15000640 1 --2- 172.21.15.102:0/4017438231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a1015bd60 0x7f3a1015f1b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3a1006fc20 tx=0x7f39f802c010 comp rx=0 tx=0).stop 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.671+0000 7f3a15000640 1 -- 172.21.15.102:0/4017438231 shutdown_connections 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.671+0000 7f3a15000640 1 --2- 172.21.15.102:0/4017438231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a1015bd60 0x7f3a1015f1b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.671+0000 7f3a15000640 1 --2- 172.21.15.102:0/4017438231 >> v2:172.21.15.102:3300/0 conn(0x7f3a10155610 0x7f3a1015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.671+0000 7f3a15000640 1 -- 172.21.15.102:0/4017438231 >> 172.21.15.102:0/4017438231 conn(0x7f3a1006ebf0 msgr2=0x7f3a1006efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:30.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.671+0000 7f3a15000640 1 -- 172.21.15.102:0/4017438231 shutdown_connections 2024-04-20T21:39:30.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.671+0000 7f3a15000640 1 -- 172.21.15.102:0/4017438231 wait complete. 2024-04-20T21:39:30.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.673+0000 7f3a15000640 1 Processor -- start 2024-04-20T21:39:30.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.673+0000 7f3a15000640 1 -- start start 2024-04-20T21:39:30.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.674+0000 7f3a15000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a10073950 0x7f3a100c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:30.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.674+0000 7f3a15000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a10155610 0x7f3a100cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:30.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.674+0000 7f3a15000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a1015bd60 0x7f3a101f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:30.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.674+0000 7f3a15000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1015b560 con 0x7f3a10073950 2024-04-20T21:39:30.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.674+0000 7f3a15000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a100c9d50 con 0x7f3a10155610 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.674+0000 7f3a15000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3a100ca030 con 0x7f3a1015bd60 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a10073950 0x7f3a100c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a10155610 0x7f3a100cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a10073950 0x7f3a100c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39546/0 (socket says 172.21.15.102:39546) 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0de00640 1 -- 172.21.15.102:0/3578960513 learned_addr learned my addr 172.21.15.102:0/3578960513 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0e800640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a1015bd60 0x7f3a101f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0d400640 1 -- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a1015bd60 msgr2=0x7f3a101f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0d400640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a1015bd60 0x7f3a101f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0d400640 1 -- 172.21.15.102:0/3578960513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a10073950 msgr2=0x7f3a100c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0d400640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a10073950 0x7f3a100c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.675+0000 7f3a0d400640 1 -- 172.21.15.102:0/3578960513 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a100c7e00 con 0x7f3a10155610 2024-04-20T21:39:30.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.676+0000 7f3a0d400640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a10155610 0x7f3a100cd450 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3a1015ba50 tx=0x7f39f8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:30.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.676+0000 7f3a06000640 1 -- 172.21.15.102:0/3578960513 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8043020 con 0x7f3a10155610 2024-04-20T21:39:30.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.676+0000 7f3a06000640 1 -- 172.21.15.102:0/3578960513 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f803e450 con 0x7f3a10155610 2024-04-20T21:39:30.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.676+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a100c6690 con 0x7f3a10155610 2024-04-20T21:39:30.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.676+0000 7f3a06000640 1 -- 172.21.15.102:0/3578960513 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8009510 con 0x7f3a10155610 2024-04-20T21:39:30.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.676+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a100c8630 con 0x7f3a10155610 2024-04-20T21:39:30.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.677+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a10074340 con 0x7f3a10155610 2024-04-20T21:39:30.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.679+0000 7f3a06000640 1 -- 172.21.15.102:0/3578960513 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39f8046020 con 0x7f3a10155610 2024-04-20T21:39:30.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.680+0000 7f3a06000640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:30.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.680+0000 7f3a06000640 1 -- 172.21.15.102:0/3578960513 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7681+0+0 (secure 0 0 0) 0x7f39f80ae8c0 con 0x7f3a10155610 2024-04-20T21:39:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.680+0000 7f3a0de00640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:30.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.681+0000 7f3a0de00640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f39ec06c8f0 0x7f39ec06edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009c00 tx=0x7f3a00009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:30.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.684+0000 7f3a06000640 1 -- 172.21.15.102:0/3578960513 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f39f804d030 con 0x7f3a10155610 2024-04-20T21:39:30.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.933+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a100c9740 con 0x7f3a10155610 2024-04-20T21:39:30.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.934+0000 7f3a06000640 1 -- 172.21.15.102:0/3578960513 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+6026 (secure 0 0 0) 0x7f39f807bf70 con 0x7f3a10155610 2024-04-20T21:39:30.933 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:30.933 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:30.821737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:30.934 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 74 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f39ec06c8f0 msgr2=0x7f39ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f39ec06c8f0 0x7f39ec06edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009c00 tx=0x7f3a00009190 comp rx=0 tx=0).stop 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a10155610 msgr2=0x7f3a100cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a10155610 0x7f3a100cd450 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3a1015ba50 tx=0x7f39f8004a40 comp rx=0 tx=0).stop 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 shutdown_connections 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a1015bd60 0x7f3a101f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a10155610 0x7f3a100cd450 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 --2- 172.21.15.102:0/3578960513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a10073950 0x7f3a100c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 >> 172.21.15.102:0/3578960513 conn(0x7f3a1006ebf0 msgr2=0x7f3a100cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 shutdown_connections 2024-04-20T21:39:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:30.937+0000 7f3a15000640 1 -- 172.21.15.102:0/3578960513 wait complete. 2024-04-20T21:39:30.954 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-04-20T21:39:30.954 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:39:30.954 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:31.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.152+0000 7f1357200640 1 Processor -- start 2024-04-20T21:39:31.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.152+0000 7f1357200640 1 -- start start 2024-04-20T21:39:31.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f1357200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1350162e40 0x7f135006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:31.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f1357200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1350163220 0x7f1350159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:31.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f1357200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f135006d2e0 con 0x7f1350163660 2024-04-20T21:39:31.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f1357200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f13500c7290 con 0x7f1350162e40 2024-04-20T21:39:31.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f1357200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f13500c7570 con 0x7f1350163220 2024-04-20T21:39:31.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f134fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1350162e40 0x7f135006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:31.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f1355200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1350163660 0x7f13500c7a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57902/0 (socket says 172.21.15.102:57902) 2024-04-20T21:39:31.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f134fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1350162e40 0x7f135006c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39548/0 (socket says 172.21.15.102:39548) 2024-04-20T21:39:31.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f1355200640 1 -- 172.21.15.102:0/835585251 learned_addr learned my addr 172.21.15.102:0/835585251 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:31.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.154+0000 7f134f400640 1 --2- 172.21.15.102:0/835585251 >> v2:172.21.15.193:3300/0 conn(0x7f1350163220 0x7f1350159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:31.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.155+0000 7f134fe00640 1 -- 172.21.15.102:0/835585251 >> v2:172.21.15.193:3300/0 conn(0x7f1350163220 msgr2=0x7f1350159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:31.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.155+0000 7f134fe00640 1 --2- 172.21.15.102:0/835585251 >> v2:172.21.15.193:3300/0 conn(0x7f1350163220 0x7f1350159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.155+0000 7f134fe00640 1 -- 172.21.15.102:0/835585251 >> v1:172.21.15.102:6789/0 conn(0x7f1350163660 legacy=0x7f13500c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:31.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.155+0000 7f134ea00640 1 -- 172.21.15.102:0/835585251 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037832138 0 0) 0x7f135006d2e0 con 0x7f1350163660 2024-04-20T21:39:31.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.155+0000 7f134fe00640 1 -- 172.21.15.102:0/835585251 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135006d5c0 con 0x7f1350162e40 2024-04-20T21:39:31.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.156+0000 7f134fe00640 1 --2- 172.21.15.102:0/835585251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350162e40 0x7f135006c9e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1350070780 tx=0x7f134002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961a7691ee5561b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:31.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.156+0000 7f134ea00640 1 -- 172.21.15.102:0/835585251 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340034070 con 0x7f1350162e40 2024-04-20T21:39:31.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.156+0000 7f134ea00640 1 -- 172.21.15.102:0/835585251 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1340030610 con 0x7f1350162e40 2024-04-20T21:39:31.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.156+0000 7f134ea00640 1 -- 172.21.15.102:0/835585251 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400309e0 con 0x7f1350162e40 2024-04-20T21:39:31.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.157+0000 7f1357200640 1 -- 172.21.15.102:0/835585251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350162e40 msgr2=0x7f135006c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:31.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.157+0000 7f1357200640 1 --2- 172.21.15.102:0/835585251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350162e40 0x7f135006c9e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1350070780 tx=0x7f134002b960 comp rx=0 tx=0).stop 2024-04-20T21:39:31.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.157+0000 7f1357200640 1 -- 172.21.15.102:0/835585251 shutdown_connections 2024-04-20T21:39:31.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.157+0000 7f1357200640 1 --2- 172.21.15.102:0/835585251 >> v2:172.21.15.193:3300/0 conn(0x7f1350163220 0x7f1350159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.157+0000 7f1357200640 1 --2- 172.21.15.102:0/835585251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350162e40 0x7f135006c9e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.157+0000 7f1357200640 1 -- 172.21.15.102:0/835585251 >> 172.21.15.102:0/835585251 conn(0x7f13500c9390 msgr2=0x7f13500c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:31.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.157+0000 7f1357200640 1 -- 172.21.15.102:0/835585251 shutdown_connections 2024-04-20T21:39:31.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.157+0000 7f1357200640 1 -- 172.21.15.102:0/835585251 wait complete. 2024-04-20T21:39:31.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.159+0000 7f1357200640 1 Processor -- start 2024-04-20T21:39:31.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.159+0000 7f1357200640 1 -- start start 2024-04-20T21:39:31.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f1357200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350162e40 0x7f13500c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:31.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f1357200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1350163220 0x7f13501f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f1357200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350163660 0x7f13501f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f1357200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13500c9080 con 0x7f1350163660 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f1357200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f135006db80 con 0x7f1350162e40 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f1357200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f135006cf20 con 0x7f1350163220 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f1355200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350163660 0x7f13501f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f134fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350162e40 0x7f13500c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f134f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1350163220 0x7f13501f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f134fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350162e40 0x7f13500c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54360/0 (socket says 172.21.15.102:54360) 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f134fe00640 1 -- 172.21.15.102:0/2249848586 learned_addr learned my addr 172.21.15.102:0/2249848586 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f1355200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350163660 0x7f13501f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39562/0 (socket says 172.21.15.102:39562) 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f134f400640 1 -- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350162e40 msgr2=0x7f13500c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f134f400640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350162e40 0x7f13500c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f134f400640 1 -- 172.21.15.102:0/2249848586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350163660 msgr2=0x7f13501f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:31.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.160+0000 7f134f400640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350163660 0x7f13501f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.161+0000 7f134f400640 1 -- 172.21.15.102:0/2249848586 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13501f65a0 con 0x7f1350163220 2024-04-20T21:39:31.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.161+0000 7f1355200640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350163660 0x7f13501f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:31.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.161+0000 7f134fe00640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350162e40 0x7f13500c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:31.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.161+0000 7f134f400640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1350163220 0x7f13501f8b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f133c00e880 tx=0x7f133c00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:31.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.161+0000 7f1347e00640 1 -- 172.21.15.102:0/2249848586 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c017070 con 0x7f1350163220 2024-04-20T21:39:31.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.161+0000 7f1347e00640 1 -- 172.21.15.102:0/2249848586 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c00c940 con 0x7f1350163220 2024-04-20T21:39:31.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.161+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135015d4f0 con 0x7f1350163220 2024-04-20T21:39:31.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.161+0000 7f1347e00640 1 -- 172.21.15.102:0/2249848586 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c00cd30 con 0x7f1350163220 2024-04-20T21:39:31.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.162+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13500c64d0 con 0x7f1350163220 2024-04-20T21:39:31.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.163+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1350074340 con 0x7f1350163220 2024-04-20T21:39:31.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.164+0000 7f1347e00640 1 -- 172.21.15.102:0/2249848586 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f133c01a070 con 0x7f1350163220 2024-04-20T21:39:31.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.165+0000 7f1347e00640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133006c8f0 0x7f133006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:31.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.165+0000 7f1347e00640 1 -- 172.21.15.102:0/2249848586 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f133c08c2e0 con 0x7f1350163220 2024-04-20T21:39:31.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.165+0000 7f134fe00640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133006c8f0 0x7f133006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:31.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.166+0000 7f134fe00640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133006c8f0 0x7f133006edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f134002b9a0 tx=0x7f134003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:31.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.170+0000 7f1347e00640 1 -- 172.21.15.102:0/2249848586 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f133c059810 con 0x7f1350163220 2024-04-20T21:39:31.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.395+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f135006f2c0 con 0x7f1350163220 2024-04-20T21:39:31.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.396+0000 7f1347e00640 1 -- 172.21.15.102:0/2249848586 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+6026 (secure 0 0 0) 0x7f133c05db10 con 0x7f1350163220 2024-04-20T21:39:31.395 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:31.395 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:30.821737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:31.395 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 74 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133006c8f0 msgr2=0x7f133006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133006c8f0 0x7f133006edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f134002b9a0 tx=0x7f134003b040 comp rx=0 tx=0).stop 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1350163220 msgr2=0x7f13501f8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1350163220 0x7f13501f8b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f133c00e880 tx=0x7f133c00ed40 comp rx=0 tx=0).stop 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 shutdown_connections 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1350163660 0x7f13501f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1350163220 0x7f13501f8b70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133006c8f0 0x7f133006edb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 --2- 172.21.15.102:0/2249848586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350162e40 0x7f13500c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.398+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 >> 172.21.15.102:0/2249848586 conn(0x7f13500c9390 msgr2=0x7f1350157b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:31.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.399+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 shutdown_connections 2024-04-20T21:39:31.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:31.399+0000 7f1357200640 1 -- 172.21.15.102:0/2249848586 wait complete. 2024-04-20T21:39:31.416 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 74, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 74, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6429": { "addr": "172.21.15.102:6835/3586768191", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 3586768191, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 3586768191, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6429, "incarnation": 74, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:39:30.821737+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6429 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:39:33.417 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:33.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.629+0000 7f3e87200640 1 Processor -- start 2024-04-20T21:39:33.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.630+0000 7f3e87200640 1 -- start start 2024-04-20T21:39:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e87200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e800cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e87200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3e8015f0e0 0x7f3e8015b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e87200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e800cb400 con 0x7f3e800cc940 2024-04-20T21:39:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e87200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3e800cc6c0 con 0x7f3e8015f0e0 2024-04-20T21:39:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e87200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3e8015f620 con 0x7f3e80155510 2024-04-20T21:39:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e7fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e800cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e7f400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f3e800cc940 0x7f3e80072ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38932/0 (socket says 172.21.15.102:38932) 2024-04-20T21:39:33.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e7f400640 1 -- 172.21.15.102:0/512030516 learned_addr learned my addr 172.21.15.102:0/512030516 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.631+0000 7f3e85200640 1 --2- 172.21.15.102:0/512030516 >> v2:172.21.15.102:3300/0 conn(0x7f3e8015f0e0 0x7f3e8015b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.632+0000 7f3e7ea00640 1 -- 172.21.15.102:0/512030516 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 713207054 0 0) 0x7f3e800cb400 con 0x7f3e800cc940 2024-04-20T21:39:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.632+0000 7f3e7fe00640 1 -- 172.21.15.102:0/512030516 >> v2:172.21.15.102:3300/0 conn(0x7f3e8015f0e0 msgr2=0x7f3e8015b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.632+0000 7f3e7fe00640 1 --2- 172.21.15.102:0/512030516 >> v2:172.21.15.102:3300/0 conn(0x7f3e8015f0e0 0x7f3e8015b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.632+0000 7f3e7fe00640 1 -- 172.21.15.102:0/512030516 >> v1:172.21.15.193:6789/0 conn(0x7f3e800cc940 legacy=0x7f3e80072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:33.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.632+0000 7f3e7fe00640 1 -- 172.21.15.102:0/512030516 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e800cb6e0 con 0x7f3e80155510 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.633+0000 7f3e7fe00640 1 --2- 172.21.15.102:0/512030516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e800cc180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e700097a0 tx=0x7f3e7002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e49edfe76295df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.633+0000 7f3e7ea00640 1 -- 172.21.15.102:0/512030516 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70035020 con 0x7f3e80155510 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.633+0000 7f3e7ea00640 1 -- 172.21.15.102:0/512030516 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e70031810 con 0x7f3e80155510 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.633+0000 7f3e7ea00640 1 -- 172.21.15.102:0/512030516 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70031be0 con 0x7f3e80155510 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.634+0000 7f3e87200640 1 -- 172.21.15.102:0/512030516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 msgr2=0x7f3e800cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.634+0000 7f3e87200640 1 --2- 172.21.15.102:0/512030516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e800cc180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e700097a0 tx=0x7f3e7002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.634+0000 7f3e87200640 1 -- 172.21.15.102:0/512030516 shutdown_connections 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.634+0000 7f3e87200640 1 --2- 172.21.15.102:0/512030516 >> v2:172.21.15.102:3300/0 conn(0x7f3e8015f0e0 0x7f3e8015b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.634+0000 7f3e87200640 1 --2- 172.21.15.102:0/512030516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e800cc180 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.634+0000 7f3e87200640 1 -- 172.21.15.102:0/512030516 >> 172.21.15.102:0/512030516 conn(0x7f3e800683e0 msgr2=0x7f3e8006ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:33.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.634+0000 7f3e87200640 1 -- 172.21.15.102:0/512030516 shutdown_connections 2024-04-20T21:39:33.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.634+0000 7f3e87200640 1 -- 172.21.15.102:0/512030516 wait complete. 2024-04-20T21:39:33.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.636+0000 7f3e87200640 1 Processor -- start 2024-04-20T21:39:33.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.636+0000 7f3e87200640 1 -- start start 2024-04-20T21:39:33.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e87200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e800cc940 0x7f3e80147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:33.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e87200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e801f0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:33.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e87200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e8015f0e0 0x7f3e801f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:33.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e87200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3e800732b0 con 0x7f3e8015f0e0 2024-04-20T21:39:33.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e87200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e800ce0b0 con 0x7f3e800cc940 2024-04-20T21:39:33.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e87200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3e80073430 con 0x7f3e80155510 2024-04-20T21:39:33.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e85200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e8015f0e0 0x7f3e801f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:33.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e85200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e8015f0e0 0x7f3e801f1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39584/0 (socket says 172.21.15.102:39584) 2024-04-20T21:39:33.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e85200640 1 -- 172.21.15.102:0/2706395903 learned_addr learned my addr 172.21.15.102:0/2706395903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:33.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e7fe00640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e800cc940 0x7f3e80147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:33.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.637+0000 7f3e7f400640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e801f0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:33.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e7fe00640 1 -- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 msgr2=0x7f3e801f0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:33.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e7fe00640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e801f0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e7fe00640 1 -- 172.21.15.102:0/2706395903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e8015f0e0 msgr2=0x7f3e801f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:33.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e7fe00640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e8015f0e0 0x7f3e801f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e7fe00640 1 -- 172.21.15.102:0/2706395903 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e80159fe0 con 0x7f3e800cc940 2024-04-20T21:39:33.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e85200640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e8015f0e0 0x7f3e801f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:33.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e7fe00640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e800cc940 0x7f3e80147580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e8015f7a0 tx=0x7f3e700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:33.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e77e00640 1 -- 172.21.15.102:0/2706395903 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70035050 con 0x7f3e800cc940 2024-04-20T21:39:33.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e77e00640 1 -- 172.21.15.102:0/2706395903 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7003bcc0 con 0x7f3e800cc940 2024-04-20T21:39:33.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e77e00640 1 -- 172.21.15.102:0/2706395903 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70006e10 con 0x7f3e800cc940 2024-04-20T21:39:33.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.638+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8015a2b0 con 0x7f3e800cc940 2024-04-20T21:39:33.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.639+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e801f4520 con 0x7f3e800cc940 2024-04-20T21:39:33.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.640+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e8004fd60 con 0x7f3e800cc940 2024-04-20T21:39:33.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.640+0000 7f3e77e00640 1 -- 172.21.15.102:0/2706395903 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e70038020 con 0x7f3e800cc940 2024-04-20T21:39:33.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.641+0000 7f3e77e00640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e6006c920 0x7f3e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:33.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.641+0000 7f3e77e00640 1 -- 172.21.15.102:0/2706395903 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f3e700ae060 con 0x7f3e800cc940 2024-04-20T21:39:33.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.642+0000 7f3e7f400640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e6006c920 0x7f3e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:33.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.642+0000 7f3e7f400640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e6006c920 0x7f3e6006ede0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e8015c0b0 tx=0x7f3e6c007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:33.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.646+0000 7f3e77e00640 1 -- 172.21.15.102:0/2706395903 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3e70040070 con 0x7f3e800cc940 2024-04-20T21:39:33.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.888+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e801593e0 con 0x7f3e800cc940 2024-04-20T21:39:33.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.889+0000 7f3e77e00640 1 -- 172.21.15.102:0/2706395903 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+6026 (secure 0 0 0) 0x7f3e7007b630 con 0x7f3e800cc940 2024-04-20T21:39:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:33.888 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:30.821737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:33.888 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 74 2024-04-20T21:39:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e6006c920 msgr2=0x7f3e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e6006c920 0x7f3e6006ede0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e8015c0b0 tx=0x7f3e6c007d50 comp rx=0 tx=0).stop 2024-04-20T21:39:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e800cc940 msgr2=0x7f3e80147580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e800cc940 0x7f3e80147580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e8015f7a0 tx=0x7f3e700046c0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 shutdown_connections 2024-04-20T21:39:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e8015f0e0 0x7f3e801f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e80155510 0x7f3e801f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3e6006c920 0x7f3e6006ede0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 --2- 172.21.15.102:0/2706395903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e800cc940 0x7f3e80147580 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.891+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 >> 172.21.15.102:0/2706395903 conn(0x7f3e800683e0 msgr2=0x7f3e80153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.892+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 shutdown_connections 2024-04-20T21:39:33.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:33.892+0000 7f3e87200640 1 -- 172.21.15.102:0/2706395903 wait complete. 2024-04-20T21:39:33.910 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:39:35.910 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:36.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.096+0000 7f69aae00640 1 Processor -- start 2024-04-20T21:39:36.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.096+0000 7f69aae00640 1 -- start start 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.097+0000 7f69aae00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f69a40cb980 0x7f69a415e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.097+0000 7f69aae00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cbd60 0x7f69a40cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.097+0000 7f69aae00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a415ef40 con 0x7f69a40ccc00 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.097+0000 7f69aae00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f69a415f7e0 con 0x7f69a40cb980 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.097+0000 7f69aae00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f69a415f220 con 0x7f69a40cbd60 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f69a40cb980 0x7f69a415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a8e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f69a40ccc00 0x7f69a40cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:38948/0 (socket says 172.21.15.102:38948) 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a8e00640 1 -- 172.21.15.102:0/563112636 learned_addr learned my addr 172.21.15.102:0/563112636 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:36.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a3400640 1 --2- 172.21.15.102:0/563112636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cbd60 0x7f69a40cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:36.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a2a00640 1 -- 172.21.15.102:0/563112636 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3850269810 0 0) 0x7f69a415ef40 con 0x7f69a40ccc00 2024-04-20T21:39:36.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a2a00640 1 -- 172.21.15.102:0/563112636 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6988003520 con 0x7f69a40ccc00 2024-04-20T21:39:36.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a3e00640 1 -- 172.21.15.102:0/563112636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cbd60 msgr2=0x7f69a40cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:36.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a3e00640 1 --2- 172.21.15.102:0/563112636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cbd60 0x7f69a40cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a3e00640 1 -- 172.21.15.102:0/563112636 >> v1:172.21.15.193:6789/0 conn(0x7f69a40ccc00 legacy=0x7f69a40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:36.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.098+0000 7f69a3e00640 1 -- 172.21.15.102:0/563112636 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a415f500 con 0x7f69a40cb980 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.100+0000 7f69a3e00640 1 --2- 172.21.15.102:0/563112636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cb980 0x7f69a415e890 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f69940097a0 tx=0x7f699402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8ecda7c086d4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.100+0000 7f69a2a00640 1 -- 172.21.15.102:0/563112636 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994035020 con 0x7f69a40cb980 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.100+0000 7f69a2a00640 1 -- 172.21.15.102:0/563112636 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6994031830 con 0x7f69a40cb980 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.100+0000 7f69a2a00640 1 -- 172.21.15.102:0/563112636 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994031c00 con 0x7f69a40cb980 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.101+0000 7f69aae00640 1 -- 172.21.15.102:0/563112636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cb980 msgr2=0x7f69a415e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.101+0000 7f69aae00640 1 --2- 172.21.15.102:0/563112636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cb980 0x7f69a415e890 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f69940097a0 tx=0x7f699402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.101+0000 7f69aae00640 1 -- 172.21.15.102:0/563112636 shutdown_connections 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.101+0000 7f69aae00640 1 --2- 172.21.15.102:0/563112636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cbd60 0x7f69a40cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.101+0000 7f69aae00640 1 --2- 172.21.15.102:0/563112636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cb980 0x7f69a415e890 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.101+0000 7f69aae00640 1 -- 172.21.15.102:0/563112636 >> 172.21.15.102:0/563112636 conn(0x7f69a4075080 msgr2=0x7f69a4075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:36.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.101+0000 7f69aae00640 1 -- 172.21.15.102:0/563112636 shutdown_connections 2024-04-20T21:39:36.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.101+0000 7f69aae00640 1 -- 172.21.15.102:0/563112636 wait complete. 2024-04-20T21:39:36.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.103+0000 7f69aae00640 1 Processor -- start 2024-04-20T21:39:36.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.103+0000 7f69aae00640 1 -- start start 2024-04-20T21:39:36.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69aae00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cb980 0x7f69a415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:36.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69aae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cbd60 0x7f69a4159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:36.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69aae00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69a40ccc00 0x7f69a415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:36.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69aae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f69a40cc1a0 con 0x7f69a40cbd60 2024-04-20T21:39:36.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69aae00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f69a40cd610 con 0x7f69a40ccc00 2024-04-20T21:39:36.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69aae00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f69a41559f0 con 0x7f69a40cb980 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cbd60 0x7f69a4159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cbd60 0x7f69a4159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39604/0 (socket says 172.21.15.102:39604) 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3400640 1 -- 172.21.15.102:0/956093919 learned_addr learned my addr 172.21.15.102:0/956093919 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3e00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cb980 0x7f69a415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a8e00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69a40ccc00 0x7f69a415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3400640 1 -- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cb980 msgr2=0x7f69a415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3400640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cb980 0x7f69a415ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3400640 1 -- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69a40ccc00 msgr2=0x7f69a415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3400640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69a40ccc00 0x7f69a415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.104+0000 7f69a3400640 1 -- 172.21.15.102:0/956093919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a415be30 con 0x7f69a40cbd60 2024-04-20T21:39:36.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.105+0000 7f69a3400640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cbd60 0x7f69a4159990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f699000e9e0 tx=0x7f699000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:36.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.105+0000 7f699be00640 1 -- 172.21.15.102:0/956093919 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990017070 con 0x7f69a40cbd60 2024-04-20T21:39:36.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.105+0000 7f699be00640 1 -- 172.21.15.102:0/956093919 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699000c9d0 con 0x7f69a40cbd60 2024-04-20T21:39:36.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.105+0000 7f699be00640 1 -- 172.21.15.102:0/956093919 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699000cdc0 con 0x7f69a40cbd60 2024-04-20T21:39:36.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.105+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a41591d0 con 0x7f69a40cbd60 2024-04-20T21:39:36.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.105+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a415b870 con 0x7f69a40cbd60 2024-04-20T21:39:36.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.107+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a4074340 con 0x7f69a40cbd60 2024-04-20T21:39:36.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.109+0000 7f699be00640 1 -- 172.21.15.102:0/956093919 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f699001a070 con 0x7f69a40cbd60 2024-04-20T21:39:36.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.110+0000 7f699be00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f698406c920 0x7f698406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:36.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.110+0000 7f699be00640 1 -- 172.21.15.102:0/956093919 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f699008c2b0 con 0x7f69a40cbd60 2024-04-20T21:39:36.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.110+0000 7f69a3e00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f698406c920 0x7f698406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:36.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.111+0000 7f69a3e00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f698406c920 0x7f698406ede0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6994036000 tx=0x7f6994008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:36.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.114+0000 7f699be00640 1 -- 172.21.15.102:0/956093919 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6990059860 con 0x7f69a40cbd60 2024-04-20T21:39:36.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.383+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69a40c7830 con 0x7f69a40cbd60 2024-04-20T21:39:36.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.384+0000 7f699be00640 1 -- 172.21.15.102:0/956093919 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+6026 (secure 0 0 0) 0x7f699005db60 con 0x7f69a40cbd60 2024-04-20T21:39:36.383 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:36.383 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:30.821737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:36.383 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 74 2024-04-20T21:39:36.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f698406c920 msgr2=0x7f698406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f698406c920 0x7f698406ede0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6994036000 tx=0x7f6994008030 comp rx=0 tx=0).stop 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cbd60 msgr2=0x7f69a4159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cbd60 0x7f69a4159990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f699000e9e0 tx=0x7f699000eea0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 shutdown_connections 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f698406c920 0x7f698406ede0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69a40ccc00 0x7f69a415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69a40cbd60 0x7f69a4159990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 --2- 172.21.15.102:0/956093919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69a40cb980 0x7f69a415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.387+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 >> 172.21.15.102:0/956093919 conn(0x7f69a4075080 msgr2=0x7f69a40b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:36.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.388+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 shutdown_connections 2024-04-20T21:39:36.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:36.389+0000 7f69aae00640 1 -- 172.21.15.102:0/956093919 wait complete. 2024-04-20T21:39:36.409 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:39:38.409 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:38.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.592+0000 7f79fdc00640 1 Processor -- start 2024-04-20T21:39:38.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.593+0000 7f79fdc00640 1 -- start start 2024-04-20T21:39:38.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.594+0000 7f79fdc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f79f8155530 0x7f79f815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.594+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cb6a0 0x7f79f80cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.594+0000 7f79fdc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f815ef60 con 0x7f79f80cc470 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.594+0000 7f79fdc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f79f80c5ca0 con 0x7f79f8155530 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.594+0000 7f79fdc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f79f80cd5f0 con 0x7f79f80cb6a0 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.594+0000 7f79f7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f79f8155530 0x7f79f815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.595+0000 7f79f7e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f79f8155530 0x7f79f815e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36004/0 (socket says 172.21.15.102:36004) 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.595+0000 7f79f7e00640 1 -- 172.21.15.102:0/2001596216 learned_addr learned my addr 172.21.15.102:0/2001596216 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.595+0000 7f79fd200640 1 --2- 172.21.15.102:0/2001596216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cb6a0 0x7f79f80cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:38.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.595+0000 7f79f7400640 1 --1- 172.21.15.102:0/2001596216 >> v1:172.21.15.102:6789/0 conn(0x7f79f80cc470 0x7f79f80cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:44116/0 (socket says 172.21.15.102:44116) 2024-04-20T21:39:38.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.595+0000 7f79f7e00640 1 -- 172.21.15.102:0/2001596216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cb6a0 msgr2=0x7f79f80cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:38.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.595+0000 7f79f7e00640 1 --2- 172.21.15.102:0/2001596216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cb6a0 0x7f79f80cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.595+0000 7f79f7e00640 1 -- 172.21.15.102:0/2001596216 >> v1:172.21.15.102:6789/0 conn(0x7f79f80cc470 legacy=0x7f79f80cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:38.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.595+0000 7f79f7e00640 1 -- 172.21.15.102:0/2001596216 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f815f240 con 0x7f79f8155530 2024-04-20T21:39:38.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.596+0000 7f79f7e00640 1 --2- 172.21.15.102:0/2001596216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f8155530 0x7f79f815e640 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009870 tx=0x7f79e802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a30398adf75e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:38.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79f6a00640 1 -- 172.21.15.102:0/2001596216 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8034070 con 0x7f79f8155530 2024-04-20T21:39:38.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79f6a00640 1 -- 172.21.15.102:0/2001596216 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e8030840 con 0x7f79f8155530 2024-04-20T21:39:38.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79f6a00640 1 -- 172.21.15.102:0/2001596216 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8030c10 con 0x7f79f8155530 2024-04-20T21:39:38.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79fdc00640 1 -- 172.21.15.102:0/2001596216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f8155530 msgr2=0x7f79f815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:38.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79fdc00640 1 --2- 172.21.15.102:0/2001596216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f8155530 0x7f79f815e640 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009870 tx=0x7f79e802b980 comp rx=0 tx=0).stop 2024-04-20T21:39:38.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79fdc00640 1 -- 172.21.15.102:0/2001596216 shutdown_connections 2024-04-20T21:39:38.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79fdc00640 1 --2- 172.21.15.102:0/2001596216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cb6a0 0x7f79f80cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79fdc00640 1 --2- 172.21.15.102:0/2001596216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f8155530 0x7f79f815e640 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79fdc00640 1 -- 172.21.15.102:0/2001596216 >> 172.21.15.102:0/2001596216 conn(0x7f79f8075080 msgr2=0x7f79f8075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:38.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79fdc00640 1 -- 172.21.15.102:0/2001596216 shutdown_connections 2024-04-20T21:39:38.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.597+0000 7f79fdc00640 1 -- 172.21.15.102:0/2001596216 wait complete. 2024-04-20T21:39:38.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.599+0000 7f79fdc00640 1 Processor -- start 2024-04-20T21:39:38.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.599+0000 7f79fdc00640 1 -- start start 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f80cb6a0 0x7f79f81f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cc470 0x7f79f8152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79f8155530 0x7f79f81e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79fdc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79f80cd280 con 0x7f79f80cb6a0 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79fdc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79f80ccfa0 con 0x7f79f8155530 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79fdc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f79f80ccbe0 con 0x7f79f80cc470 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f80cb6a0 0x7f79f81f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f80cb6a0 0x7f79f81f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36014/0 (socket says 172.21.15.102:36014) 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79f7e00640 1 -- 172.21.15.102:0/746115431 learned_addr learned my addr 172.21.15.102:0/746115431 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79f7400640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cc470 0x7f79f8152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.600+0000 7f79fd200640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79f8155530 0x7f79f81e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79f7e00640 1 -- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cc470 msgr2=0x7f79f8152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:38.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79f7e00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cc470 0x7f79f8152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79f7e00640 1 -- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79f8155530 msgr2=0x7f79f81e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:38.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79f7e00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79f8155530 0x7f79f81e53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79f7e00640 1 -- 172.21.15.102:0/746115431 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f8153730 con 0x7f79f80cb6a0 2024-04-20T21:39:38.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79f7e00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f80cb6a0 0x7f79f81f21a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f79e802b9c0 tx=0x7f79e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:38.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79efe00640 1 -- 172.21.15.102:0/746115431 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8034040 con 0x7f79f80cb6a0 2024-04-20T21:39:38.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79efe00640 1 -- 172.21.15.102:0/746115431 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e8039d90 con 0x7f79f80cb6a0 2024-04-20T21:39:38.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79efe00640 1 -- 172.21.15.102:0/746115431 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8043630 con 0x7f79f80cb6a0 2024-04-20T21:39:38.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.601+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f8153cd0 con 0x7f79f80cb6a0 2024-04-20T21:39:38.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.602+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f81f16c0 con 0x7f79f80cb6a0 2024-04-20T21:39:38.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.603+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f804fd60 con 0x7f79f80cb6a0 2024-04-20T21:39:38.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.604+0000 7f79efe00640 1 -- 172.21.15.102:0/746115431 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79e804a060 con 0x7f79f80cb6a0 2024-04-20T21:39:38.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.605+0000 7f79efe00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79d806c8f0 0x7f79d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:38.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.605+0000 7f79efe00640 1 -- 172.21.15.102:0/746115431 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f79e80ae210 con 0x7f79f80cb6a0 2024-04-20T21:39:38.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.606+0000 7f79f7400640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79d806c8f0 0x7f79d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:38.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.606+0000 7f79f7400640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79d806c8f0 0x7f79d806edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f79f8152e00 tx=0x7f79e4007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:38.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.613+0000 7f79efe00640 1 -- 172.21.15.102:0/746115431 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f79e803e070 con 0x7f79f80cb6a0 2024-04-20T21:39:38.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.862+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79f80cc9c0 con 0x7f79f80cb6a0 2024-04-20T21:39:38.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.863+0000 7f79efe00640 1 -- 172.21.15.102:0/746115431 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+6026 (secure 0 0 0) 0x7f79e807b740 con 0x7f79f80cb6a0 2024-04-20T21:39:38.862 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:38.862 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:30.821737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:38.863 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 74 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79d806c8f0 msgr2=0x7f79d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79d806c8f0 0x7f79d806edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f79f8152e00 tx=0x7f79e4007e80 comp rx=0 tx=0).stop 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f80cb6a0 msgr2=0x7f79f81f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f80cb6a0 0x7f79f81f21a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f79e802b9c0 tx=0x7f79e80046c0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 shutdown_connections 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f79d806c8f0 0x7f79d806edb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79f8155530 0x7f79f81e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79f80cc470 0x7f79f8152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 --2- 172.21.15.102:0/746115431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79f80cb6a0 0x7f79f81f21a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 >> 172.21.15.102:0/746115431 conn(0x7f79f8075080 msgr2=0x7f79f814fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 shutdown_connections 2024-04-20T21:39:38.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:38.866+0000 7f79fdc00640 1 -- 172.21.15.102:0/746115431 wait complete. 2024-04-20T21:39:38.884 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:39:40.886 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:41.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.068+0000 7f790d800640 1 Processor -- start 2024-04-20T21:39:41.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.069+0000 7f790d800640 1 -- start start 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f790d800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f790806c550 0x7f790815c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f790d800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f790815bfd0 0x7f79080ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f790d800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7908073a20 con 0x7f79081554f0 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f790d800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f790815b510 con 0x7f790815bfd0 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f790d800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7908073cc0 con 0x7f790806c550 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f7907400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f790815bfd0 0x7f79080ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f78ffe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f790806c550 0x7f790815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f7907400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f790815bfd0 0x7f79080ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36018/0 (socket says 172.21.15.102:36018) 2024-04-20T21:39:41.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f7907400640 1 -- 172.21.15.102:0/1482521262 learned_addr learned my addr 172.21.15.102:0/1482521262 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:41.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f78ffe00640 1 -- 172.21.15.102:0/1482521262 >> v2:172.21.15.102:3300/0 conn(0x7f790815bfd0 msgr2=0x7f79080ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:41.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f78ffe00640 1 --2- 172.21.15.102:0/1482521262 >> v2:172.21.15.102:3300/0 conn(0x7f790815bfd0 0x7f79080ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.071+0000 7f78ffe00640 1 -- 172.21.15.102:0/1482521262 >> v1:172.21.15.193:6789/0 conn(0x7f79081554f0 legacy=0x7f79080c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:41.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.072+0000 7f78ffe00640 1 -- 172.21.15.102:0/1482521262 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790815b7d0 con 0x7f790806c550 2024-04-20T21:39:41.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.072+0000 7f7906000640 1 -- 172.21.15.102:0/1482521262 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620754308 0 0) 0x7f7908073a20 con 0x7f79081554f0 2024-04-20T21:39:41.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.073+0000 7f78ffe00640 1 --2- 172.21.15.102:0/1482521262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790806c550 0x7f790815c810 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f790806ce30 tx=0x7f78f002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f9a92f7dbeedd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:41.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.073+0000 7f7906000640 1 -- 172.21.15.102:0/1482521262 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0035020 con 0x7f790806c550 2024-04-20T21:39:41.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.073+0000 7f7906000640 1 -- 172.21.15.102:0/1482521262 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f0031830 con 0x7f790806c550 2024-04-20T21:39:41.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.073+0000 7f7906000640 1 -- 172.21.15.102:0/1482521262 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0031c00 con 0x7f790806c550 2024-04-20T21:39:41.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.074+0000 7f790d800640 1 -- 172.21.15.102:0/1482521262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790806c550 msgr2=0x7f790815c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:41.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.074+0000 7f790d800640 1 --2- 172.21.15.102:0/1482521262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790806c550 0x7f790815c810 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f790806ce30 tx=0x7f78f002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.074+0000 7f790d800640 1 -- 172.21.15.102:0/1482521262 shutdown_connections 2024-04-20T21:39:41.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.074+0000 7f790d800640 1 --2- 172.21.15.102:0/1482521262 >> v2:172.21.15.102:3300/0 conn(0x7f790815bfd0 0x7f79080ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.074+0000 7f790d800640 1 --2- 172.21.15.102:0/1482521262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790806c550 0x7f790815c810 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.074+0000 7f790d800640 1 -- 172.21.15.102:0/1482521262 >> 172.21.15.102:0/1482521262 conn(0x7f790806ebf0 msgr2=0x7f790806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:41.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.074+0000 7f790d800640 1 -- 172.21.15.102:0/1482521262 shutdown_connections 2024-04-20T21:39:41.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.074+0000 7f790d800640 1 -- 172.21.15.102:0/1482521262 wait complete. 2024-04-20T21:39:41.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.076+0000 7f790d800640 1 Processor -- start 2024-04-20T21:39:41.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.077+0000 7f790d800640 1 -- start start 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f790d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f790806c550 0x7f790815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f790d800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79081554f0 0x7f79080c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f790d800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790815bfd0 0x7f790815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f790d800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79080ca000 con 0x7f790806c550 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f790d800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79080736e0 con 0x7f790815bfd0 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f790d800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f79081558d0 con 0x7f79081554f0 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f7906a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f790806c550 0x7f790815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f7906a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f790806c550 0x7f790815dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36028/0 (socket says 172.21.15.102:36028) 2024-04-20T21:39:41.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f7906a00640 1 -- 172.21.15.102:0/2841736399 learned_addr learned my addr 172.21.15.102:0/2841736399 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f7907400640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790815bfd0 0x7f790815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.078+0000 7f78ffe00640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79081554f0 0x7f79080c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f7907400640 1 -- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79081554f0 msgr2=0x7f79080c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f7907400640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79081554f0 0x7f79080c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f7907400640 1 -- 172.21.15.102:0/2841736399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f790806c550 msgr2=0x7f790815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f7907400640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f790806c550 0x7f790815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f7907400640 1 -- 172.21.15.102:0/2841736399 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79080c6500 con 0x7f790815bfd0 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f7906a00640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f790806c550 0x7f790815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:41.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f7907400640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790815bfd0 0x7f790815ce50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f790000eba0 tx=0x7f790000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:41.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f78fea00640 1 -- 172.21.15.102:0/2841736399 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790000cac0 con 0x7f790815bfd0 2024-04-20T21:39:41.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f78fea00640 1 -- 172.21.15.102:0/2841736399 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790000cc60 con 0x7f790815bfd0 2024-04-20T21:39:41.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.079+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79080c67e0 con 0x7f790815bfd0 2024-04-20T21:39:41.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.080+0000 7f78fea00640 1 -- 172.21.15.102:0/2841736399 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900003bb0 con 0x7f790815bfd0 2024-04-20T21:39:41.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.080+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79080c6da0 con 0x7f790815bfd0 2024-04-20T21:39:41.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.081+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7908074340 con 0x7f790815bfd0 2024-04-20T21:39:41.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.083+0000 7f78fea00640 1 -- 172.21.15.102:0/2841736399 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7900012050 con 0x7f790815bfd0 2024-04-20T21:39:41.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.084+0000 7f78fea00640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f78e406c8f0 0x7f78e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:41.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.084+0000 7f78fea00640 1 -- 172.21.15.102:0/2841736399 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f790008c5f0 con 0x7f790815bfd0 2024-04-20T21:39:41.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.084+0000 7f7906a00640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f78e406c8f0 0x7f78e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:41.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.085+0000 7f7906a00640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f78e406c8f0 0x7f78e406edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f78f8007fd0 tx=0x7f78f8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:41.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.088+0000 7f78fea00640 1 -- 172.21.15.102:0/2841736399 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7900010040 con 0x7f790815bfd0 2024-04-20T21:39:41.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.353+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f790815f0a0 con 0x7f790815bfd0 2024-04-20T21:39:41.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.354+0000 7f78fea00640 1 -- 172.21.15.102:0/2841736399 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+6026 (secure 0 0 0) 0x7f7900059bc0 con 0x7f790815bfd0 2024-04-20T21:39:41.353 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:41.353 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:30.821737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:41.354 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 74 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.356+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f78e406c8f0 msgr2=0x7f78e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.356+0000 7f790d800640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f78e406c8f0 0x7f78e406edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f78f8007fd0 tx=0x7f78f8009040 comp rx=0 tx=0).stop 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.356+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790815bfd0 msgr2=0x7f790815ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.356+0000 7f790d800640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790815bfd0 0x7f790815ce50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f790000eba0 tx=0x7f790000c490 comp rx=0 tx=0).stop 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.357+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 shutdown_connections 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.357+0000 7f790d800640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f78e406c8f0 0x7f78e406edb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.357+0000 7f790d800640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790815bfd0 0x7f790815ce50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.357+0000 7f790d800640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79081554f0 0x7f79080c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.357+0000 7f790d800640 1 --2- 172.21.15.102:0/2841736399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f790806c550 0x7f790815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.357+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 >> 172.21.15.102:0/2841736399 conn(0x7f790806ebf0 msgr2=0x7f79080ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.357+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 shutdown_connections 2024-04-20T21:39:41.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:41.357+0000 7f790d800640 1 -- 172.21.15.102:0/2841736399 wait complete. 2024-04-20T21:39:41.373 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:39:43.377 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:43.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.554+0000 7f3342a00640 1 Processor -- start 2024-04-20T21:39:43.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.554+0000 7f3342a00640 1 -- start start 2024-04-20T21:39:43.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.556+0000 7f3342a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f333c14fa50 0x7f333c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:43.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.556+0000 7f3342a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c073a10 0x7f333c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:43.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.556+0000 7f3342a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f333c1536d0 con 0x7f333c0c9eb0 2024-04-20T21:39:43.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.556+0000 7f3342a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f333c159500 con 0x7f333c14fa50 2024-04-20T21:39:43.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.556+0000 7f3342a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f333c0c76c0 con 0x7f333c073a10 2024-04-20T21:39:43.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.556+0000 7f3340a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c073a10 0x7f333c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:43.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.557+0000 7f333be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f333c14fa50 0x7f333c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:43.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.557+0000 7f3333e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f333c0c9eb0 0x7f333c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:58204/0 (socket says 172.21.15.102:58204) 2024-04-20T21:39:43.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.557+0000 7f3333e00640 1 -- 172.21.15.102:0/1955170834 learned_addr learned my addr 172.21.15.102:0/1955170834 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:43.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.557+0000 7f3340a00640 1 -- 172.21.15.102:0/1955170834 >> v2:172.21.15.193:3300/0 conn(0x7f333c14fa50 msgr2=0x7f333c0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:43.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.557+0000 7f3340a00640 1 --2- 172.21.15.102:0/1955170834 >> v2:172.21.15.193:3300/0 conn(0x7f333c14fa50 0x7f333c0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.557+0000 7f3340a00640 1 -- 172.21.15.102:0/1955170834 >> v1:172.21.15.193:6789/0 conn(0x7f333c0c9eb0 legacy=0x7f333c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:43.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.557+0000 7f3340a00640 1 -- 172.21.15.102:0/1955170834 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c0c7960 con 0x7f333c073a10 2024-04-20T21:39:43.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.558+0000 7f3340a00640 1 --2- 172.21.15.102:0/1955170834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c073a10 0x7f333c158fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3334009d90 tx=0x7f333402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d267733141dd091f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:43.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.558+0000 7f333b400640 1 -- 172.21.15.102:0/1955170834 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334039070 con 0x7f333c073a10 2024-04-20T21:39:43.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.558+0000 7f333b400640 1 -- 172.21.15.102:0/1955170834 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333402cce0 con 0x7f333c073a10 2024-04-20T21:39:43.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.558+0000 7f333b400640 1 -- 172.21.15.102:0/1955170834 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334034660 con 0x7f333c073a10 2024-04-20T21:39:43.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.559+0000 7f3342a00640 1 -- 172.21.15.102:0/1955170834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c073a10 msgr2=0x7f333c158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:43.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.559+0000 7f3342a00640 1 --2- 172.21.15.102:0/1955170834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c073a10 0x7f333c158fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3334009d90 tx=0x7f333402bf30 comp rx=0 tx=0).stop 2024-04-20T21:39:43.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.559+0000 7f3342a00640 1 -- 172.21.15.102:0/1955170834 shutdown_connections 2024-04-20T21:39:43.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.559+0000 7f3342a00640 1 --2- 172.21.15.102:0/1955170834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c073a10 0x7f333c158fc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.559+0000 7f3342a00640 1 --2- 172.21.15.102:0/1955170834 >> v2:172.21.15.193:3300/0 conn(0x7f333c14fa50 0x7f333c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.559+0000 7f3342a00640 1 -- 172.21.15.102:0/1955170834 >> 172.21.15.102:0/1955170834 conn(0x7f333c0c9390 msgr2=0x7f333c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:43.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.559+0000 7f3342a00640 1 -- 172.21.15.102:0/1955170834 shutdown_connections 2024-04-20T21:39:43.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.559+0000 7f3342a00640 1 -- 172.21.15.102:0/1955170834 wait complete. 2024-04-20T21:39:43.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.561+0000 7f3342a00640 1 Processor -- start 2024-04-20T21:39:43.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.561+0000 7f3342a00640 1 -- start start 2024-04-20T21:39:43.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f3342a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f333c073a10 0x7f333c149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:43.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f3342a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f333c0c9eb0 0x7f333c14a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:43.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f3342a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c14fa50 0x7f333c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:43.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f3342a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f333c0c7c00 con 0x7f333c073a10 2024-04-20T21:39:43.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f3342a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f333c0c3d90 con 0x7f333c0c9eb0 2024-04-20T21:39:43.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f3342a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f333c0c4030 con 0x7f333c14fa50 2024-04-20T21:39:43.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f333be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f333c073a10 0x7f333c149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:43.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f333be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f333c073a10 0x7f333c149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36038/0 (socket says 172.21.15.102:36038) 2024-04-20T21:39:43.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f333be00640 1 -- 172.21.15.102:0/3659496527 learned_addr learned my addr 172.21.15.102:0/3659496527 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:43.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f3333e00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f333c0c9eb0 0x7f333c14a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:43.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.562+0000 7f3340a00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c14fa50 0x7f333c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f333be00640 1 -- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c14fa50 msgr2=0x7f333c156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f333be00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c14fa50 0x7f333c156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f333be00640 1 -- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f333c0c9eb0 msgr2=0x7f333c14a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f333be00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f333c0c9eb0 0x7f333c14a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f333be00640 1 -- 172.21.15.102:0/3659496527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c158fe0 con 0x7f333c073a10 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f333be00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f333c073a10 0x7f333c149ca0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f332c00e9b0 tx=0x7f332c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f3338c00640 1 -- 172.21.15.102:0/3659496527 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c017070 con 0x7f333c073a10 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f3338c00640 1 -- 172.21.15.102:0/3659496527 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332c00c9a0 con 0x7f333c073a10 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f3338c00640 1 -- 172.21.15.102:0/3659496527 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c00cd90 con 0x7f333c073a10 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c1f2700 con 0x7f333c073a10 2024-04-20T21:39:43.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.563+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1f2cc0 con 0x7f333c073a10 2024-04-20T21:39:43.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.565+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c159820 con 0x7f333c073a10 2024-04-20T21:39:43.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.565+0000 7f3338c00640 1 -- 172.21.15.102:0/3659496527 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f332c01a070 con 0x7f333c073a10 2024-04-20T21:39:43.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.567+0000 7f3338c00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f331806c8f0 0x7f331806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:43.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.567+0000 7f3338c00640 1 -- 172.21.15.102:0/3659496527 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f332c08c2c0 con 0x7f333c073a10 2024-04-20T21:39:43.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.567+0000 7f3333e00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f331806c8f0 0x7f331806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:43.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.568+0000 7f3333e00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f331806c8f0 0x7f331806edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f333c14c6b0 tx=0x7f3324007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:43.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.572+0000 7f3338c00640 1 -- 172.21.15.102:0/3659496527 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f332c059880 con 0x7f333c073a10 2024-04-20T21:39:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.844+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f333c06e790 con 0x7f333c073a10 2024-04-20T21:39:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.845+0000 7f3338c00640 1 -- 172.21.15.102:0/3659496527 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+6026 (secure 0 0 0) 0x7f332c05db80 con 0x7f333c073a10 2024-04-20T21:39:43.845 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:43.845 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:30.821737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:43.845 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 74 2024-04-20T21:39:43.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f331806c8f0 msgr2=0x7f331806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f331806c8f0 0x7f331806edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f333c14c6b0 tx=0x7f3324007e80 comp rx=0 tx=0).stop 2024-04-20T21:39:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f333c073a10 msgr2=0x7f333c149ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f333c073a10 0x7f333c149ca0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f332c00e9b0 tx=0x7f332c00ee70 comp rx=0 tx=0).stop 2024-04-20T21:39:43.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 shutdown_connections 2024-04-20T21:39:43.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f333c14fa50 0x7f333c156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f331806c8f0 0x7f331806edb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f333c0c9eb0 0x7f333c14a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 --2- 172.21.15.102:0/3659496527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f333c073a10 0x7f333c149ca0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:43.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 >> 172.21.15.102:0/3659496527 conn(0x7f333c0c9390 msgr2=0x7f333c0c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:43.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.848+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 shutdown_connections 2024-04-20T21:39:43.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:43.849+0000 7f3342a00640 1 -- 172.21.15.102:0/3659496527 wait complete. 2024-04-20T21:39:43.864 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 74, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 74, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6429": { "addr": "172.21.15.102:6835/3586768191", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 3586768191, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 3586768191, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6429, "incarnation": 74, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:39:30.821737+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6429 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:39:45.865 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:46.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.070+0000 7f3fa3200640 1 Processor -- start 2024-04-20T21:39:46.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.070+0000 7f3fa3200640 1 -- start start 2024-04-20T21:39:46.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.071+0000 7f3fa3200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3f9c15ea20 0x7f3f9c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3fa3200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3f9c15ee00 0x7f3f9c0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3fa3200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3fa3200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9c0c81e0 con 0x7f3f9c15ee00 2024-04-20T21:39:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3fa3200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9c0c8f20 con 0x7f3f9c15ea20 2024-04-20T21:39:46.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3fa3200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c06cb80 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3f9be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3f9c15ea20 0x7f3f9c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3f93e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3f9c15ee00 0x7f3f9c0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3fa1200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3f9be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3f9c15ea20 0x7f3f9c06d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:50072/0 (socket says 172.21.15.102:50072) 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3f9be00640 1 -- 172.21.15.102:0/3744546050 learned_addr learned my addr 172.21.15.102:0/3744546050 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3f9be00640 1 -- 172.21.15.102:0/3744546050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15f2b0 msgr2=0x7f3f9c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.072+0000 7f3f9be00640 1 --2- 172.21.15.102:0/3744546050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.073+0000 7f3f9be00640 1 -- 172.21.15.102:0/3744546050 >> v2:172.21.15.102:3300/0 conn(0x7f3f9c15ee00 msgr2=0x7f3f9c0710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.073+0000 7f3f9be00640 1 --2- 172.21.15.102:0/3744546050 >> v2:172.21.15.102:3300/0 conn(0x7f3f9c15ee00 0x7f3f9c0710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.073+0000 7f3f9be00640 1 -- 172.21.15.102:0/3744546050 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c06ce60 con 0x7f3f9c15ea20 2024-04-20T21:39:46.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.074+0000 7f3f9be00640 1 --2- 172.21.15.102:0/3744546050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ea20 0x7f3f9c06d940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0cd8d0 tx=0x7f3f8c02b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=465616776262b224 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:46.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.074+0000 7f3f9b400640 1 -- 172.21.15.102:0/3744546050 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c034070 con 0x7f3f9c15ea20 2024-04-20T21:39:46.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.074+0000 7f3f9b400640 1 -- 172.21.15.102:0/3744546050 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c030850 con 0x7f3f9c15ea20 2024-04-20T21:39:46.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.074+0000 7f3f9b400640 1 -- 172.21.15.102:0/3744546050 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c030c20 con 0x7f3f9c15ea20 2024-04-20T21:39:46.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 -- 172.21.15.102:0/3744546050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ea20 msgr2=0x7f3f9c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:46.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 --2- 172.21.15.102:0/3744546050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ea20 0x7f3f9c06d940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0cd8d0 tx=0x7f3f8c02b990 comp rx=0 tx=0).stop 2024-04-20T21:39:46.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 -- 172.21.15.102:0/3744546050 shutdown_connections 2024-04-20T21:39:46.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 --2- 172.21.15.102:0/3744546050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 --2- 172.21.15.102:0/3744546050 >> v2:172.21.15.102:3300/0 conn(0x7f3f9c15ee00 0x7f3f9c0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 --2- 172.21.15.102:0/3744546050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ea20 0x7f3f9c06d940 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 -- 172.21.15.102:0/3744546050 >> 172.21.15.102:0/3744546050 conn(0x7f3f9c0c9390 msgr2=0x7f3f9c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:46.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 -- 172.21.15.102:0/3744546050 shutdown_connections 2024-04-20T21:39:46.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.075+0000 7f3fa3200640 1 -- 172.21.15.102:0/3744546050 wait complete. 2024-04-20T21:39:46.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.077+0000 7f3fa3200640 1 Processor -- start 2024-04-20T21:39:46.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.077+0000 7f3fa3200640 1 -- start start 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa3200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15ea20 0x7f3f9c06e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ee00 0x7f3f9c0c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c0c8b00 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c06d420 con 0x7f3f9c15ee00 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa3200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c0c8c80 con 0x7f3f9c15ea20 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36062/0 (socket says 172.21.15.102:36062) 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3fa1200640 1 -- 172.21.15.102:0/1062368429 learned_addr learned my addr 172.21.15.102:0/1062368429 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3f93e00640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ee00 0x7f3f9c0c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:46.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.078+0000 7f3f9be00640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15ea20 0x7f3f9c06e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:46.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3fa1200640 1 -- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15ea20 msgr2=0x7f3f9c06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:46.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3fa1200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15ea20 0x7f3f9c06e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3fa1200640 1 -- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ee00 msgr2=0x7f3f9c0c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:46.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3fa1200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ee00 0x7f3f9c0c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3fa1200640 1 -- 172.21.15.102:0/1062368429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c0c6b70 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3f93e00640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ee00 0x7f3f9c0c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:46.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3fa1200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c5140 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009cb0 tx=0x7f3f9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:46.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3f98c00640 1 -- 172.21.15.102:0/1062368429 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94018020 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3f98c00640 1 -- 172.21.15.102:0/1062368429 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94014480 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3f98c00640 1 -- 172.21.15.102:0/1062368429 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94014870 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.079+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9c0c6f80 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.080+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9c1e6f40 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.081+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9c074340 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.082+0000 7f3f98c00640 1 -- 172.21.15.102:0/1062368429 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f94005d10 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.082+0000 7f3f98c00640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3f7806c8f0 0x7f3f7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:46.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.083+0000 7f3f98c00640 1 -- 172.21.15.102:0/1062368429 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f3f9408cf80 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.083+0000 7f3f9be00640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3f7806c8f0 0x7f3f7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:46.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.084+0000 7f3f9be00640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3f7806c8f0 0x7f3f7806edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c036000 tx=0x7f3f8c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:46.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.087+0000 7f3f98c00640 1 -- 172.21.15.102:0/1062368429 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3f94022030 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.321+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f9c152870 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.322+0000 7f3f98c00640 1 -- 172.21.15.102:0/1062368429 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+6030 (secure 0 0 0) 0x7f3f94010040 con 0x7f3f9c15f2b0 2024-04-20T21:39:46.321 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:46.322 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:43.764571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:46.322 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 75 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.324+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3f7806c8f0 msgr2=0x7f3f7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.324+0000 7f3fa3200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3f7806c8f0 0x7f3f7806edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c036000 tx=0x7f3f8c008030 comp rx=0 tx=0).stop 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.324+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f9c15f2b0 msgr2=0x7f3f9c0c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.324+0000 7f3fa3200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c5140 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009cb0 tx=0x7f3f9400c490 comp rx=0 tx=0).stop 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.325+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 shutdown_connections 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.325+0000 7f3fa3200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f9c15f2b0 0x7f3f9c0c5140 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.325+0000 7f3fa3200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3f7806c8f0 0x7f3f7806edb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.325+0000 7f3fa3200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c15ee00 0x7f3f9c0c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.325+0000 7f3fa3200640 1 --2- 172.21.15.102:0/1062368429 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f9c15ea20 0x7f3f9c06e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.325+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 >> 172.21.15.102:0/1062368429 conn(0x7f3f9c0c9390 msgr2=0x7f3f9c1534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:46.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.325+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 shutdown_connections 2024-04-20T21:39:46.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:46.325+0000 7f3fa3200640 1 -- 172.21.15.102:0/1062368429 wait complete. 2024-04-20T21:39:46.342 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:39:48.343 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:48.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.555+0000 7f5ecb000640 1 Processor -- start 2024-04-20T21:39:48.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.555+0000 7f5ecb000640 1 -- start start 2024-04-20T21:39:48.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.557+0000 7f5ecb000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ec4151c90 0x7f5ec40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:48.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.557+0000 7f5ecb000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec4071040 con 0x7f5ec4071a70 2024-04-20T21:39:48.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.557+0000 7f5ecb000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec4071320 con 0x7f5ec40c7290 2024-04-20T21:39:48.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.557+0000 7f5ecb000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5ec40c9200 con 0x7f5ec4151c90 2024-04-20T21:39:48.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.557+0000 7f5ec3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ec4151c90 0x7f5ec40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:48.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.557+0000 7f5ebb400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f5ec40c7290 0x7f5ec4158240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55784/0 (socket says 172.21.15.102:55784) 2024-04-20T21:39:48.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.557+0000 7f5ebb400640 1 -- 172.21.15.102:0/1792293555 learned_addr learned my addr 172.21.15.102:0/1792293555 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:48.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.558+0000 7f5ec3400640 1 -- 172.21.15.102:0/1792293555 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1521054448 0 0) 0x7f5ec4071320 con 0x7f5ec40c7290 2024-04-20T21:39:48.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.558+0000 7f5ec3e00640 1 -- 172.21.15.102:0/1792293555 >> v1:172.21.15.193:6789/0 conn(0x7f5ec40c7290 legacy=0x7f5ec4158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:48.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.558+0000 7f5ec3e00640 1 -- 172.21.15.102:0/1792293555 >> v1:172.21.15.102:6789/0 conn(0x7f5ec4071a70 legacy=0x7f5ec406d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:48.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.558+0000 7f5ec3e00640 1 -- 172.21.15.102:0/1792293555 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec4071600 con 0x7f5ec4151c90 2024-04-20T21:39:48.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.559+0000 7f5ec3e00640 1 --2- 172.21.15.102:0/1792293555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec40c8cc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40097a0 tx=0x7f5eb402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f460ba6626d15d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:48.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.559+0000 7f5ec3400640 1 -- 172.21.15.102:0/1792293555 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4038070 con 0x7f5ec4151c90 2024-04-20T21:39:48.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.559+0000 7f5ec3400640 1 -- 172.21.15.102:0/1792293555 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb4031700 con 0x7f5ec4151c90 2024-04-20T21:39:48.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.559+0000 7f5ec3400640 1 -- 172.21.15.102:0/1792293555 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4031ad0 con 0x7f5ec4151c90 2024-04-20T21:39:48.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.560+0000 7f5ecb000640 1 -- 172.21.15.102:0/1792293555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 msgr2=0x7f5ec40c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:48.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.560+0000 7f5ecb000640 1 --2- 172.21.15.102:0/1792293555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec40c8cc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40097a0 tx=0x7f5eb402b820 comp rx=0 tx=0).stop 2024-04-20T21:39:48.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.560+0000 7f5ecb000640 1 -- 172.21.15.102:0/1792293555 shutdown_connections 2024-04-20T21:39:48.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.560+0000 7f5ecb000640 1 --2- 172.21.15.102:0/1792293555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec40c8cc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:48.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.560+0000 7f5ecb000640 1 -- 172.21.15.102:0/1792293555 >> 172.21.15.102:0/1792293555 conn(0x7f5ec40c9390 msgr2=0x7f5ec40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:48.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.560+0000 7f5ecb000640 1 -- 172.21.15.102:0/1792293555 shutdown_connections 2024-04-20T21:39:48.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.560+0000 7f5ecb000640 1 -- 172.21.15.102:0/1792293555 wait complete. 2024-04-20T21:39:48.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.562+0000 7f5ecb000640 1 Processor -- start 2024-04-20T21:39:48.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.562+0000 7f5ecb000640 1 -- start start 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.562+0000 7f5ecb000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ec4071a70 0x7f5ec415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ecb000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec40c7290 0x7f5ec41f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ecb000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec4146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ecb000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec4157db0 con 0x7f5ec4151c90 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ecb000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec4070df0 con 0x7f5ec40c7290 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ecb000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec40ca1f0 con 0x7f5ec4071a70 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec4146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec4146600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39660/0 (socket says 172.21.15.102:39660) 2024-04-20T21:39:48.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec9000640 1 -- 172.21.15.102:0/1854247557 learned_addr learned my addr 172.21.15.102:0/1854247557 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ebb400640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec40c7290 0x7f5ec41f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec3e00640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ec4071a70 0x7f5ec415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec9000640 1 -- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ec4071a70 msgr2=0x7f5ec415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec9000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ec4071a70 0x7f5ec415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec9000640 1 -- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec40c7290 msgr2=0x7f5ec41f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec9000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec40c7290 0x7f5ec41f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.563+0000 7f5ec9000640 1 -- 172.21.15.102:0/1854247557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec4159960 con 0x7f5ec4151c90 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.564+0000 7f5ec9000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec4146600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c8c0 tx=0x7f5ebc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.564+0000 7f5ec0c00640 1 -- 172.21.15.102:0/1854247557 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc017070 con 0x7f5ec4151c90 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.564+0000 7f5ec0c00640 1 -- 172.21.15.102:0/1854247557 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc003da0 con 0x7f5ec4151c90 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.564+0000 7f5ec0c00640 1 -- 172.21.15.102:0/1854247557 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc005290 con 0x7f5ec4151c90 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.564+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec41f3990 con 0x7f5ec4151c90 2024-04-20T21:39:48.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.564+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec4158ba0 con 0x7f5ec4151c90 2024-04-20T21:39:48.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.565+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec4069340 con 0x7f5ec4151c90 2024-04-20T21:39:48.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.566+0000 7f5ec0c00640 1 -- 172.21.15.102:0/1854247557 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ebc01a070 con 0x7f5ec4151c90 2024-04-20T21:39:48.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.567+0000 7f5ec0c00640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5ea006c920 0x7f5ea006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:48.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.567+0000 7f5ec0c00640 1 -- 172.21.15.102:0/1854247557 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f5ebc08ba80 con 0x7f5ec4151c90 2024-04-20T21:39:48.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.568+0000 7f5ec3e00640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5ea006c920 0x7f5ea006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:48.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.568+0000 7f5ec3e00640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5ea006c920 0x7f5ea006ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4035000 tx=0x7f5eb4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:48.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.571+0000 7f5ec0c00640 1 -- 172.21.15.102:0/1854247557 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5ec4069340 con 0x7f5ec4151c90 2024-04-20T21:39:48.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.811+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ec4073180 con 0x7f5ec4151c90 2024-04-20T21:39:48.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.812+0000 7f5ec0c00640 1 -- 172.21.15.102:0/1854247557 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+6030 (secure 0 0 0) 0x7f5ec4073180 con 0x7f5ec4151c90 2024-04-20T21:39:48.812 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:48.812 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:43.764571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:48.812 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 75 2024-04-20T21:39:48.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5ea006c920 msgr2=0x7f5ea006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:48.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5ea006c920 0x7f5ea006ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4035000 tx=0x7f5eb4008030 comp rx=0 tx=0).stop 2024-04-20T21:39:48.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 msgr2=0x7f5ec4146600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec4146600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c8c0 tx=0x7f5ebc00cd80 comp rx=0 tx=0).stop 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 shutdown_connections 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ec4151c90 0x7f5ec4146600 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5ea006c920 0x7f5ea006ede0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec40c7290 0x7f5ec41f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 --2- 172.21.15.102:0/1854247557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ec4071a70 0x7f5ec415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 >> 172.21.15.102:0/1854247557 conn(0x7f5ec40c9390 msgr2=0x7f5ec406e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 shutdown_connections 2024-04-20T21:39:48.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:48.815+0000 7f5ecb000640 1 -- 172.21.15.102:0/1854247557 wait complete. 2024-04-20T21:39:48.834 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:39:50.835 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:51.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.046+0000 7fc8cfa00640 1 Processor -- start 2024-04-20T21:39:51.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.046+0000 7fc8cfa00640 1 -- start start 2024-04-20T21:39:51.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8cfa00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc8c80737f0 0x7fc8c8159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8cfa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8c8073c30 0x7fc8c80c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8cfa00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80c6ec0 con 0x7fc8c813bc80 2024-04-20T21:39:51.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8cfa00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c80c6a30 con 0x7fc8c8073c30 2024-04-20T21:39:51.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8cfa00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c80c8d00 con 0x7fc8c80737f0 2024-04-20T21:39:51.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8cda00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8c8073c30 0x7fc8c80c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8cda00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc8c8073c30 0x7fc8c80c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39674/0 (socket says 172.21.15.102:39674) 2024-04-20T21:39:51.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8cda00640 1 -- 172.21.15.102:0/1167403294 learned_addr learned my addr 172.21.15.102:0/1167403294 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:51.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.048+0000 7fc8c7e00640 1 --2- 172.21.15.102:0/1167403294 >> v2:172.21.15.193:3300/0 conn(0x7fc8c80737f0 0x7fc8c8159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.049+0000 7fc8cda00640 1 -- 172.21.15.102:0/1167403294 >> v2:172.21.15.193:3300/0 conn(0x7fc8c80737f0 msgr2=0x7fc8c8159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.049+0000 7fc8cda00640 1 --2- 172.21.15.102:0/1167403294 >> v2:172.21.15.193:3300/0 conn(0x7fc8c80737f0 0x7fc8c8159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.049+0000 7fc8cda00640 1 -- 172.21.15.102:0/1167403294 >> v1:172.21.15.193:6789/0 conn(0x7fc8c813bc80 legacy=0x7fc8c80c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.049+0000 7fc8cda00640 1 -- 172.21.15.102:0/1167403294 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80c8fc0 con 0x7fc8c8073c30 2024-04-20T21:39:51.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.049+0000 7fc8c7400640 1 -- 172.21.15.102:0/1167403294 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 951672978 0 0) 0x7fc8c80c6ec0 con 0x7fc8c813bc80 2024-04-20T21:39:51.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.050+0000 7fc8cda00640 1 --2- 172.21.15.102:0/1167403294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 0x7fc8c80c98f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc8c806a220 tx=0x7fc8c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f31377493a1a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.050+0000 7fc8c7400640 1 -- 172.21.15.102:0/1167403294 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0034070 con 0x7fc8c8073c30 2024-04-20T21:39:51.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.050+0000 7fc8c7400640 1 -- 172.21.15.102:0/1167403294 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c0030860 con 0x7fc8c8073c30 2024-04-20T21:39:51.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.050+0000 7fc8c7400640 1 -- 172.21.15.102:0/1167403294 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0030c30 con 0x7fc8c8073c30 2024-04-20T21:39:51.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.051+0000 7fc8cfa00640 1 -- 172.21.15.102:0/1167403294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 msgr2=0x7fc8c80c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.051+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/1167403294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 0x7fc8c80c98f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc8c806a220 tx=0x7fc8c002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.051+0000 7fc8cfa00640 1 -- 172.21.15.102:0/1167403294 shutdown_connections 2024-04-20T21:39:51.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.051+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/1167403294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 0x7fc8c80c98f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.051+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/1167403294 >> v2:172.21.15.193:3300/0 conn(0x7fc8c80737f0 0x7fc8c8159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.051+0000 7fc8cfa00640 1 -- 172.21.15.102:0/1167403294 >> 172.21.15.102:0/1167403294 conn(0x7fc8c806ebf0 msgr2=0x7fc8c806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.051+0000 7fc8cfa00640 1 -- 172.21.15.102:0/1167403294 shutdown_connections 2024-04-20T21:39:51.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.051+0000 7fc8cfa00640 1 -- 172.21.15.102:0/1167403294 wait complete. 2024-04-20T21:39:51.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.053+0000 7fc8cfa00640 1 Processor -- start 2024-04-20T21:39:51.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.053+0000 7fc8cfa00640 1 -- start start 2024-04-20T21:39:51.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cfa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8c80737f0 0x7fc8c81520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cfa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 0x7fc8c80cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cfa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8c813bc80 0x7fc8c80cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cfa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80c66c0 con 0x7fc8c8073c30 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cfa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80c63e0 con 0x7fc8c80737f0 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cfa00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c8159560 con 0x7fc8c813bc80 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8c80737f0 0x7fc8c81520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8c80737f0 0x7fc8c81520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:60644/0 (socket says 172.21.15.102:60644) 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8c7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 0x7fc8c80cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cd000640 1 -- 172.21.15.102:0/3351752040 learned_addr learned my addr 172.21.15.102:0/3351752040 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.054+0000 7fc8cda00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8c813bc80 0x7fc8c80cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8cd000640 1 -- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8c813bc80 msgr2=0x7fc8c80cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8cd000640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8c813bc80 0x7fc8c80cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8cd000640 1 -- 172.21.15.102:0/3351752040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 msgr2=0x7fc8c80cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8cd000640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 0x7fc8c80cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8cd000640 1 -- 172.21.15.102:0/3351752040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80cdaf0 con 0x7fc8c80737f0 2024-04-20T21:39:51.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8cd000640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8c80737f0 0x7fc8c81520f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c7f0 tx=0x7fc8b800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8c4c00640 1 -- 172.21.15.102:0/3351752040 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8017070 con 0x7fc8c80737f0 2024-04-20T21:39:51.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8c4c00640 1 -- 172.21.15.102:0/3351752040 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8003da0 con 0x7fc8c80737f0 2024-04-20T21:39:51.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8c4c00640 1 -- 172.21.15.102:0/3351752040 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8005290 con 0x7fc8c80737f0 2024-04-20T21:39:51.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c81f17d0 con 0x7fc8c80737f0 2024-04-20T21:39:51.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.055+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c80ce070 con 0x7fc8c80737f0 2024-04-20T21:39:51.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.056+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c8073000 con 0x7fc8c80737f0 2024-04-20T21:39:51.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.057+0000 7fc8c4c00640 1 -- 172.21.15.102:0/3351752040 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8b801a070 con 0x7fc8c80737f0 2024-04-20T21:39:51.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.058+0000 7fc8c4c00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8a806c920 0x7fc8a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.058+0000 7fc8c4c00640 1 -- 172.21.15.102:0/3351752040 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7fc8b808ba50 con 0x7fc8c80737f0 2024-04-20T21:39:51.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.059+0000 7fc8c7e00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8a806c920 0x7fc8a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.059+0000 7fc8c7e00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8a806c920 0x7fc8a806ede0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8158e80 tx=0x7fc8b4009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.063+0000 7fc8c4c00640 1 -- 172.21.15.102:0/3351752040 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc8c8073000 con 0x7fc8c80737f0 2024-04-20T21:39:51.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.301+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8c8069340 con 0x7fc8c80737f0 2024-04-20T21:39:51.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.303+0000 7fc8c4c00640 1 -- 172.21.15.102:0/3351752040 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+6027 (secure 0 0 0) 0x7fc8c8069340 con 0x7fc8c80737f0 2024-04-20T21:39:51.302 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:51.302 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:49.366546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":14,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:51.302 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 76 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8a806c920 msgr2=0x7fc8a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8a806c920 0x7fc8a806ede0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8158e80 tx=0x7fc8b4009190 comp rx=0 tx=0).stop 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8c80737f0 msgr2=0x7fc8c81520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8c80737f0 0x7fc8c81520f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c7f0 tx=0x7fc8b800ccb0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 shutdown_connections 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8c813bc80 0x7fc8c80cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8c8073c30 0x7fc8c80cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc8a806c920 0x7fc8a806ede0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 --2- 172.21.15.102:0/3351752040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8c80737f0 0x7fc8c81520f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.305+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 >> 172.21.15.102:0/3351752040 conn(0x7fc8c806ebf0 msgr2=0x7fc8c80c98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.306+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 shutdown_connections 2024-04-20T21:39:51.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:51.306+0000 7fc8cfa00640 1 -- 172.21.15.102:0/3351752040 wait complete. 2024-04-20T21:39:51.323 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:39:53.325 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:53.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.537+0000 7fdf80c00640 1 Processor -- start 2024-04-20T21:39:53.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.537+0000 7fdf80c00640 1 -- start start 2024-04-20T21:39:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf80c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdf7c0c9eb0 0x7fdf7c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf80c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c1536d0 con 0x7fdf7c073a10 2024-04-20T21:39:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf80c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c0c76c0 con 0x7fdf7c14fa50 2024-04-20T21:39:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf80c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c0c7960 con 0x7fdf7c0c9eb0 2024-04-20T21:39:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf79e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fdf7c14fa50 0x7fdf7c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:55812/0 (socket says 172.21.15.102:55812) 2024-04-20T21:39:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf79e00640 1 -- 172.21.15.102:0/2803006780 learned_addr learned my addr 172.21.15.102:0/2803006780 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:53.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf73e00640 1 --2- 172.21.15.102:0/2803006780 >> v2:172.21.15.193:3300/0 conn(0x7fdf7c0c9eb0 0x7fdf7c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf79400640 1 -- 172.21.15.102:0/2803006780 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340883642 0 0) 0x7fdf7c0c76c0 con 0x7fdf7c14fa50 2024-04-20T21:39:53.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf79400640 1 -- 172.21.15.102:0/2803006780 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf60003520 con 0x7fdf7c14fa50 2024-04-20T21:39:53.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf79400640 1 -- 172.21.15.102:0/2803006780 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110311735 0 0) 0x7fdf74001490 con 0x7fdf7c073a10 2024-04-20T21:39:53.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.539+0000 7fdf79400640 1 -- 172.21.15.102:0/2803006780 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf7c0c76c0 con 0x7fdf7c073a10 2024-04-20T21:39:53.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.540+0000 7fdf73e00640 1 -- 172.21.15.102:0/2803006780 >> v1:172.21.15.193:6789/0 conn(0x7fdf7c14fa50 legacy=0x7fdf7c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.540+0000 7fdf73e00640 1 -- 172.21.15.102:0/2803006780 >> v1:172.21.15.102:6789/0 conn(0x7fdf7c073a10 legacy=0x7fdf7c163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.540+0000 7fdf73e00640 1 -- 172.21.15.102:0/2803006780 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0c3d90 con 0x7fdf7c0c9eb0 2024-04-20T21:39:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.541+0000 7fdf73e00640 1 --2- 172.21.15.102:0/2803006780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 0x7fdf7c0c4830 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0c7890 tx=0x7fdf6402b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94928c95704f87a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.541+0000 7fdf79400640 1 -- 172.21.15.102:0/2803006780 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64034070 con 0x7fdf7c0c9eb0 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.541+0000 7fdf79400640 1 -- 172.21.15.102:0/2803006780 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf64030890 con 0x7fdf7c0c9eb0 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.541+0000 7fdf79400640 1 -- 172.21.15.102:0/2803006780 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64030c40 con 0x7fdf7c0c9eb0 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.542+0000 7fdf80c00640 1 -- 172.21.15.102:0/2803006780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 msgr2=0x7fdf7c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.542+0000 7fdf80c00640 1 --2- 172.21.15.102:0/2803006780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 0x7fdf7c0c4830 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0c7890 tx=0x7fdf6402b9d0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.542+0000 7fdf80c00640 1 -- 172.21.15.102:0/2803006780 shutdown_connections 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.542+0000 7fdf80c00640 1 --2- 172.21.15.102:0/2803006780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 0x7fdf7c0c4830 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.542+0000 7fdf80c00640 1 -- 172.21.15.102:0/2803006780 >> 172.21.15.102:0/2803006780 conn(0x7fdf7c0c9390 msgr2=0x7fdf7c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.542+0000 7fdf80c00640 1 -- 172.21.15.102:0/2803006780 shutdown_connections 2024-04-20T21:39:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.542+0000 7fdf80c00640 1 -- 172.21.15.102:0/2803006780 wait complete. 2024-04-20T21:39:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.543+0000 7fdf80c00640 1 Processor -- start 2024-04-20T21:39:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.544+0000 7fdf80c00640 1 -- start start 2024-04-20T21:39:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.544+0000 7fdf80c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7c073a10 0x7fdf7c0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.544+0000 7fdf80c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 0x7fdf7c144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.544+0000 7fdf80c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf7c14fa50 0x7fdf7c14a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.544+0000 7fdf80c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c0c4030 con 0x7fdf7c073a10 2024-04-20T21:39:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.544+0000 7fdf80c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c0c7c00 con 0x7fdf7c0c9eb0 2024-04-20T21:39:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf80c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c0c4310 con 0x7fdf7c14fa50 2024-04-20T21:39:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf79e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7c073a10 0x7fdf7c0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf79e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7c073a10 0x7fdf7c0710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39690/0 (socket says 172.21.15.102:39690) 2024-04-20T21:39:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf79e00640 1 -- 172.21.15.102:0/1680798489 learned_addr learned my addr 172.21.15.102:0/1680798489 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf73e00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 0x7fdf7c144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf7a800640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf7c14fa50 0x7fdf7c14a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf79e00640 1 -- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf7c14fa50 msgr2=0x7fdf7c14a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf79e00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf7c14fa50 0x7fdf7c14a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf79e00640 1 -- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 msgr2=0x7fdf7c144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf79e00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 0x7fdf7c144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.545+0000 7fdf79e00640 1 -- 172.21.15.102:0/1680798489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c152740 con 0x7fdf7c073a10 2024-04-20T21:39:53.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.546+0000 7fdf79e00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7c073a10 0x7fdf7c0710a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c440 tx=0x7fdf6c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.546+0000 7fdf72000640 1 -- 172.21.15.102:0/1680798489 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c017070 con 0x7fdf7c073a10 2024-04-20T21:39:53.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.546+0000 7fdf72000640 1 -- 172.21.15.102:0/1680798489 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf6c005ce0 con 0x7fdf7c073a10 2024-04-20T21:39:53.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.546+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c145390 con 0x7fdf7c073a10 2024-04-20T21:39:53.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.546+0000 7fdf72000640 1 -- 172.21.15.102:0/1680798489 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c007270 con 0x7fdf7c073a10 2024-04-20T21:39:53.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.546+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c1529b0 con 0x7fdf7c073a10 2024-04-20T21:39:53.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.547+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7c074340 con 0x7fdf7c073a10 2024-04-20T21:39:53.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.548+0000 7fdf72000640 1 -- 172.21.15.102:0/1680798489 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf6c01a070 con 0x7fdf7c073a10 2024-04-20T21:39:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.549+0000 7fdf72000640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf5806c8f0 0x7fdf5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.549+0000 7fdf72000640 1 -- 172.21.15.102:0/1680798489 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7fdf6c08b560 con 0x7fdf7c073a10 2024-04-20T21:39:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.549+0000 7fdf73e00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf5806c8f0 0x7fdf5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.550+0000 7fdf73e00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf5806c8f0 0x7fdf5806edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdf64036000 tx=0x7fdf6403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.554+0000 7fdf72000640 1 -- 172.21.15.102:0/1680798489 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdf6c058b20 con 0x7fdf7c073a10 2024-04-20T21:39:53.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.791+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf7c145af0 con 0x7fdf7c073a10 2024-04-20T21:39:53.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.792+0000 7fdf72000640 1 -- 172.21.15.102:0/1680798489 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+6027 (secure 0 0 0) 0x7fdf6c05ce20 con 0x7fdf7c073a10 2024-04-20T21:39:53.791 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:53.792 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:49.366546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":14,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:53.792 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 76 2024-04-20T21:39:53.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf5806c8f0 msgr2=0x7fdf5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf5806c8f0 0x7fdf5806edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdf64036000 tx=0x7fdf6403b040 comp rx=0 tx=0).stop 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7c073a10 msgr2=0x7fdf7c0710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7c073a10 0x7fdf7c0710a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c440 tx=0x7fdf6c00c900 comp rx=0 tx=0).stop 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 shutdown_connections 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf7c14fa50 0x7fdf7c14a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fdf5806c8f0 0x7fdf5806edb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0c9eb0 0x7fdf7c144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 --2- 172.21.15.102:0/1680798489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7c073a10 0x7fdf7c0710a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.795+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 >> 172.21.15.102:0/1680798489 conn(0x7fdf7c0c9390 msgr2=0x7fdf7c14dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.796+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 shutdown_connections 2024-04-20T21:39:53.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:53.796+0000 7fdf80c00640 1 -- 172.21.15.102:0/1680798489 wait complete. 2024-04-20T21:39:53.814 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:39:55.815 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:56.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.025+0000 7f57ad000640 1 Processor -- start 2024-04-20T21:39:56.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.025+0000 7f57ad000640 1 -- start start 2024-04-20T21:39:56.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.026+0000 7f57ad000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f57a81532a0 0x7f57a80c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:56.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57ad000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f57a80c7e30 0x7f57a80c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:56.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57ad000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 0x7f57a815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57ad000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f57a815cae0 con 0x7f57a81532a0 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57ad000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f57a8159ff0 con 0x7f57a80c7e30 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57ad000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f57a815d4b0 con 0x7f57a8159060 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57a5e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f57a81532a0 0x7f57a80c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57a5e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f57a81532a0 0x7f57a80c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39704/0 (socket says 172.21.15.102:39704) 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57a5e00640 1 -- 172.21.15.102:0/968661167 learned_addr learned my addr 172.21.15.102:0/968661167 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57a6800640 1 --2- 172.21.15.102:0/968661167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 0x7f57a815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f579d400640 1 --2- 172.21.15.102:0/968661167 >> v2:172.21.15.193:3300/0 conn(0x7f57a80c7e30 0x7f57a80c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57a5e00640 1 -- 172.21.15.102:0/968661167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 msgr2=0x7f57a815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.027+0000 7f57a5e00640 1 --2- 172.21.15.102:0/968661167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 0x7f57a815cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.028+0000 7f57a5e00640 1 -- 172.21.15.102:0/968661167 >> v2:172.21.15.193:3300/0 conn(0x7f57a80c7e30 msgr2=0x7f57a80c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.028+0000 7f57a5e00640 1 --2- 172.21.15.102:0/968661167 >> v2:172.21.15.193:3300/0 conn(0x7f57a80c7e30 0x7f57a80c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.028+0000 7f57a5e00640 1 -- 172.21.15.102:0/968661167 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a80c9150 con 0x7f57a81532a0 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.029+0000 7f57a5e00640 1 --2- 172.21.15.102:0/968661167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a81532a0 0x7f57a80c85f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f57a80c48a0 tx=0x7f579802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f78261b543ce172 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.029+0000 7f57a5400640 1 -- 172.21.15.102:0/968661167 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798034070 con 0x7f57a81532a0 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.029+0000 7f57a5400640 1 -- 172.21.15.102:0/968661167 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5798030860 con 0x7f57a81532a0 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.029+0000 7f57a5400640 1 -- 172.21.15.102:0/968661167 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798030c30 con 0x7f57a81532a0 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.029+0000 7f57ad000640 1 -- 172.21.15.102:0/968661167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a81532a0 msgr2=0x7f57a80c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.029+0000 7f57ad000640 1 --2- 172.21.15.102:0/968661167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a81532a0 0x7f57a80c85f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f57a80c48a0 tx=0x7f579802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.030+0000 7f57ad000640 1 -- 172.21.15.102:0/968661167 shutdown_connections 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.030+0000 7f57ad000640 1 --2- 172.21.15.102:0/968661167 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 0x7f57a815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.030+0000 7f57ad000640 1 --2- 172.21.15.102:0/968661167 >> v2:172.21.15.193:3300/0 conn(0x7f57a80c7e30 0x7f57a80c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.030+0000 7f57ad000640 1 --2- 172.21.15.102:0/968661167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a81532a0 0x7f57a80c85f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.030+0000 7f57ad000640 1 -- 172.21.15.102:0/968661167 >> 172.21.15.102:0/968661167 conn(0x7f57a806f9f0 msgr2=0x7f57a806fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.030+0000 7f57ad000640 1 -- 172.21.15.102:0/968661167 shutdown_connections 2024-04-20T21:39:56.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.030+0000 7f57ad000640 1 -- 172.21.15.102:0/968661167 wait complete. 2024-04-20T21:39:56.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.031+0000 7f57ad000640 1 Processor -- start 2024-04-20T21:39:56.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.031+0000 7f57ad000640 1 -- start start 2024-04-20T21:39:56.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.032+0000 7f57ad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a80c7e30 0x7f57a81f14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:56.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.032+0000 7f57ad000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57a81532a0 0x7f57a814aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:56.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.032+0000 7f57ad000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 0x7f57a815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:56.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.032+0000 7f57ad000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f57a815c770 con 0x7f57a80c7e30 2024-04-20T21:39:56.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.032+0000 7f57ad000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57a80c9710 con 0x7f57a81532a0 2024-04-20T21:39:56.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.032+0000 7f57ad000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f57a8153680 con 0x7f57a8159060 2024-04-20T21:39:56.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a80c7e30 0x7f57a81f14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a80c7e30 0x7f57a81f14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39712/0 (socket says 172.21.15.102:39712) 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 -- 172.21.15.102:0/4079391366 learned_addr learned my addr 172.21.15.102:0/4079391366 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f579d400640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57a81532a0 0x7f57a814aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a6800640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 0x7f57a815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 -- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 msgr2=0x7f57a815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 0x7f57a815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 -- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57a81532a0 msgr2=0x7f57a814aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57a81532a0 0x7f57a814aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 -- 172.21.15.102:0/4079391366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a80739e0 con 0x7f57a80c7e30 2024-04-20T21:39:56.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.033+0000 7f57a5e00640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a80c7e30 0x7f57a81f14e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5798000c00 tx=0x7f5798004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:56.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.034+0000 7f579ea00640 1 -- 172.21.15.102:0/4079391366 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798048070 con 0x7f57a80c7e30 2024-04-20T21:39:56.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.034+0000 7f579ea00640 1 -- 172.21.15.102:0/4079391366 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5798030660 con 0x7f57a80c7e30 2024-04-20T21:39:56.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.034+0000 7f579ea00640 1 -- 172.21.15.102:0/4079391366 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798034040 con 0x7f57a80c7e30 2024-04-20T21:39:56.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.034+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a8073c80 con 0x7f57a80c7e30 2024-04-20T21:39:56.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.034+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a815ab80 con 0x7f57a80c7e30 2024-04-20T21:39:56.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.035+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a8068c30 con 0x7f57a80c7e30 2024-04-20T21:39:56.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.036+0000 7f579ea00640 1 -- 172.21.15.102:0/4079391366 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f579804b070 con 0x7f57a80c7e30 2024-04-20T21:39:56.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.037+0000 7f579ea00640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f578406c920 0x7f578406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:56.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.038+0000 7f579ea00640 1 -- 172.21.15.102:0/4079391366 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f57980ae2d0 con 0x7f57a80c7e30 2024-04-20T21:39:56.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.038+0000 7f579d400640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f578406c920 0x7f578406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:56.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.039+0000 7f579d400640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f578406c920 0x7f578406ede0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f57a815c5f0 tx=0x7f5790009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:56.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.042+0000 7f579ea00640 1 -- 172.21.15.102:0/4079391366 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5798030830 con 0x7f57a80c7e30 2024-04-20T21:39:56.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.286+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57a806e670 con 0x7f57a80c7e30 2024-04-20T21:39:56.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.287+0000 7f579ea00640 1 -- 172.21.15.102:0/4079391366 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+6027 (secure 0 0 0) 0x7f579807b800 con 0x7f57a80c7e30 2024-04-20T21:39:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:49.366546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":14,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:56.286 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 76 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f578406c920 msgr2=0x7f578406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f578406c920 0x7f578406ede0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f57a815c5f0 tx=0x7f5790009190 comp rx=0 tx=0).stop 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a80c7e30 msgr2=0x7f57a81f14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a80c7e30 0x7f57a81f14e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5798000c00 tx=0x7f5798004730 comp rx=0 tx=0).stop 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 shutdown_connections 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57a8159060 0x7f57a815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f578406c920 0x7f578406ede0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57a81532a0 0x7f57a814aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 --2- 172.21.15.102:0/4079391366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a80c7e30 0x7f57a81f14e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.289+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 >> 172.21.15.102:0/4079391366 conn(0x7f57a806f9f0 msgr2=0x7f57a8158080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.290+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 shutdown_connections 2024-04-20T21:39:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:56.290+0000 7f57ad000640 1 -- 172.21.15.102:0/4079391366 wait complete. 2024-04-20T21:39:56.309 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 76, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 76, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6429": { "addr": "172.21.15.102:6835/3586768191", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6834", "nonce": 3586768191, "type": "v2" }, { "addr": "172.21.15.102:6835", "nonce": 3586768191, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6429, "incarnation": 74, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:rejoin", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:39:49.366546+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6429 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:39:58.310 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:58.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.518+0000 7f044a400640 1 Processor -- start 2024-04-20T21:39:58.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.519+0000 7f044a400640 1 -- start start 2024-04-20T21:39:58.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.520+0000 7f044a400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f044413bc80 0x7f04440c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:58.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.520+0000 7f044a400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f04440737f0 0x7f0444159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:58.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.520+0000 7f044a400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04440c6ec0 con 0x7f0444073c30 2024-04-20T21:39:58.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.520+0000 7f044a400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f04440c6a30 con 0x7f044413bc80 2024-04-20T21:39:58.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.520+0000 7f044a400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f04440c8d00 con 0x7f04440737f0 2024-04-20T21:39:58.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.520+0000 7f0443400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f044413bc80 0x7f04440c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:58.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.520+0000 7f0443e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0444073c30 0x7f04440c98f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60936/0 (socket says 172.21.15.102:60936) 2024-04-20T21:39:58.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0443e00640 1 -- 172.21.15.102:0/1440403030 learned_addr learned my addr 172.21.15.102:0/1440403030 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:39:58.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0443400640 1 --2- 172.21.15.102:0/1440403030 >> v2:172.21.15.102:3300/0 conn(0x7f044413bc80 0x7f04440c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53386/0 (socket says 172.21.15.102:53386) 2024-04-20T21:39:58.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0442a00640 1 --2- 172.21.15.102:0/1440403030 >> v2:172.21.15.193:3300/0 conn(0x7f04440737f0 0x7f0444159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:58.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0443400640 1 -- 172.21.15.102:0/1440403030 >> v2:172.21.15.193:3300/0 conn(0x7f04440737f0 msgr2=0x7f0444159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:58.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0443400640 1 --2- 172.21.15.102:0/1440403030 >> v2:172.21.15.193:3300/0 conn(0x7f04440737f0 0x7f0444159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0443400640 1 -- 172.21.15.102:0/1440403030 >> v1:172.21.15.102:6789/0 conn(0x7f0444073c30 legacy=0x7f04440c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:58.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0443400640 1 -- 172.21.15.102:0/1440403030 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04440c8fc0 con 0x7f044413bc80 2024-04-20T21:39:58.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0442000640 1 -- 172.21.15.102:0/1440403030 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 324128451 0 0) 0x7f04440c6ec0 con 0x7f0444073c30 2024-04-20T21:39:58.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.521+0000 7f0442a00640 1 --2- 172.21.15.102:0/1440403030 >> v2:172.21.15.193:3300/0 conn(0x7f04440737f0 0x7f0444159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:58.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.522+0000 7f0443400640 1 --2- 172.21.15.102:0/1440403030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f044413bc80 0x7f04440c84e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f044406a220 tx=0x7f043402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9622ee7dc625cff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:58.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.522+0000 7f0442000640 1 -- 172.21.15.102:0/1440403030 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434035020 con 0x7f044413bc80 2024-04-20T21:39:58.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.522+0000 7f0442000640 1 -- 172.21.15.102:0/1440403030 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0434031830 con 0x7f044413bc80 2024-04-20T21:39:58.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.522+0000 7f0442000640 1 -- 172.21.15.102:0/1440403030 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434031c00 con 0x7f044413bc80 2024-04-20T21:39:58.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.523+0000 7f044a400640 1 -- 172.21.15.102:0/1440403030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f044413bc80 msgr2=0x7f04440c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:58.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.523+0000 7f044a400640 1 --2- 172.21.15.102:0/1440403030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f044413bc80 0x7f04440c84e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f044406a220 tx=0x7f043402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.523+0000 7f044a400640 1 -- 172.21.15.102:0/1440403030 shutdown_connections 2024-04-20T21:39:58.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.523+0000 7f044a400640 1 --2- 172.21.15.102:0/1440403030 >> v2:172.21.15.193:3300/0 conn(0x7f04440737f0 0x7f0444159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.523+0000 7f044a400640 1 --2- 172.21.15.102:0/1440403030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f044413bc80 0x7f04440c84e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.523+0000 7f044a400640 1 -- 172.21.15.102:0/1440403030 >> 172.21.15.102:0/1440403030 conn(0x7f044406ebf0 msgr2=0x7f044406efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:58.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.523+0000 7f044a400640 1 -- 172.21.15.102:0/1440403030 shutdown_connections 2024-04-20T21:39:58.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.523+0000 7f044a400640 1 -- 172.21.15.102:0/1440403030 wait complete. 2024-04-20T21:39:58.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.525+0000 7f044a400640 1 Processor -- start 2024-04-20T21:39:58.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.525+0000 7f044a400640 1 -- start start 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f044a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04440737f0 0x7f04441556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f044a400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0444073c30 0x7f0444155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f044a400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f044413bc80 0x7f04440ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f044a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f04440c66c0 con 0x7f04440737f0 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f044a400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04440c63e0 con 0x7f044413bc80 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f044a400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f04440c8a20 con 0x7f0444073c30 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f0443400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04440737f0 0x7f04441556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f0443400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04440737f0 0x7f04441556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53396/0 (socket says 172.21.15.102:53396) 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f0443400640 1 -- 172.21.15.102:0/611205750 learned_addr learned my addr 172.21.15.102:0/611205750 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f0443e00640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f044413bc80 0x7f04440ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.526+0000 7f0442a00640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0444073c30 0x7f0444155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f0443400640 1 -- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0444073c30 msgr2=0x7f0444155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:58.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f0443400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0444073c30 0x7f0444155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f0443400640 1 -- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f044413bc80 msgr2=0x7f04440ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:58.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f0443400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f044413bc80 0x7f04440ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f0443400640 1 -- 172.21.15.102:0/611205750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04440cd880 con 0x7f04440737f0 2024-04-20T21:39:58.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f0443400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04440737f0 0x7f04441556e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0434034040 tx=0x7f04340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:58.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f043b400640 1 -- 172.21.15.102:0/611205750 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434035040 con 0x7f04440737f0 2024-04-20T21:39:58.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f043b400640 1 -- 172.21.15.102:0/611205750 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0434004230 con 0x7f04440737f0 2024-04-20T21:39:58.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04441edf20 con 0x7f04440737f0 2024-04-20T21:39:58.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.527+0000 7f043b400640 1 -- 172.21.15.102:0/611205750 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043403a640 con 0x7f04440737f0 2024-04-20T21:39:58.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.528+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04440ccce0 con 0x7f04440737f0 2024-04-20T21:39:58.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.529+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04440c7d00 con 0x7f04440737f0 2024-04-20T21:39:58.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.530+0000 7f043b400640 1 -- 172.21.15.102:0/611205750 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0434038020 con 0x7f04440737f0 2024-04-20T21:39:58.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.531+0000 7f043b400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f042406c8f0 0x7f042406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:58.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.531+0000 7f043b400640 1 -- 172.21.15.102:0/611205750 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f04340adfe0 con 0x7f04440737f0 2024-04-20T21:39:58.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.531+0000 7f0442a00640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f042406c8f0 0x7f042406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:58.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.532+0000 7f0442a00640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f042406c8f0 0x7f042406edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f04440cf290 tx=0x7f0430009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:58.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.535+0000 7f043b400640 1 -- 172.21.15.102:0/611205750 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04440c7d00 con 0x7f04440737f0 2024-04-20T21:39:58.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.770+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04440ca570 con 0x7f04440737f0 2024-04-20T21:39:58.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.771+0000 7f043b400640 1 -- 172.21.15.102:0/611205750 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+6033 (secure 0 0 0) 0x7f04440ca570 con 0x7f04440737f0 2024-04-20T21:39:58.770 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:39:58.770 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:55.521887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:clientreplay","state_seq":16,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:39:58.770 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 77 2024-04-20T21:39:58.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f042406c8f0 msgr2=0x7f042406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:58.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f042406c8f0 0x7f042406edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f04440cf290 tx=0x7f0430009190 comp rx=0 tx=0).stop 2024-04-20T21:39:58.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04440737f0 msgr2=0x7f04441556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:58.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04440737f0 0x7f04441556e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0434034040 tx=0x7f04340046c0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 shutdown_connections 2024-04-20T21:39:58.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f042406c8f0 0x7f042406edb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f044413bc80 0x7f04440ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0444073c30 0x7f0444155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 --2- 172.21.15.102:0/611205750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04440737f0 0x7f04441556e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:58.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 >> 172.21.15.102:0/611205750 conn(0x7f044406ebf0 msgr2=0x7f044406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:58.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 shutdown_connections 2024-04-20T21:39:58.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:39:58.774+0000 7f044a400640 1 -- 172.21.15.102:0/611205750 wait complete. 2024-04-20T21:39:58.790 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:40:00.367 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:40:00.375 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:40:00.792 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:40:01.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.003+0000 7fa710600640 1 Processor -- start 2024-04-20T21:40:01.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.003+0000 7fa710600640 1 -- start start 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.004+0000 7fa710600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa7080c9eb0 0x7fa7080c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.004+0000 7fa710600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa708073a10 0x7fa708158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.004+0000 7fa710600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7081536d0 con 0x7fa70814fa50 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.004+0000 7fa710600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa708159500 con 0x7fa7080c9eb0 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.004+0000 7fa710600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa7080c76c0 con 0x7fa708073a10 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.005+0000 7fa70d200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa7080c9eb0 0x7fa7080c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.005+0000 7fa70d200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa7080c9eb0 0x7fa7080c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53412/0 (socket says 172.21.15.102:53412) 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.005+0000 7fa70d200640 1 -- 172.21.15.102:0/181018384 learned_addr learned my addr 172.21.15.102:0/181018384 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:40:01.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.005+0000 7fa70e600640 1 --2- 172.21.15.102:0/181018384 >> v2:172.21.15.193:3300/0 conn(0x7fa708073a10 0x7fa708158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.005+0000 7fa70d200640 1 -- 172.21.15.102:0/181018384 >> v2:172.21.15.193:3300/0 conn(0x7fa708073a10 msgr2=0x7fa708158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.005+0000 7fa70d200640 1 --2- 172.21.15.102:0/181018384 >> v2:172.21.15.193:3300/0 conn(0x7fa708073a10 0x7fa708158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.005+0000 7fa70d200640 1 -- 172.21.15.102:0/181018384 >> v1:172.21.15.193:6789/0 conn(0x7fa70814fa50 legacy=0x7fa7080c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.005+0000 7fa70d200640 1 -- 172.21.15.102:0/181018384 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7080c7960 con 0x7fa7080c9eb0 2024-04-20T21:40:01.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.006+0000 7fa70d200640 1 --2- 172.21.15.102:0/181018384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7080c9eb0 0x7fa7080c4830 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa708156b00 tx=0x7fa6f402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa50b4b487e5a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.007+0000 7fa707e00640 1 -- 172.21.15.102:0/181018384 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f402f850 con 0x7fa7080c9eb0 2024-04-20T21:40:01.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.007+0000 7fa707e00640 1 -- 172.21.15.102:0/181018384 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f402f9f0 con 0x7fa7080c9eb0 2024-04-20T21:40:01.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.007+0000 7fa707e00640 1 -- 172.21.15.102:0/181018384 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f402fdc0 con 0x7fa7080c9eb0 2024-04-20T21:40:01.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.008+0000 7fa710600640 1 -- 172.21.15.102:0/181018384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7080c9eb0 msgr2=0x7fa7080c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.008+0000 7fa710600640 1 --2- 172.21.15.102:0/181018384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7080c9eb0 0x7fa7080c4830 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa708156b00 tx=0x7fa6f402ba30 comp rx=0 tx=0).stop 2024-04-20T21:40:01.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.008+0000 7fa710600640 1 -- 172.21.15.102:0/181018384 shutdown_connections 2024-04-20T21:40:01.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.008+0000 7fa710600640 1 --2- 172.21.15.102:0/181018384 >> v2:172.21.15.193:3300/0 conn(0x7fa708073a10 0x7fa708158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.008+0000 7fa710600640 1 --2- 172.21.15.102:0/181018384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7080c9eb0 0x7fa7080c4830 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.008+0000 7fa710600640 1 -- 172.21.15.102:0/181018384 >> 172.21.15.102:0/181018384 conn(0x7fa7080c9390 msgr2=0x7fa7080c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.008+0000 7fa710600640 1 -- 172.21.15.102:0/181018384 shutdown_connections 2024-04-20T21:40:01.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.008+0000 7fa710600640 1 -- 172.21.15.102:0/181018384 wait complete. 2024-04-20T21:40:01.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.010+0000 7fa710600640 1 Processor -- start 2024-04-20T21:40:01.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.010+0000 7fa710600640 1 -- start start 2024-04-20T21:40:01.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa710600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708073a10 0x7fa7080c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa710600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080c9eb0 0x7fa708072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa710600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa70814fa50 0x7fa70815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa710600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa7080c7c00 con 0x7fa70814fa50 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa710600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7080c3d90 con 0x7fa708073a10 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa710600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa7080c4030 con 0x7fa7080c9eb0 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa70814fa50 0x7fa70815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa70814fa50 0x7fa70815a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53426/0 (socket says 172.21.15.102:53426) 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70e600640 1 -- 172.21.15.102:0/3837703051 learned_addr learned my addr 172.21.15.102:0/3837703051 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70d200640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080c9eb0 0x7fa708072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70dc00640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708073a10 0x7fa7080c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70e600640 1 -- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080c9eb0 msgr2=0x7fa708072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70e600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080c9eb0 0x7fa708072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70e600640 1 -- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708073a10 msgr2=0x7fa7080c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70e600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708073a10 0x7fa7080c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.011+0000 7fa70e600640 1 -- 172.21.15.102:0/3837703051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70815c380 con 0x7fa70814fa50 2024-04-20T21:40:01.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.012+0000 7fa70e600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa70814fa50 0x7fa70815a6a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa700009dd0 tx=0x7fa70000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.012+0000 7fa705600640 1 -- 172.21.15.102:0/3837703051 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700018020 con 0x7fa70814fa50 2024-04-20T21:40:01.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.012+0000 7fa705600640 1 -- 172.21.15.102:0/3837703051 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa700014480 con 0x7fa70814fa50 2024-04-20T21:40:01.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.012+0000 7fa705600640 1 -- 172.21.15.102:0/3837703051 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700014870 con 0x7fa70814fa50 2024-04-20T21:40:01.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.012+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7080c2ab0 con 0x7fa70814fa50 2024-04-20T21:40:01.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.012+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70815c8d0 con 0x7fa70814fa50 2024-04-20T21:40:01.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.013+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa708074340 con 0x7fa70814fa50 2024-04-20T21:40:01.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.014+0000 7fa705600640 1 -- 172.21.15.102:0/3837703051 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa700005d10 con 0x7fa70814fa50 2024-04-20T21:40:01.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.016+0000 7fa705600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6e806c920 0x7fa6e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.016+0000 7fa705600640 1 -- 172.21.15.102:0/3837703051 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7fa70008ccf0 con 0x7fa70814fa50 2024-04-20T21:40:01.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.016+0000 7fa70dc00640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6e806c920 0x7fa6e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.017+0000 7fa70dc00640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6e806c920 0x7fa6e806ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0016e0 tx=0x7fa6fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.021+0000 7fa705600640 1 -- 172.21.15.102:0/3837703051 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa700022070 con 0x7fa70814fa50 2024-04-20T21:40:01.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.266+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7080c7320 con 0x7fa70814fa50 2024-04-20T21:40:01.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.268+0000 7fa705600640 1 -- 172.21.15.102:0/3837703051 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+6033 (secure 0 0 0) 0x7fa700010040 con 0x7fa70814fa50 2024-04-20T21:40:01.267 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:40:01.267 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:39:55.521887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:clientreplay","state_seq":16,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:40:01.267 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 77 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.270+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6e806c920 msgr2=0x7fa6e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.270+0000 7fa710600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6e806c920 0x7fa6e806ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0016e0 tx=0x7fa6fc009040 comp rx=0 tx=0).stop 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.270+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa70814fa50 msgr2=0x7fa70815a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.270+0000 7fa710600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa70814fa50 0x7fa70815a6a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa700009dd0 tx=0x7fa70000c490 comp rx=0 tx=0).stop 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.271+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 shutdown_connections 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.271+0000 7fa710600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa70814fa50 0x7fa70815a6a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.271+0000 7fa710600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080c9eb0 0x7fa708072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.271+0000 7fa710600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa6e806c920 0x7fa6e806ede0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.271+0000 7fa710600640 1 --2- 172.21.15.102:0/3837703051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708073a10 0x7fa7080c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.271+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 >> 172.21.15.102:0/3837703051 conn(0x7fa7080c9390 msgr2=0x7fa70814be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.271+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 shutdown_connections 2024-04-20T21:40:01.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:01.271+0000 7fa710600640 1 -- 172.21.15.102:0/3837703051 wait complete. 2024-04-20T21:40:01.290 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:40:03.291 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:40:03.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.502+0000 7fde16200640 1 Processor -- start 2024-04-20T21:40:03.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.502+0000 7fde16200640 1 -- start start 2024-04-20T21:40:03.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.503+0000 7fde16200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fde10152170 0x7fde100c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.503+0000 7fde16200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fde10158770 0x7fde10070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.503+0000 7fde16200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde10071f70 con 0x7fde10158b80 2024-04-20T21:40:03.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.503+0000 7fde16200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fde100c6dc0 con 0x7fde10152170 2024-04-20T21:40:03.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.503+0000 7fde16200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fde10072210 con 0x7fde10158770 2024-04-20T21:40:03.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fde10152170 0x7fde100c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0fe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fde10158b80 0x7fde1006c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49328/0 (socket says 172.21.15.102:49328) 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0fe00640 1 -- 172.21.15.102:0/1296202882 learned_addr learned my addr 172.21.15.102:0/1296202882 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0ea00640 1 --2- 172.21.15.102:0/1296202882 >> v2:172.21.15.193:3300/0 conn(0x7fde10158770 0x7fde10070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0e000640 1 -- 172.21.15.102:0/1296202882 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368876966 0 0) 0x7fde10071f70 con 0x7fde10158b80 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0e000640 1 -- 172.21.15.102:0/1296202882 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf4003520 con 0x7fde10158b80 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0f400640 1 -- 172.21.15.102:0/1296202882 >> v2:172.21.15.193:3300/0 conn(0x7fde10158770 msgr2=0x7fde10070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0f400640 1 --2- 172.21.15.102:0/1296202882 >> v2:172.21.15.193:3300/0 conn(0x7fde10158770 0x7fde10070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0f400640 1 -- 172.21.15.102:0/1296202882 >> v1:172.21.15.193:6789/0 conn(0x7fde10158b80 legacy=0x7fde1006c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.504+0000 7fde0f400640 1 -- 172.21.15.102:0/1296202882 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde100724b0 con 0x7fde10152170 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.505+0000 7fde0f400640 1 --2- 172.21.15.102:0/1296202882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde100c8db0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fde1006de80 tx=0x7fde0002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ae5cf99ffe789 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde0e000640 1 -- 172.21.15.102:0/1296202882 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00034070 con 0x7fde10152170 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde0e000640 1 -- 172.21.15.102:0/1296202882 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde00030610 con 0x7fde10152170 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde0e000640 1 -- 172.21.15.102:0/1296202882 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde000309e0 con 0x7fde10152170 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde16200640 1 -- 172.21.15.102:0/1296202882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 msgr2=0x7fde100c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde16200640 1 --2- 172.21.15.102:0/1296202882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde100c8db0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fde1006de80 tx=0x7fde0002b960 comp rx=0 tx=0).stop 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde16200640 1 -- 172.21.15.102:0/1296202882 shutdown_connections 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde16200640 1 --2- 172.21.15.102:0/1296202882 >> v2:172.21.15.193:3300/0 conn(0x7fde10158770 0x7fde10070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde16200640 1 --2- 172.21.15.102:0/1296202882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde100c8db0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.506+0000 7fde16200640 1 -- 172.21.15.102:0/1296202882 >> 172.21.15.102:0/1296202882 conn(0x7fde100c9390 msgr2=0x7fde100c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.507+0000 7fde16200640 1 -- 172.21.15.102:0/1296202882 shutdown_connections 2024-04-20T21:40:03.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.507+0000 7fde16200640 1 -- 172.21.15.102:0/1296202882 wait complete. 2024-04-20T21:40:03.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.508+0000 7fde16200640 1 Processor -- start 2024-04-20T21:40:03.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.508+0000 7fde16200640 1 -- start start 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.509+0000 7fde16200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde1014e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.509+0000 7fde16200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde10158770 0x7fde100c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.509+0000 7fde16200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde10158b80 0x7fde101564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.509+0000 7fde16200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde100c9f20 con 0x7fde10152170 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.509+0000 7fde16200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fde10158250 con 0x7fde10158770 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.509+0000 7fde16200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fde10157f20 con 0x7fde10158b80 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.509+0000 7fde0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde1014e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde1014e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53444/0 (socket says 172.21.15.102:53444) 2024-04-20T21:40:03.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0f400640 1 -- 172.21.15.102:0/1085333650 learned_addr learned my addr 172.21.15.102:0/1085333650 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:40:03.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0fe00640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde10158b80 0x7fde101564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0ea00640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde10158770 0x7fde100c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0f400640 1 -- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde10158b80 msgr2=0x7fde101564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0f400640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde10158b80 0x7fde101564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0f400640 1 -- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde10158770 msgr2=0x7fde100c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0f400640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde10158770 0x7fde100c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0f400640 1 -- 172.21.15.102:0/1085333650 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde10146060 con 0x7fde10152170 2024-04-20T21:40:03.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.510+0000 7fde0f400640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde1014e550 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fde00009890 tx=0x7fde000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:03.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.511+0000 7fde07400640 1 -- 172.21.15.102:0/1085333650 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00046020 con 0x7fde10152170 2024-04-20T21:40:03.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.511+0000 7fde07400640 1 -- 172.21.15.102:0/1085333650 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde00030410 con 0x7fde10152170 2024-04-20T21:40:03.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.511+0000 7fde07400640 1 -- 172.21.15.102:0/1085333650 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00034040 con 0x7fde10152170 2024-04-20T21:40:03.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.511+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde10146320 con 0x7fde10152170 2024-04-20T21:40:03.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.511+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde101ef740 con 0x7fde10152170 2024-04-20T21:40:03.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.512+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde10074340 con 0x7fde10152170 2024-04-20T21:40:03.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.513+0000 7fde07400640 1 -- 172.21.15.102:0/1085333650 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde00037070 con 0x7fde10152170 2024-04-20T21:40:03.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.514+0000 7fde07400640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fddf006c8f0 0x7fddf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.514+0000 7fde07400640 1 -- 172.21.15.102:0/1085333650 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7fde000adf10 con 0x7fde10152170 2024-04-20T21:40:03.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.514+0000 7fde0ea00640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fddf006c8f0 0x7fddf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.515+0000 7fde0ea00640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fddf006c8f0 0x7fddf006edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde101481c0 tx=0x7fddfc009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:03.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.519+0000 7fde07400640 1 -- 172.21.15.102:0/1085333650 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fde00050020 con 0x7fde10152170 2024-04-20T21:40:03.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.759+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde100c3460 con 0x7fde10152170 2024-04-20T21:40:03.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.760+0000 7fde07400640 1 -- 172.21.15.102:0/1085333650 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+6034 (secure 0 0 0) 0x7fde0007b4d0 con 0x7fde10152170 2024-04-20T21:40:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:40:03.759 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:40:03.760 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 78 2024-04-20T21:40:03.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fddf006c8f0 msgr2=0x7fddf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fddf006c8f0 0x7fddf006edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde101481c0 tx=0x7fddfc009190 comp rx=0 tx=0).stop 2024-04-20T21:40:03.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 msgr2=0x7fde1014e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde1014e550 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fde00009890 tx=0x7fde000046c0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 shutdown_connections 2024-04-20T21:40:03.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde10158b80 0x7fde101564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fddf006c8f0 0x7fddf006edb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde10158770 0x7fde100c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 --2- 172.21.15.102:0/1085333650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde10152170 0x7fde1014e550 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 >> 172.21.15.102:0/1085333650 conn(0x7fde100c9390 msgr2=0x7fde1015b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:03.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 shutdown_connections 2024-04-20T21:40:03.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.763+0000 7fde16200640 1 -- 172.21.15.102:0/1085333650 wait complete. 2024-04-20T21:40:03.780 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=6421 2024-04-20T21:40:03.780 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.a 2024-04-20T21:40:03.780 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-04-20T21:40:03.780 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:40:03.780 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:40:03.786 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:40:03.786 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:40:03.933 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-04-20T21:40:03.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.965+0000 7f3d25000640 1 Processor -- start 2024-04-20T21:40:03.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.966+0000 7f3d25000640 1 -- start start 2024-04-20T21:40:03.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d25000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3d201554f0 0x7f3d2015c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d25000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3d2006c550 0x7f3d200c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d25000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d200c9e30 0x7f3d200ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d25000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3d20073a20 con 0x7f3d2006c550 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d25000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f3d2015b510 con 0x7f3d201554f0 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d25000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3d20073cc0 con 0x7f3d200c9e30 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d1d400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3d2006c550 0x7f3d200c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d1d400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3d2006c550 0x7f3d200c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53464/0 (socket says 172.21.15.102:53464) 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d1d400640 1 -- 172.21.15.102:0/2797080232 learned_addr learned my addr 172.21.15.102:0/2797080232 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d1de00640 1 --2- 172.21.15.102:0/2797080232 >> v2:172.21.15.193:3300/0 conn(0x7f3d201554f0 0x7f3d2015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.967+0000 7f3d1e800640 1 --2- 172.21.15.102:0/2797080232 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d200c9e30 0x7f3d200ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.968+0000 7f3d1d400640 1 -- 172.21.15.102:0/2797080232 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d200c9e30 msgr2=0x7f3d200ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.968+0000 7f3d1d400640 1 --2- 172.21.15.102:0/2797080232 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d200c9e30 0x7f3d200ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.968+0000 7f3d1d400640 1 -- 172.21.15.102:0/2797080232 >> v2:172.21.15.193:3300/0 conn(0x7f3d201554f0 msgr2=0x7f3d2015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:03.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.968+0000 7f3d1d400640 1 --2- 172.21.15.102:0/2797080232 >> v2:172.21.15.193:3300/0 conn(0x7f3d201554f0 0x7f3d2015c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.968+0000 7f3d1d400640 1 -- 172.21.15.102:0/2797080232 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2015b7d0 con 0x7f3d2006c550 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.969+0000 7f3d1d400640 1 --2- 172.21.15.102:0/2797080232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d2006c550 0x7f3d200c98f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3d200ceb60 tx=0x7f3d0c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cead58047233a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.969+0000 7f3d1ca00640 1 -- 172.21.15.102:0/2797080232 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c035020 con 0x7f3d2006c550 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.969+0000 7f3d1ca00640 1 -- 172.21.15.102:0/2797080232 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c02d040 con 0x7f3d2006c550 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.969+0000 7f3d1ca00640 1 -- 172.21.15.102:0/2797080232 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c031a80 con 0x7f3d2006c550 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.969+0000 7f3d25000640 1 -- 172.21.15.102:0/2797080232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d2006c550 msgr2=0x7f3d200c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.969+0000 7f3d25000640 1 --2- 172.21.15.102:0/2797080232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d2006c550 0x7f3d200c98f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3d200ceb60 tx=0x7f3d0c02b900 comp rx=0 tx=0).stop 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.970+0000 7f3d25000640 1 -- 172.21.15.102:0/2797080232 shutdown_connections 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.970+0000 7f3d25000640 1 --2- 172.21.15.102:0/2797080232 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d200c9e30 0x7f3d200ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.970+0000 7f3d25000640 1 --2- 172.21.15.102:0/2797080232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d2006c550 0x7f3d200c98f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.970+0000 7f3d25000640 1 --2- 172.21.15.102:0/2797080232 >> v2:172.21.15.193:3300/0 conn(0x7f3d201554f0 0x7f3d2015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.970+0000 7f3d25000640 1 -- 172.21.15.102:0/2797080232 >> 172.21.15.102:0/2797080232 conn(0x7f3d2006ebf0 msgr2=0x7f3d2006efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.970+0000 7f3d25000640 1 -- 172.21.15.102:0/2797080232 shutdown_connections 2024-04-20T21:40:03.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.970+0000 7f3d25000640 1 -- 172.21.15.102:0/2797080232 wait complete. 2024-04-20T21:40:03.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.971+0000 7f3d25000640 1 Processor -- start 2024-04-20T21:40:03.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.971+0000 7f3d25000640 1 -- start start 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.972+0000 7f3d25000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d2006c550 0x7f3d201f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.972+0000 7f3d25000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d200c9e30 0x7f3d201f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.972+0000 7f3d25000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d201554f0 0x7f3d2014fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.972+0000 7f3d25000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3d201558d0 con 0x7f3d200c9e30 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.972+0000 7f3d25000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d200736e0 con 0x7f3d2006c550 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.972+0000 7f3d25000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3d20155a50 con 0x7f3d201554f0 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d200c9e30 0x7f3d201f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d200c9e30 0x7f3d201f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53470/0 (socket says 172.21.15.102:53470) 2024-04-20T21:40:03.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 -- 172.21.15.102:0/1275733601 learned_addr learned my addr 172.21.15.102:0/1275733601 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:40:03.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1e800640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d201554f0 0x7f3d2014fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1de00640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d2006c550 0x7f3d201f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 -- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d201554f0 msgr2=0x7f3d2014fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d201554f0 0x7f3d2014fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 -- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d2006c550 msgr2=0x7f3d201f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:03.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d2006c550 0x7f3d201f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:03.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 -- 172.21.15.102:0/1275733601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d201f2910 con 0x7f3d200c9e30 2024-04-20T21:40:03.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.973+0000 7f3d1d400640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d200c9e30 0x7f3d201f4740 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c000c00 tx=0x7f3d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:03.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.974+0000 7f3d16000640 1 -- 172.21.15.102:0/1275733601 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c035040 con 0x7f3d200c9e30 2024-04-20T21:40:03.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.974+0000 7f3d16000640 1 -- 172.21.15.102:0/1275733601 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c0399f0 con 0x7f3d200c9e30 2024-04-20T21:40:03.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.974+0000 7f3d16000640 1 -- 172.21.15.102:0/1275733601 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c039dc0 con 0x7f3d200c9e30 2024-04-20T21:40:03.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.974+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d201f5220 con 0x7f3d200c9e30 2024-04-20T21:40:03.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.974+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2015e690 con 0x7f3d200c9e30 2024-04-20T21:40:03.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.975+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d2015f860 con 0x7f3d200c9e30 2024-04-20T21:40:03.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.976+0000 7f3d16000640 1 -- 172.21.15.102:0/1275733601 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d0c038020 con 0x7f3d200c9e30 2024-04-20T21:40:03.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.977+0000 7f3d16000640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3d0006c8f0 0x7f3d0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:03.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.977+0000 7f3d16000640 1 -- 172.21.15.102:0/1275733601 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f3d0c0ae270 con 0x7f3d200c9e30 2024-04-20T21:40:03.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.977+0000 7f3d1de00640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3d0006c8f0 0x7f3d0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:03.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.978+0000 7f3d1de00640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3d0006c8f0 0x7f3d0006edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009aa0 tx=0x7f3d10009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:03.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:03.981+0000 7f3d16000640 1 -- 172.21.15.102:0/1275733601 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3d2015f860 con 0x7f3d200c9e30 2024-04-20T21:40:04.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.218+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d20074340 con 0x7f3d200c9e30 2024-04-20T21:40:04.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.219+0000 7f3d16000640 1 -- 172.21.15.102:0/1275733601 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+6034 (secure 0 0 0) 0x7f3d0c07b7a0 con 0x7f3d200c9e30 2024-04-20T21:40:04.218 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:40:04.218 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:40:04.219 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 78 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.221+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3d0006c8f0 msgr2=0x7f3d0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.221+0000 7f3d25000640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3d0006c8f0 0x7f3d0006edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009aa0 tx=0x7f3d10009190 comp rx=0 tx=0).stop 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.221+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d200c9e30 msgr2=0x7f3d201f4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.221+0000 7f3d25000640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d200c9e30 0x7f3d201f4740 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c000c00 tx=0x7f3d0c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.222+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 shutdown_connections 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.222+0000 7f3d25000640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3d201554f0 0x7f3d2014fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.222+0000 7f3d25000640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d200c9e30 0x7f3d201f4740 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.222+0000 7f3d25000640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3d0006c8f0 0x7f3d0006edb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.222+0000 7f3d25000640 1 --2- 172.21.15.102:0/1275733601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d2006c550 0x7f3d201f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.222+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 >> 172.21.15.102:0/1275733601 conn(0x7f3d2006ebf0 msgr2=0x7f3d2006efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.222+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 shutdown_connections 2024-04-20T21:40:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:04.222+0000 7f3d25000640 1 -- 172.21.15.102:0/1275733601 wait complete. 2024-04-20T21:40:04.238 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-04-20T21:40:06.239 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:40:06.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.448+0000 7f54faa00640 1 Processor -- start 2024-04-20T21:40:06.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.448+0000 7f54faa00640 1 -- start start 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.449+0000 7f54faa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f4152290 0x7f54f40cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.449+0000 7f54faa00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f54f40c8330 0x7f54f40c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54faa00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f415c220 con 0x7f54f40c3700 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54faa00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f54f40c9180 con 0x7f54f40c8330 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54faa00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f54f40c4840 con 0x7f54f4152290 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54f8c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f54f40c3700 0x7f54f406e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60960/0 (socket says 172.21.15.102:60960) 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54f8c00640 1 -- 172.21.15.102:0/3028166077 learned_addr learned my addr 172.21.15.102:0/3028166077 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54fa000640 1 --2- 172.21.15.102:0/3028166077 >> v2:172.21.15.193:3300/0 conn(0x7f54f40c8330 0x7f54f40c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:06.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54f9600640 1 --2- 172.21.15.102:0/3028166077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f4152290 0x7f54f40cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:06.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54f9600640 1 -- 172.21.15.102:0/3028166077 >> v2:172.21.15.193:3300/0 conn(0x7f54f40c8330 msgr2=0x7f54f40c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:06.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54f9600640 1 --2- 172.21.15.102:0/3028166077 >> v2:172.21.15.193:3300/0 conn(0x7f54f40c8330 0x7f54f40c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54f9600640 1 -- 172.21.15.102:0/3028166077 >> v1:172.21.15.102:6789/0 conn(0x7f54f40c3700 legacy=0x7f54f406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:06.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54f3e00640 1 -- 172.21.15.102:0/3028166077 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001382727 0 0) 0x7f54f415c220 con 0x7f54f40c3700 2024-04-20T21:40:06.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.450+0000 7f54f9600640 1 -- 172.21.15.102:0/3028166077 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f40c4560 con 0x7f54f4152290 2024-04-20T21:40:06.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.452+0000 7f54f9600640 1 --2- 172.21.15.102:0/3028166077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f4152290 0x7f54f40cd310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f54e40097a0 tx=0x7f54e402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f112225e76d8143b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:06.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.452+0000 7f54f3e00640 1 -- 172.21.15.102:0/3028166077 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4034070 con 0x7f54f4152290 2024-04-20T21:40:06.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.452+0000 7f54f3e00640 1 -- 172.21.15.102:0/3028166077 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e4030610 con 0x7f54f4152290 2024-04-20T21:40:06.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.452+0000 7f54f3e00640 1 -- 172.21.15.102:0/3028166077 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e40309e0 con 0x7f54f4152290 2024-04-20T21:40:06.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.452+0000 7f54faa00640 1 -- 172.21.15.102:0/3028166077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f4152290 msgr2=0x7f54f40cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:06.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.452+0000 7f54faa00640 1 --2- 172.21.15.102:0/3028166077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f4152290 0x7f54f40cd310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f54e40097a0 tx=0x7f54e402b960 comp rx=0 tx=0).stop 2024-04-20T21:40:06.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.453+0000 7f54faa00640 1 -- 172.21.15.102:0/3028166077 shutdown_connections 2024-04-20T21:40:06.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.453+0000 7f54faa00640 1 --2- 172.21.15.102:0/3028166077 >> v2:172.21.15.193:3300/0 conn(0x7f54f40c8330 0x7f54f40c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.453+0000 7f54faa00640 1 --2- 172.21.15.102:0/3028166077 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f4152290 0x7f54f40cd310 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.453+0000 7f54faa00640 1 -- 172.21.15.102:0/3028166077 >> 172.21.15.102:0/3028166077 conn(0x7f54f40cc820 msgr2=0x7f54f40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:06.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.453+0000 7f54faa00640 1 -- 172.21.15.102:0/3028166077 shutdown_connections 2024-04-20T21:40:06.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.453+0000 7f54faa00640 1 -- 172.21.15.102:0/3028166077 wait complete. 2024-04-20T21:40:06.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.454+0000 7f54faa00640 1 Processor -- start 2024-04-20T21:40:06.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.455+0000 7f54faa00640 1 -- start start 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.455+0000 7f54faa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f40c3700 0x7f54f41e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.455+0000 7f54faa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54f40c8330 0x7f54f40c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54faa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54f4152290 0x7f54f40c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54faa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f54f415ba20 con 0x7f54f4152290 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54faa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54f415b740 con 0x7f54f40c8330 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54faa00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f54f415bd00 con 0x7f54f40c3700 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54fa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54f4152290 0x7f54f40c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54f8c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54f40c8330 0x7f54f40c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54f9600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f40c3700 0x7f54f41e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54fa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54f4152290 0x7f54f40c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53482/0 (socket says 172.21.15.102:53482) 2024-04-20T21:40:06.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54fa000640 1 -- 172.21.15.102:0/3732375685 learned_addr learned my addr 172.21.15.102:0/3732375685 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:40:06.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54fa000640 1 -- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f40c3700 msgr2=0x7f54f41e3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:06.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54fa000640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f40c3700 0x7f54f41e3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54fa000640 1 -- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54f40c8330 msgr2=0x7f54f40c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:06.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54fa000640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54f40c8330 0x7f54f40c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.456+0000 7f54fa000640 1 -- 172.21.15.102:0/3732375685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f40cc400 con 0x7f54f4152290 2024-04-20T21:40:06.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.457+0000 7f54fa000640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54f4152290 0x7f54f40c2d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00e930 tx=0x7f54ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:06.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.457+0000 7f54f1600640 1 -- 172.21.15.102:0/3732375685 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec017070 con 0x7f54f4152290 2024-04-20T21:40:06.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.457+0000 7f54f1600640 1 -- 172.21.15.102:0/3732375685 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ec00c980 con 0x7f54f4152290 2024-04-20T21:40:06.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.457+0000 7f54f1600640 1 -- 172.21.15.102:0/3732375685 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00cd70 con 0x7f54f4152290 2024-04-20T21:40:06.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.457+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f4145750 con 0x7f54f4152290 2024-04-20T21:40:06.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.457+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f4145d10 con 0x7f54f4152290 2024-04-20T21:40:06.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.458+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f404fd60 con 0x7f54f4152290 2024-04-20T21:40:06.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.459+0000 7f54f1600640 1 -- 172.21.15.102:0/3732375685 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54ec01a070 con 0x7f54f4152290 2024-04-20T21:40:06.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.460+0000 7f54f1600640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54d806c8f0 0x7f54d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:06.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.461+0000 7f54f1600640 1 -- 172.21.15.102:0/3732375685 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f54ec08c6a0 con 0x7f54f4152290 2024-04-20T21:40:06.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.461+0000 7f54f9600640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54d806c8f0 0x7f54d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:06.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.461+0000 7f54f9600640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54d806c8f0 0x7f54d806edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f54e4036000 tx=0x7f54e403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:06.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.465+0000 7f54f1600640 1 -- 172.21.15.102:0/3732375685 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f54ec059ca0 con 0x7f54f4152290 2024-04-20T21:40:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.712+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54f4146440 con 0x7f54f4152290 2024-04-20T21:40:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.713+0000 7f54f1600640 1 -- 172.21.15.102:0/3732375685 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7f54ec05dfa0 con 0x7f54f4152290 2024-04-20T21:40:06.712 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:40:06.712 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:40:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:40:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54d806c8f0 msgr2=0x7f54d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54d806c8f0 0x7f54d806edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f54e4036000 tx=0x7f54e403b040 comp rx=0 tx=0).stop 2024-04-20T21:40:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54f4152290 msgr2=0x7f54f40c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54f4152290 0x7f54f40c2d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00e930 tx=0x7f54ec00edf0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 shutdown_connections 2024-04-20T21:40:06.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54f4152290 0x7f54f40c2d80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54d806c8f0 0x7f54d806edb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54f40c8330 0x7f54f40c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 --2- 172.21.15.102:0/3732375685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54f40c3700 0x7f54f41e3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:06.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 >> 172.21.15.102:0/3732375685 conn(0x7f54f40cc820 msgr2=0x7f54f40c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:06.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 shutdown_connections 2024-04-20T21:40:06.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:40:06.716+0000 7f54faa00640 1 -- 172.21.15.102:0/3732375685 wait complete. 2024-04-20T21:40:06.733 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-04-20T21:40:06.733 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 59 secs before thrashing 2024-04-20T21:40:08.187 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:40:08.188+0000 7f8d75c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:40:12.932 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 84% (112/133) Updating files: 85% (114/133) Updating files: 86% (115/133) Updating files: 87% (116/133) Updating files: 88% (118/133) Updating files: 89% (119/133) Updating files: 90% (120/133) Updating files: 90% (121/133) Updating files: 91% (122/133) Updating files: 92% (123/133) Updating files: 93% (124/133) Updating files: 94% (126/133) Updating files: 95% (127/133) Updating files: 96% (128/133) Updating files: 97% (130/133) Updating files: 98% (131/133) Updating files: 99% (132/133) Updating files: 100% (133/133) Updating files: 100% (133/133), done. 2024-04-20T21:40:13.060 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 2b32297267d v0.13 2024-04-20T21:40:13.075 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.13 2024-04-20T21:40:13.115 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:40:13.115 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 14 -eq 48 ']' 2024-04-20T21:40:13.115 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.14 2024-04-20T21:40:13.115 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.14 2024-04-20T21:40:13.115 INFO:tasks.workunit.client.0.smithi102.stdout:v0.14 2024-04-20T21:40:13.115 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.14 2024-04-20T21:40:22.212 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 45% (69/151) Updating files: 46% (70/151) Updating files: 47% (71/151) Updating files: 48% (73/151) Updating files: 49% (74/151) Updating files: 50% (76/151) Updating files: 51% (78/151) Updating files: 52% (79/151) Updating files: 53% (81/151) Updating files: 54% (82/151) Updating files: 55% (84/151) Updating files: 56% (85/151) Updating files: 56% (86/151) Updating files: 57% (87/151) Updating files: 58% (88/151) Updating files: 59% (90/151) Updating files: 60% (91/151) Updating files: 61% (93/151) Updating files: 62% (94/151) Updating files: 63% (96/151) Updating files: 64% (97/151) Updating files: 65% (99/151) Updating files: 66% (100/151) Updating files: 67% (102/151) Updating files: 68% (103/151) Updating files: 69% (105/151) Updating files: 70% (106/151) Updating files: 71% (108/151) Updating files: 72% (109/151) Updating files: 73% (111/151) Updating files: 74% (112/151) Updating files: 75% (114/151) Updating files: 76% (115/151) Updating files: 77% (117/151) Updating files: 78% (118/151) Updating files: 79% (120/151) Updating files: 80% (121/151) Updating files: 81% (123/151) Updating files: 82% (124/151) Updating files: 83% (126/151) Updating files: 84% (127/151) Updating files: 85% (129/151) Updating files: 86% (130/151) Updating files: 87% (132/151) Updating files: 88% (133/151) Updating files: 89% (135/151) Updating files: 90% (136/151) Updating files: 91% (138/151) Updating files: 92% (139/151) Updating files: 93% (141/151) Updating files: 94% (142/151) Updating files: 95% (144/151) Updating files: 96% (145/151) Updating files: 97% (147/151) Updating files: 98% (148/151) Updating files: 99% (150/151) Updating files: 100% (151/151) Updating files: 100% (151/151), done. 2024-04-20T21:40:22.247 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 1ec1e8d6fb6 v0.14 2024-04-20T21:40:22.261 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.14 2024-04-20T21:40:22.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:40:22.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 15 -eq 48 ']' 2024-04-20T21:40:22.396 INFO:tasks.workunit.client.0.smithi102.stdout:v0.15 2024-04-20T21:40:22.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.15 2024-04-20T21:40:22.397 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.15 2024-04-20T21:40:22.397 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.15 2024-04-20T21:40:25.219 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 73% (82/112) Updating files: 74% (83/112) Updating files: 75% (84/112) Updating files: 76% (86/112) Updating files: 77% (87/112) Updating files: 78% (88/112) Updating files: 79% (89/112) Updating files: 80% (90/112) Updating files: 81% (91/112) Updating files: 82% (92/112) Updating files: 83% (93/112) Updating files: 84% (95/112) Updating files: 85% (96/112) Updating files: 86% (97/112) Updating files: 87% (98/112) Updating files: 88% (99/112) Updating files: 89% (100/112) Updating files: 90% (101/112) Updating files: 91% (102/112) Updating files: 92% (104/112) Updating files: 93% (105/112) Updating files: 94% (106/112) Updating files: 95% (107/112) Updating files: 96% (108/112) Updating files: 97% (109/112) Updating files: 98% (110/112) Updating files: 99% (111/112) Updating files: 100% (112/112) Updating files: 100% (112/112), done. 2024-04-20T21:40:25.264 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at ab3c421eb95 v0.15 2024-04-20T21:40:25.275 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.15 2024-04-20T21:40:25.846 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:40:25.847 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 16 -eq 48 ']' 2024-04-20T21:40:25.847 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.16 2024-04-20T21:40:25.847 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.16 2024-04-20T21:40:25.847 INFO:tasks.workunit.client.0.smithi102.stdout:v0.16 2024-04-20T21:40:25.847 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.16 2024-04-20T21:40:28.530 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 58% (63/107) Updating files: 59% (64/107) Updating files: 60% (65/107) Updating files: 61% (66/107) Updating files: 62% (67/107) Updating files: 63% (68/107) Updating files: 64% (69/107) Updating files: 65% (70/107) Updating files: 66% (71/107) Updating files: 67% (72/107) Updating files: 68% (73/107) Updating files: 69% (74/107) Updating files: 70% (75/107) Updating files: 71% (76/107) Updating files: 72% (78/107) Updating files: 73% (79/107) Updating files: 74% (80/107) Updating files: 75% (81/107) Updating files: 76% (82/107) Updating files: 77% (83/107) Updating files: 78% (84/107) Updating files: 79% (85/107) Updating files: 80% (86/107) Updating files: 81% (87/107) Updating files: 82% (88/107) Updating files: 83% (89/107) Updating files: 84% (90/107) Updating files: 85% (91/107) Updating files: 86% (93/107) Updating files: 87% (94/107) Updating files: 88% (95/107) Updating files: 89% (96/107) Updating files: 90% (97/107) Updating files: 91% (98/107) Updating files: 92% (99/107) Updating files: 93% (100/107) Updating files: 94% (101/107) Updating files: 95% (102/107) Updating files: 96% (103/107) Updating files: 97% (104/107) Updating files: 98% (105/107) Updating files: 99% (106/107) Updating files: 100% (107/107) Updating files: 100% (107/107), done. 2024-04-20T21:40:28.614 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 38dbaa5175c v0.16 2024-04-20T21:40:28.618 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.16 2024-04-20T21:40:29.246 INFO:tasks.workunit.client.0.smithi102.stdout:v0.17 2024-04-20T21:40:29.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:40:29.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 17 -eq 48 ']' 2024-04-20T21:40:29.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.17 2024-04-20T21:40:29.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.17 2024-04-20T21:40:29.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.17 2024-04-20T21:40:30.493 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:40:30.501 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:40:32.113 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 46% (100/217) Updating files: 47% (102/217) Updating files: 48% (105/217) Updating files: 49% (107/217) Updating files: 50% (109/217) Updating files: 51% (111/217) Updating files: 52% (113/217) Updating files: 53% (116/217) Updating files: 54% (118/217) Updating files: 55% (120/217) Updating files: 56% (122/217) Updating files: 57% (124/217) Updating files: 58% (126/217) Updating files: 59% (129/217) Updating files: 60% (131/217) Updating files: 61% (133/217) Updating files: 62% (135/217) Updating files: 63% (137/217) Updating files: 64% (139/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 94% (206/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2024-04-20T21:40:32.180 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at dac3a8c35fb v0.17 2024-04-20T21:40:32.189 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.17 2024-04-20T21:40:32.342 INFO:tasks.workunit.client.0.smithi102.stdout:v0.18 2024-04-20T21:40:32.342 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:40:32.342 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 18 -eq 48 ']' 2024-04-20T21:40:32.342 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.18 2024-04-20T21:40:32.342 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.18 2024-04-20T21:40:32.342 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.18 2024-04-20T21:40:45.486 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 6% (48/779) Updating files: 7% (55/779) Updating files: 8% (63/779) Updating files: 9% (71/779) Updating files: 10% (78/779) Updating files: 11% (86/779) Updating files: 12% (94/779) Updating files: 13% (102/779) Updating files: 14% (110/779) Updating files: 14% (116/779) Updating files: 15% (117/779) Updating files: 16% (125/779) Updating files: 17% (133/779) Updating files: 18% (141/779) Updating files: 19% (149/779) Updating files: 20% (156/779) Updating files: 21% (164/779) Updating files: 22% (172/779) Updating files: 23% (180/779) Updating files: 24% (187/779) Updating files: 25% (195/779) Updating files: 26% (203/779) Updating files: 26% (207/779) Updating files: 27% (211/779) Updating files: 28% (219/779) Updating files: 29% (226/779) Updating files: 30% (234/779) Updating files: 31% (242/779) Updating files: 32% (250/779) Updating files: 32% (254/779) Updating files: 33% (258/779) Updating files: 34% (265/779) Updating files: 35% (273/779) Updating files: 36% (281/779) Updating files: 37% (289/779) Updating files: 38% (297/779) Updating files: 39% (304/779) Updating files: 40% (312/779) Updating files: 41% (320/779) Updating files: 42% (328/779) Updating files: 43% (335/779) Updating files: 44% (343/779) Updating files: 45% (351/779) Updating files: 46% (359/779) Updating files: 47% (367/779) Updating files: 48% (374/779) Updating files: 48% (381/779) Updating files: 49% (382/779) Updating files: 50% (390/779) Updating files: 51% (398/779) Updating files: 52% (406/779) Updating files: 53% (413/779) Updating files: 54% (421/779) Updating files: 55% (429/779) Updating files: 56% (437/779) Updating files: 57% (445/779) Updating files: 58% (452/779) Updating files: 59% (460/779) Updating files: 60% (468/779) Updating files: 61% (476/779) Updating files: 62% (483/779) Updating files: 63% (491/779) Updating files: 63% (493/779) Updating files: 64% (499/779) Updating files: 65% (507/779) Updating files: 66% (515/779) Updating files: 67% (522/779) Updating files: 68% (530/779) Updating files: 69% (538/779) Updating files: 70% (546/779) Updating files: 71% (554/779) Updating files: 72% (561/779) Updating files: 73% (569/779) Updating files: 74% (577/779) Updating files: 74% (580/779) Updating files: 75% (585/779) Updating files: 75% (587/779) Updating files: 76% (593/779) Updating files: 77% (600/779) Updating files: 78% (608/779) Updating files: 79% (616/779) Updating files: 80% (624/779) Updating files: 81% (631/779) Updating files: 82% (639/779) Updating files: 83% (647/779) Updating files: 84% (655/779) Updating files: 85% (663/779) Updating files: 86% (670/779) Updating files: 87% (678/779) Updating files: 88% (686/779) Updating files: 89% (694/779) Updating files: 90% (702/779) Updating files: 91% (709/779) Updating files: 92% (717/779) Updating files: 93% (725/779) Updating files: 94% (733/779) Updating files: 94% (739/779) Updating files: 95% (741/779) Updating files: 96% (748/779) Updating files: 97% (756/779) Updating files: 98% (764/779) Updating files: 99% (772/779) Updating files: 100% (779/779) Updating files: 100% (779/779), done. 2024-04-20T21:40:45.550 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 2f10137986a v0.18 2024-04-20T21:40:45.564 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.18 2024-04-20T21:40:45.696 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:40:45.696 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 19 -eq 48 ']' 2024-04-20T21:40:45.696 INFO:tasks.workunit.client.0.smithi102.stdout:v0.19 2024-04-20T21:40:45.697 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.19 2024-04-20T21:40:45.697 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.19 2024-04-20T21:40:45.697 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.19 2024-04-20T21:40:51.679 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 21% (46/217) Updating files: 22% (48/217) Updating files: 23% (50/217) Updating files: 24% (53/217) Updating files: 25% (55/217) Updating files: 26% (57/217) Updating files: 27% (59/217) Updating files: 28% (61/217) Updating files: 29% (63/217) Updating files: 30% (66/217) Updating files: 31% (68/217) Updating files: 32% (70/217) Updating files: 33% (72/217) Updating files: 34% (74/217) Updating files: 34% (75/217) Updating files: 35% (76/217) Updating files: 36% (79/217) Updating files: 37% (81/217) Updating files: 38% (83/217) Updating files: 39% (85/217) Updating files: 40% (87/217) Updating files: 41% (89/217) Updating files: 42% (92/217) Updating files: 43% (94/217) Updating files: 44% (96/217) Updating files: 45% (98/217) Updating files: 46% (100/217) Updating files: 47% (102/217) Updating files: 48% (105/217) Updating files: 49% (107/217) Updating files: 50% (109/217) Updating files: 51% (111/217) Updating files: 52% (113/217) Updating files: 53% (116/217) Updating files: 54% (118/217) Updating files: 55% (120/217) Updating files: 56% (122/217) Updating files: 57% (124/217) Updating files: 58% (126/217) Updating files: 59% (129/217) Updating files: 60% (131/217) Updating files: 61% (133/217) Updating files: 62% (135/217) Updating files: 63% (137/217) Updating files: 64% (139/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 94% (206/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2024-04-20T21:40:51.740 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 98f5be53f85 v0.19 2024-04-20T21:40:51.752 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.19 2024-04-20T21:40:52.052 INFO:tasks.workunit.client.0.smithi102.stdout:v0.20 2024-04-20T21:40:52.053 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:40:52.053 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 20 -eq 48 ']' 2024-04-20T21:40:52.053 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.20 2024-04-20T21:40:52.053 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.20 2024-04-20T21:40:52.053 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.20 2024-04-20T21:40:54.910 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 56% (139/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 92% (227/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2024-04-20T21:40:54.968 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 266d2076b6c debian: fix up debian scripts 2024-04-20T21:40:54.976 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.20 2024-04-20T21:40:55.169 INFO:tasks.workunit.client.0.smithi102.stdout:v0.21 2024-04-20T21:40:55.169 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:40:55.169 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 21 -eq 48 ']' 2024-04-20T21:40:55.169 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.21 2024-04-20T21:40:55.169 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.21 2024-04-20T21:40:55.169 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.21 2024-04-20T21:41:00.618 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:41:00.626 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:41:05.734 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:05.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.947+0000 7f0af4e00640 1 Processor -- start 2024-04-20T21:41:05.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.947+0000 7f0af4e00640 1 -- start start 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0af4e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0af4e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0af0073a10 0x7f0af0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0af4e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af01536d0 con 0x7f0af014fa50 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0af4e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0af0159500 con 0x7f0af0073a10 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0af4e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0af00c76c0 con 0x7f0af00c9eb0 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0aede00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0af014fa50 0x7f0af00c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53282/0 (socket says 172.21.15.102:53282) 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0aede00640 1 -- 172.21.15.102:0/533320283 learned_addr learned my addr 172.21.15.102:0/533320283 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0aee800640 1 --2- 172.21.15.102:0/533320283 >> v2:172.21.15.193:3300/0 conn(0x7f0af0073a10 0x7f0af0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.949+0000 7f0ae5400640 1 --2- 172.21.15.102:0/533320283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.950+0000 7f0aed400640 1 -- 172.21.15.102:0/533320283 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504985393 0 0) 0x7f0ae0001110 con 0x7f0af014fa50 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.950+0000 7f0ae5400640 1 -- 172.21.15.102:0/533320283 >> v2:172.21.15.193:3300/0 conn(0x7f0af0073a10 msgr2=0x7f0af0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.950+0000 7f0ae5400640 1 --2- 172.21.15.102:0/533320283 >> v2:172.21.15.193:3300/0 conn(0x7f0af0073a10 0x7f0af0158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.950+0000 7f0ae5400640 1 -- 172.21.15.102:0/533320283 >> v1:172.21.15.102:6789/0 conn(0x7f0af014fa50 legacy=0x7f0af00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.950+0000 7f0ae5400640 1 -- 172.21.15.102:0/533320283 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af00c7960 con 0x7f0af00c9eb0 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.950+0000 7f0aee800640 1 --2- 172.21.15.102:0/533320283 >> v2:172.21.15.193:3300/0 conn(0x7f0af0073a10 0x7f0af0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.951+0000 7f0ae5400640 1 --2- 172.21.15.102:0/533320283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af00c4830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80097a0 tx=0x7f0ad802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd5dc249b7c1418 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.951+0000 7f0aed400640 1 -- 172.21.15.102:0/533320283 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8034070 con 0x7f0af00c9eb0 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.951+0000 7f0aed400640 1 -- 172.21.15.102:0/533320283 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad8030800 con 0x7f0af00c9eb0 2024-04-20T21:41:05.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.951+0000 7f0aed400640 1 -- 172.21.15.102:0/533320283 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8030bd0 con 0x7f0af00c9eb0 2024-04-20T21:41:05.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.952+0000 7f0af4e00640 1 -- 172.21.15.102:0/533320283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 msgr2=0x7f0af00c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.952+0000 7f0af4e00640 1 --2- 172.21.15.102:0/533320283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af00c4830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80097a0 tx=0x7f0ad802b940 comp rx=0 tx=0).stop 2024-04-20T21:41:05.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.952+0000 7f0af4e00640 1 -- 172.21.15.102:0/533320283 shutdown_connections 2024-04-20T21:41:05.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.952+0000 7f0af4e00640 1 --2- 172.21.15.102:0/533320283 >> v2:172.21.15.193:3300/0 conn(0x7f0af0073a10 0x7f0af0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.952+0000 7f0af4e00640 1 --2- 172.21.15.102:0/533320283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af00c4830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.952+0000 7f0af4e00640 1 -- 172.21.15.102:0/533320283 >> 172.21.15.102:0/533320283 conn(0x7f0af00c9390 msgr2=0x7f0af00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:05.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.952+0000 7f0af4e00640 1 -- 172.21.15.102:0/533320283 shutdown_connections 2024-04-20T21:41:05.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.952+0000 7f0af4e00640 1 -- 172.21.15.102:0/533320283 wait complete. 2024-04-20T21:41:05.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.954+0000 7f0af4e00640 1 Processor -- start 2024-04-20T21:41:05.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.954+0000 7f0af4e00640 1 -- start start 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0af4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0af0073a10 0x7f0af01eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0af4e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af01571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0af4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af014fa50 0x7f0af0157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0af4e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0af00c7c00 con 0x7f0af014fa50 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0af4e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0af00c3d90 con 0x7f0af0073a10 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0af4e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0af00c4030 con 0x7f0af00c9eb0 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0aee800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af014fa50 0x7f0af0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0aede00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0af0073a10 0x7f0af01eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0ae5400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af01571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0aee800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af014fa50 0x7f0af0157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45468/0 (socket says 172.21.15.102:45468) 2024-04-20T21:41:05.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.956+0000 7f0aee800640 1 -- 172.21.15.102:0/1483668096 learned_addr learned my addr 172.21.15.102:0/1483668096 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0aede00640 1 -- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 msgr2=0x7f0af01571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0aede00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af01571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0aede00640 1 -- 172.21.15.102:0/1483668096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af014fa50 msgr2=0x7f0af0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0aede00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af014fa50 0x7f0af0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0aede00640 1 -- 172.21.15.102:0/1483668096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af014c100 con 0x7f0af0073a10 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0aee800640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af014fa50 0x7f0af0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0ae5400640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af01571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0aede00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0af0073a10 0x7f0af01eeec0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c780 tx=0x7f0ae000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0ae6a00640 1 -- 172.21.15.102:0/1483668096 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0018070 con 0x7f0af0073a10 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0ae6a00640 1 -- 172.21.15.102:0/1483668096 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae00070d0 con 0x7f0af0073a10 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0ae6a00640 1 -- 172.21.15.102:0/1483668096 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00074c0 con 0x7f0af0073a10 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af014c3a0 con 0x7f0af0073a10 2024-04-20T21:41:05.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.957+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af014c940 con 0x7f0af0073a10 2024-04-20T21:41:05.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.958+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af00c56b0 con 0x7f0af0073a10 2024-04-20T21:41:05.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.959+0000 7f0ae6a00640 1 -- 172.21.15.102:0/1483668096 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ae001b070 con 0x7f0af0073a10 2024-04-20T21:41:05.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.960+0000 7f0ae6a00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0acc06c8f0 0x7f0acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.960+0000 7f0ae6a00640 1 -- 172.21.15.102:0/1483668096 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f0ae008b6b0 con 0x7f0af0073a10 2024-04-20T21:41:05.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.961+0000 7f0ae5400640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0acc06c8f0 0x7f0acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.961+0000 7f0ae5400640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0acc06c8f0 0x7f0acc06edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8036000 tx=0x7f0ad8008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:05.965+0000 7f0ae6a00640 1 -- 172.21.15.102:0/1483668096 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0af00c56b0 con 0x7f0af0073a10 2024-04-20T21:41:06.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.224+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0af01568e0 con 0x7f0af0073a10 2024-04-20T21:41:06.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.225+0000 7f0ae6a00640 1 -- 172.21.15.102:0/1483668096 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7f0af01536d0 con 0x7f0af0073a10 2024-04-20T21:41:06.224 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:06.224 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:06.224 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.227+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0acc06c8f0 msgr2=0x7f0acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.227+0000 7f0af4e00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0acc06c8f0 0x7f0acc06edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8036000 tx=0x7f0ad8008030 comp rx=0 tx=0).stop 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.227+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0af0073a10 msgr2=0x7f0af01eeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.227+0000 7f0af4e00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0af0073a10 0x7f0af01eeec0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c780 tx=0x7f0ae000cc40 comp rx=0 tx=0).stop 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.228+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 shutdown_connections 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.228+0000 7f0af4e00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af014fa50 0x7f0af0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.228+0000 7f0af4e00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0af00c9eb0 0x7f0af01571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.228+0000 7f0af4e00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0acc06c8f0 0x7f0acc06edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.228+0000 7f0af4e00640 1 --2- 172.21.15.102:0/1483668096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0af0073a10 0x7f0af01eeec0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.228+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 >> 172.21.15.102:0/1483668096 conn(0x7f0af00c9390 msgr2=0x7f0af006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.228+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 shutdown_connections 2024-04-20T21:41:06.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.228+0000 7f0af4e00640 1 -- 172.21.15.102:0/1483668096 wait complete. 2024-04-20T21:41:06.244 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:41:06.245 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:41:06.245 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:41:06.290 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:41:06.290 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:06.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.481+0000 7f5c0d400640 1 Processor -- start 2024-04-20T21:41:06.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.481+0000 7f5c0d400640 1 -- start start 2024-04-20T21:41:06.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.482+0000 7f5c0d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c080c75a0 0x7f5c080c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.482+0000 7f5c0d400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c08153560 con 0x7f5c08153180 2024-04-20T21:41:06.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.482+0000 7f5c0d400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c080c7a70 con 0x7f5c08073c70 2024-04-20T21:41:06.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.482+0000 7f5c0d400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5c080c7d30 con 0x7f5c080c75a0 2024-04-20T21:41:06.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c07e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5c08153180 0x7f5c08158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53298/0 (socket says 172.21.15.102:53298) 2024-04-20T21:41:06.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c07e00640 1 -- 172.21.15.102:0/2014503968 learned_addr learned my addr 172.21.15.102:0/2014503968 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:06.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c07400640 1 --2- 172.21.15.102:0/2014503968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c080c75a0 0x7f5c080c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c06a00640 1 -- 172.21.15.102:0/2014503968 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 863026552 0 0) 0x7f5c08153560 con 0x7f5c08153180 2024-04-20T21:41:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c06a00640 1 -- 172.21.15.102:0/2014503968 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bec003520 con 0x7f5c08153180 2024-04-20T21:41:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c06a00640 1 -- 172.21.15.102:0/2014503968 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797502131 0 0) 0x7f5c080c7a70 con 0x7f5c08073c70 2024-04-20T21:41:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c06a00640 1 -- 172.21.15.102:0/2014503968 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c08153560 con 0x7f5c08073c70 2024-04-20T21:41:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c07400640 1 -- 172.21.15.102:0/2014503968 >> v1:172.21.15.193:6789/0 conn(0x7f5c08073c70 legacy=0x7f5c081637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.483+0000 7f5c07400640 1 -- 172.21.15.102:0/2014503968 >> v1:172.21.15.102:6789/0 conn(0x7f5c08153180 legacy=0x7f5c08158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.484+0000 7f5c07400640 1 -- 172.21.15.102:0/2014503968 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0815c4d0 con 0x7f5c080c75a0 2024-04-20T21:41:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.485+0000 7f5c07400640 1 --2- 172.21.15.102:0/2014503968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c080c75a0 0x7f5c080c5f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00097d0 tx=0x7f5bf002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9843cf8ab6f8467 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.485+0000 7f5c06a00640 1 -- 172.21.15.102:0/2014503968 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0035020 con 0x7f5c080c75a0 2024-04-20T21:41:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.485+0000 7f5c06a00640 1 -- 172.21.15.102:0/2014503968 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0031890 con 0x7f5c080c75a0 2024-04-20T21:41:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.485+0000 7f5c06a00640 1 -- 172.21.15.102:0/2014503968 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0031c40 con 0x7f5c080c75a0 2024-04-20T21:41:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.485+0000 7f5c0d400640 1 -- 172.21.15.102:0/2014503968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c080c75a0 msgr2=0x7f5c080c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.485+0000 7f5c0d400640 1 --2- 172.21.15.102:0/2014503968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c080c75a0 0x7f5c080c5f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00097d0 tx=0x7f5bf002b930 comp rx=0 tx=0).stop 2024-04-20T21:41:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.486+0000 7f5c0d400640 1 -- 172.21.15.102:0/2014503968 shutdown_connections 2024-04-20T21:41:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.486+0000 7f5c0d400640 1 --2- 172.21.15.102:0/2014503968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c080c75a0 0x7f5c080c5f30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.486+0000 7f5c0d400640 1 -- 172.21.15.102:0/2014503968 >> 172.21.15.102:0/2014503968 conn(0x7f5c080cc820 msgr2=0x7f5c080ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.486+0000 7f5c0d400640 1 -- 172.21.15.102:0/2014503968 shutdown_connections 2024-04-20T21:41:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.486+0000 7f5c0d400640 1 -- 172.21.15.102:0/2014503968 wait complete. 2024-04-20T21:41:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.487+0000 7f5c0d400640 1 Processor -- start 2024-04-20T21:41:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.487+0000 7f5c0d400640 1 -- start start 2024-04-20T21:41:06.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c0d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c08073c70 0x7f5c081ef370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c0d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c080c75a0 0x7f5c08070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c0d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c08153180 0x7f5c0815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c0d400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08159550 con 0x7f5c080c75a0 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c0d400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0815ca90 con 0x7f5c08073c70 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c0d400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0815d290 con 0x7f5c08153180 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c07400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c080c75a0 0x7f5c08070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c07400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c080c75a0 0x7f5c08070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45478/0 (socket says 172.21.15.102:45478) 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.488+0000 7f5c07400640 1 -- 172.21.15.102:0/1105042804 learned_addr learned my addr 172.21.15.102:0/1105042804 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5c07e00640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c08073c70 0x7f5c081ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5c0ca00640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c08153180 0x7f5c0815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5c07400640 1 -- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c08153180 msgr2=0x7f5c0815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5c07400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c08153180 0x7f5c0815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5c07400640 1 -- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c08073c70 msgr2=0x7f5c081ef370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5c07400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c08073c70 0x7f5c081ef370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5c07400640 1 -- 172.21.15.102:0/1105042804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c081f2310 con 0x7f5c080c75a0 2024-04-20T21:41:06.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5c07400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c080c75a0 0x7f5c08070eb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00098f0 tx=0x7f5bf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.489+0000 7f5bffe00640 1 -- 172.21.15.102:0/1105042804 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0035040 con 0x7f5c080c75a0 2024-04-20T21:41:06.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.490+0000 7f5bffe00640 1 -- 172.21.15.102:0/1105042804 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0003820 con 0x7f5c080c75a0 2024-04-20T21:41:06.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.490+0000 7f5bffe00640 1 -- 172.21.15.102:0/1105042804 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0003bd0 con 0x7f5c080c75a0 2024-04-20T21:41:06.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.490+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c081f25d0 con 0x7f5c080c75a0 2024-04-20T21:41:06.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.490+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c081f2b90 con 0x7f5c080c75a0 2024-04-20T21:41:06.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.491+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0804fd60 con 0x7f5c080c75a0 2024-04-20T21:41:06.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.492+0000 7f5bffe00640 1 -- 172.21.15.102:0/1105042804 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bf0038020 con 0x7f5c080c75a0 2024-04-20T21:41:06.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.492+0000 7f5bffe00640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5be406c8f0 0x7f5be406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.492+0000 7f5bffe00640 1 -- 172.21.15.102:0/1105042804 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f5bf00ae5a0 con 0x7f5c080c75a0 2024-04-20T21:41:06.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.493+0000 7f5c07e00640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5be406c8f0 0x7f5be406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.493+0000 7f5c07e00640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5be406c8f0 0x7f5be406edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8000f60 tx=0x7f5bf8007d70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.497+0000 7f5bffe00640 1 -- 172.21.15.102:0/1105042804 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5bf002d040 con 0x7f5c080c75a0 2024-04-20T21:41:06.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.744+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c080c8050 con 0x7f5c080c75a0 2024-04-20T21:41:06.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.746+0000 7f5bffe00640 1 -- 172.21.15.102:0/1105042804 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7f5bf007ba50 con 0x7f5c080c75a0 2024-04-20T21:41:06.745 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:06.745 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:06.746 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:06.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5be406c8f0 msgr2=0x7f5be406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5be406c8f0 0x7f5be406edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8000f60 tx=0x7f5bf8007d70 comp rx=0 tx=0).stop 2024-04-20T21:41:06.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c080c75a0 msgr2=0x7f5c08070eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c080c75a0 0x7f5c08070eb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00098f0 tx=0x7f5bf00046c0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 shutdown_connections 2024-04-20T21:41:06.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c08153180 0x7f5c0815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c080c75a0 0x7f5c08070eb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5be406c8f0 0x7f5be406edb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 --2- 172.21.15.102:0/1105042804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c08073c70 0x7f5c081ef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 >> 172.21.15.102:0/1105042804 conn(0x7f5c080cc820 msgr2=0x7f5c0806f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 shutdown_connections 2024-04-20T21:41:06.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:06.749+0000 7f5c0d400640 1 -- 172.21.15.102:0/1105042804 wait complete. 2024-04-20T21:41:06.766 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:41:08.767 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:08.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.971+0000 7f283a400640 1 Processor -- start 2024-04-20T21:41:08.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.972+0000 7f283a400640 1 -- start start 2024-04-20T21:41:08.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.973+0000 7f283a400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f28340ca2b0 0x7f283415b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.973+0000 7f283a400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2834068350 0x7f28341557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.973+0000 7f283a400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f283415b480 con 0x7f28340ca690 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.973+0000 7f283a400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f28340c9200 con 0x7f28340ca2b0 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.973+0000 7f283a400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f28340c94e0 con 0x7f2834068350 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.973+0000 7f2833400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f28340ca2b0 0x7f283415b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.973+0000 7f2833400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f28340ca2b0 0x7f283415b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39970/0 (socket says 172.21.15.102:39970) 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.973+0000 7f2833400640 1 -- 172.21.15.102:0/1307146355 learned_addr learned my addr 172.21.15.102:0/1307146355 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.974+0000 7f2833e00640 1 --2- 172.21.15.102:0/1307146355 >> v2:172.21.15.193:3300/0 conn(0x7f2834068350 0x7f28341557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.974+0000 7f2833400640 1 -- 172.21.15.102:0/1307146355 >> v2:172.21.15.193:3300/0 conn(0x7f2834068350 msgr2=0x7f28341557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.974+0000 7f2833400640 1 --2- 172.21.15.102:0/1307146355 >> v2:172.21.15.193:3300/0 conn(0x7f2834068350 0x7f28341557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:08.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.974+0000 7f2833400640 1 -- 172.21.15.102:0/1307146355 >> v1:172.21.15.193:6789/0 conn(0x7f28340ca690 legacy=0x7f28341550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:08.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.974+0000 7f2833400640 1 -- 172.21.15.102:0/1307146355 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28340c97c0 con 0x7f28340ca2b0 2024-04-20T21:41:08.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.975+0000 7f2833400640 1 --2- 172.21.15.102:0/1307146355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28340ca2b0 0x7f283415b980 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2834068070 tx=0x7f282402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d954a5fb2376238 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:08.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f2832000640 1 -- 172.21.15.102:0/1307146355 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282402f850 con 0x7f28340ca2b0 2024-04-20T21:41:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f2832000640 1 -- 172.21.15.102:0/1307146355 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282402f9f0 con 0x7f28340ca2b0 2024-04-20T21:41:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f2832000640 1 -- 172.21.15.102:0/1307146355 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282402fdc0 con 0x7f28340ca2b0 2024-04-20T21:41:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f283a400640 1 -- 172.21.15.102:0/1307146355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28340ca2b0 msgr2=0x7f283415b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f283a400640 1 --2- 172.21.15.102:0/1307146355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28340ca2b0 0x7f283415b980 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2834068070 tx=0x7f282402ba30 comp rx=0 tx=0).stop 2024-04-20T21:41:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f283a400640 1 -- 172.21.15.102:0/1307146355 shutdown_connections 2024-04-20T21:41:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f283a400640 1 --2- 172.21.15.102:0/1307146355 >> v2:172.21.15.193:3300/0 conn(0x7f2834068350 0x7f28341557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f283a400640 1 --2- 172.21.15.102:0/1307146355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28340ca2b0 0x7f283415b980 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f283a400640 1 -- 172.21.15.102:0/1307146355 >> 172.21.15.102:0/1307146355 conn(0x7f283406bb20 msgr2=0x7f283406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f283a400640 1 -- 172.21.15.102:0/1307146355 shutdown_connections 2024-04-20T21:41:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.976+0000 7f283a400640 1 -- 172.21.15.102:0/1307146355 wait complete. 2024-04-20T21:41:08.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.978+0000 7f283a400640 1 Processor -- start 2024-04-20T21:41:08.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.978+0000 7f283a400640 1 -- start start 2024-04-20T21:41:08.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.979+0000 7f283a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2834068350 0x7f28341f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:08.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.979+0000 7f283a400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28340ca2b0 0x7f2834156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:08.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.979+0000 7f283a400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28340ca690 0x7f2834156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:08.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.979+0000 7f283a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2834071740 con 0x7f2834068350 2024-04-20T21:41:08.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.979+0000 7f283a400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28340c8190 con 0x7f28340ca690 2024-04-20T21:41:08.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.979+0000 7f283a400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f28340718c0 con 0x7f28340ca2b0 2024-04-20T21:41:08.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.979+0000 7f2833400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2834068350 0x7f28341f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2834068350 0x7f28341f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39980/0 (socket says 172.21.15.102:39980) 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833400640 1 -- 172.21.15.102:0/2463944942 learned_addr learned my addr 172.21.15.102:0/2463944942 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833e00640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28340ca690 0x7f2834156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2832a00640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28340ca2b0 0x7f2834156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833400640 1 -- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28340ca2b0 msgr2=0x7f2834156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28340ca2b0 0x7f2834156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833400640 1 -- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28340ca690 msgr2=0x7f2834156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28340ca690 0x7f2834156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:08.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833400640 1 -- 172.21.15.102:0/2463944942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2834156a80 con 0x7f2834068350 2024-04-20T21:41:08.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.980+0000 7f2833400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2834068350 0x7f28341f4ce0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f282402ba70 tx=0x7f2824004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:08.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.981+0000 7f282b400640 1 -- 172.21.15.102:0/2463944942 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824037d10 con 0x7f2834068350 2024-04-20T21:41:08.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.981+0000 7f282b400640 1 -- 172.21.15.102:0/2463944942 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2824046070 con 0x7f2834068350 2024-04-20T21:41:08.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.981+0000 7f282b400640 1 -- 172.21.15.102:0/2463944942 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824041640 con 0x7f2834068350 2024-04-20T21:41:08.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.981+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28340c4d10 con 0x7f2834068350 2024-04-20T21:41:08.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.981+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28340c5190 con 0x7f2834068350 2024-04-20T21:41:08.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.982+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2834068c30 con 0x7f2834068350 2024-04-20T21:41:08.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.983+0000 7f282b400640 1 -- 172.21.15.102:0/2463944942 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2824040070 con 0x7f2834068350 2024-04-20T21:41:08.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.984+0000 7f282b400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f281006c920 0x7f281006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:08.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.984+0000 7f282b400640 1 -- 172.21.15.102:0/2463944942 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f28240ae8d0 con 0x7f2834068350 2024-04-20T21:41:08.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.984+0000 7f2832a00640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f281006c920 0x7f281006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:08.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.985+0000 7f2832a00640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f281006c920 0x7f281006ede0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2834151d70 tx=0x7f281c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:08.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:08.988+0000 7f282b400640 1 -- 172.21.15.102:0/2463944942 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2824033030 con 0x7f2834068350 2024-04-20T21:41:09.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.228+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28340c8430 con 0x7f2834068350 2024-04-20T21:41:09.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.229+0000 7f282b400640 1 -- 172.21.15.102:0/2463944942 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7f282407be00 con 0x7f2834068350 2024-04-20T21:41:09.229 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:09.229 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:09.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.232+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f281006c920 msgr2=0x7f281006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:09.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.232+0000 7f283a400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f281006c920 0x7f281006ede0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2834151d70 tx=0x7f281c009040 comp rx=0 tx=0).stop 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2834068350 msgr2=0x7f28341f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2834068350 0x7f28341f4ce0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f282402ba70 tx=0x7f2824004730 comp rx=0 tx=0).stop 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 shutdown_connections 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f281006c920 0x7f281006ede0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28340ca690 0x7f2834156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28340ca2b0 0x7f2834156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 --2- 172.21.15.102:0/2463944942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2834068350 0x7f28341f4ce0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 >> 172.21.15.102:0/2463944942 conn(0x7f283406bb20 msgr2=0x7f28340cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 shutdown_connections 2024-04-20T21:41:09.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:09.233+0000 7f283a400640 1 -- 172.21.15.102:0/2463944942 wait complete. 2024-04-20T21:41:09.250 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:41:11.251 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.460+0000 7fe844c00640 1 Processor -- start 2024-04-20T21:41:11.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.460+0000 7fe844c00640 1 -- start start 2024-04-20T21:41:11.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe844c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe84014fa50 0x7fe8400c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:11.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe844c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe8400c9eb0 0x7fe840158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:11.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe844c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 0x7fe8400c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:11.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe844c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe840159500 con 0x7fe8400c9eb0 2024-04-20T21:41:11.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe844c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fe8401536d0 con 0x7fe84014fa50 2024-04-20T21:41:11.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe844c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe8400c76c0 con 0x7fe840073a10 2024-04-20T21:41:11.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe835400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe8400c9eb0 0x7fe840158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:11.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe835400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe8400c9eb0 0x7fe840158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39984/0 (socket says 172.21.15.102:39984) 2024-04-20T21:41:11.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe835400640 1 -- 172.21.15.102:0/1201009083 learned_addr learned my addr 172.21.15.102:0/1201009083 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe83de00640 1 --2- 172.21.15.102:0/1201009083 >> v2:172.21.15.193:3300/0 conn(0x7fe84014fa50 0x7fe8400c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.462+0000 7fe83e800640 1 --2- 172.21.15.102:0/1201009083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 0x7fe8400c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.463+0000 7fe835400640 1 -- 172.21.15.102:0/1201009083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 msgr2=0x7fe8400c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.463+0000 7fe835400640 1 --2- 172.21.15.102:0/1201009083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 0x7fe8400c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.463+0000 7fe835400640 1 -- 172.21.15.102:0/1201009083 >> v2:172.21.15.193:3300/0 conn(0x7fe84014fa50 msgr2=0x7fe8400c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.463+0000 7fe835400640 1 --2- 172.21.15.102:0/1201009083 >> v2:172.21.15.193:3300/0 conn(0x7fe84014fa50 0x7fe8400c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.463+0000 7fe835400640 1 -- 172.21.15.102:0/1201009083 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8400c7960 con 0x7fe8400c9eb0 2024-04-20T21:41:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.463+0000 7fe83de00640 1 --2- 172.21.15.102:0/1201009083 >> v2:172.21.15.193:3300/0 conn(0x7fe84014fa50 0x7fe8400c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:11.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.464+0000 7fe835400640 1 --2- 172.21.15.102:0/1201009083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe840158fc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe8400c7b30 tx=0x7fe82802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a85982547a0a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:11.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.464+0000 7fe83d400640 1 -- 172.21.15.102:0/1201009083 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828034070 con 0x7fe8400c9eb0 2024-04-20T21:41:11.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.464+0000 7fe83d400640 1 -- 172.21.15.102:0/1201009083 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe828030850 con 0x7fe8400c9eb0 2024-04-20T21:41:11.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.464+0000 7fe83d400640 1 -- 172.21.15.102:0/1201009083 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828030c20 con 0x7fe8400c9eb0 2024-04-20T21:41:11.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 -- 172.21.15.102:0/1201009083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 msgr2=0x7fe840158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:11.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 --2- 172.21.15.102:0/1201009083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe840158fc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe8400c7b30 tx=0x7fe82802b990 comp rx=0 tx=0).stop 2024-04-20T21:41:11.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 -- 172.21.15.102:0/1201009083 shutdown_connections 2024-04-20T21:41:11.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 --2- 172.21.15.102:0/1201009083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 0x7fe8400c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 --2- 172.21.15.102:0/1201009083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe840158fc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 --2- 172.21.15.102:0/1201009083 >> v2:172.21.15.193:3300/0 conn(0x7fe84014fa50 0x7fe8400c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 -- 172.21.15.102:0/1201009083 >> 172.21.15.102:0/1201009083 conn(0x7fe8400c9390 msgr2=0x7fe8400c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:11.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 -- 172.21.15.102:0/1201009083 shutdown_connections 2024-04-20T21:41:11.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.465+0000 7fe844c00640 1 -- 172.21.15.102:0/1201009083 wait complete. 2024-04-20T21:41:11.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.467+0000 7fe844c00640 1 Processor -- start 2024-04-20T21:41:11.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.467+0000 7fe844c00640 1 -- start start 2024-04-20T21:41:11.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe844c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 0x7fe840158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:11.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe844c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe8401444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:11.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe844c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe84014fa50 0x7fe84015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:11.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe844c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8400c7c00 con 0x7fe8400c9eb0 2024-04-20T21:41:11.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe844c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8400c3d90 con 0x7fe84014fa50 2024-04-20T21:41:11.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe844c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe8400c4030 con 0x7fe840073a10 2024-04-20T21:41:11.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe835400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe8401444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:11.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe835400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe8401444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39992/0 (socket says 172.21.15.102:39992) 2024-04-20T21:41:11.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe835400640 1 -- 172.21.15.102:0/1782710248 learned_addr learned my addr 172.21.15.102:0/1782710248 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:11.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe83de00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 0x7fe840158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:11.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe83e800640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe84014fa50 0x7fe84015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:11.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe835400640 1 -- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 msgr2=0x7fe840158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:11.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe835400640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 0x7fe840158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe835400640 1 -- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe84014fa50 msgr2=0x7fe84015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:11.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.468+0000 7fe835400640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe84014fa50 0x7fe84015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.469+0000 7fe835400640 1 -- 172.21.15.102:0/1782710248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe840145940 con 0x7fe8400c9eb0 2024-04-20T21:41:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.469+0000 7fe835400640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe8401444c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe82802b9d0 tx=0x7fe828004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.469+0000 7fe836a00640 1 -- 172.21.15.102:0/1782710248 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828048070 con 0x7fe8400c9eb0 2024-04-20T21:41:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.469+0000 7fe836a00640 1 -- 172.21.15.102:0/1782710248 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe828030650 con 0x7fe8400c9eb0 2024-04-20T21:41:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.469+0000 7fe836a00640 1 -- 172.21.15.102:0/1782710248 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828034040 con 0x7fe8400c9eb0 2024-04-20T21:41:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.469+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe840144f80 con 0x7fe8400c9eb0 2024-04-20T21:41:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.469+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8401f29d0 con 0x7fe8400c9eb0 2024-04-20T21:41:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.471+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe840074340 con 0x7fe8400c9eb0 2024-04-20T21:41:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.471+0000 7fe836a00640 1 -- 172.21.15.102:0/1782710248 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe828037070 con 0x7fe8400c9eb0 2024-04-20T21:41:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.472+0000 7fe836a00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe81c06c8f0 0x7fe81c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.472+0000 7fe836a00640 1 -- 172.21.15.102:0/1782710248 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7fe8280ae120 con 0x7fe8400c9eb0 2024-04-20T21:41:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.473+0000 7fe83de00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe81c06c8f0 0x7fe81c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.473+0000 7fe83de00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe81c06c8f0 0x7fe81c06edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe830009bd0 tx=0x7fe83000b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:11.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.477+0000 7fe836a00640 1 -- 172.21.15.102:0/1782710248 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe840145940 con 0x7fe8400c9eb0 2024-04-20T21:41:11.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.715+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8400c7320 con 0x7fe8400c9eb0 2024-04-20T21:41:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.716+0000 7fe836a00640 1 -- 172.21.15.102:0/1782710248 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7fe828030820 con 0x7fe8400c9eb0 2024-04-20T21:41:11.716 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:11.717 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:11.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe81c06c8f0 msgr2=0x7fe81c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe81c06c8f0 0x7fe81c06edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe830009bd0 tx=0x7fe83000b040 comp rx=0 tx=0).stop 2024-04-20T21:41:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 msgr2=0x7fe8401444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe8401444c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe82802b9d0 tx=0x7fe828004730 comp rx=0 tx=0).stop 2024-04-20T21:41:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 shutdown_connections 2024-04-20T21:41:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe81c06c8f0 0x7fe81c06edb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe84014fa50 0x7fe84015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8400c9eb0 0x7fe8401444c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 --2- 172.21.15.102:0/1782710248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe840073a10 0x7fe840158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 >> 172.21.15.102:0/1782710248 conn(0x7fe8400c9390 msgr2=0x7fe84014c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 shutdown_connections 2024-04-20T21:41:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:11.719+0000 7fe844c00640 1 -- 172.21.15.102:0/1782710248 wait complete. 2024-04-20T21:41:11.737 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:41:13.738 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:13.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.946+0000 7f1899600640 1 Processor -- start 2024-04-20T21:41:13.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.946+0000 7f1899600640 1 -- start start 2024-04-20T21:41:13.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f1899600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f189414fa50 0x7f18940c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:13.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f1899600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 0x7f1894158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:13.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f1899600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18941536d0 con 0x7f1894073a10 2024-04-20T21:41:13.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f1899600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1894159500 con 0x7f189414fa50 2024-04-20T21:41:13.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f1899600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f18940c76c0 con 0x7f18940c9eb0 2024-04-20T21:41:13.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f1892a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f189414fa50 0x7f18940c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f1893400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1894073a10 0x7f18940c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49164/0 (socket says 172.21.15.102:49164) 2024-04-20T21:41:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f1893400640 1 -- 172.21.15.102:0/2937653723 learned_addr learned my addr 172.21.15.102:0/2937653723 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.948+0000 7f188a000640 1 --2- 172.21.15.102:0/2937653723 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 0x7f1894158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.949+0000 7f1892a00640 1 -- 172.21.15.102:0/2937653723 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 msgr2=0x7f1894158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.949+0000 7f1892a00640 1 --2- 172.21.15.102:0/2937653723 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 0x7f1894158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.949+0000 7f1892a00640 1 -- 172.21.15.102:0/2937653723 >> v1:172.21.15.102:6789/0 conn(0x7f1894073a10 legacy=0x7f18940c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:13.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.949+0000 7f1892a00640 1 -- 172.21.15.102:0/2937653723 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18940c7960 con 0x7f189414fa50 2024-04-20T21:41:13.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.950+0000 7f1892a00640 1 --2- 172.21.15.102:0/2937653723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f189414fa50 0x7f18940c4830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f18940c7b30 tx=0x7f188402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb155ffce44ff968 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:13.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.950+0000 7f1892000640 1 -- 172.21.15.102:0/2937653723 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884034070 con 0x7f189414fa50 2024-04-20T21:41:13.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.950+0000 7f1892000640 1 -- 172.21.15.102:0/2937653723 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1884030800 con 0x7f189414fa50 2024-04-20T21:41:13.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.950+0000 7f1892000640 1 -- 172.21.15.102:0/2937653723 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884030bd0 con 0x7f189414fa50 2024-04-20T21:41:13.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.951+0000 7f1899600640 1 -- 172.21.15.102:0/2937653723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f189414fa50 msgr2=0x7f18940c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:13.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.951+0000 7f1899600640 1 --2- 172.21.15.102:0/2937653723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f189414fa50 0x7f18940c4830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f18940c7b30 tx=0x7f188402b940 comp rx=0 tx=0).stop 2024-04-20T21:41:13.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.951+0000 7f1899600640 1 -- 172.21.15.102:0/2937653723 shutdown_connections 2024-04-20T21:41:13.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.951+0000 7f1899600640 1 --2- 172.21.15.102:0/2937653723 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 0x7f1894158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.951+0000 7f1899600640 1 --2- 172.21.15.102:0/2937653723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f189414fa50 0x7f18940c4830 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.951+0000 7f1899600640 1 -- 172.21.15.102:0/2937653723 >> 172.21.15.102:0/2937653723 conn(0x7f18940c9390 msgr2=0x7f18940c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.951+0000 7f1899600640 1 -- 172.21.15.102:0/2937653723 shutdown_connections 2024-04-20T21:41:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.951+0000 7f1899600640 1 -- 172.21.15.102:0/2937653723 wait complete. 2024-04-20T21:41:13.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.953+0000 7f1899600640 1 Processor -- start 2024-04-20T21:41:13.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.953+0000 7f1899600640 1 -- start start 2024-04-20T21:41:13.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1899600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1894073a10 0x7f18941f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1899600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 0x7f18941575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1899600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f189414fa50 0x7f1894157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1899600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f18940c7c00 con 0x7f1894073a10 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1899600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f18940c3d90 con 0x7f189414fa50 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1899600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f18940c4030 con 0x7f18940c9eb0 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1892a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1894073a10 0x7f18941f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1892a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1894073a10 0x7f18941f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40018/0 (socket says 172.21.15.102:40018) 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1892a00640 1 -- 172.21.15.102:0/1236127323 learned_addr learned my addr 172.21.15.102:0/1236127323 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f188a000640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 0x7f18941575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.954+0000 7f1893400640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f189414fa50 0x7f1894157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f1892a00640 1 -- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 msgr2=0x7f18941575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:13.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f1892a00640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 0x7f18941575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:13.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f1892a00640 1 -- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f189414fa50 msgr2=0x7f1894157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:13.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f1892a00640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f189414fa50 0x7f1894157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:13.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f1892a00640 1 -- 172.21.15.102:0/1236127323 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18941f2f50 con 0x7f1894073a10 2024-04-20T21:41:13.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f1892a00640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1894073a10 0x7f18941f2460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1884009890 tx=0x7f18840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:13.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f188b400640 1 -- 172.21.15.102:0/1236127323 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884046020 con 0x7f1894073a10 2024-04-20T21:41:13.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f188b400640 1 -- 172.21.15.102:0/1236127323 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1884030600 con 0x7f1894073a10 2024-04-20T21:41:13.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f188b400640 1 -- 172.21.15.102:0/1236127323 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884034040 con 0x7f1894073a10 2024-04-20T21:41:13.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18940cb8b0 con 0x7f1894073a10 2024-04-20T21:41:13.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.955+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18940cbe70 con 0x7f1894073a10 2024-04-20T21:41:13.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.957+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1894074340 con 0x7f1894073a10 2024-04-20T21:41:13.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.957+0000 7f188b400640 1 -- 172.21.15.102:0/1236127323 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1884037070 con 0x7f1894073a10 2024-04-20T21:41:13.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.958+0000 7f188b400640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f187006c8f0 0x7f187006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:13.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.959+0000 7f188b400640 1 -- 172.21.15.102:0/1236127323 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f18840ae1f0 con 0x7f1894073a10 2024-04-20T21:41:13.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.959+0000 7f188a000640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f187006c8f0 0x7f187006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:13.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.959+0000 7f188a000640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f187006c8f0 0x7f187006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f189414f7b0 tx=0x7f187c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:13.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:13.963+0000 7f188b400640 1 -- 172.21.15.102:0/1236127323 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1884050020 con 0x7f1894073a10 2024-04-20T21:41:14.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.205+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18941568e0 con 0x7f1894073a10 2024-04-20T21:41:14.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.206+0000 7f188b400640 1 -- 172.21.15.102:0/1236127323 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7f188407b7b0 con 0x7f1894073a10 2024-04-20T21:41:14.205 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:14.206 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:14.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f187006c8f0 msgr2=0x7f187006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:14.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f187006c8f0 0x7f187006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f189414f7b0 tx=0x7f187c009190 comp rx=0 tx=0).stop 2024-04-20T21:41:14.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1894073a10 msgr2=0x7f18941f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:14.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1894073a10 0x7f18941f2460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1884009890 tx=0x7f18840046c0 comp rx=0 tx=0).stop 2024-04-20T21:41:14.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 shutdown_connections 2024-04-20T21:41:14.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f187006c8f0 0x7f187006edb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:14.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f189414fa50 0x7f1894157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:14.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18940c9eb0 0x7f18941575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:14.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 --2- 172.21.15.102:0/1236127323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1894073a10 0x7f18941f2460 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:14.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 >> 172.21.15.102:0/1236127323 conn(0x7f18940c9390 msgr2=0x7f18940c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:14.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 shutdown_connections 2024-04-20T21:41:14.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:14.209+0000 7f1899600640 1 -- 172.21.15.102:0/1236127323 wait complete. 2024-04-20T21:41:14.227 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:41:16.227 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:16.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.391+0000 7ff7a2000640 1 Processor -- start 2024-04-20T21:41:16.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.391+0000 7ff7a2000640 1 -- start start 2024-04-20T21:41:16.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a2000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff79c155530 0x7ff79c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:16.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a2000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c0cc470 0x7ff79c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:16.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a2000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79c15ef60 con 0x7ff79c0cdfc0 2024-04-20T21:41:16.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a2000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff79c0c5ca0 con 0x7ff79c155530 2024-04-20T21:41:16.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a2000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff79c0cd5f0 con 0x7ff79c0cc470 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a0c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff79c155530 0x7ff79c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff79be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c0cc470 0x7ff79c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a1600640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff79c0cdfc0 0x7ff79c0cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:51354/0 (socket says 172.21.15.102:51354) 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff79be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c0cc470 0x7ff79c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:34640/0 (socket says 172.21.15.102:34640) 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a0c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff79c155530 0x7ff79c15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:32930/0 (socket says 172.21.15.102:32930) 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff7a1600640 1 -- 172.21.15.102:0/565991559 learned_addr learned my addr 172.21.15.102:0/565991559 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff79be00640 1 -- 172.21.15.102:0/565991559 >> v2:172.21.15.193:3300/0 conn(0x7ff79c155530 msgr2=0x7ff79c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff79be00640 1 --2- 172.21.15.102:0/565991559 >> v2:172.21.15.193:3300/0 conn(0x7ff79c155530 0x7ff79c15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff79be00640 1 -- 172.21.15.102:0/565991559 >> v1:172.21.15.193:6789/0 conn(0x7ff79c0cdfc0 legacy=0x7ff79c0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:16.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.393+0000 7ff79be00640 1 -- 172.21.15.102:0/565991559 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c15f240 con 0x7ff79c0cc470 2024-04-20T21:41:16.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.394+0000 7ff79be00640 1 --2- 172.21.15.102:0/565991559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c0cc470 0x7ff79c0cda80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7840097a0 tx=0x7ff78402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87c902df1cbf150 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff79b400640 1 -- 172.21.15.102:0/565991559 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784035020 con 0x7ff79c0cc470 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff79b400640 1 -- 172.21.15.102:0/565991559 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff784031830 con 0x7ff79c0cc470 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff79b400640 1 -- 172.21.15.102:0/565991559 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784031c00 con 0x7ff79c0cc470 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff7a2000640 1 -- 172.21.15.102:0/565991559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c0cc470 msgr2=0x7ff79c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff7a2000640 1 --2- 172.21.15.102:0/565991559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c0cc470 0x7ff79c0cda80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7840097a0 tx=0x7ff78402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff7a2000640 1 -- 172.21.15.102:0/565991559 shutdown_connections 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff7a2000640 1 --2- 172.21.15.102:0/565991559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c0cc470 0x7ff79c0cda80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff7a2000640 1 --2- 172.21.15.102:0/565991559 >> v2:172.21.15.193:3300/0 conn(0x7ff79c155530 0x7ff79c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.395+0000 7ff7a2000640 1 -- 172.21.15.102:0/565991559 >> 172.21.15.102:0/565991559 conn(0x7ff79c075080 msgr2=0x7ff79c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.396+0000 7ff7a2000640 1 -- 172.21.15.102:0/565991559 shutdown_connections 2024-04-20T21:41:16.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.396+0000 7ff7a2000640 1 -- 172.21.15.102:0/565991559 wait complete. 2024-04-20T21:41:16.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.397+0000 7ff7a2000640 1 Processor -- start 2024-04-20T21:41:16.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.397+0000 7ff7a2000640 1 -- start start 2024-04-20T21:41:16.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.398+0000 7ff7a2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff79c0cc470 0x7ff79c1e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.398+0000 7ff7a2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff79c0cdfc0 0x7ff79c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.398+0000 7ff7a2000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c155530 0x7ff79c1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.398+0000 7ff7a2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c0cd280 con 0x7ff79c0cdfc0 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.398+0000 7ff7a2000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c0ccfa0 con 0x7ff79c0cc470 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.398+0000 7ff7a2000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff79c0ccbe0 con 0x7ff79c155530 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff79c0cc470 0x7ff79c1e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff79c0cc470 0x7ff79c1e5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:32936/0 (socket says 172.21.15.102:32936) 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 -- 172.21.15.102:0/3204328316 learned_addr learned my addr 172.21.15.102:0/3204328316 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff79be00640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff79c0cdfc0 0x7ff79c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a1600640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c155530 0x7ff79c1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 -- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c155530 msgr2=0x7ff79c1590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c155530 0x7ff79c1590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 -- 172.21.15.102:0/3204328316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff79c0cdfc0 msgr2=0x7ff79c15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff79c0cdfc0 0x7ff79c15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 -- 172.21.15.102:0/3204328316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c1e4680 con 0x7ff79c0cc470 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff7a0c00640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff79c0cc470 0x7ff79c1e5a80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00e870 tx=0x7ff78c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff798c00640 1 -- 172.21.15.102:0/3204328316 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c017070 con 0x7ff79c0cc470 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.399+0000 7ff798c00640 1 -- 172.21.15.102:0/3204328316 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff78c00c940 con 0x7ff79c0cc470 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.400+0000 7ff798c00640 1 -- 172.21.15.102:0/3204328316 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00cd30 con 0x7ff79c0cc470 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.400+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79c1e4940 con 0x7ff79c0cc470 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.400+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79c1e4f00 con 0x7ff79c0cc470 2024-04-20T21:41:16.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.401+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff79c1e58a0 con 0x7ff79c0cc470 2024-04-20T21:41:16.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.402+0000 7ff798c00640 1 -- 172.21.15.102:0/3204328316 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff78c01a070 con 0x7ff79c0cc470 2024-04-20T21:41:16.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.402+0000 7ff798c00640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff77806c920 0x7ff77806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:16.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.403+0000 7ff798c00640 1 -- 172.21.15.102:0/3204328316 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7ff78c08c5a0 con 0x7ff79c0cc470 2024-04-20T21:41:16.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.403+0000 7ff79be00640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff77806c920 0x7ff77806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:16.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.403+0000 7ff79be00640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff77806c920 0x7ff77806ede0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff784036000 tx=0x7ff784008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:16.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.407+0000 7ff798c00640 1 -- 172.21.15.102:0/3204328316 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff79c1e58a0 con 0x7ff79c0cc470 2024-04-20T21:41:16.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.634+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff79c04fd60 con 0x7ff79c0cc470 2024-04-20T21:41:16.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.635+0000 7ff798c00640 1 -- 172.21.15.102:0/3204328316 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7ff78c059a40 con 0x7ff79c0cc470 2024-04-20T21:41:16.635 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:16.635 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:16.635 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff77806c920 msgr2=0x7ff77806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff77806c920 0x7ff77806ede0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff784036000 tx=0x7ff784008030 comp rx=0 tx=0).stop 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff79c0cc470 msgr2=0x7ff79c1e5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff79c0cc470 0x7ff79c1e5a80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00e870 tx=0x7ff78c00ed30 comp rx=0 tx=0).stop 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 shutdown_connections 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff79c155530 0x7ff79c1590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff79c0cdfc0 0x7ff79c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff77806c920 0x7ff77806ede0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 --2- 172.21.15.102:0/3204328316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff79c0cc470 0x7ff79c1e5a80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 >> 172.21.15.102:0/3204328316 conn(0x7ff79c075080 msgr2=0x7ff79c0d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 shutdown_connections 2024-04-20T21:41:16.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:16.638+0000 7ff7a2000640 1 -- 172.21.15.102:0/3204328316 wait complete. 2024-04-20T21:41:16.655 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:41:18.655 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:18.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.865+0000 7fbcbde00640 1 Processor -- start 2024-04-20T21:41:18.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.866+0000 7fbcbde00640 1 -- start start 2024-04-20T21:41:18.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.867+0000 7fbcbde00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbcb8153180 0x7fbcb8158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:18.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.867+0000 7fbcbde00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb80c75a0 0x7fbcb81637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:18.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.867+0000 7fbcbde00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb8153560 con 0x7fbcb8159380 2024-04-20T21:41:18.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.867+0000 7fbcbde00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb80c7a70 con 0x7fbcb8153180 2024-04-20T21:41:18.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.867+0000 7fbcbde00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb80c7d30 con 0x7fbcb80c75a0 2024-04-20T21:41:18.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcbca00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbcb8153180 0x7fbcb8158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:18.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcafe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbcb8159380 0x7fbcb80c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50904/0 (socket says 172.21.15.102:50904) 2024-04-20T21:41:18.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcafe00640 1 -- 172.21.15.102:0/2129149241 learned_addr learned my addr 172.21.15.102:0/2129149241 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:18.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcbd400640 1 --2- 172.21.15.102:0/2129149241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb80c75a0 0x7fbcb81637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:18.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcbca00640 1 -- 172.21.15.102:0/2129149241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb80c75a0 msgr2=0x7fbcb81637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:18.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcbca00640 1 --2- 172.21.15.102:0/2129149241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb80c75a0 0x7fbcb81637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:18.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcbca00640 1 -- 172.21.15.102:0/2129149241 >> v1:172.21.15.102:6789/0 conn(0x7fbcb8159380 legacy=0x7fbcb80c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:18.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcbca00640 1 -- 172.21.15.102:0/2129149241 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb815c4d0 con 0x7fbcb8153180 2024-04-20T21:41:18.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.868+0000 7fbcb7e00640 1 -- 172.21.15.102:0/2129149241 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2589040912 0 0) 0x7fbcb8153560 con 0x7fbcb8159380 2024-04-20T21:41:18.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.869+0000 7fbcbca00640 1 --2- 172.21.15.102:0/2129149241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb8153180 0x7fbcb8158e40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097a0 tx=0x7fbca802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea52b19c0ccd28b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:18.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcb7e00640 1 -- 172.21.15.102:0/2129149241 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8035020 con 0x7fbcb8153180 2024-04-20T21:41:18.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcb7e00640 1 -- 172.21.15.102:0/2129149241 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca8031810 con 0x7fbcb8153180 2024-04-20T21:41:18.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcb7e00640 1 -- 172.21.15.102:0/2129149241 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8031be0 con 0x7fbcb8153180 2024-04-20T21:41:18.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcbde00640 1 -- 172.21.15.102:0/2129149241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb8153180 msgr2=0x7fbcb8158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:18.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcbde00640 1 --2- 172.21.15.102:0/2129149241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb8153180 0x7fbcb8158e40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097a0 tx=0x7fbca802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:41:18.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcbde00640 1 -- 172.21.15.102:0/2129149241 shutdown_connections 2024-04-20T21:41:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcbde00640 1 --2- 172.21.15.102:0/2129149241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb80c75a0 0x7fbcb81637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcbde00640 1 --2- 172.21.15.102:0/2129149241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb8153180 0x7fbcb8158e40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcbde00640 1 -- 172.21.15.102:0/2129149241 >> 172.21.15.102:0/2129149241 conn(0x7fbcb80cc820 msgr2=0x7fbcb80ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcbde00640 1 -- 172.21.15.102:0/2129149241 shutdown_connections 2024-04-20T21:41:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.870+0000 7fbcbde00640 1 -- 172.21.15.102:0/2129149241 wait complete. 2024-04-20T21:41:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.872+0000 7fbcbde00640 1 Processor -- start 2024-04-20T21:41:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.872+0000 7fbcbde00640 1 -- start start 2024-04-20T21:41:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.873+0000 7fbcbde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb80c75a0 0x7fbcb80c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.873+0000 7fbcbde00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb8153180 0x7fbcb80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.873+0000 7fbcbde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb8159380 0x7fbcb8070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.873+0000 7fbcbde00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb815d290 con 0x7fbcb8159380 2024-04-20T21:41:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.873+0000 7fbcbde00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb815ca90 con 0x7fbcb80c75a0 2024-04-20T21:41:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.873+0000 7fbcbde00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb80c6be0 con 0x7fbcb8153180 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb80c75a0 0x7fbcb80c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcafe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb8153180 0x7fbcb80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb8159380 0x7fbcb8070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb80c75a0 0x7fbcb80c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54506/0 (socket says 172.21.15.102:54506) 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcafe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb8153180 0x7fbcb80cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:60042/0 (socket says 172.21.15.102:60042) 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 -- 172.21.15.102:0/3583379136 learned_addr learned my addr 172.21.15.102:0/3583379136 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 -- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb8153180 msgr2=0x7fbcb80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb8153180 0x7fbcb80cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 -- 172.21.15.102:0/3583379136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb8159380 msgr2=0x7fbcb8070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb8159380 0x7fbcb8070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 -- 172.21.15.102:0/3583379136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb8147ad0 con 0x7fbcb80c75a0 2024-04-20T21:41:18.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcafe00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb8153180 0x7fbcb80cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:18.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.874+0000 7fbcbca00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb80c75a0 0x7fbcb80c59f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbca80098c0 tx=0x7fbca80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:18.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.875+0000 7fbcb5600640 1 -- 172.21.15.102:0/3583379136 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8041070 con 0x7fbcb80c75a0 2024-04-20T21:41:18.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.875+0000 7fbcb5600640 1 -- 172.21.15.102:0/3583379136 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca803ad00 con 0x7fbcb80c75a0 2024-04-20T21:41:18.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.875+0000 7fbcb5600640 1 -- 172.21.15.102:0/3583379136 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8035040 con 0x7fbcb80c75a0 2024-04-20T21:41:18.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.875+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb8147d90 con 0x7fbcb80c75a0 2024-04-20T21:41:18.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.875+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb81464e0 con 0x7fbcb80c75a0 2024-04-20T21:41:18.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.876+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb804fd60 con 0x7fbcb80c75a0 2024-04-20T21:41:18.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.877+0000 7fbcb5600640 1 -- 172.21.15.102:0/3583379136 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbca804c060 con 0x7fbcb80c75a0 2024-04-20T21:41:18.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.878+0000 7fbcb5600640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbc9406c8f0 0x7fbc9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:18.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.878+0000 7fbcb5600640 1 -- 172.21.15.102:0/3583379136 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7fbca80addf0 con 0x7fbcb80c75a0 2024-04-20T21:41:18.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.878+0000 7fbcafe00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbc9406c8f0 0x7fbc9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:18.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.879+0000 7fbcafe00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbc9406c8f0 0x7fbc9406edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8071bf0 tx=0x7fbca0007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:18.883+0000 7fbcb5600640 1 -- 172.21.15.102:0/3583379136 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbca807b3b0 con 0x7fbcb80c75a0 2024-04-20T21:41:19.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.121+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcb80c69c0 con 0x7fbcb80c75a0 2024-04-20T21:41:19.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.122+0000 7fbcb5600640 1 -- 172.21.15.102:0/3583379136 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7fbca807f6b0 con 0x7fbcb80c75a0 2024-04-20T21:41:19.122 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:19.122 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:19.122 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:19.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbc9406c8f0 msgr2=0x7fbc9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:19.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbc9406c8f0 0x7fbc9406edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8071bf0 tx=0x7fbca0007d50 comp rx=0 tx=0).stop 2024-04-20T21:41:19.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb80c75a0 msgr2=0x7fbcb80c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:19.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb80c75a0 0x7fbcb80c59f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbca80098c0 tx=0x7fbca80046c0 comp rx=0 tx=0).stop 2024-04-20T21:41:19.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 shutdown_connections 2024-04-20T21:41:19.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb8159380 0x7fbcb8070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:19.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb8153180 0x7fbcb80cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:19.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbc9406c8f0 0x7fbc9406edb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:19.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 --2- 172.21.15.102:0/3583379136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb80c75a0 0x7fbcb80c59f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:19.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 >> 172.21.15.102:0/3583379136 conn(0x7fbcb80cc820 msgr2=0x7fbcb80c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:19.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 shutdown_connections 2024-04-20T21:41:19.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:19.125+0000 7fbcbde00640 1 -- 172.21.15.102:0/3583379136 wait complete. 2024-04-20T21:41:19.144 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:41:21.144 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:21.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.360+0000 7fbd39600640 1 Processor -- start 2024-04-20T21:41:21.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.360+0000 7fbd39600640 1 -- start start 2024-04-20T21:41:21.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.362+0000 7fbd39600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbd340c8db0 0x7fbd34070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:21.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.362+0000 7fbd39600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd34158e80 con 0x7fbd34071330 2024-04-20T21:41:21.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.362+0000 7fbd39600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd340c91c0 con 0x7fbd34152170 2024-04-20T21:41:21.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.362+0000 7fbd39600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fbd340c6dc0 con 0x7fbd340c8db0 2024-04-20T21:41:21.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.362+0000 7fbd33400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbd34071330 0x7fbd341628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50912/0 (socket says 172.21.15.102:50912) 2024-04-20T21:41:21.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.362+0000 7fbd33400640 1 -- 172.21.15.102:0/2176348145 learned_addr learned my addr 172.21.15.102:0/2176348145 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:21.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.362+0000 7fbd2a000640 1 --2- 172.21.15.102:0/2176348145 >> v2:172.21.15.193:3300/0 conn(0x7fbd340c8db0 0x7fbd34070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:21.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.363+0000 7fbd32000640 1 -- 172.21.15.102:0/2176348145 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280825153 0 0) 0x7fbd340c91c0 con 0x7fbd34152170 2024-04-20T21:41:21.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.363+0000 7fbd32000640 1 -- 172.21.15.102:0/2176348145 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd18003520 con 0x7fbd34152170 2024-04-20T21:41:21.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.363+0000 7fbd32000640 1 -- 172.21.15.102:0/2176348145 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 669614876 0 0) 0x7fbd34158e80 con 0x7fbd34071330 2024-04-20T21:41:21.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.363+0000 7fbd2a000640 1 -- 172.21.15.102:0/2176348145 >> v1:172.21.15.193:6789/0 conn(0x7fbd34152170 legacy=0x7fbd34158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:21.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.363+0000 7fbd2a000640 1 -- 172.21.15.102:0/2176348145 >> v1:172.21.15.102:6789/0 conn(0x7fbd34071330 legacy=0x7fbd341628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:21.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.363+0000 7fbd2a000640 1 -- 172.21.15.102:0/2176348145 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd34071f70 con 0x7fbd340c8db0 2024-04-20T21:41:21.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.364+0000 7fbd2a000640 1 --2- 172.21.15.102:0/2176348145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd340c8db0 0x7fbd34070df0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0097a0 tx=0x7fbd1c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d8f0b655e2ff477 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:21.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.364+0000 7fbd32000640 1 -- 172.21.15.102:0/2176348145 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c034070 con 0x7fbd340c8db0 2024-04-20T21:41:21.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.364+0000 7fbd32000640 1 -- 172.21.15.102:0/2176348145 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c030610 con 0x7fbd340c8db0 2024-04-20T21:41:21.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.364+0000 7fbd32000640 1 -- 172.21.15.102:0/2176348145 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0309c0 con 0x7fbd340c8db0 2024-04-20T21:41:21.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.365+0000 7fbd39600640 1 -- 172.21.15.102:0/2176348145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd340c8db0 msgr2=0x7fbd34070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:21.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.365+0000 7fbd39600640 1 --2- 172.21.15.102:0/2176348145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd340c8db0 0x7fbd34070df0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0097a0 tx=0x7fbd1c02b960 comp rx=0 tx=0).stop 2024-04-20T21:41:21.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.365+0000 7fbd39600640 1 -- 172.21.15.102:0/2176348145 shutdown_connections 2024-04-20T21:41:21.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.365+0000 7fbd39600640 1 --2- 172.21.15.102:0/2176348145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd340c8db0 0x7fbd34070df0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:21.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.365+0000 7fbd39600640 1 -- 172.21.15.102:0/2176348145 >> 172.21.15.102:0/2176348145 conn(0x7fbd340c9390 msgr2=0x7fbd340c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:21.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.365+0000 7fbd39600640 1 -- 172.21.15.102:0/2176348145 shutdown_connections 2024-04-20T21:41:21.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.365+0000 7fbd39600640 1 -- 172.21.15.102:0/2176348145 wait complete. 2024-04-20T21:41:21.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.367+0000 7fbd39600640 1 Processor -- start 2024-04-20T21:41:21.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.367+0000 7fbd39600640 1 -- start start 2024-04-20T21:41:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd39600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd34071330 0x7fbd34157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd39600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340c8db0 0x7fbd3415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd39600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd34152170 0x7fbd3415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd39600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340724b0 con 0x7fbd340c8db0 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd39600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34072210 con 0x7fbd34152170 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd39600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbd34158250 con 0x7fbd34071330 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd2a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340c8db0 0x7fbd3415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd2a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340c8db0 0x7fbd3415a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54182/0 (socket says 172.21.15.102:54182) 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd2a000640 1 -- 172.21.15.102:0/3721188411 learned_addr learned my addr 172.21.15.102:0/3721188411 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd33400640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd34152170 0x7fbd3415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.368+0000 7fbd32a00640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd34071330 0x7fbd34157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2a000640 1 -- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd34071330 msgr2=0x7fbd34157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2a000640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd34071330 0x7fbd34157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2a000640 1 -- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd34152170 msgr2=0x7fbd3415a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:21.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2a000640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd34152170 0x7fbd3415a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:21.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2a000640 1 -- 172.21.15.102:0/3721188411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd34151ca0 con 0x7fbd340c8db0 2024-04-20T21:41:21.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2a000640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340c8db0 0x7fbd3415a340 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c000c00 tx=0x7fbd1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:21.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2b400640 1 -- 172.21.15.102:0/3721188411 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c048070 con 0x7fbd340c8db0 2024-04-20T21:41:21.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2b400640 1 -- 172.21.15.102:0/3721188411 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c030410 con 0x7fbd340c8db0 2024-04-20T21:41:21.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd34150bb0 con 0x7fbd340c8db0 2024-04-20T21:41:21.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd2b400640 1 -- 172.21.15.102:0/3721188411 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c034050 con 0x7fbd340c8db0 2024-04-20T21:41:21.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.369+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3414fc90 con 0x7fbd340c8db0 2024-04-20T21:41:21.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.371+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd34074340 con 0x7fbd340c8db0 2024-04-20T21:41:21.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.372+0000 7fbd2b400640 1 -- 172.21.15.102:0/3721188411 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd1c04b070 con 0x7fbd340c8db0 2024-04-20T21:41:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.373+0000 7fbd2b400640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbd1006c8f0 0x7fbd1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.373+0000 7fbd2b400640 1 -- 172.21.15.102:0/3721188411 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7fbd1c0adf50 con 0x7fbd340c8db0 2024-04-20T21:41:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.373+0000 7fbd32a00640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbd1006c8f0 0x7fbd1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.374+0000 7fbd32a00640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbd1006c8f0 0x7fbd1006edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbd24001ea0 tx=0x7fbd24009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:21.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.378+0000 7fbd2b400640 1 -- 172.21.15.102:0/3721188411 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd1c07b480 con 0x7fbd340c8db0 2024-04-20T21:41:21.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.620+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd3406da40 con 0x7fbd340c8db0 2024-04-20T21:41:21.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.621+0000 7fbd2b400640 1 -- 172.21.15.102:0/3721188411 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7fbd1c07f780 con 0x7fbd340c8db0 2024-04-20T21:41:21.621 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:21.621 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:21.621 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbd1006c8f0 msgr2=0x7fbd1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbd1006c8f0 0x7fbd1006edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbd24001ea0 tx=0x7fbd24009040 comp rx=0 tx=0).stop 2024-04-20T21:41:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340c8db0 msgr2=0x7fbd3415a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340c8db0 0x7fbd3415a340 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c000c00 tx=0x7fbd1c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:41:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 shutdown_connections 2024-04-20T21:41:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbd1006c8f0 0x7fbd1006edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd34152170 0x7fbd3415a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340c8db0 0x7fbd3415a340 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 --2- 172.21.15.102:0/3721188411 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd34071330 0x7fbd34157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.624+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 >> 172.21.15.102:0/3721188411 conn(0x7fbd340c9390 msgr2=0x7fbd3406f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.625+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 shutdown_connections 2024-04-20T21:41:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:21.625+0000 7fbd39600640 1 -- 172.21.15.102:0/3721188411 wait complete. 2024-04-20T21:41:21.643 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:41:23.645 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:23.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.859+0000 7f22bac00640 1 Processor -- start 2024-04-20T21:41:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.859+0000 7f22bac00640 1 -- start start 2024-04-20T21:41:23.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.861+0000 7f22bac00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f22b406c550 0x7f22b415c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:23.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22bac00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22b4073a20 con 0x7f22b415bfd0 2024-04-20T21:41:23.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22bac00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22b415b510 con 0x7f22b41554f0 2024-04-20T21:41:23.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22bac00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f22b4073cc0 con 0x7f22b406c550 2024-04-20T21:41:23.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22b3400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f22b406c550 0x7f22b415c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:23.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22b3400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f22b406c550 0x7f22b415c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54544/0 (socket says 172.21.15.102:54544) 2024-04-20T21:41:23.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22b3400640 1 -- 172.21.15.102:0/444267449 learned_addr learned my addr 172.21.15.102:0/444267449 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:23.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22b3400640 1 -- 172.21.15.102:0/444267449 >> v1:172.21.15.193:6789/0 conn(0x7f22b41554f0 legacy=0x7f22b40c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:23.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22b3400640 1 -- 172.21.15.102:0/444267449 >> v1:172.21.15.102:6789/0 conn(0x7f22b415bfd0 legacy=0x7f22b40ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:23.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22b3400640 1 -- 172.21.15.102:0/444267449 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b415b7d0 con 0x7f22b406c550 2024-04-20T21:41:23.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.862+0000 7f22b2a00640 1 -- 172.21.15.102:0/444267449 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420642388 0 0) 0x7f22b415b510 con 0x7f22b41554f0 2024-04-20T21:41:23.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.863+0000 7f22b3400640 1 --2- 172.21.15.102:0/444267449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 0x7f22b415c810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009870 tx=0x7f22a002ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bb3ac8db33d4b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:23.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.864+0000 7f22b2a00640 1 -- 172.21.15.102:0/444267449 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a002f850 con 0x7f22b406c550 2024-04-20T21:41:23.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.864+0000 7f22b2a00640 1 -- 172.21.15.102:0/444267449 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a002f9f0 con 0x7f22b406c550 2024-04-20T21:41:23.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.864+0000 7f22b2a00640 1 -- 172.21.15.102:0/444267449 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a002fdc0 con 0x7f22b406c550 2024-04-20T21:41:23.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.864+0000 7f22bac00640 1 -- 172.21.15.102:0/444267449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 msgr2=0x7f22b415c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.864+0000 7f22bac00640 1 --2- 172.21.15.102:0/444267449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 0x7f22b415c810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009870 tx=0x7f22a002ba30 comp rx=0 tx=0).stop 2024-04-20T21:41:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.864+0000 7f22bac00640 1 -- 172.21.15.102:0/444267449 shutdown_connections 2024-04-20T21:41:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.864+0000 7f22bac00640 1 --2- 172.21.15.102:0/444267449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 0x7f22b415c810 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.864+0000 7f22bac00640 1 -- 172.21.15.102:0/444267449 >> 172.21.15.102:0/444267449 conn(0x7f22b406ebf0 msgr2=0x7f22b406efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.865+0000 7f22bac00640 1 -- 172.21.15.102:0/444267449 shutdown_connections 2024-04-20T21:41:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.865+0000 7f22bac00640 1 -- 172.21.15.102:0/444267449 wait complete. 2024-04-20T21:41:23.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.866+0000 7f22bac00640 1 Processor -- start 2024-04-20T21:41:23.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.867+0000 7f22bac00640 1 -- start start 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.867+0000 7f22bac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 0x7f22b4153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.867+0000 7f22bac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22b41554f0 0x7f22b40c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.867+0000 7f22bac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b415bfd0 0x7f22b414d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.867+0000 7f22bac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22b40ca000 con 0x7f22b415bfd0 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.867+0000 7f22bac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22b40736e0 con 0x7f22b406c550 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.867+0000 7f22bac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f22b41558d0 con 0x7f22b41554f0 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b415bfd0 0x7f22b414d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b3400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22b41554f0 0x7f22b40c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b415bfd0 0x7f22b414d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54188/0 (socket says 172.21.15.102:54188) 2024-04-20T21:41:23.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b8c00640 1 -- 172.21.15.102:0/36808862 learned_addr learned my addr 172.21.15.102:0/36808862 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b3e00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 0x7f22b4153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b8c00640 1 -- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22b41554f0 msgr2=0x7f22b40c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b8c00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22b41554f0 0x7f22b40c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b8c00640 1 -- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 msgr2=0x7f22b4153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b8c00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 0x7f22b4153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b8c00640 1 -- 172.21.15.102:0/36808862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b41e52b0 con 0x7f22b415bfd0 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b3400640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22b41554f0 0x7f22b40c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.868+0000 7f22b3e00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 0x7f22b4153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.869+0000 7f22b8c00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b415bfd0 0x7f22b414d850 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c3c0 tx=0x7f22ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.869+0000 7f22abe00640 1 -- 172.21.15.102:0/36808862 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac017070 con 0x7f22b415bfd0 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.869+0000 7f22abe00640 1 -- 172.21.15.102:0/36808862 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac0052d0 con 0x7f22b415bfd0 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.869+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b41f1140 con 0x7f22b415bfd0 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.869+0000 7f22abe00640 1 -- 172.21.15.102:0/36808862 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac003850 con 0x7f22b415bfd0 2024-04-20T21:41:23.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.869+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b41f1700 con 0x7f22b415bfd0 2024-04-20T21:41:23.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.870+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b4074340 con 0x7f22b415bfd0 2024-04-20T21:41:23.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.871+0000 7f22abe00640 1 -- 172.21.15.102:0/36808862 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22ac01a070 con 0x7f22b415bfd0 2024-04-20T21:41:23.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.872+0000 7f22abe00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f229406c920 0x7f229406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:23.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.872+0000 7f22abe00640 1 -- 172.21.15.102:0/36808862 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7767+0+0 (secure 0 0 0) 0x7f22ac08b620 con 0x7f22b415bfd0 2024-04-20T21:41:23.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.873+0000 7f22b3e00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f229406c920 0x7f229406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:23.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.873+0000 7f22b3e00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f229406c920 0x7f229406ede0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f22a40012e0 tx=0x7f22a4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:23.877+0000 7f22abe00640 1 -- 172.21.15.102:0/36808862 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f22ac058b60 con 0x7f22b415bfd0 2024-04-20T21:41:24.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.122+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22b40c96d0 con 0x7f22b415bfd0 2024-04-20T21:41:24.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.123+0000 7f22abe00640 1 -- 172.21.15.102:0/36808862 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+6862 (secure 0 0 0) 0x7f22ac05ce60 con 0x7f22b415bfd0 2024-04-20T21:41:24.122 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:24.123 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:40:02.424745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":18,"addr":"172.21.15.102:6835/3586768191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3586768191},{"type":"v1","addr":"172.21.15.102:6835","nonce":3586768191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":1}]} 2024-04-20T21:41:24.123 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 79 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f229406c920 msgr2=0x7f229406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f229406c920 0x7f229406ede0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f22a40012e0 tx=0x7f22a4009040 comp rx=0 tx=0).stop 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b415bfd0 msgr2=0x7f22b414d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b415bfd0 0x7f22b414d850 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c3c0 tx=0x7f22ac00c880 comp rx=0 tx=0).stop 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 shutdown_connections 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b415bfd0 0x7f22b414d850 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22b41554f0 0x7f22b40c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f229406c920 0x7f229406ede0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 --2- 172.21.15.102:0/36808862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22b406c550 0x7f22b4153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 >> 172.21.15.102:0/36808862 conn(0x7f22b406ebf0 msgr2=0x7f22b415a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:24.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 shutdown_connections 2024-04-20T21:41:24.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:24.126+0000 7f22bac00640 1 -- 172.21.15.102:0/36808862 wait complete. 2024-04-20T21:41:24.143 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:41:26.144 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:26.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.356+0000 7f41a3000640 1 Processor -- start 2024-04-20T21:41:26.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.356+0000 7f41a3000640 1 -- start start 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f41a3000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f419c14fa50 0x7f419c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f41a3000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f419c073a10 0x7f419c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f41a3000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f419c1536d0 con 0x7f419c0c9eb0 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f41a3000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f419c159500 con 0x7f419c073a10 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f41a3000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f419c0c76c0 con 0x7f419c14fa50 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f419b400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f419c0c9eb0 0x7f419c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:50936/0 (socket says 172.21.15.102:50936) 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f419b400640 1 -- 172.21.15.102:0/669121850 learned_addr learned my addr 172.21.15.102:0/669121850 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f41a1000640 1 --2- 172.21.15.102:0/669121850 >> v2:172.21.15.102:3300/0 conn(0x7f419c073a10 0x7f419c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.358+0000 7f419be00640 1 --2- 172.21.15.102:0/669121850 >> v2:172.21.15.193:3300/0 conn(0x7f419c14fa50 0x7f419c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.359+0000 7f419aa00640 1 -- 172.21.15.102:0/669121850 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4054386640 0 0) 0x7f4188001110 con 0x7f419c0c9eb0 2024-04-20T21:41:26.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.359+0000 7f419aa00640 1 -- 172.21.15.102:0/669121850 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4180003520 con 0x7f419c0c9eb0 2024-04-20T21:41:26.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.359+0000 7f41a1000640 1 -- 172.21.15.102:0/669121850 >> v2:172.21.15.193:3300/0 conn(0x7f419c14fa50 msgr2=0x7f419c0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:26.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.359+0000 7f41a1000640 1 --2- 172.21.15.102:0/669121850 >> v2:172.21.15.193:3300/0 conn(0x7f419c14fa50 0x7f419c0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.359+0000 7f41a1000640 1 -- 172.21.15.102:0/669121850 >> v1:172.21.15.102:6789/0 conn(0x7f419c0c9eb0 legacy=0x7f419c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.359+0000 7f41a1000640 1 -- 172.21.15.102:0/669121850 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c0c7960 con 0x7f419c073a10 2024-04-20T21:41:26.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.360+0000 7f41a1000640 1 --2- 172.21.15.102:0/669121850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c073a10 0x7f419c158fc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f419c0706a0 tx=0x7f419402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c064a8e929c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:26.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.360+0000 7f419aa00640 1 -- 172.21.15.102:0/669121850 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194039070 con 0x7f419c073a10 2024-04-20T21:41:26.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.360+0000 7f419aa00640 1 -- 172.21.15.102:0/669121850 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419402cd00 con 0x7f419c073a10 2024-04-20T21:41:26.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.360+0000 7f419aa00640 1 -- 172.21.15.102:0/669121850 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194034640 con 0x7f419c073a10 2024-04-20T21:41:26.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.361+0000 7f41a3000640 1 -- 172.21.15.102:0/669121850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c073a10 msgr2=0x7f419c158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.361+0000 7f41a3000640 1 --2- 172.21.15.102:0/669121850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c073a10 0x7f419c158fc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f419c0706a0 tx=0x7f419402bf50 comp rx=0 tx=0).stop 2024-04-20T21:41:26.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.361+0000 7f41a3000640 1 -- 172.21.15.102:0/669121850 shutdown_connections 2024-04-20T21:41:26.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.361+0000 7f41a3000640 1 --2- 172.21.15.102:0/669121850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c073a10 0x7f419c158fc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.361+0000 7f41a3000640 1 --2- 172.21.15.102:0/669121850 >> v2:172.21.15.193:3300/0 conn(0x7f419c14fa50 0x7f419c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.361+0000 7f41a3000640 1 -- 172.21.15.102:0/669121850 >> 172.21.15.102:0/669121850 conn(0x7f419c0c9390 msgr2=0x7f419c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:26.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.361+0000 7f41a3000640 1 -- 172.21.15.102:0/669121850 shutdown_connections 2024-04-20T21:41:26.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.361+0000 7f41a3000640 1 -- 172.21.15.102:0/669121850 wait complete. 2024-04-20T21:41:26.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.363+0000 7f41a3000640 1 Processor -- start 2024-04-20T21:41:26.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.363+0000 7f41a3000640 1 -- start start 2024-04-20T21:41:26.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a3000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f419c073a10 0x7f419c071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a3000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f419c0c9eb0 0x7f419c06ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a3000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c14fa50 0x7f419c14a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a3000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f419c0c7c00 con 0x7f419c14fa50 2024-04-20T21:41:26.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a3000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f419c0c3d90 con 0x7f419c073a10 2024-04-20T21:41:26.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a3000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f419c0c4030 con 0x7f419c0c9eb0 2024-04-20T21:41:26.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c14fa50 0x7f419c14a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c14fa50 0x7f419c14a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54208/0 (socket says 172.21.15.102:54208) 2024-04-20T21:41:26.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.364+0000 7f41a1000640 1 -- 172.21.15.102:0/3175080750 learned_addr learned my addr 172.21.15.102:0/3175080750 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:26.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f419b400640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f419c0c9eb0 0x7f419c06ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f419be00640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f419c073a10 0x7f419c071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f41a1000640 1 -- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f419c0c9eb0 msgr2=0x7f419c06ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f41a1000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f419c0c9eb0 0x7f419c06ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f41a1000640 1 -- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f419c073a10 msgr2=0x7f419c071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f41a1000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f419c073a10 0x7f419c071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f41a1000640 1 -- 172.21.15.102:0/3175080750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c072310 con 0x7f419c14fa50 2024-04-20T21:41:26.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f419b400640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f419c0c9eb0 0x7f419c06ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:26.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f41a1000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c14fa50 0x7f419c14a3d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4194009e80 tx=0x7f4194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:26.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.365+0000 7f4193e00640 1 -- 172.21.15.102:0/3175080750 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194039050 con 0x7f419c14fa50 2024-04-20T21:41:26.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.366+0000 7f4193e00640 1 -- 172.21.15.102:0/3175080750 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4194045070 con 0x7f419c14fa50 2024-04-20T21:41:26.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.366+0000 7f4193e00640 1 -- 172.21.15.102:0/3175080750 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194009450 con 0x7f419c14fa50 2024-04-20T21:41:26.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.366+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419c0725f0 con 0x7f419c14fa50 2024-04-20T21:41:26.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.366+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419c072bb0 con 0x7f419c14fa50 2024-04-20T21:41:26.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.367+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f419c074340 con 0x7f419c14fa50 2024-04-20T21:41:26.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.368+0000 7f4193e00640 1 -- 172.21.15.102:0/3175080750 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f419403e830 con 0x7f419c14fa50 2024-04-20T21:41:26.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.369+0000 7f4193e00640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f417c06c8f0 0x7f417c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.369+0000 7f4193e00640 1 -- 172.21.15.102:0/3175080750 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f41940aefd0 con 0x7f419c14fa50 2024-04-20T21:41:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.369+0000 7f419be00640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f417c06c8f0 0x7f417c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.370+0000 7f419be00640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f417c06c8f0 0x7f417c06edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f418c009aa0 tx=0x7f418c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.374+0000 7f4193e00640 1 -- 172.21.15.102:0/3175080750 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f419407cce0 con 0x7f419c14fa50 2024-04-20T21:41:26.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.613+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f419c0c7320 con 0x7f419c14fa50 2024-04-20T21:41:26.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.614+0000 7f4193e00640 1 -- 172.21.15.102:0/3175080750 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+6026 (secure 0 0 0) 0x7f419407c5e0 con 0x7f419c14fa50 2024-04-20T21:41:26.613 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:26.613 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:25.855269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:26.614 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 80 2024-04-20T21:41:26.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f417c06c8f0 msgr2=0x7f417c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f417c06c8f0 0x7f417c06edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f418c009aa0 tx=0x7f418c009190 comp rx=0 tx=0).stop 2024-04-20T21:41:26.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c14fa50 msgr2=0x7f419c14a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c14fa50 0x7f419c14a3d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4194009e80 tx=0x7f4194004a40 comp rx=0 tx=0).stop 2024-04-20T21:41:26.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 shutdown_connections 2024-04-20T21:41:26.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f419c14fa50 0x7f419c14a3d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f419c0c9eb0 0x7f419c06ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f417c06c8f0 0x7f417c06edb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 --2- 172.21.15.102:0/3175080750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f419c073a10 0x7f419c071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 >> 172.21.15.102:0/3175080750 conn(0x7f419c0c9390 msgr2=0x7f419c0c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:26.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 shutdown_connections 2024-04-20T21:41:26.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.617+0000 7f41a3000640 1 -- 172.21.15.102:0/3175080750 wait complete. 2024-04-20T21:41:26.636 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:41:26.637 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:41:26.637 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:26.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.838+0000 7f0638800640 1 Processor -- start 2024-04-20T21:41:26.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.838+0000 7f0638800640 1 -- start start 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.839+0000 7f0638800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f063014fa50 0x7f06300c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.839+0000 7f0638800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0638800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06301536d0 con 0x7f06300c9eb0 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0638800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f0630159500 con 0x7f063014fa50 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0638800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f06300c76c0 con 0x7f0630073a10 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0635400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f06300c9eb0 0x7f06300c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:42950/0 (socket says 172.21.15.102:42950) 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0635400640 1 -- 172.21.15.102:0/3841045350 learned_addr learned my addr 172.21.15.102:0/3841045350 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0636800640 1 --2- 172.21.15.102:0/3841045350 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0635e00640 1 --2- 172.21.15.102:0/3841045350 >> v2:172.21.15.102:3300/0 conn(0x7f063014fa50 0x7f06300c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0634a00640 1 -- 172.21.15.102:0/3841045350 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227076022 0 0) 0x7f0620001110 con 0x7f06300c9eb0 2024-04-20T21:41:26.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0634a00640 1 -- 172.21.15.102:0/3841045350 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0618003520 con 0x7f06300c9eb0 2024-04-20T21:41:26.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.840+0000 7f0636800640 1 -- 172.21.15.102:0/3841045350 >> v2:172.21.15.102:3300/0 conn(0x7f063014fa50 msgr2=0x7f06300c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:26.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.841+0000 7f0636800640 1 --2- 172.21.15.102:0/3841045350 >> v2:172.21.15.102:3300/0 conn(0x7f063014fa50 0x7f06300c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.841+0000 7f0636800640 1 -- 172.21.15.102:0/3841045350 >> v1:172.21.15.193:6789/0 conn(0x7f06300c9eb0 legacy=0x7f06300c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.841+0000 7f0636800640 1 -- 172.21.15.102:0/3841045350 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06300c7960 con 0x7f0630073a10 2024-04-20T21:41:26.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.841+0000 7f0635e00640 1 --2- 172.21.15.102:0/3841045350 >> v2:172.21.15.102:3300/0 conn(0x7f063014fa50 0x7f06300c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.842+0000 7f0636800640 1 --2- 172.21.15.102:0/3841045350 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630158fc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f062c009d90 tx=0x7f062c02bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42b0c12fba1ec14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.842+0000 7f0634a00640 1 -- 172.21.15.102:0/3841045350 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c039070 con 0x7f0630073a10 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.842+0000 7f0634a00640 1 -- 172.21.15.102:0/3841045350 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c02cce0 con 0x7f0630073a10 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.842+0000 7f0634a00640 1 -- 172.21.15.102:0/3841045350 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c034660 con 0x7f0630073a10 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.843+0000 7f0638800640 1 -- 172.21.15.102:0/3841045350 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 msgr2=0x7f0630158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.843+0000 7f0638800640 1 --2- 172.21.15.102:0/3841045350 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630158fc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f062c009d90 tx=0x7f062c02bf30 comp rx=0 tx=0).stop 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.843+0000 7f0638800640 1 -- 172.21.15.102:0/3841045350 shutdown_connections 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.843+0000 7f0638800640 1 --2- 172.21.15.102:0/3841045350 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630158fc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.843+0000 7f0638800640 1 --2- 172.21.15.102:0/3841045350 >> v2:172.21.15.102:3300/0 conn(0x7f063014fa50 0x7f06300c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.843+0000 7f0638800640 1 -- 172.21.15.102:0/3841045350 >> 172.21.15.102:0/3841045350 conn(0x7f06300c9390 msgr2=0x7f06300c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.843+0000 7f0638800640 1 -- 172.21.15.102:0/3841045350 shutdown_connections 2024-04-20T21:41:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.843+0000 7f0638800640 1 -- 172.21.15.102:0/3841045350 wait complete. 2024-04-20T21:41:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.845+0000 7f0638800640 1 Processor -- start 2024-04-20T21:41:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.845+0000 7f0638800640 1 -- start start 2024-04-20T21:41:26.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0638800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0638800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06300c9eb0 0x7f06301f1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0638800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063014fa50 0x7f06301f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0638800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06300c7c00 con 0x7f063014fa50 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0638800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06300c3d90 con 0x7f06300c9eb0 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0638800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f06300c4030 con 0x7f0630073a10 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0636800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063014fa50 0x7f06301f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0635e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0635400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06300c9eb0 0x7f06301f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0636800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063014fa50 0x7f06301f1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54218/0 (socket says 172.21.15.102:54218) 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0636800640 1 -- 172.21.15.102:0/354308952 learned_addr learned my addr 172.21.15.102:0/354308952 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0635e00640 1 -- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06300c9eb0 msgr2=0x7f06301f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0635e00640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06300c9eb0 0x7f06301f1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0635e00640 1 -- 172.21.15.102:0/354308952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063014fa50 msgr2=0x7f06301f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0635e00640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063014fa50 0x7f06301f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0635e00640 1 -- 172.21.15.102:0/354308952 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0630072bc0 con 0x7f0630073a10 2024-04-20T21:41:26.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.846+0000 7f0636800640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063014fa50 0x7f06301f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:26.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.847+0000 7f0635e00640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630155580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f062400e900 tx=0x7f062400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:26.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.847+0000 7f062a000640 1 -- 172.21.15.102:0/354308952 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624017070 con 0x7f0630073a10 2024-04-20T21:41:26.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.847+0000 7f062a000640 1 -- 172.21.15.102:0/354308952 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062400ca00 con 0x7f0630073a10 2024-04-20T21:41:26.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.847+0000 7f062a000640 1 -- 172.21.15.102:0/354308952 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062400cdf0 con 0x7f0630073a10 2024-04-20T21:41:26.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.847+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06300cb880 con 0x7f0630073a10 2024-04-20T21:41:26.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.847+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f06300cbe40 con 0x7f0630073a10 2024-04-20T21:41:26.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.848+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063014d510 con 0x7f0630073a10 2024-04-20T21:41:26.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.849+0000 7f062a000640 1 -- 172.21.15.102:0/354308952 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f062401a070 con 0x7f0630073a10 2024-04-20T21:41:26.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.850+0000 7f062a000640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c8f0 0x7f061406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:26.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.850+0000 7f062a000640 1 -- 172.21.15.102:0/354308952 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f062408c590 con 0x7f0630073a10 2024-04-20T21:41:26.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.851+0000 7f0635400640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c8f0 0x7f061406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:26.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.851+0000 7f0635400640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c8f0 0x7f061406edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0630072350 tx=0x7f0620009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:26.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:26.855+0000 7f062a000640 1 -- 172.21.15.102:0/354308952 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f063014d510 con 0x7f0630073a10 2024-04-20T21:41:27.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.102+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06300c5990 con 0x7f0630073a10 2024-04-20T21:41:27.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.103+0000 7f062a000640 1 -- 172.21.15.102:0/354308952 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+6026 (secure 0 0 0) 0x7f06300c5990 con 0x7f0630073a10 2024-04-20T21:41:27.103 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:27.103 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:25.855269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:27.103 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 80 2024-04-20T21:41:27.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c8f0 msgr2=0x7f061406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:27.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c8f0 0x7f061406edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0630072350 tx=0x7f0620009040 comp rx=0 tx=0).stop 2024-04-20T21:41:27.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 msgr2=0x7f0630155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:27.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630155580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f062400e900 tx=0x7f062400edc0 comp rx=0 tx=0).stop 2024-04-20T21:41:27.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 shutdown_connections 2024-04-20T21:41:27.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063014fa50 0x7f06301f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:27.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c8f0 0x7f061406edb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:27.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06300c9eb0 0x7f06301f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:27.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 --2- 172.21.15.102:0/354308952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0630073a10 0x7f0630155580 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:27.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 >> 172.21.15.102:0/354308952 conn(0x7f06300c9390 msgr2=0x7f06300c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:27.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 shutdown_connections 2024-04-20T21:41:27.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:27.106+0000 7f0638800640 1 -- 172.21.15.102:0/354308952 wait complete. 2024-04-20T21:41:27.125 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 80, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 80, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6035": { "addr": "172.21.15.102:6833/2637555410", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 2637555410, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 2637555410, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6035, "incarnation": 80, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:41:25.855269+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6035 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:41:29.126 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:29.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.337+0000 7fa3d2000640 1 Processor -- start 2024-04-20T21:41:29.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.338+0000 7fa3d2000640 1 -- start start 2024-04-20T21:41:29.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3d2000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc0caf70 0x7fa3cc0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:29.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3d2000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3cc0c8b10 0x7fa3cc0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3d2000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3cc0c8fd0 con 0x7fa3cc073550 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3d2000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa3cc0c6d10 con 0x7fa3cc0c8b10 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3d2000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa3cc0c6100 con 0x7fa3cc0caf70 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3d1600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3cc0c8b10 0x7fa3cc0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3cbe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa3cc073550 0x7fa3cc0c8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48002/0 (socket says 172.21.15.102:48002) 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3cbe00640 1 -- 172.21.15.102:0/1957243488 learned_addr learned my addr 172.21.15.102:0/1957243488 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.339+0000 7fa3d0c00640 1 --2- 172.21.15.102:0/1957243488 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc0caf70 0x7fa3cc0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.340+0000 7fa3d1600640 1 -- 172.21.15.102:0/1957243488 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc0caf70 msgr2=0x7fa3cc0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.340+0000 7fa3d1600640 1 --2- 172.21.15.102:0/1957243488 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc0caf70 0x7fa3cc0ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.340+0000 7fa3d1600640 1 -- 172.21.15.102:0/1957243488 >> v1:172.21.15.102:6789/0 conn(0x7fa3cc073550 legacy=0x7fa3cc0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.340+0000 7fa3d1600640 1 -- 172.21.15.102:0/1957243488 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc0c63e0 con 0x7fa3cc0c8b10 2024-04-20T21:41:29.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.340+0000 7fa3cb400640 1 -- 172.21.15.102:0/1957243488 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2871113301 0 0) 0x7fa3cc0c8fd0 con 0x7fa3cc073550 2024-04-20T21:41:29.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.341+0000 7fa3d1600640 1 --2- 172.21.15.102:0/1957243488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0c8b10 0x7fa3cc0cf090 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40097a0 tx=0x7fa3c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1459e970e472a431 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:29.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.341+0000 7fa3cb400640 1 -- 172.21.15.102:0/1957243488 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4035020 con 0x7fa3cc0c8b10 2024-04-20T21:41:29.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.341+0000 7fa3cb400640 1 -- 172.21.15.102:0/1957243488 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c4031830 con 0x7fa3cc0c8b10 2024-04-20T21:41:29.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.341+0000 7fa3cb400640 1 -- 172.21.15.102:0/1957243488 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4031c00 con 0x7fa3cc0c8b10 2024-04-20T21:41:29.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.342+0000 7fa3d2000640 1 -- 172.21.15.102:0/1957243488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0c8b10 msgr2=0x7fa3cc0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:29.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.342+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1957243488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0c8b10 0x7fa3cc0cf090 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40097a0 tx=0x7fa3c402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.342+0000 7fa3d2000640 1 -- 172.21.15.102:0/1957243488 shutdown_connections 2024-04-20T21:41:29.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.342+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1957243488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0c8b10 0x7fa3cc0cf090 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.342+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1957243488 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc0caf70 0x7fa3cc0ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.342+0000 7fa3d2000640 1 -- 172.21.15.102:0/1957243488 >> 172.21.15.102:0/1957243488 conn(0x7fa3cc06eaa0 msgr2=0x7fa3cc06ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:29.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.342+0000 7fa3d2000640 1 -- 172.21.15.102:0/1957243488 shutdown_connections 2024-04-20T21:41:29.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.342+0000 7fa3d2000640 1 -- 172.21.15.102:0/1957243488 wait complete. 2024-04-20T21:41:29.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.343+0000 7fa3d2000640 1 Processor -- start 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.344+0000 7fa3d2000640 1 -- start start 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.344+0000 7fa3d2000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc073550 0x7fa3cc13d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.344+0000 7fa3d2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3cc0c8b10 0x7fa3cc1f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.344+0000 7fa3d2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0caf70 0x7fa3cc155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.344+0000 7fa3d2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc0cb350 con 0x7fa3cc0caf70 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.344+0000 7fa3d2000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc0c69a0 con 0x7fa3cc0c8b10 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.344+0000 7fa3d2000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa3cc13e200 con 0x7fa3cc073550 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0caf70 0x7fa3cc155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc073550 0x7fa3cc13d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3cc0c8b10 0x7fa3cc1f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0caf70 0x7fa3cc155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52990/0 (socket says 172.21.15.102:52990) 2024-04-20T21:41:29.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d1600640 1 -- 172.21.15.102:0/1735848460 learned_addr learned my addr 172.21.15.102:0/1735848460 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d0c00640 1 -- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3cc0c8b10 msgr2=0x7fa3cc1f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d0c00640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3cc0c8b10 0x7fa3cc1f1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d0c00640 1 -- 172.21.15.102:0/1735848460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0caf70 msgr2=0x7fa3cc155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d0c00640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0caf70 0x7fa3cc155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d0c00640 1 -- 172.21.15.102:0/1735848460 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc156000 con 0x7fa3cc073550 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.345+0000 7fa3d1600640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0caf70 0x7fa3cc155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.346+0000 7fa3d0c00640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc073550 0x7fa3cc13d8c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00ebf0 tx=0x7fa3bc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.346+0000 7fa3c8c00640 1 -- 172.21.15.102:0/1735848460 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc018020 con 0x7fa3cc073550 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.346+0000 7fa3c8c00640 1 -- 172.21.15.102:0/1735848460 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc00cda0 con 0x7fa3cc073550 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.346+0000 7fa3c8c00640 1 -- 172.21.15.102:0/1735848460 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0146c0 con 0x7fa3cc073550 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.346+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3cc1562a0 con 0x7fa3cc073550 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.346+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3cc1f0b70 con 0x7fa3cc073550 2024-04-20T21:41:29.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.347+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3cc04fd60 con 0x7fa3cc073550 2024-04-20T21:41:29.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.348+0000 7fa3c8c00640 1 -- 172.21.15.102:0/1735848460 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3bc005d10 con 0x7fa3cc073550 2024-04-20T21:41:29.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.349+0000 7fa3c8c00640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3ac06c920 0x7fa3ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:29.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.349+0000 7fa3c8c00640 1 -- 172.21.15.102:0/1735848460 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7fa3bc08cf90 con 0x7fa3cc073550 2024-04-20T21:41:29.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.349+0000 7fa3cbe00640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3ac06c920 0x7fa3ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:29.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.350+0000 7fa3cbe00640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3ac06c920 0x7fa3ac06ede0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc152a60 tx=0x7fa3b8007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:29.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.354+0000 7fa3c8c00640 1 -- 172.21.15.102:0/1735848460 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa3bc022030 con 0x7fa3cc073550 2024-04-20T21:41:29.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.592+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3cc0c8760 con 0x7fa3cc073550 2024-04-20T21:41:29.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.594+0000 7fa3c8c00640 1 -- 172.21.15.102:0/1735848460 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+6026 (secure 0 0 0) 0x7fa3bc010040 con 0x7fa3cc073550 2024-04-20T21:41:29.594 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:29.594 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:25.855269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 80 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3ac06c920 msgr2=0x7fa3ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3ac06c920 0x7fa3ac06ede0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc152a60 tx=0x7fa3b8007d50 comp rx=0 tx=0).stop 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc073550 msgr2=0x7fa3cc13d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc073550 0x7fa3cc13d8c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00ebf0 tx=0x7fa3bc00c490 comp rx=0 tx=0).stop 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 shutdown_connections 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc0caf70 0x7fa3cc155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3ac06c920 0x7fa3ac06ede0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3cc0c8b10 0x7fa3cc1f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 --2- 172.21.15.102:0/1735848460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3cc073550 0x7fa3cc13d8c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.595+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 >> 172.21.15.102:0/1735848460 conn(0x7fa3cc06eaa0 msgr2=0x7fa3cc0c9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.596+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 shutdown_connections 2024-04-20T21:41:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:29.596+0000 7fa3d2000640 1 -- 172.21.15.102:0/1735848460 wait complete. 2024-04-20T21:41:29.614 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:41:30.985 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:41:30.992 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:41:31.616 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:31.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.794+0000 7f8c3a600640 1 Processor -- start 2024-04-20T21:41:31.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.794+0000 7f8c3a600640 1 -- start start 2024-04-20T21:41:31.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c3a600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c3415be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c3a600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8c340c60c0 0x7f8c34073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c3a600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c3415e7b0 con 0x7f8c340c5cc0 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c3a600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8c3415f0c0 con 0x7f8c340c60c0 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c3a600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8c34155940 con 0x7f8c34155560 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c33e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8c340c60c0 0x7f8c34073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c33e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8c340c60c0 0x7f8c34073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52994/0 (socket says 172.21.15.102:52994) 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c33e00640 1 -- 172.21.15.102:0/2531500750 learned_addr learned my addr 172.21.15.102:0/2531500750 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.796+0000 7f8c33400640 1 --2- 172.21.15.102:0/2531500750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c3415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.797+0000 7f8c33e00640 1 -- 172.21.15.102:0/2531500750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 msgr2=0x7f8c3415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.797+0000 7f8c33e00640 1 --2- 172.21.15.102:0/2531500750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c3415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.797+0000 7f8c33e00640 1 -- 172.21.15.102:0/2531500750 >> v1:172.21.15.193:6789/0 conn(0x7f8c340c5cc0 legacy=0x7f8c340ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.797+0000 7f8c33e00640 1 -- 172.21.15.102:0/2531500750 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c340cbea0 con 0x7f8c340c60c0 2024-04-20T21:41:31.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.797+0000 7f8c33400640 1 --2- 172.21.15.102:0/2531500750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c3415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:31.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.798+0000 7f8c33e00640 1 --2- 172.21.15.102:0/2531500750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c340c60c0 0x7f8c34073230 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009e50 tx=0x7f8c2c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aad2e2114abfe9e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.798+0000 7f8c32000640 1 -- 172.21.15.102:0/2531500750 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c02cb90 con 0x7f8c340c60c0 2024-04-20T21:41:31.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.798+0000 7f8c32000640 1 -- 172.21.15.102:0/2531500750 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2c02cd30 con 0x7f8c340c60c0 2024-04-20T21:41:31.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.798+0000 7f8c32000640 1 -- 172.21.15.102:0/2531500750 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c034610 con 0x7f8c340c60c0 2024-04-20T21:41:31.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.799+0000 7f8c3a600640 1 -- 172.21.15.102:0/2531500750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c340c60c0 msgr2=0x7f8c34073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.799+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2531500750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c340c60c0 0x7f8c34073230 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009e50 tx=0x7f8c2c02bff0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.799+0000 7f8c3a600640 1 -- 172.21.15.102:0/2531500750 shutdown_connections 2024-04-20T21:41:31.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.799+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2531500750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c340c60c0 0x7f8c34073230 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.799+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2531500750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c3415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.799+0000 7f8c3a600640 1 -- 172.21.15.102:0/2531500750 >> 172.21.15.102:0/2531500750 conn(0x7f8c3406cd40 msgr2=0x7f8c3406d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.799+0000 7f8c3a600640 1 -- 172.21.15.102:0/2531500750 shutdown_connections 2024-04-20T21:41:31.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.799+0000 7f8c3a600640 1 -- 172.21.15.102:0/2531500750 wait complete. 2024-04-20T21:41:31.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.801+0000 7f8c3a600640 1 Processor -- start 2024-04-20T21:41:31.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.801+0000 7f8c3a600640 1 -- start start 2024-04-20T21:41:31.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c3a600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c340c5cc0 0x7f8c3415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c3a600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c34145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c3a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3414a540 0x7f8c3415c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c3a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3415ed50 con 0x7f8c3414a540 2024-04-20T21:41:31.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c3a600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3415ea70 con 0x7f8c340c5cc0 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c3a600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8c340cbb60 con 0x7f8c34155560 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3414a540 0x7f8c3415c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c33400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c340c5cc0 0x7f8c3415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c32a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c34145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3414a540 0x7f8c3415c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53000/0 (socket says 172.21.15.102:53000) 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c33e00640 1 -- 172.21.15.102:0/2806946518 learned_addr learned my addr 172.21.15.102:0/2806946518 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.802+0000 7f8c33400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c340c5cc0 0x7f8c3415adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37578/0 (socket says 172.21.15.102:37578) 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c33e00640 1 -- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 msgr2=0x7f8c34145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c33e00640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c34145ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c33e00640 1 -- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c340c5cc0 msgr2=0x7f8c3415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c33e00640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c340c5cc0 0x7f8c3415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c33e00640 1 -- 172.21.15.102:0/2806946518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3415b390 con 0x7f8c3414a540 2024-04-20T21:41:31.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c33e00640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3414a540 0x7f8c3415c810 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c036040 tx=0x7f8c2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c2b400640 1 -- 172.21.15.102:0/2806946518 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c043020 con 0x7f8c3414a540 2024-04-20T21:41:31.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c2b400640 1 -- 172.21.15.102:0/2806946518 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2c03e450 con 0x7f8c3414a540 2024-04-20T21:41:31.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c2b400640 1 -- 172.21.15.102:0/2806946518 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c009510 con 0x7f8c3414a540 2024-04-20T21:41:31.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3415b670 con 0x7f8c3414a540 2024-04-20T21:41:31.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.803+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3415bc30 con 0x7f8c3414a540 2024-04-20T21:41:31.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.805+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c34074340 con 0x7f8c3414a540 2024-04-20T21:41:31.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.806+0000 7f8c2b400640 1 -- 172.21.15.102:0/2806946518 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c2c046020 con 0x7f8c3414a540 2024-04-20T21:41:31.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.806+0000 7f8c2b400640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c1406c920 0x7f8c1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.806+0000 7f8c2b400640 1 -- 172.21.15.102:0/2806946518 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f8c2c0ae6f0 con 0x7f8c3414a540 2024-04-20T21:41:31.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.807+0000 7f8c33400640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c1406c920 0x7f8c1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.807+0000 7f8c33400640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c1406c920 0x7f8c1406ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c24006e80 tx=0x7f8c24008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:31.816+0000 7f8c2b400640 1 -- 172.21.15.102:0/2806946518 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c2c04d070 con 0x7f8c3414a540 2024-04-20T21:41:32.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.071+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c341496a0 con 0x7f8c3414a540 2024-04-20T21:41:32.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.072+0000 7f8c2b400640 1 -- 172.21.15.102:0/2806946518 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+6026 (secure 0 0 0) 0x7f8c2c07bbd0 con 0x7f8c3414a540 2024-04-20T21:41:32.071 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:32.071 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:25.855269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:32.071 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 80 2024-04-20T21:41:32.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.075+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c1406c920 msgr2=0x7f8c1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.075+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c1406c920 0x7f8c1406ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c24006e80 tx=0x7f8c24008040 comp rx=0 tx=0).stop 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.075+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3414a540 msgr2=0x7f8c3415c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.075+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3414a540 0x7f8c3415c810 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c036040 tx=0x7f8c2c004a40 comp rx=0 tx=0).stop 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.075+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 shutdown_connections 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.075+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3414a540 0x7f8c3415c810 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.075+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c34155560 0x7f8c34145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.075+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c1406c920 0x7f8c1406ede0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.076+0000 7f8c3a600640 1 --2- 172.21.15.102:0/2806946518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c340c5cc0 0x7f8c3415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.076+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 >> 172.21.15.102:0/2806946518 conn(0x7f8c3406cd40 msgr2=0x7f8c340b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.076+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 shutdown_connections 2024-04-20T21:41:32.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:32.076+0000 7f8c3a600640 1 -- 172.21.15.102:0/2806946518 wait complete. 2024-04-20T21:41:32.099 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:41:34.100 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:34.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.278+0000 7f2062e00640 1 Processor -- start 2024-04-20T21:41:34.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.278+0000 7f2062e00640 1 -- start start 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.279+0000 7f2062e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f205c0cb980 0x7f205c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.279+0000 7f2062e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f205c0cbd60 0x7f205c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f2062e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f205c15ef40 con 0x7f205c0ccc00 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f2062e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f205c15f7e0 con 0x7f205c0cb980 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f2062e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f205c15f220 con 0x7f205c0cbd60 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f205be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f205c0cb980 0x7f205c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f2060e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f205c0ccc00 0x7f205c0cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43446/0 (socket says 172.21.15.102:43446) 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f2060e00640 1 -- 172.21.15.102:0/2412370699 learned_addr learned my addr 172.21.15.102:0/2412370699 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:34.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f2053e00640 1 --2- 172.21.15.102:0/2412370699 >> v2:172.21.15.193:3300/0 conn(0x7f205c0cbd60 0x7f205c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:34.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f205be00640 1 -- 172.21.15.102:0/2412370699 >> v2:172.21.15.193:3300/0 conn(0x7f205c0cbd60 msgr2=0x7f205c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:34.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f205be00640 1 --2- 172.21.15.102:0/2412370699 >> v2:172.21.15.193:3300/0 conn(0x7f205c0cbd60 0x7f205c0cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f205be00640 1 -- 172.21.15.102:0/2412370699 >> v1:172.21.15.193:6789/0 conn(0x7f205c0ccc00 legacy=0x7f205c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:34.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f205b400640 1 -- 172.21.15.102:0/2412370699 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 261927941 0 0) 0x7f205c15ef40 con 0x7f205c0ccc00 2024-04-20T21:41:34.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.280+0000 7f205be00640 1 -- 172.21.15.102:0/2412370699 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c15f500 con 0x7f205c0cb980 2024-04-20T21:41:34.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.282+0000 7f205be00640 1 --2- 172.21.15.102:0/2412370699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c15e890 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f205c0ce6a0 tx=0x7f204c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a664c9cfb0529b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:34.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.282+0000 7f205b400640 1 -- 172.21.15.102:0/2412370699 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c035020 con 0x7f205c0cb980 2024-04-20T21:41:34.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.282+0000 7f205b400640 1 -- 172.21.15.102:0/2412370699 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c031830 con 0x7f205c0cb980 2024-04-20T21:41:34.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.282+0000 7f205b400640 1 -- 172.21.15.102:0/2412370699 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c031c00 con 0x7f205c0cb980 2024-04-20T21:41:34.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.282+0000 7f2062e00640 1 -- 172.21.15.102:0/2412370699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 msgr2=0x7f205c15e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:34.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.282+0000 7f2062e00640 1 --2- 172.21.15.102:0/2412370699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c15e890 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f205c0ce6a0 tx=0x7f204c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.283+0000 7f2062e00640 1 -- 172.21.15.102:0/2412370699 shutdown_connections 2024-04-20T21:41:34.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.283+0000 7f2062e00640 1 --2- 172.21.15.102:0/2412370699 >> v2:172.21.15.193:3300/0 conn(0x7f205c0cbd60 0x7f205c0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.283+0000 7f2062e00640 1 --2- 172.21.15.102:0/2412370699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c15e890 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.283+0000 7f2062e00640 1 -- 172.21.15.102:0/2412370699 >> 172.21.15.102:0/2412370699 conn(0x7f205c075080 msgr2=0x7f205c075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:34.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.283+0000 7f2062e00640 1 -- 172.21.15.102:0/2412370699 shutdown_connections 2024-04-20T21:41:34.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.283+0000 7f2062e00640 1 -- 172.21.15.102:0/2412370699 wait complete. 2024-04-20T21:41:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.285+0000 7f2062e00640 1 Processor -- start 2024-04-20T21:41:34.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.285+0000 7f2062e00640 1 -- start start 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f2062e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c1e4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f2062e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f205c0cbd60 0x7f205c1e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f2062e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f205c0ccc00 0x7f205c1f1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f2062e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f205c0cc1d0 con 0x7f205c0cb980 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f2062e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f205c0cd610 con 0x7f205c0cbd60 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f2062e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f205c1559f0 con 0x7f205c0ccc00 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f205be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c1e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f205be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c1e4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53016/0 (socket says 172.21.15.102:53016) 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f205be00640 1 -- 172.21.15.102:0/4161120418 learned_addr learned my addr 172.21.15.102:0/4161120418 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f2053e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f205c0cbd60 0x7f205c1e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.287+0000 7f2060e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f205c0ccc00 0x7f205c1f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f205be00640 1 -- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f205c0ccc00 msgr2=0x7f205c1f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f205be00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f205c0ccc00 0x7f205c1f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f205be00640 1 -- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f205c0cbd60 msgr2=0x7f205c1e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:34.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f205be00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f205c0cbd60 0x7f205c1e6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f205be00640 1 -- 172.21.15.102:0/4161120418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c1e5000 con 0x7f205c0cb980 2024-04-20T21:41:34.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f205be00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c1e4990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f204c02bee0 tx=0x7f204c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:34.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f2058c00640 1 -- 172.21.15.102:0/4161120418 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c041070 con 0x7f205c0cb980 2024-04-20T21:41:34.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f2058c00640 1 -- 172.21.15.102:0/4161120418 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c03f070 con 0x7f205c0cb980 2024-04-20T21:41:34.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f2058c00640 1 -- 172.21.15.102:0/4161120418 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c035040 con 0x7f205c0cb980 2024-04-20T21:41:34.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.288+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c1e52e0 con 0x7f205c0cb980 2024-04-20T21:41:34.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.289+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c159610 con 0x7f205c0cb980 2024-04-20T21:41:34.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.290+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c0c43a0 con 0x7f205c0cb980 2024-04-20T21:41:34.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.291+0000 7f2058c00640 1 -- 172.21.15.102:0/4161120418 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f204c038020 con 0x7f205c0cb980 2024-04-20T21:41:34.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.291+0000 7f2058c00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f203806c8f0 0x7f203806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:34.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.292+0000 7f2058c00640 1 -- 172.21.15.102:0/4161120418 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f204c0adcb0 con 0x7f205c0cb980 2024-04-20T21:41:34.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.292+0000 7f2053e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f203806c8f0 0x7f203806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:34.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.292+0000 7f2053e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f203806c8f0 0x7f203806edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f205c15ec90 tx=0x7f2044009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:34.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.298+0000 7f2058c00640 1 -- 172.21.15.102:0/4161120418 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f205c0c43a0 con 0x7f205c0cb980 2024-04-20T21:41:34.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.544+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f205c074340 con 0x7f205c0cb980 2024-04-20T21:41:34.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.545+0000 7f2058c00640 1 -- 172.21.15.102:0/4161120418 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+6026 (secure 0 0 0) 0x7f204c031800 con 0x7f205c0cb980 2024-04-20T21:41:34.544 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:34.544 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:25.855269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:34.544 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 80 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f203806c8f0 msgr2=0x7f203806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f203806c8f0 0x7f203806edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f205c15ec90 tx=0x7f2044009190 comp rx=0 tx=0).stop 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 msgr2=0x7f205c1e4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c1e4990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f204c02bee0 tx=0x7f204c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 shutdown_connections 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f205c0ccc00 0x7f205c1f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f203806c8f0 0x7f203806edb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f205c0cbd60 0x7f205c1e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 --2- 172.21.15.102:0/4161120418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c0cb980 0x7f205c1e4990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:34.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 >> 172.21.15.102:0/4161120418 conn(0x7f205c075080 msgr2=0x7f205c14f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:34.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 shutdown_connections 2024-04-20T21:41:34.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:34.548+0000 7f2062e00640 1 -- 172.21.15.102:0/4161120418 wait complete. 2024-04-20T21:41:34.564 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:41:36.565 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:36.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.753+0000 7eff88200640 1 Processor -- start 2024-04-20T21:41:36.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.753+0000 7eff88200640 1 -- start start 2024-04-20T21:41:36.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.755+0000 7eff88200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff80151c90 0x7eff800c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.755+0000 7eff88200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff800c7290 0x7eff80158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.755+0000 7eff88200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7eff80071a70 0x7eff8006d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.755+0000 7eff88200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7eff800c9200 con 0x7eff800c7290 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.755+0000 7eff88200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7eff80070df0 con 0x7eff80071a70 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.755+0000 7eff88200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7eff80157db0 con 0x7eff80151c90 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff84e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff800c7290 0x7eff80158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff85800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff80151c90 0x7eff800c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff84e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff800c7290 0x7eff80158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53018/0 (socket says 172.21.15.102:53018) 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff86200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7eff80071a70 0x7eff8006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff84e00640 1 -- 172.21.15.102:0/1684924779 learned_addr learned my addr 172.21.15.102:0/1684924779 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff85800640 1 -- 172.21.15.102:0/1684924779 >> v2:172.21.15.193:3300/0 conn(0x7eff80071a70 msgr2=0x7eff8006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff85800640 1 --2- 172.21.15.102:0/1684924779 >> v2:172.21.15.193:3300/0 conn(0x7eff80071a70 0x7eff8006d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff85800640 1 -- 172.21.15.102:0/1684924779 >> v2:172.21.15.102:3300/0 conn(0x7eff800c7290 msgr2=0x7eff80158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff85800640 1 --2- 172.21.15.102:0/1684924779 >> v2:172.21.15.102:3300/0 conn(0x7eff800c7290 0x7eff80158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:36.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff85800640 1 -- 172.21.15.102:0/1684924779 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff80158780 con 0x7eff80151c90 2024-04-20T21:41:36.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.756+0000 7eff84e00640 1 --2- 172.21.15.102:0/1684924779 >> v2:172.21.15.102:3300/0 conn(0x7eff800c7290 0x7eff80158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.757+0000 7eff85800640 1 --2- 172.21.15.102:0/1684924779 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff80151c90 0x7eff800c8cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7eff8006e2b0 tx=0x7eff7402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63d66b33950a493 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.757+0000 7eff7fe00640 1 -- 172.21.15.102:0/1684924779 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74035020 con 0x7eff80151c90 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.757+0000 7eff7fe00640 1 -- 172.21.15.102:0/1684924779 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff74031830 con 0x7eff80151c90 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff7fe00640 1 -- 172.21.15.102:0/1684924779 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74031c00 con 0x7eff80151c90 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 -- 172.21.15.102:0/1684924779 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff80151c90 msgr2=0x7eff800c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 --2- 172.21.15.102:0/1684924779 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff80151c90 0x7eff800c8cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7eff8006e2b0 tx=0x7eff7402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 -- 172.21.15.102:0/1684924779 shutdown_connections 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 --2- 172.21.15.102:0/1684924779 >> v2:172.21.15.193:3300/0 conn(0x7eff80071a70 0x7eff8006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 --2- 172.21.15.102:0/1684924779 >> v2:172.21.15.102:3300/0 conn(0x7eff800c7290 0x7eff80158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:36.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 --2- 172.21.15.102:0/1684924779 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff80151c90 0x7eff800c8cc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:36.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 -- 172.21.15.102:0/1684924779 >> 172.21.15.102:0/1684924779 conn(0x7eff800c9390 msgr2=0x7eff800c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:36.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 -- 172.21.15.102:0/1684924779 shutdown_connections 2024-04-20T21:41:36.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.758+0000 7eff88200640 1 -- 172.21.15.102:0/1684924779 wait complete. 2024-04-20T21:41:36.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.760+0000 7eff88200640 1 Processor -- start 2024-04-20T21:41:36.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.760+0000 7eff88200640 1 -- start start 2024-04-20T21:41:36.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff88200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff80071a70 0x7eff801569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:36.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff88200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff800c7290 0x7eff800cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:36.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff88200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff80151c90 0x7eff800cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:36.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff88200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff80071600 con 0x7eff80071a70 2024-04-20T21:41:36.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff88200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7eff80071320 con 0x7eff80151c90 2024-04-20T21:41:36.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff88200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7eff80157a40 con 0x7eff800c7290 2024-04-20T21:41:36.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff85800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff80071a70 0x7eff801569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:36.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff86200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff80151c90 0x7eff800cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:36.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff84e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff800c7290 0x7eff800cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:36.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff86200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff80151c90 0x7eff800cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37598/0 (socket says 172.21.15.102:37598) 2024-04-20T21:41:36.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff86200640 1 -- 172.21.15.102:0/635795086 learned_addr learned my addr 172.21.15.102:0/635795086 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:36.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.761+0000 7eff84e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff800c7290 0x7eff800cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:53050/0 (socket says 172.21.15.102:53050) 2024-04-20T21:41:36.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff85800640 1 -- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff800c7290 msgr2=0x7eff800cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:36.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff85800640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff800c7290 0x7eff800cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:36.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff85800640 1 -- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff80151c90 msgr2=0x7eff800cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:36.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff85800640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff80151c90 0x7eff800cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:36.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff85800640 1 -- 172.21.15.102:0/635795086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8014f610 con 0x7eff80071a70 2024-04-20T21:41:36.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff85800640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff80071a70 0x7eff801569b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7eff74033000 tx=0x7eff740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:36.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff7d600640 1 -- 172.21.15.102:0/635795086 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7403a8f0 con 0x7eff80071a70 2024-04-20T21:41:36.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff7d600640 1 -- 172.21.15.102:0/635795086 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff7403aad0 con 0x7eff80071a70 2024-04-20T21:41:36.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.762+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff801505c0 con 0x7eff80071a70 2024-04-20T21:41:36.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.763+0000 7eff7d600640 1 -- 172.21.15.102:0/635795086 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740443f0 con 0x7eff80071a70 2024-04-20T21:41:36.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.763+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff80150b80 con 0x7eff80071a70 2024-04-20T21:41:36.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.764+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff80070310 con 0x7eff80071a70 2024-04-20T21:41:36.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.765+0000 7eff7d600640 1 -- 172.21.15.102:0/635795086 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff7404b060 con 0x7eff80071a70 2024-04-20T21:41:36.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.766+0000 7eff7d600640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff6006c8f0 0x7eff6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:36.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.766+0000 7eff7d600640 1 -- 172.21.15.102:0/635795086 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7eff740adc90 con 0x7eff80071a70 2024-04-20T21:41:36.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.766+0000 7eff84e00640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff6006c8f0 0x7eff6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:36.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.767+0000 7eff84e00640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff6006c8f0 0x7eff6006edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7eff8014ad30 tx=0x7eff6c009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:36.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:36.775+0000 7eff7d600640 1 -- 172.21.15.102:0/635795086 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7eff80070310 con 0x7eff80071a70 2024-04-20T21:41:37.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.019+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff800ca1f0 con 0x7eff80071a70 2024-04-20T21:41:37.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.020+0000 7eff7d600640 1 -- 172.21.15.102:0/635795086 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+6026 (secure 0 0 0) 0x7eff800ca1f0 con 0x7eff80071a70 2024-04-20T21:41:37.018 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:37.019 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:25.855269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:37.019 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 80 2024-04-20T21:41:37.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff6006c8f0 msgr2=0x7eff6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff6006c8f0 0x7eff6006edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7eff8014ad30 tx=0x7eff6c009240 comp rx=0 tx=0).stop 2024-04-20T21:41:37.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff80071a70 msgr2=0x7eff801569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff80071a70 0x7eff801569b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7eff74033000 tx=0x7eff740046c0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 shutdown_connections 2024-04-20T21:41:37.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7eff6006c8f0 0x7eff6006edb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff80151c90 0x7eff800cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff800c7290 0x7eff800cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 --2- 172.21.15.102:0/635795086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff80071a70 0x7eff801569b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 >> 172.21.15.102:0/635795086 conn(0x7eff800c9390 msgr2=0x7eff8014ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 shutdown_connections 2024-04-20T21:41:37.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:37.023+0000 7eff88200640 1 -- 172.21.15.102:0/635795086 wait complete. 2024-04-20T21:41:37.037 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:41:39.038 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:39.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.248+0000 7f4fd0800640 1 Processor -- start 2024-04-20T21:41:39.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.249+0000 7f4fd0800640 1 -- start start 2024-04-20T21:41:39.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.250+0000 7f4fd0800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4fc8073a10 0x7f4fc80c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.250+0000 7f4fd0800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc81536d0 con 0x7f4fc80c9eb0 2024-04-20T21:41:39.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.250+0000 7f4fd0800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc80c76c0 con 0x7f4fc814fa50 2024-04-20T21:41:39.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.250+0000 7f4fd0800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f4fc80c7960 con 0x7f4fc8073a10 2024-04-20T21:41:39.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.250+0000 7f4fcd400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4fc80c9eb0 0x7f4fc8161930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58808/0 (socket says 172.21.15.102:58808) 2024-04-20T21:41:39.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.250+0000 7f4fcd400640 1 -- 172.21.15.102:0/758585067 learned_addr learned my addr 172.21.15.102:0/758585067 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:39.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.251+0000 7f4fce800640 1 --2- 172.21.15.102:0/758585067 >> v2:172.21.15.193:3300/0 conn(0x7f4fc8073a10 0x7f4fc80c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.251+0000 7f4fcca00640 1 -- 172.21.15.102:0/758585067 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2680965232 0 0) 0x7f4fb8001110 con 0x7f4fc80c9eb0 2024-04-20T21:41:39.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.251+0000 7f4fcca00640 1 -- 172.21.15.102:0/758585067 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fb0003520 con 0x7f4fc80c9eb0 2024-04-20T21:41:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.251+0000 7f4fcca00640 1 -- 172.21.15.102:0/758585067 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197207840 0 0) 0x7f4fc80c76c0 con 0x7f4fc814fa50 2024-04-20T21:41:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.251+0000 7f4fcca00640 1 -- 172.21.15.102:0/758585067 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fb8001110 con 0x7f4fc814fa50 2024-04-20T21:41:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.251+0000 7f4fce800640 1 -- 172.21.15.102:0/758585067 >> v1:172.21.15.193:6789/0 conn(0x7f4fc814fa50 legacy=0x7f4fc80c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.251+0000 7f4fce800640 1 -- 172.21.15.102:0/758585067 >> v1:172.21.15.102:6789/0 conn(0x7f4fc80c9eb0 legacy=0x7f4fc8161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.251+0000 7f4fce800640 1 -- 172.21.15.102:0/758585067 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc80c3d90 con 0x7f4fc8073a10 2024-04-20T21:41:39.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.253+0000 7f4fce800640 1 --2- 172.21.15.102:0/758585067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc80c4830 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80c7890 tx=0x7f4fc402bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd8112fc743a8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.253+0000 7f4fcca00640 1 -- 172.21.15.102:0/758585067 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4039070 con 0x7f4fc8073a10 2024-04-20T21:41:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.253+0000 7f4fcca00640 1 -- 172.21.15.102:0/758585067 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc402cd80 con 0x7f4fc8073a10 2024-04-20T21:41:39.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.253+0000 7f4fcca00640 1 -- 172.21.15.102:0/758585067 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4034660 con 0x7f4fc8073a10 2024-04-20T21:41:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.254+0000 7f4fd0800640 1 -- 172.21.15.102:0/758585067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 msgr2=0x7f4fc80c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.254+0000 7f4fd0800640 1 --2- 172.21.15.102:0/758585067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc80c4830 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80c7890 tx=0x7f4fc402bfd0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.254+0000 7f4fd0800640 1 -- 172.21.15.102:0/758585067 shutdown_connections 2024-04-20T21:41:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.254+0000 7f4fd0800640 1 --2- 172.21.15.102:0/758585067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc80c4830 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.254+0000 7f4fd0800640 1 -- 172.21.15.102:0/758585067 >> 172.21.15.102:0/758585067 conn(0x7f4fc80c9390 msgr2=0x7f4fc80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.254+0000 7f4fd0800640 1 -- 172.21.15.102:0/758585067 shutdown_connections 2024-04-20T21:41:39.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.254+0000 7f4fd0800640 1 -- 172.21.15.102:0/758585067 wait complete. 2024-04-20T21:41:39.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.255+0000 7f4fd0800640 1 Processor -- start 2024-04-20T21:41:39.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.256+0000 7f4fd0800640 1 -- start start 2024-04-20T21:41:39.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.256+0000 7f4fd0800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.256+0000 7f4fd0800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc80c9eb0 0x7f4fc81e3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.256+0000 7f4fd0800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fc814fa50 0x7f4fc81f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.256+0000 7f4fd0800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80c4030 con 0x7f4fc814fa50 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.256+0000 7f4fd0800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80c7c00 con 0x7f4fc8073a10 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.256+0000 7f4fd0800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc80c4310 con 0x7f4fc80c9eb0 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc80c9eb0 0x7f4fc81e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc8071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:44608/0 (socket says 172.21.15.102:44608) 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcde00640 1 -- 172.21.15.102:0/825495642 learned_addr learned my addr 172.21.15.102:0/825495642 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:39.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fce800640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fc814fa50 0x7f4fc81f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcde00640 1 -- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc80c9eb0 msgr2=0x7f4fc81e3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcde00640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc80c9eb0 0x7f4fc81e3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcde00640 1 -- 172.21.15.102:0/825495642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fc814fa50 msgr2=0x7f4fc81f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcde00640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fc814fa50 0x7f4fc81f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcde00640 1 -- 172.21.15.102:0/825495642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc814af60 con 0x7f4fc8073a10 2024-04-20T21:41:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fcd400640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc80c9eb0 0x7f4fc81e3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.257+0000 7f4fce800640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fc814fa50 0x7f4fc81f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:39.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.258+0000 7f4fcde00640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc8071ae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c430 tx=0x7f4fbc00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.258+0000 7f4fc2000640 1 -- 172.21.15.102:0/825495642 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc017070 con 0x7f4fc8073a10 2024-04-20T21:41:39.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.258+0000 7f4fc2000640 1 -- 172.21.15.102:0/825495642 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fbc0052f0 con 0x7f4fc8073a10 2024-04-20T21:41:39.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.258+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc8159670 con 0x7f4fc8073a10 2024-04-20T21:41:39.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.258+0000 7f4fc2000640 1 -- 172.21.15.102:0/825495642 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc003830 con 0x7f4fc8073a10 2024-04-20T21:41:39.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.258+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc814a4f0 con 0x7f4fc8073a10 2024-04-20T21:41:39.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.259+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc80c3250 con 0x7f4fc8073a10 2024-04-20T21:41:39.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.260+0000 7f4fc2000640 1 -- 172.21.15.102:0/825495642 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fbc01a070 con 0x7f4fc8073a10 2024-04-20T21:41:39.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.261+0000 7f4fc2000640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4fac06c8f0 0x7f4fac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.261+0000 7f4fc2000640 1 -- 172.21.15.102:0/825495642 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f4fbc08b7d0 con 0x7f4fc8073a10 2024-04-20T21:41:39.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.262+0000 7f4fcd400640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4fac06c8f0 0x7f4fac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.262+0000 7f4fcd400640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4fac06c8f0 0x7f4fac06edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4fc806ef10 tx=0x7f4fb8007d70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.267+0000 7f4fc2000640 1 -- 172.21.15.102:0/825495642 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4fc80c3250 con 0x7f4fc8073a10 2024-04-20T21:41:39.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.500+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fc8074340 con 0x7f4fc8073a10 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.501+0000 7f4fc2000640 1 -- 172.21.15.102:0/825495642 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6030 (secure 0 0 0) 0x7f4fbc058c30 con 0x7f4fc8073a10 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:38.534140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:39.501 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 81 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4fac06c8f0 msgr2=0x7f4fac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4fac06c8f0 0x7f4fac06edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4fc806ef10 tx=0x7f4fb8007d70 comp rx=0 tx=0).stop 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 msgr2=0x7f4fc8071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc8071ae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c430 tx=0x7f4fbc00c8f0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 shutdown_connections 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fc814fa50 0x7f4fc81f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc80c9eb0 0x7f4fc81e3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f4fac06c8f0 0x7f4fac06edb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 --2- 172.21.15.102:0/825495642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc8073a10 0x7f4fc8071ae0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 >> 172.21.15.102:0/825495642 conn(0x7f4fc80c9390 msgr2=0x7f4fc814df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 shutdown_connections 2024-04-20T21:41:39.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:39.504+0000 7f4fd0800640 1 -- 172.21.15.102:0/825495642 wait complete. 2024-04-20T21:41:39.523 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 81, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 81, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6035": { "addr": "172.21.15.102:6833/2637555410", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 2637555410, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 2637555410, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6035, "incarnation": 80, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 25 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:41:38.534140+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6035 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:41:41.524 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:41.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.739+0000 7f73ab600640 1 Processor -- start 2024-04-20T21:41:41.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.739+0000 7f73ab600640 1 -- start start 2024-04-20T21:41:41.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f73ab600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f73a40c7900 0x7f73a415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:41.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f73ab600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73a40cd940 0x7f73a4074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f73ab600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40ce310 0x7f73a40753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f73ab600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f73a415f940 con 0x7f73a40cd940 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f73ab600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f73a415eb60 con 0x7f73a40c7900 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f73ab600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f73a415ee00 con 0x7f73a40ce310 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f739be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73a40cd940 0x7f73a4074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f73a8c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f73a40c7900 0x7f73a415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f739be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f73a40cd940 0x7f73a4074ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36978/0 (socket says 172.21.15.102:36978) 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f739be00640 1 -- 172.21.15.102:0/2540155123 learned_addr learned my addr 172.21.15.102:0/2540155123 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.741+0000 7f73a9600640 1 --2- 172.21.15.102:0/2540155123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40ce310 0x7f73a40753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.742+0000 7f739be00640 1 -- 172.21.15.102:0/2540155123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40ce310 msgr2=0x7f73a40753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.742+0000 7f739be00640 1 --2- 172.21.15.102:0/2540155123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40ce310 0x7f73a40753e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.742+0000 7f739be00640 1 -- 172.21.15.102:0/2540155123 >> v2:172.21.15.193:3300/0 conn(0x7f73a40c7900 msgr2=0x7f73a415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.742+0000 7f739be00640 1 --2- 172.21.15.102:0/2540155123 >> v2:172.21.15.193:3300/0 conn(0x7f73a40c7900 0x7f73a415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.742+0000 7f739be00640 1 -- 172.21.15.102:0/2540155123 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a415f0a0 con 0x7f73a40cd940 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.743+0000 7f739be00640 1 --2- 172.21.15.102:0/2540155123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a4074ea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f73a40cd910 tx=0x7f738c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc720a535449300 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.743+0000 7f73a3e00640 1 -- 172.21.15.102:0/2540155123 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c035020 con 0x7f73a40cd940 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.743+0000 7f73a3e00640 1 -- 172.21.15.102:0/2540155123 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738c02d040 con 0x7f73a40cd940 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.743+0000 7f73a3e00640 1 -- 172.21.15.102:0/2540155123 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c031a80 con 0x7f73a40cd940 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 -- 172.21.15.102:0/2540155123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 msgr2=0x7f73a4074ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 --2- 172.21.15.102:0/2540155123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a4074ea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f73a40cd910 tx=0x7f738c02b900 comp rx=0 tx=0).stop 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 -- 172.21.15.102:0/2540155123 shutdown_connections 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 --2- 172.21.15.102:0/2540155123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40ce310 0x7f73a40753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 --2- 172.21.15.102:0/2540155123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a4074ea0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 --2- 172.21.15.102:0/2540155123 >> v2:172.21.15.193:3300/0 conn(0x7f73a40c7900 0x7f73a415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 -- 172.21.15.102:0/2540155123 >> 172.21.15.102:0/2540155123 conn(0x7f73a406dad0 msgr2=0x7f73a406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 -- 172.21.15.102:0/2540155123 shutdown_connections 2024-04-20T21:41:41.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.744+0000 7f73ab600640 1 -- 172.21.15.102:0/2540155123 wait complete. 2024-04-20T21:41:41.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.745+0000 7f73ab600640 1 Processor -- start 2024-04-20T21:41:41.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.746+0000 7f73ab600640 1 -- start start 2024-04-20T21:41:41.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.746+0000 7f73ab600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40c7900 0x7f73a41e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:41.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.746+0000 7f73ab600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a41f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:41.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.746+0000 7f73ab600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73a40ce310 0x7f73a41f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:41.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.746+0000 7f73ab600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73a40c7ce0 con 0x7f73a40cd940 2024-04-20T21:41:41.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.746+0000 7f73ab600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73a415f660 con 0x7f73a40ce310 2024-04-20T21:41:41.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.746+0000 7f73ab600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f73a40c7e60 con 0x7f73a40c7900 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a41f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a41f0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36988/0 (socket says 172.21.15.102:36988) 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 -- 172.21.15.102:0/1349840831 learned_addr learned my addr 172.21.15.102:0/1349840831 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f73a8c00640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40c7900 0x7f73a41e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f73a9600640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73a40ce310 0x7f73a41f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 -- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40c7900 msgr2=0x7f73a41e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40c7900 0x7f73a41e64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 -- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73a40ce310 msgr2=0x7f73a41f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73a40ce310 0x7f73a41f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 -- 172.21.15.102:0/1349840831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a414fdd0 con 0x7f73a40cd940 2024-04-20T21:41:41.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.747+0000 7f739be00640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a41f0cb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f738c02b940 tx=0x7f738c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:41.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.748+0000 7f73a1600640 1 -- 172.21.15.102:0/1349840831 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c040070 con 0x7f73a40cd940 2024-04-20T21:41:41.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.748+0000 7f73a1600640 1 -- 172.21.15.102:0/1349840831 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738c0398e0 con 0x7f73a40cd940 2024-04-20T21:41:41.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.748+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a41500b0 con 0x7f73a40cd940 2024-04-20T21:41:41.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.748+0000 7f73a1600640 1 -- 172.21.15.102:0/1349840831 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c035040 con 0x7f73a40cd940 2024-04-20T21:41:41.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.748+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41f4470 con 0x7f73a40cd940 2024-04-20T21:41:41.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.749+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a4074340 con 0x7f73a40cd940 2024-04-20T21:41:41.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.750+0000 7f73a1600640 1 -- 172.21.15.102:0/1349840831 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f738c038020 con 0x7f73a40cd940 2024-04-20T21:41:41.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.751+0000 7f73a1600640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f738006c920 0x7f738006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:41.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.751+0000 7f73a1600640 1 -- 172.21.15.102:0/1349840831 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f738c0adde0 con 0x7f73a40cd940 2024-04-20T21:41:41.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.751+0000 7f73a8c00640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f738006c920 0x7f738006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:41.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.752+0000 7f73a8c00640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f738006c920 0x7f738006ede0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7394009ac0 tx=0x7f7394009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:41.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.756+0000 7f73a1600640 1 -- 172.21.15.102:0/1349840831 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f738c07b350 con 0x7f73a40cd940 2024-04-20T21:41:41.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.996+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73a40c8560 con 0x7f73a40cd940 2024-04-20T21:41:41.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:41.997+0000 7f73a1600640 1 -- 172.21.15.102:0/1349840831 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6030 (secure 0 0 0) 0x7f738c07f650 con 0x7f73a40cd940 2024-04-20T21:41:41.997 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:41.997 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:38.534140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:41.997 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 81 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f738006c920 msgr2=0x7f738006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f738006c920 0x7f738006ede0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7394009ac0 tx=0x7f7394009190 comp rx=0 tx=0).stop 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 msgr2=0x7f73a41f0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a41f0cb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f738c02b940 tx=0x7f738c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 shutdown_connections 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f738006c920 0x7f738006ede0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73a40ce310 0x7f73a41f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73a40cd940 0x7f73a41f0cb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:41.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 --2- 172.21.15.102:0/1349840831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73a40c7900 0x7f73a41e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:42.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 >> 172.21.15.102:0/1349840831 conn(0x7f73a406dad0 msgr2=0x7f73a40c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:42.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 shutdown_connections 2024-04-20T21:41:42.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:42.000+0000 7f73ab600640 1 -- 172.21.15.102:0/1349840831 wait complete. 2024-04-20T21:41:42.017 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:41:44.018 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:44.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.218+0000 7f110a200640 1 Processor -- start 2024-04-20T21:41:44.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.219+0000 7f110a200640 1 -- start start 2024-04-20T21:41:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.220+0000 7f110a200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1104152290 0x7f11040cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.220+0000 7f110a200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f110415c220 con 0x7f11040c8330 2024-04-20T21:41:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.220+0000 7f110a200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11040c4560 con 0x7f11040c3700 2024-04-20T21:41:44.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.220+0000 7f110a200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f11040c9180 con 0x7f1104152290 2024-04-20T21:41:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.220+0000 7f1109800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f11040c8330 0x7f1104162650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58812/0 (socket says 172.21.15.102:58812) 2024-04-20T21:41:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.220+0000 7f1109800640 1 -- 172.21.15.102:0/1203428547 learned_addr learned my addr 172.21.15.102:0/1203428547 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.220+0000 7f1108e00640 1 --2- 172.21.15.102:0/1203428547 >> v2:172.21.15.193:3300/0 conn(0x7f1104152290 0x7f11040cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.221+0000 7f1103e00640 1 -- 172.21.15.102:0/1203428547 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620039053 0 0) 0x7f11040c4560 con 0x7f11040c3700 2024-04-20T21:41:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.221+0000 7f1103e00640 1 -- 172.21.15.102:0/1203428547 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10e8005730 con 0x7f11040c3700 2024-04-20T21:41:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.221+0000 7f1103e00640 1 -- 172.21.15.102:0/1203428547 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 667035032 0 0) 0x7f110415c220 con 0x7f11040c8330 2024-04-20T21:41:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.221+0000 7f1103e00640 1 -- 172.21.15.102:0/1203428547 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11040c4560 con 0x7f11040c8330 2024-04-20T21:41:44.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.221+0000 7f1108e00640 1 -- 172.21.15.102:0/1203428547 >> v1:172.21.15.193:6789/0 conn(0x7f11040c3700 legacy=0x7f110406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:44.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.221+0000 7f1108e00640 1 -- 172.21.15.102:0/1203428547 >> v1:172.21.15.102:6789/0 conn(0x7f11040c8330 legacy=0x7f1104162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:44.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.221+0000 7f1108e00640 1 -- 172.21.15.102:0/1203428547 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110415b460 con 0x7f1104152290 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.222+0000 7f1108e00640 1 --2- 172.21.15.102:0/1203428547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152290 0x7f11040cd310 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f10f40097a0 tx=0x7f10f402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cf2206824a456 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.222+0000 7f1103e00640 1 -- 172.21.15.102:0/1203428547 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f402f810 con 0x7f1104152290 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.222+0000 7f1103e00640 1 -- 172.21.15.102:0/1203428547 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f402f9b0 con 0x7f1104152290 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.222+0000 7f1103e00640 1 -- 172.21.15.102:0/1203428547 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f402fd80 con 0x7f1104152290 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.223+0000 7f110a200640 1 -- 172.21.15.102:0/1203428547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152290 msgr2=0x7f11040cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.223+0000 7f110a200640 1 --2- 172.21.15.102:0/1203428547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152290 0x7f11040cd310 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f10f40097a0 tx=0x7f10f402b9f0 comp rx=0 tx=0).stop 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.223+0000 7f110a200640 1 -- 172.21.15.102:0/1203428547 shutdown_connections 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.223+0000 7f110a200640 1 --2- 172.21.15.102:0/1203428547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1104152290 0x7f11040cd310 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.223+0000 7f110a200640 1 -- 172.21.15.102:0/1203428547 >> 172.21.15.102:0/1203428547 conn(0x7f11040cc820 msgr2=0x7f11040ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.223+0000 7f110a200640 1 -- 172.21.15.102:0/1203428547 shutdown_connections 2024-04-20T21:41:44.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.223+0000 7f110a200640 1 -- 172.21.15.102:0/1203428547 wait complete. 2024-04-20T21:41:44.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.225+0000 7f110a200640 1 Processor -- start 2024-04-20T21:41:44.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.225+0000 7f110a200640 1 -- start start 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f110a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11040c3700 0x7f1104158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f110a200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11040c8330 0x7f11041f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f110a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1104152290 0x7f11041f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f110a200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f110415ba20 con 0x7f1104152290 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f110a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11040c4840 con 0x7f11040c3700 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f110a200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f110415bd00 con 0x7f11040c8330 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f1108e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11040c3700 0x7f1104158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f1109800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1104152290 0x7f11041f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f10fbe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11040c8330 0x7f11041f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f1108e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11040c3700 0x7f1104158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:44652/0 (socket says 172.21.15.102:44652) 2024-04-20T21:41:44.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.226+0000 7f1108e00640 1 -- 172.21.15.102:0/3323641619 learned_addr learned my addr 172.21.15.102:0/3323641619 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1108e00640 1 -- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11040c8330 msgr2=0x7f11041f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1108e00640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11040c8330 0x7f11041f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1108e00640 1 -- 172.21.15.102:0/3323641619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1104152290 msgr2=0x7f11041f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1108e00640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1104152290 0x7f11041f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1108e00640 1 -- 172.21.15.102:0/3323641619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1104154e10 con 0x7f11040c3700 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f10fbe00640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11040c8330 0x7f11041f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1109800640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1104152290 0x7f11041f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1108e00640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11040c3700 0x7f1104158390 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f10f402c000 tx=0x7f10f4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1101600640 1 -- 172.21.15.102:0/3323641619 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4047070 con 0x7f11040c3700 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1101600640 1 -- 172.21.15.102:0/3323641619 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f40378f0 con 0x7f11040c3700 2024-04-20T21:41:44.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f1101600640 1 -- 172.21.15.102:0/3323641619 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4037ce0 con 0x7f11040c3700 2024-04-20T21:41:44.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11041f4510 con 0x7f11040c3700 2024-04-20T21:41:44.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.227+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11041f4ad0 con 0x7f11040c3700 2024-04-20T21:41:44.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.228+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110404fd60 con 0x7f11040c3700 2024-04-20T21:41:44.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.229+0000 7f1101600640 1 -- 172.21.15.102:0/3323641619 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10f402d3a0 con 0x7f11040c3700 2024-04-20T21:41:44.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.230+0000 7f1101600640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e006c920 0x7f10e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:44.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.230+0000 7f1101600640 1 -- 172.21.15.102:0/3323641619 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f10f40ade00 con 0x7f11040c3700 2024-04-20T21:41:44.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.230+0000 7f10fbe00640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e006c920 0x7f10e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:44.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.231+0000 7f10fbe00640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e006c920 0x7f10e006ede0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f10ec007fd0 tx=0x7f10ec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:44.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.235+0000 7f1101600640 1 -- 172.21.15.102:0/3323641619 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f10f4033030 con 0x7f11040c3700 2024-04-20T21:41:44.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.496+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11041f0fa0 con 0x7f11040c3700 2024-04-20T21:41:44.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.497+0000 7f1101600640 1 -- 172.21.15.102:0/3323641619 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6030 (secure 0 0 0) 0x7f10f407b2e0 con 0x7f11040c3700 2024-04-20T21:41:44.497 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:44.497 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:38.534140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:44.497 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 81 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e006c920 msgr2=0x7f10e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e006c920 0x7f10e006ede0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f10ec007fd0 tx=0x7f10ec009040 comp rx=0 tx=0).stop 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11040c3700 msgr2=0x7f1104158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11040c3700 0x7f1104158390 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f10f402c000 tx=0x7f10f4004730 comp rx=0 tx=0).stop 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 shutdown_connections 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1104152290 0x7f11041f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11040c8330 0x7f11041f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f10e006c920 0x7f10e006ede0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 --2- 172.21.15.102:0/3323641619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11040c3700 0x7f1104158390 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:44.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 >> 172.21.15.102:0/3323641619 conn(0x7f11040cc820 msgr2=0x7f110406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:44.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 shutdown_connections 2024-04-20T21:41:44.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:44.500+0000 7f110a200640 1 -- 172.21.15.102:0/3323641619 wait complete. 2024-04-20T21:41:44.516 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:41:46.517 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.714+0000 7ff4c2400640 1 Processor -- start 2024-04-20T21:41:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.714+0000 7ff4c2400640 1 -- start start 2024-04-20T21:41:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.715+0000 7ff4c2400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c68e0 0x7ff4bc16b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.715+0000 7ff4c2400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff4bc06f0b0 0x7ff4bc16b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.715+0000 7ff4c2400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4bc1659c0 con 0x7ff4bc1655e0 2024-04-20T21:41:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.715+0000 7ff4c2400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff4bc0c7df0 con 0x7ff4bc06f0b0 2024-04-20T21:41:46.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.715+0000 7ff4c2400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc0c80d0 con 0x7ff4bc0c68e0 2024-04-20T21:41:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.716+0000 7ff4c1000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff4bc1655e0 0x7ff4bc06e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58822/0 (socket says 172.21.15.102:58822) 2024-04-20T21:41:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.716+0000 7ff4c1000640 1 -- 172.21.15.102:0/3084140456 learned_addr learned my addr 172.21.15.102:0/3084140456 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:41:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.716+0000 7ff4bbe00640 1 --2- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c68e0 0x7ff4bc16b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.716+0000 7ff4c1a00640 1 --2- 172.21.15.102:0/3084140456 >> v2:172.21.15.193:3300/0 conn(0x7ff4bc06f0b0 0x7ff4bc16b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:46.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.716+0000 7ff4bb400640 1 -- 172.21.15.102:0/3084140456 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2940877993 0 0) 0x7ff4bc1659c0 con 0x7ff4bc1655e0 2024-04-20T21:41:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.768+0000 7ff4bb400640 1 -- 172.21.15.102:0/3084140456 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4a4003500 con 0x7ff4bc1655e0 2024-04-20T21:41:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.769+0000 7ff4c1a00640 1 -- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c68e0 msgr2=0x7ff4bc16b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.769+0000 7ff4c1a00640 1 --2- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c68e0 0x7ff4bc16b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.769+0000 7ff4c1a00640 1 -- 172.21.15.102:0/3084140456 >> v1:172.21.15.102:6789/0 conn(0x7ff4bc1655e0 legacy=0x7ff4bc06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.769+0000 7ff4c1a00640 1 -- 172.21.15.102:0/3084140456 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc0c3460 con 0x7ff4bc06f0b0 2024-04-20T21:41:46.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.769+0000 7ff4bbe00640 1 --2- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c68e0 0x7ff4bc16b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:46.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.771+0000 7ff4c1a00640 1 --2- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc16b760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009d90 tx=0x7ff4b402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6af876f690b43cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.772+0000 7ff4bb400640 1 -- 172.21.15.102:0/3084140456 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4039070 con 0x7ff4bc06f0b0 2024-04-20T21:41:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.772+0000 7ff4bb400640 1 -- 172.21.15.102:0/3084140456 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b402cc50 con 0x7ff4bc06f0b0 2024-04-20T21:41:46.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.772+0000 7ff4bb400640 1 -- 172.21.15.102:0/3084140456 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4034440 con 0x7ff4bc06f0b0 2024-04-20T21:41:46.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.773+0000 7ff4c2400640 1 -- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 msgr2=0x7ff4bc16b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:46.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.773+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc16b760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009d90 tx=0x7ff4b402bea0 comp rx=0 tx=0).stop 2024-04-20T21:41:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.773+0000 7ff4c2400640 1 -- 172.21.15.102:0/3084140456 shutdown_connections 2024-04-20T21:41:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.773+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc16b760 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.773+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3084140456 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c68e0 0x7ff4bc16b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.773+0000 7ff4c2400640 1 -- 172.21.15.102:0/3084140456 >> 172.21.15.102:0/3084140456 conn(0x7ff4bc0cc820 msgr2=0x7ff4bc0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.774+0000 7ff4c2400640 1 -- 172.21.15.102:0/3084140456 shutdown_connections 2024-04-20T21:41:46.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.774+0000 7ff4c2400640 1 -- 172.21.15.102:0/3084140456 wait complete. 2024-04-20T21:41:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.776+0000 7ff4c2400640 1 Processor -- start 2024-04-20T21:41:46.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.776+0000 7ff4c2400640 1 -- start start 2024-04-20T21:41:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc2027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:46.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4bc1655e0 0x7ff4bc0c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c2400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c3980 0x7ff4bc205b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc06c2a0 con 0x7ff4bc1655e0 2024-04-20T21:41:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c2400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc06c120 con 0x7ff4bc06f0b0 2024-04-20T21:41:46.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c2400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc16bca0 con 0x7ff4bc0c3980 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4bc1655e0 0x7ff4bc0c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc2027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c1a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c3980 0x7ff4bc205b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc2027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:44660/0 (socket says 172.21.15.102:44660) 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4bc1655e0 0x7ff4bc0c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37004/0 (socket says 172.21.15.102:37004) 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.778+0000 7ff4c1000640 1 -- 172.21.15.102:0/3615934540 learned_addr learned my addr 172.21.15.102:0/3615934540 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4c1a00640 1 -- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 msgr2=0x7ff4bc2027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4c1a00640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc2027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4c1a00640 1 -- 172.21.15.102:0/3615934540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4bc1655e0 msgr2=0x7ff4bc0c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4c1a00640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4bc1655e0 0x7ff4bc0c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4c1a00640 1 -- 172.21.15.102:0/3615934540 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc201d30 con 0x7ff4bc0c3980 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4c1000640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc2027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:46.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4c1a00640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c3980 0x7ff4bc205b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4033000 tx=0x7ff4b4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:46.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4b8c00640 1 -- 172.21.15.102:0/3615934540 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4044020 con 0x7ff4bc0c3980 2024-04-20T21:41:46.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4b8c00640 1 -- 172.21.15.102:0/3615934540 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b402ca50 con 0x7ff4bc0c3980 2024-04-20T21:41:46.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4b8c00640 1 -- 172.21.15.102:0/3615934540 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4039040 con 0x7ff4bc0c3980 2024-04-20T21:41:46.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.779+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc206090 con 0x7ff4bc0c3980 2024-04-20T21:41:46.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.780+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc16a450 con 0x7ff4bc0c3980 2024-04-20T21:41:46.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.781+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4bc04fd60 con 0x7ff4bc0c3980 2024-04-20T21:41:46.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.782+0000 7ff4b8c00640 1 -- 172.21.15.102:0/3615934540 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4b4047020 con 0x7ff4bc0c3980 2024-04-20T21:41:46.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.783+0000 7ff4b8c00640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff49c06c8f0 0x7ff49c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:46.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.783+0000 7ff4b8c00640 1 -- 172.21.15.102:0/3615934540 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7ff4b40aeac0 con 0x7ff4bc0c3980 2024-04-20T21:41:46.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.783+0000 7ff4c1000640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff49c06c8f0 0x7ff49c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:46.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.784+0000 7ff4c1000640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff49c06c8f0 0x7ff49c06edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0005880 tx=0x7ff4b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:46.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:46.788+0000 7ff4b8c00640 1 -- 172.21.15.102:0/3615934540 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff4b404e030 con 0x7ff4bc0c3980 2024-04-20T21:41:47.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.028+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4bc0c83e0 con 0x7ff4bc0c3980 2024-04-20T21:41:47.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.029+0000 7ff4b8c00640 1 -- 172.21.15.102:0/3615934540 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+6027 (secure 0 0 0) 0x7ff4b407c030 con 0x7ff4bc0c3980 2024-04-20T21:41:47.029 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:47.029 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:44.410474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:47.029 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 82 2024-04-20T21:41:47.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff49c06c8f0 msgr2=0x7ff49c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:47.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff49c06c8f0 0x7ff49c06edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0005880 tx=0x7ff4b0009040 comp rx=0 tx=0).stop 2024-04-20T21:41:47.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c3980 msgr2=0x7ff4bc205b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:47.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c3980 0x7ff4bc205b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4033000 tx=0x7ff4b4004a40 comp rx=0 tx=0).stop 2024-04-20T21:41:47.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 shutdown_connections 2024-04-20T21:41:47.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4bc0c3980 0x7ff4bc205b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:47.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4bc1655e0 0x7ff4bc0c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:47.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff49c06c8f0 0x7ff49c06edb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:47.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 --2- 172.21.15.102:0/3615934540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4bc06f0b0 0x7ff4bc2027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:47.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 >> 172.21.15.102:0/3615934540 conn(0x7ff4bc0cc820 msgr2=0x7ff4bc0cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:47.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 shutdown_connections 2024-04-20T21:41:47.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:47.032+0000 7ff4c2400640 1 -- 172.21.15.102:0/3615934540 wait complete. 2024-04-20T21:41:47.048 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:41:49.049 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:49.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.256+0000 7f56d2e00640 1 Processor -- start 2024-04-20T21:41:49.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.257+0000 7f56d2e00640 1 -- start start 2024-04-20T21:41:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.258+0000 7f56d2e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f56cc152290 0x7f56cc0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:49.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.258+0000 7f56d2e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c8330 0x7f56cc0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.258+0000 7f56d2e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56cc15c220 con 0x7f56cc0c3700 2024-04-20T21:41:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.258+0000 7f56d2e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f56cc0c9180 con 0x7f56cc152290 2024-04-20T21:41:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.258+0000 7f56d2e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc0c4840 con 0x7f56cc0c8330 2024-04-20T21:41:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56d1a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f56cc152290 0x7f56cc0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56d1a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f56cc152290 0x7f56cc0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57836/0 (socket says 172.21.15.102:57836) 2024-04-20T21:41:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56d1a00640 1 -- 172.21.15.102:0/1539955345 learned_addr learned my addr 172.21.15.102:0/1539955345 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:49.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56d2400640 1 --2- 172.21.15.102:0/1539955345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c8330 0x7f56cc0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:49.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56d1a00640 1 -- 172.21.15.102:0/1539955345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c8330 msgr2=0x7f56cc0c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:49.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56d1a00640 1 --2- 172.21.15.102:0/1539955345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c8330 0x7f56cc0c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56d1a00640 1 -- 172.21.15.102:0/1539955345 >> v1:172.21.15.193:6789/0 conn(0x7f56cc0c3700 legacy=0x7f56cc06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:49.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56d1a00640 1 -- 172.21.15.102:0/1539955345 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc0c4560 con 0x7f56cc152290 2024-04-20T21:41:49.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.259+0000 7f56cbe00640 1 -- 172.21.15.102:0/1539955345 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1688591898 0 0) 0x7f56cc15c220 con 0x7f56cc0c3700 2024-04-20T21:41:49.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.260+0000 7f56d1a00640 1 --2- 172.21.15.102:0/1539955345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc152290 0x7f56cc0cd310 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009870 tx=0x7f56c002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112d29539333ffe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:49.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.260+0000 7f56cbe00640 1 -- 172.21.15.102:0/1539955345 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c002f840 con 0x7f56cc152290 2024-04-20T21:41:49.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.260+0000 7f56cbe00640 1 -- 172.21.15.102:0/1539955345 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c002f9e0 con 0x7f56cc152290 2024-04-20T21:41:49.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.260+0000 7f56cbe00640 1 -- 172.21.15.102:0/1539955345 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c002fdb0 con 0x7f56cc152290 2024-04-20T21:41:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.261+0000 7f56d2e00640 1 -- 172.21.15.102:0/1539955345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc152290 msgr2=0x7f56cc0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.261+0000 7f56d2e00640 1 --2- 172.21.15.102:0/1539955345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc152290 0x7f56cc0cd310 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009870 tx=0x7f56c002ba20 comp rx=0 tx=0).stop 2024-04-20T21:41:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.261+0000 7f56d2e00640 1 -- 172.21.15.102:0/1539955345 shutdown_connections 2024-04-20T21:41:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.261+0000 7f56d2e00640 1 --2- 172.21.15.102:0/1539955345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c8330 0x7f56cc0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.261+0000 7f56d2e00640 1 --2- 172.21.15.102:0/1539955345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc152290 0x7f56cc0cd310 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.261+0000 7f56d2e00640 1 -- 172.21.15.102:0/1539955345 >> 172.21.15.102:0/1539955345 conn(0x7f56cc0cc820 msgr2=0x7f56cc0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.261+0000 7f56d2e00640 1 -- 172.21.15.102:0/1539955345 shutdown_connections 2024-04-20T21:41:49.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.261+0000 7f56d2e00640 1 -- 172.21.15.102:0/1539955345 wait complete. 2024-04-20T21:41:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.263+0000 7f56d2e00640 1 Processor -- start 2024-04-20T21:41:49.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.263+0000 7f56d2e00640 1 -- start start 2024-04-20T21:41:49.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.264+0000 7f56d2e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c3700 0x7f56cc148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:49.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.264+0000 7f56d2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc0c8330 0x7f56cc0c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:49.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.264+0000 7f56d2e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56cc152290 0x7f56cc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.264+0000 7f56d2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc15ba20 con 0x7f56cc0c8330 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.264+0000 7f56d2e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc15b740 con 0x7f56cc152290 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.264+0000 7f56d2e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc15bd00 con 0x7f56cc0c3700 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.264+0000 7f56d1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc0c8330 0x7f56cc0c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56cc152290 0x7f56cc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c3700 0x7f56cc148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c3700 0x7f56cc148440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:40580/0 (socket says 172.21.15.102:40580) 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56cc152290 0x7f56cc0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:37478/0 (socket says 172.21.15.102:37478) 2024-04-20T21:41:49.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 -- 172.21.15.102:0/4225775109 learned_addr learned my addr 172.21.15.102:0/4225775109 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:49.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 -- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56cc152290 msgr2=0x7f56cc0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:49.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56cc152290 0x7f56cc0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 -- 172.21.15.102:0/4225775109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc0c8330 msgr2=0x7f56cc0c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:49.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc0c8330 0x7f56cc0c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 -- 172.21.15.102:0/4225775109 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc156bd0 con 0x7f56cc0c3700 2024-04-20T21:41:49.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d2400640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56cc152290 0x7f56cc0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:49.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.265+0000 7f56d1a00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c3700 0x7f56cc148440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56c002ba60 tx=0x7f56c0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:49.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.266+0000 7f56c9600640 1 -- 172.21.15.102:0/4225775109 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0047070 con 0x7f56cc0c3700 2024-04-20T21:41:49.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.266+0000 7f56c9600640 1 -- 172.21.15.102:0/4225775109 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c0037bc0 con 0x7f56cc0c3700 2024-04-20T21:41:49.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.266+0000 7f56c9600640 1 -- 172.21.15.102:0/4225775109 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0042480 con 0x7f56cc0c3700 2024-04-20T21:41:49.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.266+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc0cbbe0 con 0x7f56cc0c3700 2024-04-20T21:41:49.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.266+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc14e3e0 con 0x7f56cc0c3700 2024-04-20T21:41:49.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.267+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc04fd60 con 0x7f56cc0c3700 2024-04-20T21:41:49.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.268+0000 7f56c9600640 1 -- 172.21.15.102:0/4225775109 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56c004a070 con 0x7f56cc0c3700 2024-04-20T21:41:49.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.269+0000 7f56c9600640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f56ac06c8f0 0x7f56ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:49.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.269+0000 7f56c9600640 1 -- 172.21.15.102:0/4225775109 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f56c00ae6d0 con 0x7f56cc0c3700 2024-04-20T21:41:49.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.269+0000 7f56d1000640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f56ac06c8f0 0x7f56ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:49.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.270+0000 7f56d1000640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f56ac06c8f0 0x7f56ac06edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f56cc1f2000 tx=0x7f56b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:49.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.273+0000 7f56c9600640 1 -- 172.21.15.102:0/4225775109 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f56c0033030 con 0x7f56cc0c3700 2024-04-20T21:41:49.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.512+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56cc06dcb0 con 0x7f56cc0c3700 2024-04-20T21:41:49.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.514+0000 7f56c9600640 1 -- 172.21.15.102:0/4225775109 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+6027 (secure 0 0 0) 0x7f56c007bc30 con 0x7f56cc0c3700 2024-04-20T21:41:49.513 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:49.513 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:44.410474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:41:49.513 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 82 2024-04-20T21:41:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f56ac06c8f0 msgr2=0x7f56ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f56ac06c8f0 0x7f56ac06edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f56cc1f2000 tx=0x7f56b8009040 comp rx=0 tx=0).stop 2024-04-20T21:41:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c3700 msgr2=0x7f56cc148440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c3700 0x7f56cc148440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56c002ba60 tx=0x7f56c0004730 comp rx=0 tx=0).stop 2024-04-20T21:41:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 shutdown_connections 2024-04-20T21:41:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f56ac06c8f0 0x7f56ac06edb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56cc152290 0x7f56cc0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56cc0c8330 0x7f56cc0c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 --2- 172.21.15.102:0/4225775109 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56cc0c3700 0x7f56cc148440 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:49.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.516+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 >> 172.21.15.102:0/4225775109 conn(0x7f56cc0cc820 msgr2=0x7f56cc06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:49.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.517+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 shutdown_connections 2024-04-20T21:41:49.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:49.517+0000 7f56d2e00640 1 -- 172.21.15.102:0/4225775109 wait complete. 2024-04-20T21:41:49.534 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:41:51.535 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:51.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.745+0000 7fbf61000640 1 Processor -- start 2024-04-20T21:41:51.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.745+0000 7fbf61000640 1 -- start start 2024-04-20T21:41:51.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.746+0000 7fbf61000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c152290 0x7fbf5c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:51.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.746+0000 7fbf61000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbf5c0c3700 0x7fbf5c06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.746+0000 7fbf61000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf5c15c220 con 0x7fbf5c06eb70 2024-04-20T21:41:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.746+0000 7fbf61000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fbf5c0c9180 con 0x7fbf5c0c3700 2024-04-20T21:41:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.746+0000 7fbf61000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5c0c4840 con 0x7fbf5c152290 2024-04-20T21:41:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.747+0000 7fbf5aa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbf5c0c3700 0x7fbf5c06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.747+0000 7fbf5aa00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbf5c0c3700 0x7fbf5c06e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57846/0 (socket says 172.21.15.102:57846) 2024-04-20T21:41:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.747+0000 7fbf5aa00640 1 -- 172.21.15.102:0/1831802241 learned_addr learned my addr 172.21.15.102:0/1831802241 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:51.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.747+0000 7fbf5b400640 1 --2- 172.21.15.102:0/1831802241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c152290 0x7fbf5c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.747+0000 7fbf5aa00640 1 -- 172.21.15.102:0/1831802241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c152290 msgr2=0x7fbf5c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.747+0000 7fbf5aa00640 1 --2- 172.21.15.102:0/1831802241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c152290 0x7fbf5c0cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.747+0000 7fbf5aa00640 1 -- 172.21.15.102:0/1831802241 >> v1:172.21.15.193:6789/0 conn(0x7fbf5c06eb70 legacy=0x7fbf5c0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.747+0000 7fbf5aa00640 1 -- 172.21.15.102:0/1831802241 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c0c4560 con 0x7fbf5c0c3700 2024-04-20T21:41:51.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.748+0000 7fbf5aa00640 1 --2- 172.21.15.102:0/1831802241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c0c3700 0x7fbf5c06e630 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009870 tx=0x7fbf4802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e02097d778f725 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.748+0000 7fbf5a000640 1 -- 172.21.15.102:0/1831802241 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4802f840 con 0x7fbf5c0c3700 2024-04-20T21:41:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.748+0000 7fbf5a000640 1 -- 172.21.15.102:0/1831802241 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4802f9e0 con 0x7fbf5c0c3700 2024-04-20T21:41:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.748+0000 7fbf5a000640 1 -- 172.21.15.102:0/1831802241 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4802fdb0 con 0x7fbf5c0c3700 2024-04-20T21:41:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.749+0000 7fbf61000640 1 -- 172.21.15.102:0/1831802241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c0c3700 msgr2=0x7fbf5c06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.749+0000 7fbf61000640 1 --2- 172.21.15.102:0/1831802241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c0c3700 0x7fbf5c06e630 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009870 tx=0x7fbf4802ba20 comp rx=0 tx=0).stop 2024-04-20T21:41:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.749+0000 7fbf61000640 1 -- 172.21.15.102:0/1831802241 shutdown_connections 2024-04-20T21:41:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.749+0000 7fbf61000640 1 --2- 172.21.15.102:0/1831802241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c0c3700 0x7fbf5c06e630 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.749+0000 7fbf61000640 1 --2- 172.21.15.102:0/1831802241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c152290 0x7fbf5c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.749+0000 7fbf61000640 1 -- 172.21.15.102:0/1831802241 >> 172.21.15.102:0/1831802241 conn(0x7fbf5c0cc820 msgr2=0x7fbf5c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.749+0000 7fbf61000640 1 -- 172.21.15.102:0/1831802241 shutdown_connections 2024-04-20T21:41:51.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.749+0000 7fbf61000640 1 -- 172.21.15.102:0/1831802241 wait complete. 2024-04-20T21:41:51.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.751+0000 7fbf61000640 1 Processor -- start 2024-04-20T21:41:51.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.751+0000 7fbf61000640 1 -- start start 2024-04-20T21:41:51.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf61000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c06eb70 0x7fbf5c1e4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:51.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf61000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c0c3700 0x7fbf5c071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:51.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf61000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c152290 0x7fbf5c0724c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:51.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf61000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c15ba20 con 0x7fbf5c152290 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf61000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c15b740 con 0x7fbf5c0c3700 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf61000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5c15bd00 con 0x7fbf5c06eb70 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf5be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c152290 0x7fbf5c0724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf5be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c152290 0x7fbf5c0724c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57852/0 (socket says 172.21.15.102:57852) 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf5be00640 1 -- 172.21.15.102:0/1208069094 learned_addr learned my addr 172.21.15.102:0/1208069094 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf5aa00640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c0c3700 0x7fbf5c071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.752+0000 7fbf5b400640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c06eb70 0x7fbf5c1e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf5be00640 1 -- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c06eb70 msgr2=0x7fbf5c1e4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf5be00640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c06eb70 0x7fbf5c1e4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf5be00640 1 -- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c0c3700 msgr2=0x7fbf5c071f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf5be00640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c0c3700 0x7fbf5c071f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf5be00640 1 -- 172.21.15.102:0/1208069094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c1e24c0 con 0x7fbf5c152290 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf5be00640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c152290 0x7fbf5c0724c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400c680 tx=0x7fbf5400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:51.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf53400640 1 -- 172.21.15.102:0/1208069094 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54003b70 con 0x7fbf5c152290 2024-04-20T21:41:51.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf53400640 1 -- 172.21.15.102:0/1208069094 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf54003d10 con 0x7fbf5c152290 2024-04-20T21:41:51.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf53400640 1 -- 172.21.15.102:0/1208069094 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54005310 con 0x7fbf5c152290 2024-04-20T21:41:51.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c0cc5a0 con 0x7fbf5c152290 2024-04-20T21:41:51.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.753+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c1e2a60 con 0x7fbf5c152290 2024-04-20T21:41:51.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.754+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf5c04fd60 con 0x7fbf5c152290 2024-04-20T21:41:51.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.755+0000 7fbf53400640 1 -- 172.21.15.102:0/1208069094 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf540054b0 con 0x7fbf5c152290 2024-04-20T21:41:51.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.756+0000 7fbf53400640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:51.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.756+0000 7fbf53400640 1 -- 172.21.15.102:0/1208069094 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7fbf5408c160 con 0x7fbf5c152290 2024-04-20T21:41:51.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.757+0000 7fbf5b400640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:51.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.757+0000 7fbf5b400640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009b40 tx=0x7fbf4c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:51.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:51.761+0000 7fbf53400640 1 -- 172.21.15.102:0/1208069094 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbf54059640 con 0x7fbf5c152290 2024-04-20T21:41:52.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.001+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf5c151ab0 con 0x7fbf5c152290 2024-04-20T21:41:52.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.002+0000 7fbf53400640 1 -- 172.21.15.102:0/1208069094 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+6034 (secure 0 0 0) 0x7fbf5408b070 con 0x7fbf5c152290 2024-04-20T21:41:52.001 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:52.001 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:51.395204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":29,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6035,"qdb_cluster":[6035]},"id":1}]} 2024-04-20T21:41:52.001 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 83 2024-04-20T21:41:52.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 msgr2=0x7fbf3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:52.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009b40 tx=0x7fbf4c009190 comp rx=0 tx=0).stop 2024-04-20T21:41:52.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c152290 msgr2=0x7fbf5c0724c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c152290 0x7fbf5c0724c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400c680 tx=0x7fbf5400cb40 comp rx=0 tx=0).stop 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 shutdown_connections 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf5c152290 0x7fbf5c0724c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf3c06c8f0 0x7fbf3c06edb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c0c3700 0x7fbf5c071f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 --2- 172.21.15.102:0/1208069094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf5c06eb70 0x7fbf5c1e4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.004+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 >> 172.21.15.102:0/1208069094 conn(0x7fbf5c0cc820 msgr2=0x7fbf5c0c88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.005+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 shutdown_connections 2024-04-20T21:41:52.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.005+0000 7fbf61000640 1 -- 172.21.15.102:0/1208069094 wait complete. 2024-04-20T21:41:52.023 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=6429 2024-04-20T21:41:52.023 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.c 2024-04-20T21:41:52.023 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:41:52.023 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:41:52.023 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:41:52.031 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:41:52.031 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:52.181 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:41:52.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.206+0000 7f065fc00640 1 Processor -- start 2024-04-20T21:41:52.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.206+0000 7f065fc00640 1 -- start start 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f065fc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0658155610 0x7f065815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f065fc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f065815bd60 0x7f065815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f065fc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658073950 0x7f06580ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f065fc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f065815b8d0 con 0x7f065815bd60 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f065fc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f065815f6f0 con 0x7f0658155610 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f065fc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f065815c180 con 0x7f0658073950 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f0657e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f065815bd60 0x7f065815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f065dc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658073950 0x7f06580ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:52.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f065d200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0658155610 0x7f065815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:52.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f0657e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f065815bd60 0x7f065815f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57890/0 (socket says 172.21.15.102:57890) 2024-04-20T21:41:52.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.208+0000 7f0657e00640 1 -- 172.21.15.102:0/2521495702 learned_addr learned my addr 172.21.15.102:0/2521495702 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:52.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.209+0000 7f065dc00640 1 -- 172.21.15.102:0/2521495702 >> v2:172.21.15.193:3300/0 conn(0x7f0658155610 msgr2=0x7f065815c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:52.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.209+0000 7f065dc00640 1 --2- 172.21.15.102:0/2521495702 >> v2:172.21.15.193:3300/0 conn(0x7f0658155610 0x7f065815c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.209+0000 7f065dc00640 1 -- 172.21.15.102:0/2521495702 >> v2:172.21.15.102:3300/0 conn(0x7f065815bd60 msgr2=0x7f065815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:52.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.209+0000 7f065dc00640 1 --2- 172.21.15.102:0/2521495702 >> v2:172.21.15.102:3300/0 conn(0x7f065815bd60 0x7f065815f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.209+0000 7f065dc00640 1 -- 172.21.15.102:0/2521495702 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065815b2c0 con 0x7f0658073950 2024-04-20T21:41:52.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.210+0000 7f065dc00640 1 --2- 172.21.15.102:0/2521495702 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658073950 0x7f06580ceb10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0658073400 tx=0x7f065002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f35c23ca7efceb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.210+0000 7f0657400640 1 -- 172.21.15.102:0/2521495702 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650039070 con 0x7f0658073950 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.210+0000 7f0657400640 1 -- 172.21.15.102:0/2521495702 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065002cc60 con 0x7f0658073950 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.210+0000 7f0657400640 1 -- 172.21.15.102:0/2521495702 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650034660 con 0x7f0658073950 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 -- 172.21.15.102:0/2521495702 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658073950 msgr2=0x7f06580ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 --2- 172.21.15.102:0/2521495702 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658073950 0x7f06580ceb10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0658073400 tx=0x7f065002beb0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 -- 172.21.15.102:0/2521495702 shutdown_connections 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 --2- 172.21.15.102:0/2521495702 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658073950 0x7f06580ceb10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 --2- 172.21.15.102:0/2521495702 >> v2:172.21.15.102:3300/0 conn(0x7f065815bd60 0x7f065815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 --2- 172.21.15.102:0/2521495702 >> v2:172.21.15.193:3300/0 conn(0x7f0658155610 0x7f065815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 -- 172.21.15.102:0/2521495702 >> 172.21.15.102:0/2521495702 conn(0x7f065806ebf0 msgr2=0x7f065806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 -- 172.21.15.102:0/2521495702 shutdown_connections 2024-04-20T21:41:52.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.211+0000 7f065fc00640 1 -- 172.21.15.102:0/2521495702 wait complete. 2024-04-20T21:41:52.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.213+0000 7f065fc00640 1 Processor -- start 2024-04-20T21:41:52.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.213+0000 7f065fc00640 1 -- start start 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065fc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0658073950 0x7f06580c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065fc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658155610 0x7f06581f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065fc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065815bd60 0x7f0658154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065fc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f065815b560 con 0x7f065815bd60 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065fc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06580c9d50 con 0x7f0658073950 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065fc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f06580ca030 con 0x7f0658155610 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065815bd60 0x7f0658154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065815bd60 0x7f0658154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57906/0 (socket says 172.21.15.102:57906) 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065dc00640 1 -- 172.21.15.102:0/2172057196 learned_addr learned my addr 172.21.15.102:0/2172057196 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f0657e00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658155610 0x7f06581f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:52.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065d200640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0658073950 0x7f06580c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:52.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065dc00640 1 -- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658155610 msgr2=0x7f06581f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:52.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065dc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658155610 0x7f06581f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065dc00640 1 -- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0658073950 msgr2=0x7f06580c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065dc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0658073950 0x7f06580c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.214+0000 7f065dc00640 1 -- 172.21.15.102:0/2172057196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06581e4800 con 0x7f065815bd60 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.215+0000 7f065dc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065815bd60 0x7f0658154040 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0650037040 tx=0x7f0650004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.215+0000 7f0654c00640 1 -- 172.21.15.102:0/2172057196 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650039050 con 0x7f065815bd60 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.215+0000 7f0654c00640 1 -- 172.21.15.102:0/2172057196 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0650045070 con 0x7f065815bd60 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.215+0000 7f0654c00640 1 -- 172.21.15.102:0/2172057196 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650009450 con 0x7f065815bd60 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.215+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06581f1e30 con 0x7f065815bd60 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.215+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06581e4d80 con 0x7f065815bd60 2024-04-20T21:41:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.216+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0658074340 con 0x7f065815bd60 2024-04-20T21:41:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.217+0000 7f0654c00640 1 -- 172.21.15.102:0/2172057196 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f065003e850 con 0x7f065815bd60 2024-04-20T21:41:52.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.219+0000 7f0654c00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f063806c8f0 0x7f063806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:52.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.219+0000 7f0654c00640 1 -- 172.21.15.102:0/2172057196 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7853+0+0 (secure 0 0 0) 0x7f06500aefe0 con 0x7f065815bd60 2024-04-20T21:41:52.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.219+0000 7f065d200640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f063806c8f0 0x7f063806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:52.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.220+0000 7f065d200640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f063806c8f0 0x7f063806edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f064c009b40 tx=0x7f064c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:52.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.223+0000 7f0654c00640 1 -- 172.21.15.102:0/2172057196 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f065007c6a0 con 0x7f065815bd60 2024-04-20T21:41:52.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.465+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06580c9740 con 0x7f065815bd60 2024-04-20T21:41:52.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.467+0000 7f0654c00640 1 -- 172.21.15.102:0/2172057196 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+6862 (secure 0 0 0) 0x7f06500809a0 con 0x7f065815bd60 2024-04-20T21:41:52.466 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:41:52.466 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2459661167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2459661167},{"type":"v1","addr":"172.21.15.102:6835","nonce":2459661167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:41:51.395204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6035":{"gid":6035,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":29,"addr":"172.21.15.102:6833/2637555410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2637555410},{"type":"v1","addr":"172.21.15.102:6833","nonce":2637555410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6035,"qdb_cluster":[6035]},"id":1}]} 2024-04-20T21:41:52.466 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 84 2024-04-20T21:41:52.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f063806c8f0 msgr2=0x7f063806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f063806c8f0 0x7f063806edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f064c009b40 tx=0x7f064c009190 comp rx=0 tx=0).stop 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065815bd60 msgr2=0x7f0658154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065815bd60 0x7f0658154040 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0650037040 tx=0x7f0650004a40 comp rx=0 tx=0).stop 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 shutdown_connections 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065815bd60 0x7f0658154040 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0658155610 0x7f06581f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f063806c8f0 0x7f063806edb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 --2- 172.21.15.102:0/2172057196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0658073950 0x7f06580c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 >> 172.21.15.102:0/2172057196 conn(0x7f065806ebf0 msgr2=0x7f065806cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.469+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 shutdown_connections 2024-04-20T21:41:52.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:41:52.470+0000 7f065fc00640 1 -- 172.21.15.102:0/2172057196 wait complete. 2024-04-20T21:41:52.487 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:41:52.488 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 49 secs before thrashing 2024-04-20T21:41:57.147 INFO:tasks.ceph.mds.a.smithi102.stderr:terminate called after throwing an instance of 'std::out_of_range' 2024-04-20T21:41:57.147 INFO:tasks.ceph.mds.a.smithi102.stderr: what(): map::at 2024-04-20T21:41:57.148 INFO:tasks.ceph.mds.a.smithi102.stderr:*** Caught signal (Aborted) ** 2024-04-20T21:41:57.148 INFO:tasks.ceph.mds.a.smithi102.stderr: in thread 7f53e3000640 thread_name:MR_Finisher 2024-04-20T21:41:57.148 INFO:tasks.ceph.mds.a.smithi102.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f53f163e6f0] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f53f168b94c] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 3: raise() 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 4: abort() 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7f53f1aa1b21] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7f53f1aad52c] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7f53f1aad597] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7f53f1aad7f9] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7f53f1aa4533] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x55d9c5c92d86] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x55d9c5d99cf7] 2024-04-20T21:41:57.149 INFO:tasks.ceph.mds.a.smithi102.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x55d9c5dda98f] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x55d9c5d4cfef] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x1486) [0x55d9c5d2f7f6] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x55d9c5d27ac6] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55d9c5d28411] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 17: (Locker::file_update_finish(CInode*, boost::intrusive_ptr&, unsigned int, client_t, boost::intrusive_ptr const&)+0x1b1) [0x55d9c5d36021] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 18: ceph-mds(+0x34494d) [0x55d9c5d3694d] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 19: (MDSContext::complete(int)+0x5c) [0x55d9c5e2e2ac] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 20: (MDSIOContextBase::complete(int)+0x31c) [0x55d9c5e2e6fc] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 21: (MDSLogContextBase::complete(int)+0x53) [0x55d9c5e2e853] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 22: (Finisher::finisher_thread_entry()+0x175) [0x7f53f2030455] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 23: /lib64/libc.so.6(+0x89c02) [0x7f53f1689c02] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr: 24: /lib64/libc.so.6(+0x10ec40) [0x7f53f170ec40] 2024-04-20T21:41:57.150 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-04-20T21:41:57.150+0000 7f53e3000640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: in thread 7f53e3000640 thread_name:MR_Finisher 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f53f163e6f0] 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f53f168b94c] 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: 3: raise() 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: 4: abort() 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7f53f1aa1b21] 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7f53f1aad52c] 2024-04-20T21:41:57.151 INFO:tasks.ceph.mds.a.smithi102.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7f53f1aad597] 2024-04-20T21:41:57.152 INFO:tasks.ceph.mds.a.smithi102.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7f53f1aad7f9] 2024-04-20T21:41:57.152 INFO:tasks.ceph.mds.a.smithi102.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7f53f1aa4533] 2024-04-20T21:41:57.152 INFO:tasks.ceph.mds.a.smithi102.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x55d9c5c92d86] 2024-04-20T21:41:57.152 INFO:tasks.ceph.mds.a.smithi102.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x55d9c5d99cf7] 2024-04-20T21:41:57.152 INFO:tasks.ceph.mds.a.smithi102.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x55d9c5dda98f] 2024-04-20T21:41:57.152 INFO:tasks.ceph.mds.a.smithi102.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x55d9c5d4cfef] 2024-04-20T21:41:57.152 INFO:tasks.ceph.mds.a.smithi102.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x1486) [0x55d9c5d2f7f6] 2024-04-20T21:41:57.152 INFO:tasks.ceph.mds.a.smithi102.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x55d9c5d27ac6] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55d9c5d28411] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 17: (Locker::file_update_finish(CInode*, boost::intrusive_ptr&, unsigned int, client_t, boost::intrusive_ptr const&)+0x1b1) [0x55d9c5d36021] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 18: ceph-mds(+0x34494d) [0x55d9c5d3694d] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 19: (MDSContext::complete(int)+0x5c) [0x55d9c5e2e2ac] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 20: (MDSIOContextBase::complete(int)+0x31c) [0x55d9c5e2e6fc] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 21: (MDSLogContextBase::complete(int)+0x53) [0x55d9c5e2e853] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 22: (Finisher::finisher_thread_entry()+0x175) [0x7f53f2030455] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 23: /lib64/libc.so.6(+0x89c02) [0x7f53f1689c02] 2024-04-20T21:41:57.153 INFO:tasks.ceph.mds.a.smithi102.stderr: 24: /lib64/libc.so.6(+0x10ec40) [0x7f53f170ec40] 2024-04-20T21:41:57.154 INFO:tasks.ceph.mds.a.smithi102.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:41:57.154 INFO:tasks.ceph.mds.a.smithi102.stderr: 2024-04-20T21:41:57.182 INFO:tasks.ceph.mds.a.smithi102.stderr: 0> 2024-04-20T21:41:57.150+0000 7f53e3000640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:41:57.182 INFO:tasks.ceph.mds.a.smithi102.stderr: in thread 7f53e3000640 thread_name:MR_Finisher 2024-04-20T21:41:57.182 INFO:tasks.ceph.mds.a.smithi102.stderr: 2024-04-20T21:41:57.182 INFO:tasks.ceph.mds.a.smithi102.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f53f163e6f0] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f53f168b94c] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 3: raise() 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 4: abort() 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7f53f1aa1b21] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7f53f1aad52c] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7f53f1aad597] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7f53f1aad7f9] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7f53f1aa4533] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x55d9c5c92d86] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x55d9c5d99cf7] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x55d9c5dda98f] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x55d9c5d4cfef] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x1486) [0x55d9c5d2f7f6] 2024-04-20T21:41:57.183 INFO:tasks.ceph.mds.a.smithi102.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x55d9c5d27ac6] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55d9c5d28411] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 17: (Locker::file_update_finish(CInode*, boost::intrusive_ptr&, unsigned int, client_t, boost::intrusive_ptr const&)+0x1b1) [0x55d9c5d36021] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 18: ceph-mds(+0x34494d) [0x55d9c5d3694d] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 19: (MDSContext::complete(int)+0x5c) [0x55d9c5e2e2ac] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 20: (MDSIOContextBase::complete(int)+0x31c) [0x55d9c5e2e6fc] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 21: (MDSLogContextBase::complete(int)+0x53) [0x55d9c5e2e853] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 22: (Finisher::finisher_thread_entry()+0x175) [0x7f53f2030455] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 23: /lib64/libc.so.6(+0x89c02) [0x7f53f1689c02] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 24: /lib64/libc.so.6(+0x10ec40) [0x7f53f170ec40] 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:41:57.184 INFO:tasks.ceph.mds.a.smithi102.stderr: 2024-04-20T21:41:57.193 INFO:tasks.ceph.mds.a.smithi102.stderr: -28> 2024-04-20T21:41:57.150+0000 7f53e3000640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:41:57.193 INFO:tasks.ceph.mds.a.smithi102.stderr: in thread 7f53e3000640 thread_name:MR_Finisher 2024-04-20T21:41:57.193 INFO:tasks.ceph.mds.a.smithi102.stderr: 2024-04-20T21:41:57.193 INFO:tasks.ceph.mds.a.smithi102.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f53f163e6f0] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f53f168b94c] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 3: raise() 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 4: abort() 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7f53f1aa1b21] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7f53f1aad52c] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7f53f1aad597] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7f53f1aad7f9] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7f53f1aa4533] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x55d9c5c92d86] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x55d9c5d99cf7] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x55d9c5dda98f] 2024-04-20T21:41:57.194 INFO:tasks.ceph.mds.a.smithi102.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x55d9c5d4cfef] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x1486) [0x55d9c5d2f7f6] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x55d9c5d27ac6] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55d9c5d28411] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 17: (Locker::file_update_finish(CInode*, boost::intrusive_ptr&, unsigned int, client_t, boost::intrusive_ptr const&)+0x1b1) [0x55d9c5d36021] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 18: ceph-mds(+0x34494d) [0x55d9c5d3694d] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 19: (MDSContext::complete(int)+0x5c) [0x55d9c5e2e2ac] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 20: (MDSIOContextBase::complete(int)+0x31c) [0x55d9c5e2e6fc] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 21: (MDSLogContextBase::complete(int)+0x53) [0x55d9c5e2e853] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 22: (Finisher::finisher_thread_entry()+0x175) [0x7f53f2030455] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 23: /lib64/libc.so.6(+0x89c02) [0x7f53f1689c02] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 24: /lib64/libc.so.6(+0x10ec40) [0x7f53f170ec40] 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:41:57.195 INFO:tasks.ceph.mds.a.smithi102.stderr: 2024-04-20T21:41:58.078 INFO:tasks.ceph.mds.a.smithi102.stderr:daemon-helper: command crashed with signal 6 2024-04-20T21:42:01.109 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:42:01.117 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:42:02.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-20T21:42:08.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-04-20T21:42:15.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~14s 2024-04-20T21:42:22.526 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2024-04-20T21:42:29.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~27s 2024-04-20T21:42:31.235 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:42:31.243 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:42:36.152 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~34s 2024-04-20T21:42:41.488 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:41.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.700+0000 7fb34d000640 1 Processor -- start 2024-04-20T21:42:41.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.700+0000 7fb34d000640 1 -- start start 2024-04-20T21:42:41.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.701+0000 7fb34d000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb34806c550 0x7fb34815c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:41.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.701+0000 7fb34d000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb348073a20 con 0x7fb3481554f0 2024-04-20T21:42:41.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.701+0000 7fb34d000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34815b510 con 0x7fb34815bfd0 2024-04-20T21:42:41.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.701+0000 7fb34d000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb348073cc0 con 0x7fb34806c550 2024-04-20T21:42:41.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb345400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb34806c550 0x7fb34815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:41.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb345e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb3481554f0 0x7fb3480c98f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48880/0 (socket says 172.21.15.102:48880) 2024-04-20T21:42:41.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb345400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb34806c550 0x7fb34815c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32936/0 (socket says 172.21.15.102:32936) 2024-04-20T21:42:41.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb345e00640 1 -- 172.21.15.102:0/3627886363 learned_addr learned my addr 172.21.15.102:0/3627886363 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:42:41.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb344a00640 1 -- 172.21.15.102:0/3627886363 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092310479 0 0) 0x7fb348073a20 con 0x7fb3481554f0 2024-04-20T21:42:41.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb344a00640 1 -- 172.21.15.102:0/3627886363 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb32c003520 con 0x7fb3481554f0 2024-04-20T21:42:41.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb344a00640 1 -- 172.21.15.102:0/3627886363 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838154069 0 0) 0x7fb34815b510 con 0x7fb34815bfd0 2024-04-20T21:42:41.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb345400640 1 -- 172.21.15.102:0/3627886363 >> v1:172.21.15.193:6789/0 conn(0x7fb34815bfd0 legacy=0x7fb3480ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:41.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb345400640 1 -- 172.21.15.102:0/3627886363 >> v1:172.21.15.102:6789/0 conn(0x7fb3481554f0 legacy=0x7fb3480c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:41.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.702+0000 7fb345400640 1 -- 172.21.15.102:0/3627886363 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34815b7d0 con 0x7fb34806c550 2024-04-20T21:42:41.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.704+0000 7fb345400640 1 --2- 172.21.15.102:0/3627886363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34806c550 0x7fb34815c810 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb3340097a0 tx=0x7fb33402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be27b4a548edb895 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:41.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.704+0000 7fb344a00640 1 -- 172.21.15.102:0/3627886363 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334034070 con 0x7fb34806c550 2024-04-20T21:42:41.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.704+0000 7fb344a00640 1 -- 172.21.15.102:0/3627886363 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb334030610 con 0x7fb34806c550 2024-04-20T21:42:41.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.704+0000 7fb344a00640 1 -- 172.21.15.102:0/3627886363 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3340309c0 con 0x7fb34806c550 2024-04-20T21:42:41.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.705+0000 7fb34d000640 1 -- 172.21.15.102:0/3627886363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34806c550 msgr2=0x7fb34815c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:41.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.705+0000 7fb34d000640 1 --2- 172.21.15.102:0/3627886363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34806c550 0x7fb34815c810 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb3340097a0 tx=0x7fb33402b960 comp rx=0 tx=0).stop 2024-04-20T21:42:41.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.705+0000 7fb34d000640 1 -- 172.21.15.102:0/3627886363 shutdown_connections 2024-04-20T21:42:41.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.705+0000 7fb34d000640 1 --2- 172.21.15.102:0/3627886363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34806c550 0x7fb34815c810 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:41.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.705+0000 7fb34d000640 1 -- 172.21.15.102:0/3627886363 >> 172.21.15.102:0/3627886363 conn(0x7fb34806ebf0 msgr2=0x7fb34806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:41.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.705+0000 7fb34d000640 1 -- 172.21.15.102:0/3627886363 shutdown_connections 2024-04-20T21:42:41.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.705+0000 7fb34d000640 1 -- 172.21.15.102:0/3627886363 wait complete. 2024-04-20T21:42:41.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.707+0000 7fb34d000640 1 Processor -- start 2024-04-20T21:42:41.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.707+0000 7fb34d000640 1 -- start start 2024-04-20T21:42:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb34d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34806c550 0x7fb348158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb34d000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3481554f0 0x7fb34815df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb34d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34815bfd0 0x7fb34815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb34d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb3480ca000 con 0x7fb34815bfd0 2024-04-20T21:42:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb34d000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3480736e0 con 0x7fb34806c550 2024-04-20T21:42:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb34d000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb3481558d0 con 0x7fb3481554f0 2024-04-20T21:42:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb345e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34806c550 0x7fb348158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:41.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb346800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34815bfd0 0x7fb34815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb346800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34815bfd0 0x7fb34815e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32950/0 (socket says 172.21.15.102:32950) 2024-04-20T21:42:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb345400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3481554f0 0x7fb34815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:41.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.708+0000 7fb346800640 1 -- 172.21.15.102:0/4058580309 learned_addr learned my addr 172.21.15.102:0/4058580309 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb346800640 1 -- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3481554f0 msgr2=0x7fb34815df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb346800640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3481554f0 0x7fb34815df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb346800640 1 -- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34806c550 msgr2=0x7fb348158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb346800640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34806c550 0x7fb348158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb346800640 1 -- 172.21.15.102:0/4058580309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3481568b0 con 0x7fb34815bfd0 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb346800640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34815bfd0 0x7fb34815e4a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c790 tx=0x7fb34000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb33e000640 1 -- 172.21.15.102:0/4058580309 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340003da0 con 0x7fb34815bfd0 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb33e000640 1 -- 172.21.15.102:0/4058580309 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb340005070 con 0x7fb34815bfd0 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb33e000640 1 -- 172.21.15.102:0/4058580309 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340005460 con 0x7fb34815bfd0 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.709+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb348155ca0 con 0x7fb34815bfd0 2024-04-20T21:42:41.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.710+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb348156260 con 0x7fb34815bfd0 2024-04-20T21:42:41.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.711+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb348074340 con 0x7fb34815bfd0 2024-04-20T21:42:41.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.712+0000 7fb33e000640 1 -- 172.21.15.102:0/4058580309 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb340005600 con 0x7fb34815bfd0 2024-04-20T21:42:41.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.713+0000 7fb33e000640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32806c8f0 0x7fb32806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:41.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.713+0000 7fb33e000640 1 -- 172.21.15.102:0/4058580309 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 7939+0+0 (secure 0 0 0) 0x7fb34008c140 con 0x7fb34815bfd0 2024-04-20T21:42:41.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.713+0000 7fb345e00640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32806c8f0 0x7fb32806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:41.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.714+0000 7fb345e00640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32806c8f0 0x7fb32806edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb338001200 tx=0x7fb338000bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:41.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:41.718+0000 7fb33e000640 1 -- 172.21.15.102:0/4058580309 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb3400596f0 con 0x7fb34815bfd0 2024-04-20T21:42:42.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.104+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3480c96d0 con 0x7fb34815bfd0 2024-04-20T21:42:42.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.105+0000 7fb33e000640 1 -- 172.21.15.102:0/4058580309 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+6034 (secure 0 0 0) 0x7fb34005d9f0 con 0x7fb34815bfd0 2024-04-20T21:42:42.104 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:42.104 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:36.539799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6835/2459661167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2459661167},{"type":"v1","addr":"172.21.15.102:6835","nonce":2459661167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":1}]} 2024-04-20T21:42:42.104 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 88 2024-04-20T21:42:42.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32806c8f0 msgr2=0x7fb32806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32806c8f0 0x7fb32806edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb338001200 tx=0x7fb338000bc0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34815bfd0 msgr2=0x7fb34815e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34815bfd0 0x7fb34815e4a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c790 tx=0x7fb34000cc50 comp rx=0 tx=0).stop 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 shutdown_connections 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb34815bfd0 0x7fb34815e4a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3481554f0 0x7fb34815df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb32806c8f0 0x7fb32806edb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 --2- 172.21.15.102:0/4058580309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34806c550 0x7fb348158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.107+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 >> 172.21.15.102:0/4058580309 conn(0x7fb34806ebf0 msgr2=0x7fb34806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.108+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 shutdown_connections 2024-04-20T21:42:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.108+0000 7fb34d000640 1 -- 172.21.15.102:0/4058580309 wait complete. 2024-04-20T21:42:42.123 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:42:42.124 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:42:42.124 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:42:42.155 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:42:42.155 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:42.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.392+0000 7f33c5e00640 1 Processor -- start 2024-04-20T21:42:42.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.392+0000 7f33c5e00640 1 -- start start 2024-04-20T21:42:42.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.393+0000 7f33c5e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c00c4040 0x7f33c015e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.393+0000 7f33c5e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f33c015eac0 0x7f33c0165910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c5e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c015f3c0 con 0x7f33c0155110 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c5e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f33c00c7c30 con 0x7f33c015eac0 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c5e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f33c006ea00 con 0x7f33c00c4040 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c5400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f33c015eac0 0x7f33c0165910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c4a00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f33c0155110 0x7f33c006ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43702/0 (socket says 172.21.15.102:43702) 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c4a00640 1 -- 172.21.15.102:0/3502444859 learned_addr learned my addr 172.21.15.102:0/3502444859 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33bfe00640 1 --2- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c00c4040 0x7f33c015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33bf400640 1 -- 172.21.15.102:0/3502444859 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2332889529 0 0) 0x7f33c015f3c0 con 0x7f33c0155110 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33bf400640 1 -- 172.21.15.102:0/3502444859 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33a4003520 con 0x7f33c0155110 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c5400640 1 -- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c00c4040 msgr2=0x7f33c015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c5400640 1 --2- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c00c4040 0x7f33c015e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c5400640 1 -- 172.21.15.102:0/3502444859 >> v1:172.21.15.193:6789/0 conn(0x7f33c0155110 legacy=0x7f33c006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33c5400640 1 -- 172.21.15.102:0/3502444859 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c006e6d0 con 0x7f33c015eac0 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.394+0000 7f33bfe00640 1 --2- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c00c4040 0x7f33c015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.395+0000 7f33c5400640 1 --2- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c015eac0 0x7f33c0165910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f33c00c7db0 tx=0x7f33b802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4067419f76044e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33bf400640 1 -- 172.21.15.102:0/3502444859 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8034070 con 0x7f33c015eac0 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33bf400640 1 -- 172.21.15.102:0/3502444859 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b8030800 con 0x7f33c015eac0 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33bf400640 1 -- 172.21.15.102:0/3502444859 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8030bd0 con 0x7f33c015eac0 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33c5e00640 1 -- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c015eac0 msgr2=0x7f33c0165910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c015eac0 0x7f33c0165910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f33c00c7db0 tx=0x7f33b802b940 comp rx=0 tx=0).stop 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33c5e00640 1 -- 172.21.15.102:0/3502444859 shutdown_connections 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c015eac0 0x7f33c0165910 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3502444859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c00c4040 0x7f33c015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33c5e00640 1 -- 172.21.15.102:0/3502444859 >> 172.21.15.102:0/3502444859 conn(0x7f33c00cc820 msgr2=0x7f33c00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.396+0000 7f33c5e00640 1 -- 172.21.15.102:0/3502444859 shutdown_connections 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.397+0000 7f33c5e00640 1 -- 172.21.15.102:0/3502444859 wait complete. 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.398+0000 7f33c5e00640 1 Processor -- start 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.398+0000 7f33c5e00640 1 -- start start 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.399+0000 7f33c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c00c4040 0x7f33c01f1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.399+0000 7f33c5e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c0155110 0x7f33c015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.399+0000 7f33c5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c015eac0 0x7f33c015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.399+0000 7f33c5e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f33c006f5a0 con 0x7f33c015eac0 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.399+0000 7f33c5e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00c8160 con 0x7f33c00c4040 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.399+0000 7f33c5e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f33c006f720 con 0x7f33c0155110 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33c4a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c00c4040 0x7f33c01f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c0155110 0x7f33c015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c0155110 0x7f33c015ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:41206/0 (socket says 172.21.15.102:41206) 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 -- 172.21.15.102:0/3707364208 learned_addr learned my addr 172.21.15.102:0/3707364208 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33c5400640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c015eac0 0x7f33c015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 -- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c00c4040 msgr2=0x7f33c01f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c00c4040 0x7f33c01f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 -- 172.21.15.102:0/3707364208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c015eac0 msgr2=0x7f33c015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c015eac0 0x7f33c015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 -- 172.21.15.102:0/3707364208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c015da70 con 0x7f33c0155110 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33c4a00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c00c4040 0x7f33c01f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.400+0000 7f33bfe00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c0155110 0x7f33c015ce40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f33a800ec10 tx=0x7f33a800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.401+0000 7f33bcc00640 1 -- 172.21.15.102:0/3707364208 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8017070 con 0x7f33c0155110 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.401+0000 7f33bcc00640 1 -- 172.21.15.102:0/3707364208 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a800cc30 con 0x7f33c0155110 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.401+0000 7f33bcc00640 1 -- 172.21.15.102:0/3707364208 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8013470 con 0x7f33c0155110 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.401+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c01f0a80 con 0x7f33c0155110 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.401+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c015cbd0 con 0x7f33c0155110 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.402+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c004fd60 con 0x7f33c0155110 2024-04-20T21:42:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.403+0000 7f33bcc00640 1 -- 172.21.15.102:0/3707364208 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33a801a070 con 0x7f33c0155110 2024-04-20T21:42:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.404+0000 7f33bcc00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f339c06c8f0 0x7f339c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.404+0000 7f33bcc00640 1 -- 172.21.15.102:0/3707364208 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 7939+0+0 (secure 0 0 0) 0x7f33a808c6e0 con 0x7f33c0155110 2024-04-20T21:42:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.404+0000 7f33c4a00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f339c06c8f0 0x7f339c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.404+0000 7f33c4a00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f339c06c8f0 0x7f339c06edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005c00 tx=0x7f33b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.409+0000 7f33bcc00640 1 -- 172.21.15.102:0/3707364208 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f33a8059b60 con 0x7f33c0155110 2024-04-20T21:42:42.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.653+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c01589b0 con 0x7f33c0155110 2024-04-20T21:42:42.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.654+0000 7f33bcc00640 1 -- 172.21.15.102:0/3707364208 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+6034 (secure 0 0 0) 0x7f33a805de60 con 0x7f33c0155110 2024-04-20T21:42:42.653 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:42.653 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:36.539799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6835/2459661167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2459661167},{"type":"v1","addr":"172.21.15.102:6835","nonce":2459661167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":1}]} 2024-04-20T21:42:42.653 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 88 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f339c06c8f0 msgr2=0x7f339c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f339c06c8f0 0x7f339c06edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005c00 tx=0x7f33b0009040 comp rx=0 tx=0).stop 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c0155110 msgr2=0x7f33c015ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c0155110 0x7f33c015ce40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f33a800ec10 tx=0x7f33a800c520 comp rx=0 tx=0).stop 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 shutdown_connections 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c015eac0 0x7f33c015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33c0155110 0x7f33c015ce40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f339c06c8f0 0x7f339c06edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 --2- 172.21.15.102:0/3707364208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33c00c4040 0x7f33c01f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 >> 172.21.15.102:0/3707364208 conn(0x7f33c00cc820 msgr2=0x7f33c0152930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 shutdown_connections 2024-04-20T21:42:42.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:42.657+0000 7f33c5e00640 1 -- 172.21.15.102:0/3707364208 wait complete. 2024-04-20T21:42:42.674 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:42:42.864 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~41s 2024-04-20T21:42:44.675 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:44.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.886+0000 7fa332c00640 1 Processor -- start 2024-04-20T21:42:44.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.886+0000 7fa332c00640 1 -- start start 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.887+0000 7fa332c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa32c153180 0x7fa32c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa332c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa32c153560 con 0x7fa32c159380 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa332c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa32c0c7a70 con 0x7fa32c0c75a0 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa332c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa32c0c7d30 con 0x7fa32c153180 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa330e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa32c159380 0x7fa32c0c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48890/0 (socket says 172.21.15.102:48890) 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa330e00640 1 -- 172.21.15.102:0/971352692 learned_addr learned my addr 172.21.15.102:0/971352692 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa332200640 1 --1- 172.21.15.102:0/971352692 >> v1:172.21.15.193:6789/0 conn(0x7fa32c0c75a0 0x7fa32c1637c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:43706/0 (socket says 172.21.15.102:43706) 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa331800640 1 --2- 172.21.15.102:0/971352692 >> v2:172.21.15.102:3300/0 conn(0x7fa32c153180 0x7fa32c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa32be00640 1 -- 172.21.15.102:0/971352692 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605437979 0 0) 0x7fa32c0c7a70 con 0x7fa32c0c75a0 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.888+0000 7fa32be00640 1 -- 172.21.15.102:0/971352692 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa314003520 con 0x7fa32c0c75a0 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.889+0000 7fa331800640 1 -- 172.21.15.102:0/971352692 >> v1:172.21.15.193:6789/0 conn(0x7fa32c0c75a0 legacy=0x7fa32c1637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.889+0000 7fa331800640 1 -- 172.21.15.102:0/971352692 >> v1:172.21.15.102:6789/0 conn(0x7fa32c159380 legacy=0x7fa32c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.889+0000 7fa331800640 1 -- 172.21.15.102:0/971352692 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c15c4d0 con 0x7fa32c153180 2024-04-20T21:42:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.890+0000 7fa331800640 1 --2- 172.21.15.102:0/971352692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c153180 0x7fa32c158e40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097a0 tx=0x7fa32002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec8f20a59f3106b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.890+0000 7fa32be00640 1 -- 172.21.15.102:0/971352692 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320034070 con 0x7fa32c153180 2024-04-20T21:42:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.890+0000 7fa32be00640 1 -- 172.21.15.102:0/971352692 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa320030610 con 0x7fa32c153180 2024-04-20T21:42:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.890+0000 7fa32be00640 1 -- 172.21.15.102:0/971352692 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200309c0 con 0x7fa32c153180 2024-04-20T21:42:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.891+0000 7fa332c00640 1 -- 172.21.15.102:0/971352692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c153180 msgr2=0x7fa32c158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.891+0000 7fa332c00640 1 --2- 172.21.15.102:0/971352692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c153180 0x7fa32c158e40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097a0 tx=0x7fa32002b960 comp rx=0 tx=0).stop 2024-04-20T21:42:44.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.891+0000 7fa332c00640 1 -- 172.21.15.102:0/971352692 shutdown_connections 2024-04-20T21:42:44.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.891+0000 7fa332c00640 1 --2- 172.21.15.102:0/971352692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c153180 0x7fa32c158e40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.891+0000 7fa332c00640 1 -- 172.21.15.102:0/971352692 >> 172.21.15.102:0/971352692 conn(0x7fa32c0cc820 msgr2=0x7fa32c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.891+0000 7fa332c00640 1 -- 172.21.15.102:0/971352692 shutdown_connections 2024-04-20T21:42:44.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.891+0000 7fa332c00640 1 -- 172.21.15.102:0/971352692 wait complete. 2024-04-20T21:42:44.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.893+0000 7fa332c00640 1 Processor -- start 2024-04-20T21:42:44.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.893+0000 7fa332c00640 1 -- start start 2024-04-20T21:42:44.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa332c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c0c75a0 0x7fa32c1e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa332c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa32c153180 0x7fa32c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa332c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa32c159380 0x7fa32c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa332c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c15d290 con 0x7fa32c0c75a0 2024-04-20T21:42:44.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa332c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c15ca90 con 0x7fa32c159380 2024-04-20T21:42:44.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa332c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa32c0c6be0 con 0x7fa32c153180 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa331800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c0c75a0 0x7fa32c1e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa331800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c0c75a0 0x7fa32c1e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32970/0 (socket says 172.21.15.102:32970) 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa331800640 1 -- 172.21.15.102:0/2679242804 learned_addr learned my addr 172.21.15.102:0/2679242804 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa330e00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa32c153180 0x7fa32c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa332200640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa32c159380 0x7fa32c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa331800640 1 -- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa32c153180 msgr2=0x7fa32c0caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa331800640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa32c153180 0x7fa32c0caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa331800640 1 -- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa32c159380 msgr2=0x7fa32c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa331800640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa32c159380 0x7fa32c0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.894+0000 7fa331800640 1 -- 172.21.15.102:0/2679242804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c15a000 con 0x7fa32c0c75a0 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.895+0000 7fa331800640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c0c75a0 0x7fa32c1e5ac0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097a0 tx=0x7fa3200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.895+0000 7fa329600640 1 -- 172.21.15.102:0/2679242804 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320047070 con 0x7fa32c0c75a0 2024-04-20T21:42:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.895+0000 7fa329600640 1 -- 172.21.15.102:0/2679242804 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa320030410 con 0x7fa32c0c75a0 2024-04-20T21:42:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.895+0000 7fa329600640 1 -- 172.21.15.102:0/2679242804 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320034040 con 0x7fa32c0c75a0 2024-04-20T21:42:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.895+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32c15a2a0 con 0x7fa32c0c75a0 2024-04-20T21:42:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.895+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32c15a840 con 0x7fa32c0c75a0 2024-04-20T21:42:44.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.897+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32c04fd60 con 0x7fa32c0c75a0 2024-04-20T21:42:44.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.897+0000 7fa329600640 1 -- 172.21.15.102:0/2679242804 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa32004a070 con 0x7fa32c0c75a0 2024-04-20T21:42:44.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.898+0000 7fa329600640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa30c06c8f0 0x7fa30c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.898+0000 7fa329600640 1 -- 172.21.15.102:0/2679242804 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 7939+0+0 (secure 0 0 0) 0x7fa3200add40 con 0x7fa32c0c75a0 2024-04-20T21:42:44.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.899+0000 7fa330e00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa30c06c8f0 0x7fa30c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.899+0000 7fa330e00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa30c06c8f0 0x7fa30c06edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa32c1590b0 tx=0x7fa318007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:44.903+0000 7fa329600640 1 -- 172.21.15.102:0/2679242804 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa32007b130 con 0x7fa32c0c75a0 2024-04-20T21:42:45.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.135+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa32c0c69c0 con 0x7fa32c0c75a0 2024-04-20T21:42:45.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.136+0000 7fa329600640 1 -- 172.21.15.102:0/2679242804 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+6034 (secure 0 0 0) 0x7fa32007f430 con 0x7fa32c0c75a0 2024-04-20T21:42:45.135 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:45.135 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:36.539799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6835/2459661167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2459661167},{"type":"v1","addr":"172.21.15.102:6835","nonce":2459661167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":1}]} 2024-04-20T21:42:45.136 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 88 2024-04-20T21:42:45.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa30c06c8f0 msgr2=0x7fa30c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa30c06c8f0 0x7fa30c06edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa32c1590b0 tx=0x7fa318007e80 comp rx=0 tx=0).stop 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c0c75a0 msgr2=0x7fa32c1e5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c0c75a0 0x7fa32c1e5ac0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097a0 tx=0x7fa3200046c0 comp rx=0 tx=0).stop 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 shutdown_connections 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa30c06c8f0 0x7fa30c06edb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa32c159380 0x7fa32c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa32c153180 0x7fa32c0caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 --2- 172.21.15.102:0/2679242804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32c0c75a0 0x7fa32c1e5ac0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 >> 172.21.15.102:0/2679242804 conn(0x7fa32c0cc820 msgr2=0x7fa32c148c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 shutdown_connections 2024-04-20T21:42:45.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:45.139+0000 7fa332c00640 1 -- 172.21.15.102:0/2679242804 wait complete. 2024-04-20T21:42:45.156 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:42:47.156 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:47.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.362+0000 7f4403000640 1 Processor -- start 2024-04-20T21:42:47.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.362+0000 7f4403000640 1 -- start start 2024-04-20T21:42:47.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.363+0000 7f4403000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f43fc152290 0x7f43fc0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:47.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.363+0000 7f4403000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f43fc0c3700 0x7f43fc06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:47.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.363+0000 7f4403000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43fc15c220 con 0x7f43fc06eb70 2024-04-20T21:42:47.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.363+0000 7f4403000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f43fc0c9180 con 0x7f43fc152290 2024-04-20T21:42:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.363+0000 7f4403000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f43fc0c4840 con 0x7f43fc0c3700 2024-04-20T21:42:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.363+0000 7f4401c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f43fc152290 0x7f43fc0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.364+0000 7f4401c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f43fc152290 0x7f43fc0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32980/0 (socket says 172.21.15.102:32980) 2024-04-20T21:42:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.364+0000 7f4401c00640 1 -- 172.21.15.102:0/2789648832 learned_addr learned my addr 172.21.15.102:0/2789648832 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.364+0000 7f4401200640 1 --2- 172.21.15.102:0/2789648832 >> v2:172.21.15.193:3300/0 conn(0x7f43fc0c3700 0x7f43fc06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.364+0000 7f4401c00640 1 -- 172.21.15.102:0/2789648832 >> v2:172.21.15.193:3300/0 conn(0x7f43fc0c3700 msgr2=0x7f43fc06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:47.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.364+0000 7f4401c00640 1 --2- 172.21.15.102:0/2789648832 >> v2:172.21.15.193:3300/0 conn(0x7f43fc0c3700 0x7f43fc06e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.364+0000 7f4401c00640 1 -- 172.21.15.102:0/2789648832 >> v1:172.21.15.193:6789/0 conn(0x7f43fc06eb70 legacy=0x7f43fc0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.364+0000 7f4401c00640 1 -- 172.21.15.102:0/2789648832 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc0c4560 con 0x7f43fc152290 2024-04-20T21:42:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.365+0000 7f4401c00640 1 --2- 172.21.15.102:0/2789648832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc0cd310 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f43fc069910 tx=0x7f43ec02ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977d76407dc99714 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.365+0000 7f43fbe00640 1 -- 172.21.15.102:0/2789648832 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec02f860 con 0x7f43fc152290 2024-04-20T21:42:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.365+0000 7f43fbe00640 1 -- 172.21.15.102:0/2789648832 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ec02fa00 con 0x7f43fc152290 2024-04-20T21:42:47.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.365+0000 7f43fbe00640 1 -- 172.21.15.102:0/2789648832 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec02fdd0 con 0x7f43fc152290 2024-04-20T21:42:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.366+0000 7f4403000640 1 -- 172.21.15.102:0/2789648832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 msgr2=0x7f43fc0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.366+0000 7f4403000640 1 --2- 172.21.15.102:0/2789648832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc0cd310 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f43fc069910 tx=0x7f43ec02ba40 comp rx=0 tx=0).stop 2024-04-20T21:42:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.366+0000 7f4403000640 1 -- 172.21.15.102:0/2789648832 shutdown_connections 2024-04-20T21:42:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.366+0000 7f4403000640 1 --2- 172.21.15.102:0/2789648832 >> v2:172.21.15.193:3300/0 conn(0x7f43fc0c3700 0x7f43fc06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.366+0000 7f4403000640 1 --2- 172.21.15.102:0/2789648832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc0cd310 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.366+0000 7f4403000640 1 -- 172.21.15.102:0/2789648832 >> 172.21.15.102:0/2789648832 conn(0x7f43fc0cc820 msgr2=0x7f43fc0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:47.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.366+0000 7f4403000640 1 -- 172.21.15.102:0/2789648832 shutdown_connections 2024-04-20T21:42:47.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.366+0000 7f4403000640 1 -- 172.21.15.102:0/2789648832 wait complete. 2024-04-20T21:42:47.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.368+0000 7f4403000640 1 Processor -- start 2024-04-20T21:42:47.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.368+0000 7f4403000640 1 -- start start 2024-04-20T21:42:47.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4403000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc06eb70 0x7f43fc072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:47.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4403000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0c3700 0x7f43fc150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:47.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4403000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:47.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4403000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc15ba20 con 0x7f43fc152290 2024-04-20T21:42:47.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4403000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc15b740 con 0x7f43fc0c3700 2024-04-20T21:42:47.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4403000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f43fc15bd00 con 0x7f43fc06eb70 2024-04-20T21:42:47.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4402600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:47.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4402600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc1f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32984/0 (socket says 172.21.15.102:32984) 2024-04-20T21:42:47.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4401200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0c3700 0x7f43fc150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:47.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4402600640 1 -- 172.21.15.102:0/2009794750 learned_addr learned my addr 172.21.15.102:0/2009794750 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:47.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.369+0000 7f4401c00640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc06eb70 0x7f43fc072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f4402600640 1 -- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc06eb70 msgr2=0x7f43fc072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f4402600640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc06eb70 0x7f43fc072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f4402600640 1 -- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0c3700 msgr2=0x7f43fc150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f4402600640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0c3700 0x7f43fc150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f4402600640 1 -- 172.21.15.102:0/2009794750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc143410 con 0x7f43fc152290 2024-04-20T21:42:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f4402600640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc1f4c60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c3c0 tx=0x7f43f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f43f9600640 1 -- 172.21.15.102:0/2009794750 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f40052d0 con 0x7f43fc152290 2024-04-20T21:42:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f43f9600640 1 -- 172.21.15.102:0/2009794750 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f4005470 con 0x7f43fc152290 2024-04-20T21:42:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f43f9600640 1 -- 172.21.15.102:0/2009794750 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4007240 con 0x7f43fc152290 2024-04-20T21:42:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.370+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc1519e0 con 0x7f43fc152290 2024-04-20T21:42:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.371+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc1592f0 con 0x7f43fc152290 2024-04-20T21:42:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.372+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43fc04fd60 con 0x7f43fc152290 2024-04-20T21:42:47.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.373+0000 7f43f9600640 1 -- 172.21.15.102:0/2009794750 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43f40073e0 con 0x7f43fc152290 2024-04-20T21:42:47.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.374+0000 7f43f9600640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f43dc06c920 0x7f43dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:47.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.374+0000 7f43f9600640 1 -- 172.21.15.102:0/2009794750 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 7939+0+0 (secure 0 0 0) 0x7f43f408b6e0 con 0x7f43fc152290 2024-04-20T21:42:47.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.374+0000 7f4401c00640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f43dc06c920 0x7f43dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:47.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.375+0000 7f4401c00640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f43dc06c920 0x7f43dc06ede0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f43ec036000 tx=0x7f43ec008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:47.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.378+0000 7f43f9600640 1 -- 172.21.15.102:0/2009794750 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f43f4058bf0 con 0x7f43fc152290 2024-04-20T21:42:47.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.620+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43fc06dcb0 con 0x7f43fc152290 2024-04-20T21:42:47.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.621+0000 7f43f9600640 1 -- 172.21.15.102:0/2009794750 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+6034 (secure 0 0 0) 0x7f43f405cef0 con 0x7f43fc152290 2024-04-20T21:42:47.621 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:47.621 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:36.539799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6835/2459661167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2459661167},{"type":"v1","addr":"172.21.15.102:6835","nonce":2459661167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":1}]} 2024-04-20T21:42:47.621 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 88 2024-04-20T21:42:47.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.623+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f43dc06c920 msgr2=0x7f43dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:47.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.623+0000 7f4403000640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f43dc06c920 0x7f43dc06ede0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f43ec036000 tx=0x7f43ec008030 comp rx=0 tx=0).stop 2024-04-20T21:42:47.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.623+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 msgr2=0x7f43fc1f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:47.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.623+0000 7f4403000640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc1f4c60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c3c0 tx=0x7f43f400c880 comp rx=0 tx=0).stop 2024-04-20T21:42:47.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.624+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 shutdown_connections 2024-04-20T21:42:47.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.624+0000 7f4403000640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43fc152290 0x7f43fc1f4c60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.624+0000 7f4403000640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f43dc06c920 0x7f43dc06ede0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.624+0000 7f4403000640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0c3700 0x7f43fc150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.624+0000 7f4403000640 1 --2- 172.21.15.102:0/2009794750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc06eb70 0x7f43fc072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:47.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.624+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 >> 172.21.15.102:0/2009794750 conn(0x7f43fc0cc820 msgr2=0x7f43fc0c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:47.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.624+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 shutdown_connections 2024-04-20T21:42:47.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:47.624+0000 7f4403000640 1 -- 172.21.15.102:0/2009794750 wait complete. 2024-04-20T21:42:47.642 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:42:49.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~48s 2024-04-20T21:42:49.643 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:49.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.849+0000 7f0f68c00640 1 Processor -- start 2024-04-20T21:42:49.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.849+0000 7f0f68c00640 1 -- start start 2024-04-20T21:42:49.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.851+0000 7f0f68c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64163660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:49.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.851+0000 7f0f68c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f6415c220 con 0x7f0f640c3700 2024-04-20T21:42:49.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.851+0000 7f0f68c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f640c4560 con 0x7f0f64152290 2024-04-20T21:42:49.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.851+0000 7f0f68c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0f640c9180 con 0x7f0f640cd310 2024-04-20T21:42:49.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.851+0000 7f0f62a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0f640c3700 0x7f0f640c8330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57978/0 (socket says 172.21.15.102:57978) 2024-04-20T21:42:49.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.851+0000 7f0f62a00640 1 -- 172.21.15.102:0/2304451114 learned_addr learned my addr 172.21.15.102:0/2304451114 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:42:49.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.851+0000 7f0f63e00640 1 --2- 172.21.15.102:0/2304451114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:49.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.851+0000 7f0f62000640 1 -- 172.21.15.102:0/2304451114 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569099292 0 0) 0x7f0f6415c220 con 0x7f0f640c3700 2024-04-20T21:42:49.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.852+0000 7f0f62000640 1 -- 172.21.15.102:0/2304451114 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f48005730 con 0x7f0f640c3700 2024-04-20T21:42:49.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.852+0000 7f0f62000640 1 -- 172.21.15.102:0/2304451114 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292317837 0 0) 0x7f0f640c4560 con 0x7f0f64152290 2024-04-20T21:42:49.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.852+0000 7f0f63e00640 1 -- 172.21.15.102:0/2304451114 >> v1:172.21.15.193:6789/0 conn(0x7f0f64152290 legacy=0x7f0f6406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:49.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.852+0000 7f0f63e00640 1 -- 172.21.15.102:0/2304451114 >> v1:172.21.15.102:6789/0 conn(0x7f0f640c3700 legacy=0x7f0f640c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:49.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.852+0000 7f0f63e00640 1 -- 172.21.15.102:0/2304451114 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6415b460 con 0x7f0f640cd310 2024-04-20T21:42:49.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.853+0000 7f0f63e00640 1 --2- 172.21.15.102:0/2304451114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64163660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009dc0 tx=0x7f0f5c02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a02c4894f5f3a9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.853+0000 7f0f62000640 1 -- 172.21.15.102:0/2304451114 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c039070 con 0x7f0f640cd310 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.853+0000 7f0f62000640 1 -- 172.21.15.102:0/2304451114 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5c02cd50 con 0x7f0f640cd310 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.853+0000 7f0f62000640 1 -- 172.21.15.102:0/2304451114 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c034640 con 0x7f0f640cd310 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.853+0000 7f0f68c00640 1 -- 172.21.15.102:0/2304451114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 msgr2=0x7f0f64163660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.853+0000 7f0f68c00640 1 --2- 172.21.15.102:0/2304451114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64163660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009dc0 tx=0x7f0f5c02bfa0 comp rx=0 tx=0).stop 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.854+0000 7f0f68c00640 1 -- 172.21.15.102:0/2304451114 shutdown_connections 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.854+0000 7f0f68c00640 1 --2- 172.21.15.102:0/2304451114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64163660 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.854+0000 7f0f68c00640 1 -- 172.21.15.102:0/2304451114 >> 172.21.15.102:0/2304451114 conn(0x7f0f640cc820 msgr2=0x7f0f640ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.854+0000 7f0f68c00640 1 -- 172.21.15.102:0/2304451114 shutdown_connections 2024-04-20T21:42:49.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.854+0000 7f0f68c00640 1 -- 172.21.15.102:0/2304451114 wait complete. 2024-04-20T21:42:49.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.855+0000 7f0f68c00640 1 Processor -- start 2024-04-20T21:42:49.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.855+0000 7f0f68c00640 1 -- start start 2024-04-20T21:42:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.856+0000 7f0f68c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f640c3700 0x7f0f64150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.856+0000 7f0f68c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.856+0000 7f0f68c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f64152290 0x7f0f640ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.856+0000 7f0f68c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640cd760 con 0x7f0f640c3700 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.856+0000 7f0f68c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640c4840 con 0x7f0f64152290 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.856+0000 7f0f68c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6415ba20 con 0x7f0f640cd310 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f640c3700 0x7f0f64150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f64152290 0x7f0f640ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f62a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f64152290 0x7f0f640ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:41184/0 (socket says 172.21.15.102:41184) 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63e00640 1 -- 172.21.15.102:0/4073210039 learned_addr learned my addr 172.21.15.102:0/4073210039 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63400640 1 -- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 msgr2=0x7f0f64144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63400640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63400640 1 -- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f64152290 msgr2=0x7f0f640ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63400640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f64152290 0x7f0f640ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63400640 1 -- 172.21.15.102:0/4073210039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f64144a90 con 0x7f0f640c3700 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63e00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f64152290 0x7f0f640ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f62a00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.857+0000 7f0f63400640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f640c3700 0x7f0f64150860 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c440 tx=0x7f0f5400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.858+0000 7f0f5b400640 1 -- 172.21.15.102:0/4073210039 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54017070 con 0x7f0f640c3700 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.858+0000 7f0f5b400640 1 -- 172.21.15.102:0/4073210039 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f540035f0 con 0x7f0f640c3700 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.858+0000 7f0f5b400640 1 -- 172.21.15.102:0/4073210039 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540039e0 con 0x7f0f640c3700 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.858+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f64144d50 con 0x7f0f640c3700 2024-04-20T21:42:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.858+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f64145310 con 0x7f0f640c3700 2024-04-20T21:42:49.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.859+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6404fd60 con 0x7f0f640c3700 2024-04-20T21:42:49.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.860+0000 7f0f5b400640 1 -- 172.21.15.102:0/4073210039 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f5401a070 con 0x7f0f640c3700 2024-04-20T21:42:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.861+0000 7f0f5b400640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0f4406c8f0 0x7f0f4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.861+0000 7f0f5b400640 1 -- 172.21.15.102:0/4073210039 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 7939+0+0 (secure 0 0 0) 0x7f0f5408b880 con 0x7f0f640c3700 2024-04-20T21:42:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.861+0000 7f0f62a00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0f4406c8f0 0x7f0f4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.862+0000 7f0f62a00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0f4406c8f0 0x7f0f4406edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0f640735c0 tx=0x7f0f50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:49.865+0000 7f0f5b400640 1 -- 172.21.15.102:0/4073210039 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0f54058cb0 con 0x7f0f640c3700 2024-04-20T21:42:50.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.088+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f6406dcb0 con 0x7f0f640c3700 2024-04-20T21:42:50.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.089+0000 7f0f5b400640 1 -- 172.21.15.102:0/4073210039 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+6034 (secure 0 0 0) 0x7f0f5405cfb0 con 0x7f0f640c3700 2024-04-20T21:42:50.088 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:50.088 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:36.539799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6835/2459661167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2459661167},{"type":"v1","addr":"172.21.15.102:6835","nonce":2459661167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":1}]} 2024-04-20T21:42:50.088 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 88 2024-04-20T21:42:50.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0f4406c8f0 msgr2=0x7f0f4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:50.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0f4406c8f0 0x7f0f4406edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0f640735c0 tx=0x7f0f50009040 comp rx=0 tx=0).stop 2024-04-20T21:42:50.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f640c3700 msgr2=0x7f0f64150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:50.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f640c3700 0x7f0f64150860 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c440 tx=0x7f0f5400c840 comp rx=0 tx=0).stop 2024-04-20T21:42:50.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 shutdown_connections 2024-04-20T21:42:50.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f0f4406c8f0 0x7f0f4406edb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:50.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f64152290 0x7f0f640ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:50.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0f640cd310 0x7f0f64144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:50.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 --2- 172.21.15.102:0/4073210039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f640c3700 0x7f0f64150860 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:50.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 >> 172.21.15.102:0/4073210039 conn(0x7f0f640cc820 msgr2=0x7f0f6406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:50.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 shutdown_connections 2024-04-20T21:42:50.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:50.091+0000 7f0f68c00640 1 -- 172.21.15.102:0/4073210039 wait complete. 2024-04-20T21:42:50.108 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:42:52.109 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:52.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.320+0000 7f4323000640 1 Processor -- start 2024-04-20T21:42:52.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.320+0000 7f4323000640 1 -- start start 2024-04-20T21:42:52.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f4323000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f431c14fa50 0x7f431c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:52.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f4323000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f431c0c9eb0 0x7f431c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:52.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f4323000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c073a10 0x7f431c0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:52.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f4323000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f431c159500 con 0x7f431c14fa50 2024-04-20T21:42:52.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f4323000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f431c1536d0 con 0x7f431c0c9eb0 2024-04-20T21:42:52.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f4323000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f431c0c76c0 con 0x7f431c073a10 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f431be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f431c14fa50 0x7f431c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f4321000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c073a10 0x7f431c0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f431be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f431c14fa50 0x7f431c0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36390/0 (socket says 172.21.15.102:36390) 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f4321000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c073a10 0x7f431c0c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:47004/0 (socket says 172.21.15.102:47004) 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f431be00640 1 -- 172.21.15.102:0/3562126475 learned_addr learned my addr 172.21.15.102:0/3562126475 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.322+0000 7f431b400640 1 --2- 172.21.15.102:0/3562126475 >> v2:172.21.15.193:3300/0 conn(0x7f431c0c9eb0 0x7f431c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.323+0000 7f431be00640 1 -- 172.21.15.102:0/3562126475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c073a10 msgr2=0x7f431c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.323+0000 7f431be00640 1 --2- 172.21.15.102:0/3562126475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c073a10 0x7f431c0c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.323+0000 7f431be00640 1 -- 172.21.15.102:0/3562126475 >> v2:172.21.15.193:3300/0 conn(0x7f431c0c9eb0 msgr2=0x7f431c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.323+0000 7f431be00640 1 --2- 172.21.15.102:0/3562126475 >> v2:172.21.15.193:3300/0 conn(0x7f431c0c9eb0 0x7f431c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.323+0000 7f431be00640 1 -- 172.21.15.102:0/3562126475 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c0c7960 con 0x7f431c14fa50 2024-04-20T21:42:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.323+0000 7f4321000640 1 --2- 172.21.15.102:0/3562126475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c073a10 0x7f431c0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.323+0000 7f431b400640 1 --2- 172.21.15.102:0/3562126475 >> v2:172.21.15.193:3300/0 conn(0x7f431c0c9eb0 0x7f431c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.324+0000 7f431be00640 1 --2- 172.21.15.102:0/3562126475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c14fa50 0x7f431c0c4830 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f431c0c7b30 tx=0x7f430c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e7ae3ed9245b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.324+0000 7f431aa00640 1 -- 172.21.15.102:0/3562126475 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c034070 con 0x7f431c14fa50 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.324+0000 7f431aa00640 1 -- 172.21.15.102:0/3562126475 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c030850 con 0x7f431c14fa50 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.324+0000 7f431aa00640 1 -- 172.21.15.102:0/3562126475 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c030c20 con 0x7f431c14fa50 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 -- 172.21.15.102:0/3562126475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c14fa50 msgr2=0x7f431c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 --2- 172.21.15.102:0/3562126475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c14fa50 0x7f431c0c4830 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f431c0c7b30 tx=0x7f430c02b990 comp rx=0 tx=0).stop 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 -- 172.21.15.102:0/3562126475 shutdown_connections 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 --2- 172.21.15.102:0/3562126475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c073a10 0x7f431c0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 --2- 172.21.15.102:0/3562126475 >> v2:172.21.15.193:3300/0 conn(0x7f431c0c9eb0 0x7f431c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 --2- 172.21.15.102:0/3562126475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c14fa50 0x7f431c0c4830 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 -- 172.21.15.102:0/3562126475 >> 172.21.15.102:0/3562126475 conn(0x7f431c0c9390 msgr2=0x7f431c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:52.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 -- 172.21.15.102:0/3562126475 shutdown_connections 2024-04-20T21:42:52.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.325+0000 7f4323000640 1 -- 172.21.15.102:0/3562126475 wait complete. 2024-04-20T21:42:52.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.327+0000 7f4323000640 1 Processor -- start 2024-04-20T21:42:52.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.328+0000 7f4323000640 1 -- start start 2024-04-20T21:42:52.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f4323000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f431c073a10 0x7f431c0cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f4323000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c0c9eb0 0x7f431c1edba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f4323000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c14fa50 0x7f431c14cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f4323000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f431c0c7c00 con 0x7f431c0c9eb0 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f4323000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f431c0c3d90 con 0x7f431c073a10 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f4323000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f431c0c4030 con 0x7f431c14fa50 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f431b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c0c9eb0 0x7f431c1edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f431b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c0c9eb0 0x7f431c1edba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36400/0 (socket says 172.21.15.102:36400) 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f431b400640 1 -- 172.21.15.102:0/1615225152 learned_addr learned my addr 172.21.15.102:0/1615225152 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f431be00640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f431c073a10 0x7f431c0cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.329+0000 7f4321000640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c14fa50 0x7f431c14cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f431b400640 1 -- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c14fa50 msgr2=0x7f431c14cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:52.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f431b400640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c14fa50 0x7f431c14cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f431b400640 1 -- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f431c073a10 msgr2=0x7f431c0cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:52.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f431b400640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f431c073a10 0x7f431c0cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f431b400640 1 -- 172.21.15.102:0/1615225152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c1f1580 con 0x7f431c0c9eb0 2024-04-20T21:42:52.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f431be00640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f431c073a10 0x7f431c0cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:52.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f431b400640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c0c9eb0 0x7f431c1edba0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f430800ebe0 tx=0x7f430800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:52.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f4313e00640 1 -- 172.21.15.102:0/1615225152 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308017070 con 0x7f431c0c9eb0 2024-04-20T21:42:52.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f4313e00640 1 -- 172.21.15.102:0/1615225152 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430800cc30 con 0x7f431c0c9eb0 2024-04-20T21:42:52.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c0cb650 con 0x7f431c0c9eb0 2024-04-20T21:42:52.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f4313e00640 1 -- 172.21.15.102:0/1615225152 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308013470 con 0x7f431c0c9eb0 2024-04-20T21:42:52.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.330+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1f0aa0 con 0x7f431c0c9eb0 2024-04-20T21:42:52.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.331+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c074340 con 0x7f431c0c9eb0 2024-04-20T21:42:52.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.332+0000 7f4313e00640 1 -- 172.21.15.102:0/1615225152 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f430801a070 con 0x7f431c0c9eb0 2024-04-20T21:42:52.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.333+0000 7f4313e00640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f42fc06c8f0 0x7f42fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:52.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.334+0000 7f4313e00640 1 -- 172.21.15.102:0/1615225152 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 7939+0+0 (secure 0 0 0) 0x7f430808c700 con 0x7f431c0c9eb0 2024-04-20T21:42:52.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.334+0000 7f431be00640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f42fc06c8f0 0x7f42fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:52.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.334+0000 7f431be00640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f42fc06c8f0 0x7f42fc06edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f430c036000 tx=0x7f430c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:52.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.339+0000 7f4313e00640 1 -- 172.21.15.102:0/1615225152 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4308059b80 con 0x7f431c0c9eb0 2024-04-20T21:42:52.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.579+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f431c15d450 con 0x7f431c0c9eb0 2024-04-20T21:42:52.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.580+0000 7f4313e00640 1 -- 172.21.15.102:0/1615225152 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+6034 (secure 0 0 0) 0x7f430805de80 con 0x7f431c0c9eb0 2024-04-20T21:42:52.579 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:52.579 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:36.539799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6835/2459661167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2459661167},{"type":"v1","addr":"172.21.15.102:6835","nonce":2459661167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":1}]} 2024-04-20T21:42:52.579 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 88 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f42fc06c8f0 msgr2=0x7f42fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f42fc06c8f0 0x7f42fc06edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f430c036000 tx=0x7f430c03b040 comp rx=0 tx=0).stop 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c0c9eb0 msgr2=0x7f431c1edba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c0c9eb0 0x7f431c1edba0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f430800ebe0 tx=0x7f430800c520 comp rx=0 tx=0).stop 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 shutdown_connections 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f431c14fa50 0x7f431c14cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f431c0c9eb0 0x7f431c1edba0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f42fc06c8f0 0x7f42fc06edb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 --2- 172.21.15.102:0/1615225152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f431c073a10 0x7f431c0cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:52.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 >> 172.21.15.102:0/1615225152 conn(0x7f431c0c9390 msgr2=0x7f431c14be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:52.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 shutdown_connections 2024-04-20T21:42:52.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:52.583+0000 7f4323000640 1 -- 172.21.15.102:0/1615225152 wait complete. 2024-04-20T21:42:52.601 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:42:54.602 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:54.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.805+0000 7f7e6e000640 1 Processor -- start 2024-04-20T21:42:54.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.805+0000 7f7e6e000640 1 -- start start 2024-04-20T21:42:54.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e6e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7e6814fa50 0x7f7e680c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:54.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e6e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e68073a10 0x7f7e68158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:54.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e6e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e681536d0 con 0x7f7e680c9eb0 2024-04-20T21:42:54.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e6e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f7e68159500 con 0x7f7e6814fa50 2024-04-20T21:42:54.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e6e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7e680c76c0 con 0x7f7e68073a10 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e67400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7e6814fa50 0x7f7e680c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e66a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7e680c9eb0 0x7f7e680c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57994/0 (socket says 172.21.15.102:57994) 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e66a00640 1 -- 172.21.15.102:0/1379753449 learned_addr learned my addr 172.21.15.102:0/1379753449 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.807+0000 7f7e67e00640 1 --2- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e68073a10 0x7f7e68158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.808+0000 7f7e67400640 1 -- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e68073a10 msgr2=0x7f7e68158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.808+0000 7f7e67400640 1 --2- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e68073a10 0x7f7e68158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.808+0000 7f7e67400640 1 -- 172.21.15.102:0/1379753449 >> v1:172.21.15.102:6789/0 conn(0x7f7e680c9eb0 legacy=0x7f7e680c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.808+0000 7f7e66000640 1 -- 172.21.15.102:0/1379753449 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416571112 0 0) 0x7f7e54001110 con 0x7f7e680c9eb0 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.808+0000 7f7e67400640 1 -- 172.21.15.102:0/1379753449 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e680c7960 con 0x7f7e6814fa50 2024-04-20T21:42:54.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.808+0000 7f7e67e00640 1 --2- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e68073a10 0x7f7e68158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:54.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.809+0000 7f7e67400640 1 --2- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 0x7f7e680c4830 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7e580097a0 tx=0x7f7e5802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e9ed4cde0ec29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:54.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.809+0000 7f7e66000640 1 -- 172.21.15.102:0/1379753449 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58034070 con 0x7f7e6814fa50 2024-04-20T21:42:54.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.809+0000 7f7e66000640 1 -- 172.21.15.102:0/1379753449 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e58030800 con 0x7f7e6814fa50 2024-04-20T21:42:54.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.809+0000 7f7e66000640 1 -- 172.21.15.102:0/1379753449 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58030bd0 con 0x7f7e6814fa50 2024-04-20T21:42:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.810+0000 7f7e6e000640 1 -- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 msgr2=0x7f7e680c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.810+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 0x7f7e680c4830 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7e580097a0 tx=0x7f7e5802b940 comp rx=0 tx=0).stop 2024-04-20T21:42:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.810+0000 7f7e6e000640 1 -- 172.21.15.102:0/1379753449 shutdown_connections 2024-04-20T21:42:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.810+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e68073a10 0x7f7e68158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.810+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1379753449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 0x7f7e680c4830 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.810+0000 7f7e6e000640 1 -- 172.21.15.102:0/1379753449 >> 172.21.15.102:0/1379753449 conn(0x7f7e680c9390 msgr2=0x7f7e680c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.810+0000 7f7e6e000640 1 -- 172.21.15.102:0/1379753449 shutdown_connections 2024-04-20T21:42:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.810+0000 7f7e6e000640 1 -- 172.21.15.102:0/1379753449 wait complete. 2024-04-20T21:42:54.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.812+0000 7f7e6e000640 1 Processor -- start 2024-04-20T21:42:54.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.812+0000 7f7e6e000640 1 -- start start 2024-04-20T21:42:54.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e6e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e68073a10 0x7f7e68157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:54.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e6e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e680c9eb0 0x7f7e68147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:54.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e6e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 0x7f7e680cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:54.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e6e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7e680c7c00 con 0x7f7e68073a10 2024-04-20T21:42:54.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e6e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e680c3d90 con 0x7f7e6814fa50 2024-04-20T21:42:54.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e6e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7e680c4030 con 0x7f7e680c9eb0 2024-04-20T21:42:54.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e68073a10 0x7f7e68157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:54.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e68073a10 0x7f7e68157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36402/0 (socket says 172.21.15.102:36402) 2024-04-20T21:42:54.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e67400640 1 -- 172.21.15.102:0/1772318534 learned_addr learned my addr 172.21.15.102:0/1772318534 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:54.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e67e00640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 0x7f7e680cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:54.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.813+0000 7f7e66a00640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e680c9eb0 0x7f7e68147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.814+0000 7f7e67400640 1 -- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e680c9eb0 msgr2=0x7f7e68147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.814+0000 7f7e67400640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e680c9eb0 0x7f7e68147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.814+0000 7f7e67400640 1 -- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 msgr2=0x7f7e680cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.814+0000 7f7e67400640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 0x7f7e680cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.814+0000 7f7e67400640 1 -- 172.21.15.102:0/1772318534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e68148c70 con 0x7f7e68073a10 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.814+0000 7f7e67e00640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 0x7f7e680cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.814+0000 7f7e67400640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e68073a10 0x7f7e68157440 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7e5802bf50 tx=0x7f7e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.814+0000 7f7e5f400640 1 -- 172.21.15.102:0/1772318534 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58034040 con 0x7f7e68073a10 2024-04-20T21:42:54.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.815+0000 7f7e5f400640 1 -- 172.21.15.102:0/1772318534 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e58030600 con 0x7f7e68073a10 2024-04-20T21:42:54.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.815+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e68148f70 con 0x7f7e68073a10 2024-04-20T21:42:54.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.815+0000 7f7e5f400640 1 -- 172.21.15.102:0/1772318534 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58006e10 con 0x7f7e68073a10 2024-04-20T21:42:54.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.815+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e680cc190 con 0x7f7e68073a10 2024-04-20T21:42:54.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.816+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e68069340 con 0x7f7e68073a10 2024-04-20T21:42:54.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.817+0000 7f7e5f400640 1 -- 172.21.15.102:0/1772318534 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e58037070 con 0x7f7e68073a10 2024-04-20T21:42:54.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.818+0000 7f7e5f400640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e4806c8f0 0x7f7e4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:54.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.818+0000 7f7e5f400640 1 -- 172.21.15.102:0/1772318534 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 7939+0+0 (secure 0 0 0) 0x7f7e580ae410 con 0x7f7e68073a10 2024-04-20T21:42:54.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.818+0000 7f7e66a00640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e4806c8f0 0x7f7e4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:54.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.819+0000 7f7e66a00640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e4806c8f0 0x7f7e4806edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e680cb070 tx=0x7f7e54007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:54.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:54.822+0000 7f7e5f400640 1 -- 172.21.15.102:0/1772318534 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7e68069340 con 0x7f7e68073a10 2024-04-20T21:42:55.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.065+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e68074340 con 0x7f7e68073a10 2024-04-20T21:42:55.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.066+0000 7f7e5f400640 1 -- 172.21.15.102:0/1772318534 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+6034 (secure 0 0 0) 0x7f7e68148f70 con 0x7f7e68073a10 2024-04-20T21:42:55.065 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:55.065 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:36.539799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6835/2459661167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2459661167},{"type":"v1","addr":"172.21.15.102:6835","nonce":2459661167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":1}]} 2024-04-20T21:42:55.066 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 88 2024-04-20T21:42:55.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e4806c8f0 msgr2=0x7f7e4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:55.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e4806c8f0 0x7f7e4806edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e680cb070 tx=0x7f7e54007b50 comp rx=0 tx=0).stop 2024-04-20T21:42:55.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e68073a10 msgr2=0x7f7e68157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:55.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e68073a10 0x7f7e68157440 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7e5802bf50 tx=0x7f7e580046c0 comp rx=0 tx=0).stop 2024-04-20T21:42:55.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 shutdown_connections 2024-04-20T21:42:55.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f7e4806c8f0 0x7f7e4806edb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:55.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6814fa50 0x7f7e680cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:55.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e680c9eb0 0x7f7e68147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:55.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 --2- 172.21.15.102:0/1772318534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e68073a10 0x7f7e68157440 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:55.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 >> 172.21.15.102:0/1772318534 conn(0x7f7e680c9390 msgr2=0x7f7e6814d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:55.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 shutdown_connections 2024-04-20T21:42:55.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:55.069+0000 7f7e6e000640 1 -- 172.21.15.102:0/1772318534 wait complete. 2024-04-20T21:42:55.087 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:42:56.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~54s 2024-04-20T21:42:57.088 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:57.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.296+0000 7ff6ad000640 1 Processor -- start 2024-04-20T21:42:57.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.296+0000 7ff6ad000640 1 -- start start 2024-04-20T21:42:57.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.298+0000 7ff6ad000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a80c9eb0 0x7ff6a80c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.298+0000 7ff6ad000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff6a8073a10 0x7ff6a8158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.298+0000 7ff6ad000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a81536d0 con 0x7ff6a814fa50 2024-04-20T21:42:57.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.298+0000 7ff6ad000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a8159500 con 0x7ff6a8073a10 2024-04-20T21:42:57.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.298+0000 7ff6ad000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a80c76c0 con 0x7ff6a80c9eb0 2024-04-20T21:42:57.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.298+0000 7ff6a5e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff6a814fa50 0x7ff6a80c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58010/0 (socket says 172.21.15.102:58010) 2024-04-20T21:42:57.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.298+0000 7ff6a5e00640 1 -- 172.21.15.102:0/1773308669 learned_addr learned my addr 172.21.15.102:0/1773308669 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:42:57.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.298+0000 7ff6a6800640 1 --2- 172.21.15.102:0/1773308669 >> v2:172.21.15.102:3300/0 conn(0x7ff6a8073a10 0x7ff6a8158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.299+0000 7ff6a5400640 1 --2- 172.21.15.102:0/1773308669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a80c9eb0 0x7ff6a80c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.299+0000 7ff6a4a00640 1 -- 172.21.15.102:0/1773308669 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 30914466 0 0) 0x7ff698001490 con 0x7ff6a814fa50 2024-04-20T21:42:57.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.299+0000 7ff6a4a00640 1 -- 172.21.15.102:0/1773308669 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff68c003520 con 0x7ff6a814fa50 2024-04-20T21:42:57.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.299+0000 7ff6a6800640 1 -- 172.21.15.102:0/1773308669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a80c9eb0 msgr2=0x7ff6a80c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:57.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.299+0000 7ff6a6800640 1 --2- 172.21.15.102:0/1773308669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a80c9eb0 0x7ff6a80c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.299+0000 7ff6a6800640 1 -- 172.21.15.102:0/1773308669 >> v1:172.21.15.102:6789/0 conn(0x7ff6a814fa50 legacy=0x7ff6a80c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.299+0000 7ff6a6800640 1 -- 172.21.15.102:0/1773308669 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a80c7960 con 0x7ff6a8073a10 2024-04-20T21:42:57.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.300+0000 7ff6a6800640 1 --2- 172.21.15.102:0/1773308669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a8158fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00097a0 tx=0x7ff6a002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43572ff514bf877e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:57.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.300+0000 7ff6a4a00640 1 -- 172.21.15.102:0/1773308669 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0034070 con 0x7ff6a8073a10 2024-04-20T21:42:57.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.300+0000 7ff6a4a00640 1 -- 172.21.15.102:0/1773308669 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a0030800 con 0x7ff6a8073a10 2024-04-20T21:42:57.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.300+0000 7ff6a4a00640 1 -- 172.21.15.102:0/1773308669 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0030bd0 con 0x7ff6a8073a10 2024-04-20T21:42:57.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.301+0000 7ff6ad000640 1 -- 172.21.15.102:0/1773308669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 msgr2=0x7ff6a8158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.301+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1773308669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a8158fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00097a0 tx=0x7ff6a002b940 comp rx=0 tx=0).stop 2024-04-20T21:42:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.301+0000 7ff6ad000640 1 -- 172.21.15.102:0/1773308669 shutdown_connections 2024-04-20T21:42:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.301+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1773308669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a8158fc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.301+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1773308669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a80c9eb0 0x7ff6a80c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.301+0000 7ff6ad000640 1 -- 172.21.15.102:0/1773308669 >> 172.21.15.102:0/1773308669 conn(0x7ff6a80c9390 msgr2=0x7ff6a80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.301+0000 7ff6ad000640 1 -- 172.21.15.102:0/1773308669 shutdown_connections 2024-04-20T21:42:57.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.301+0000 7ff6ad000640 1 -- 172.21.15.102:0/1773308669 wait complete. 2024-04-20T21:42:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.304+0000 7ff6ad000640 1 Processor -- start 2024-04-20T21:42:57.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.304+0000 7ff6ad000640 1 -- start start 2024-04-20T21:42:57.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6ad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a80cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6ad000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6a80c9eb0 0x7ff6a81485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6ad000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a814fa50 0x7ff6a8148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6ad000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a80c7c00 con 0x7ff6a8073a10 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6ad000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a80c3d90 con 0x7ff6a80c9eb0 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6ad000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a80c4030 con 0x7ff6a814fa50 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a80cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a80cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36416/0 (socket says 172.21.15.102:36416) 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6a5e00640 1 -- 172.21.15.102:0/1363517490 learned_addr learned my addr 172.21.15.102:0/1363517490 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.305+0000 7ff6a6800640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a814fa50 0x7ff6a8148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff6a5400640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6a80c9eb0 0x7ff6a81485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff6a5e00640 1 -- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a814fa50 msgr2=0x7ff6a8148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff6a5e00640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a814fa50 0x7ff6a8148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff6a5e00640 1 -- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6a80c9eb0 msgr2=0x7ff6a81485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff6a5e00640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6a80c9eb0 0x7ff6a81485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff6a5e00640 1 -- 172.21.15.102:0/1363517490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a8158d70 con 0x7ff6a8073a10 2024-04-20T21:42:57.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff6a5e00640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a80cc070 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff69800cb80 tx=0x7ff69800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:57.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff69e000640 1 -- 172.21.15.102:0/1363517490 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698017070 con 0x7ff6a8073a10 2024-04-20T21:42:57.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff69e000640 1 -- 172.21.15.102:0/1363517490 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff69800ad40 con 0x7ff6a8073a10 2024-04-20T21:42:57.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff69e000640 1 -- 172.21.15.102:0/1363517490 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698012640 con 0x7ff6a8073a10 2024-04-20T21:42:57.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.306+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a8159050 con 0x7ff6a8073a10 2024-04-20T21:42:57.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.307+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a814aff0 con 0x7ff6a8073a10 2024-04-20T21:42:57.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.308+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a80c56b0 con 0x7ff6a8073a10 2024-04-20T21:42:57.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.309+0000 7ff69e000640 1 -- 172.21.15.102:0/1363517490 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff69801a070 con 0x7ff6a8073a10 2024-04-20T21:42:57.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.310+0000 7ff69e000640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff68806c920 0x7ff68806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.311+0000 7ff69e000640 1 -- 172.21.15.102:0/1363517490 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7ff69808c2f0 con 0x7ff6a8073a10 2024-04-20T21:42:57.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.311+0000 7ff6a5400640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff68806c920 0x7ff68806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.311+0000 7ff6a5400640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff68806c920 0x7ff68806ede0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff6a814c9e0 tx=0x7ff694009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:57.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.315+0000 7ff69e000640 1 -- 172.21.15.102:0/1363517490 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff6a80c56b0 con 0x7ff6a8073a10 2024-04-20T21:42:57.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.553+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6a8074340 con 0x7ff6a8073a10 2024-04-20T21:42:57.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.554+0000 7ff69e000640 1 -- 172.21.15.102:0/1363517490 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5199 (secure 0 0 0) 0x7ff6a81536d0 con 0x7ff6a8073a10 2024-04-20T21:42:57.554 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:57.554 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:55.885106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:42:57.554 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 89 2024-04-20T21:42:57.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff68806c920 msgr2=0x7ff68806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff68806c920 0x7ff68806ede0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff6a814c9e0 tx=0x7ff694009190 comp rx=0 tx=0).stop 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 msgr2=0x7ff6a80cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a80cc070 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff69800cb80 tx=0x7ff69800a430 comp rx=0 tx=0).stop 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 shutdown_connections 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6a814fa50 0x7ff6a8148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff68806c920 0x7ff68806ede0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6a80c9eb0 0x7ff6a81485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 --2- 172.21.15.102:0/1363517490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6a8073a10 0x7ff6a80cc070 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 >> 172.21.15.102:0/1363517490 conn(0x7ff6a80c9390 msgr2=0x7ff6a814cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 shutdown_connections 2024-04-20T21:42:57.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.557+0000 7ff6ad000640 1 -- 172.21.15.102:0/1363517490 wait complete. 2024-04-20T21:42:57.575 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:42:57.575 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:42:57.575 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:57.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.779+0000 7fa65ba00640 1 Processor -- start 2024-04-20T21:42:57.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.779+0000 7fa65ba00640 1 -- start start 2024-04-20T21:42:57.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.780+0000 7fa65ba00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa65415ea20 0x7fa65406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.780+0000 7fa65ba00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa6540710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.780+0000 7fa65ba00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6540c8f20 con 0x7fa65415f2b0 2024-04-20T21:42:57.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.780+0000 7fa65ba00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa65406cb80 con 0x7fa65415ea20 2024-04-20T21:42:57.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.780+0000 7fa65ba00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa65406ce60 con 0x7fa65415ee00 2024-04-20T21:42:57.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa653e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa6540710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa659a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa65415f2b0 0x7fa6540c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58012/0 (socket says 172.21.15.102:58012) 2024-04-20T21:42:57.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa659a00640 1 -- 172.21.15.102:0/4077555587 learned_addr learned my addr 172.21.15.102:0/4077555587 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:42:57.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa659000640 1 --2- 172.21.15.102:0/4077555587 >> v2:172.21.15.193:3300/0 conn(0x7fa65415ea20 0x7fa65406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa653e00640 1 -- 172.21.15.102:0/4077555587 >> v2:172.21.15.193:3300/0 conn(0x7fa65415ea20 msgr2=0x7fa65406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa653e00640 1 --2- 172.21.15.102:0/4077555587 >> v2:172.21.15.193:3300/0 conn(0x7fa65415ea20 0x7fa65406d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa653e00640 1 -- 172.21.15.102:0/4077555587 >> v1:172.21.15.102:6789/0 conn(0x7fa65415f2b0 legacy=0x7fa6540c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa653e00640 1 -- 172.21.15.102:0/4077555587 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65406d140 con 0x7fa65415ee00 2024-04-20T21:42:57.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.781+0000 7fa653400640 1 -- 172.21.15.102:0/4077555587 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844433769 0 0) 0x7fa6540c8f20 con 0x7fa65415f2b0 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.783+0000 7fa653e00640 1 --2- 172.21.15.102:0/4077555587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa6540710a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097a0 tx=0x7fa64002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c31e942b0de50a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.783+0000 7fa653400640 1 -- 172.21.15.102:0/4077555587 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640034070 con 0x7fa65415ee00 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.783+0000 7fa653400640 1 -- 172.21.15.102:0/4077555587 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa640030610 con 0x7fa65415ee00 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.783+0000 7fa653400640 1 -- 172.21.15.102:0/4077555587 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400309e0 con 0x7fa65415ee00 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.783+0000 7fa65ba00640 1 -- 172.21.15.102:0/4077555587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 msgr2=0x7fa6540710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.783+0000 7fa65ba00640 1 --2- 172.21.15.102:0/4077555587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa6540710a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097a0 tx=0x7fa64002b960 comp rx=0 tx=0).stop 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.784+0000 7fa65ba00640 1 -- 172.21.15.102:0/4077555587 shutdown_connections 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.784+0000 7fa65ba00640 1 --2- 172.21.15.102:0/4077555587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa6540710a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.784+0000 7fa65ba00640 1 --2- 172.21.15.102:0/4077555587 >> v2:172.21.15.193:3300/0 conn(0x7fa65415ea20 0x7fa65406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.784+0000 7fa65ba00640 1 -- 172.21.15.102:0/4077555587 >> 172.21.15.102:0/4077555587 conn(0x7fa6540c9390 msgr2=0x7fa6540c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.784+0000 7fa65ba00640 1 -- 172.21.15.102:0/4077555587 shutdown_connections 2024-04-20T21:42:57.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.784+0000 7fa65ba00640 1 -- 172.21.15.102:0/4077555587 wait complete. 2024-04-20T21:42:57.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.786+0000 7fa65ba00640 1 Processor -- start 2024-04-20T21:42:57.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.786+0000 7fa65ba00640 1 -- start start 2024-04-20T21:42:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa65ba00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa65415ea20 0x7fa6540cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa65ba00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa654156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa65ba00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65415f2b0 0x7fa6541572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa65ba00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa6540c8c80 con 0x7fa65415f2b0 2024-04-20T21:42:57.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa65ba00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6540c8b00 con 0x7fa65415ea20 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa65ba00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa654154e80 con 0x7fa65415ee00 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65415f2b0 0x7fa6541572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65415f2b0 0x7fa6541572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36426/0 (socket says 172.21.15.102:36426) 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659a00640 1 -- 172.21.15.102:0/1657072036 learned_addr learned my addr 172.21.15.102:0/1657072036 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659000640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa65415ea20 0x7fa6540cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa653e00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa654156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659a00640 1 -- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 msgr2=0x7fa654156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659a00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa654156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659a00640 1 -- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa65415ea20 msgr2=0x7fa6540cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659a00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa65415ea20 0x7fa6540cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659a00640 1 -- 172.21.15.102:0/1657072036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa654149300 con 0x7fa65415f2b0 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.787+0000 7fa659000640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa65415ea20 0x7fa6540cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:57.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.788+0000 7fa659a00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65415f2b0 0x7fa6541572e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00c6d0 tx=0x7fa64c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.788+0000 7fa650c00640 1 -- 172.21.15.102:0/1657072036 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c018070 con 0x7fa65415f2b0 2024-04-20T21:42:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.788+0000 7fa650c00640 1 -- 172.21.15.102:0/1657072036 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c004d60 con 0x7fa65415f2b0 2024-04-20T21:42:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.788+0000 7fa650c00640 1 -- 172.21.15.102:0/1657072036 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00a880 con 0x7fa65415f2b0 2024-04-20T21:42:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.788+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6541546e0 con 0x7fa65415f2b0 2024-04-20T21:42:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.788+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa654155fe0 con 0x7fa65415f2b0 2024-04-20T21:42:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.789+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa654074340 con 0x7fa65415f2b0 2024-04-20T21:42:57.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.790+0000 7fa650c00640 1 -- 172.21.15.102:0/1657072036 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa64c00ac30 con 0x7fa65415f2b0 2024-04-20T21:42:57.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.791+0000 7fa650c00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa63406c8f0 0x7fa63406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:57.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.791+0000 7fa650c00640 1 -- 172.21.15.102:0/1657072036 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7fa64c08bc20 con 0x7fa65415f2b0 2024-04-20T21:42:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.792+0000 7fa659000640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa63406c8f0 0x7fa63406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.792+0000 7fa659000640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa63406c8f0 0x7fa63406edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa648009f30 tx=0x7fa648009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:57.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:57.795+0000 7fa650c00640 1 -- 172.21.15.102:0/1657072036 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa64c059050 con 0x7fa65415f2b0 2024-04-20T21:42:58.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.026+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa65414a2b0 con 0x7fa65415f2b0 2024-04-20T21:42:58.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.027+0000 7fa650c00640 1 -- 172.21.15.102:0/1657072036 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5199 (secure 0 0 0) 0x7fa64c05d350 con 0x7fa65415f2b0 2024-04-20T21:42:58.027 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:42:58.027 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:55.885106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:42:58.027 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 89 2024-04-20T21:42:58.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa63406c8f0 msgr2=0x7fa63406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:58.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa63406c8f0 0x7fa63406edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa648009f30 tx=0x7fa648009190 comp rx=0 tx=0).stop 2024-04-20T21:42:58.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65415f2b0 msgr2=0x7fa6541572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:58.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65415f2b0 0x7fa6541572e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00c6d0 tx=0x7fa64c00cb90 comp rx=0 tx=0).stop 2024-04-20T21:42:58.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 shutdown_connections 2024-04-20T21:42:58.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa65415f2b0 0x7fa6541572e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:58.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa65415ee00 0x7fa654156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:58.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa63406c8f0 0x7fa63406edb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:58.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 --2- 172.21.15.102:0/1657072036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa65415ea20 0x7fa6540cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:58.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 >> 172.21.15.102:0/1657072036 conn(0x7fa6540c9390 msgr2=0x7fa654153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:58.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 shutdown_connections 2024-04-20T21:42:58.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:42:58.030+0000 7fa65ba00640 1 -- 172.21.15.102:0/1657072036 wait complete. 2024-04-20T21:42:58.048 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4285": { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 89, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:42:55.885106+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4285 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:43:00.049 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.258+0000 7f9713800640 1 Processor -- start 2024-04-20T21:43:00.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.258+0000 7f9713800640 1 -- start start 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9713800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f970c13bc80 0x7f970c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9713800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c0737f0 0x7f970c159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9713800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f970c073c30 0x7f970c0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9713800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f970c0c6a30 con 0x7f970c13bc80 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9713800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f970c0c6ec0 con 0x7f970c073c30 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9713800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f970c0c8d00 con 0x7f970c0737f0 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9710e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f970c13bc80 0x7f970c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9710e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f970c13bc80 0x7f970c0c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49164/0 (socket says 172.21.15.102:49164) 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9710e00640 1 -- 172.21.15.102:0/1971448706 learned_addr learned my addr 172.21.15.102:0/1971448706 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f970be00640 1 --2- 172.21.15.102:0/1971448706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c0737f0 0x7f970c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.260+0000 7f9711800640 1 --2- 172.21.15.102:0/1971448706 >> v2:172.21.15.193:3300/0 conn(0x7f970c073c30 0x7f970c0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.261+0000 7f9710e00640 1 -- 172.21.15.102:0/1971448706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c0737f0 msgr2=0x7f970c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:00.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.261+0000 7f9710e00640 1 --2- 172.21.15.102:0/1971448706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c0737f0 0x7f970c159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.261+0000 7f9710e00640 1 -- 172.21.15.102:0/1971448706 >> v2:172.21.15.193:3300/0 conn(0x7f970c073c30 msgr2=0x7f970c0c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:00.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.261+0000 7f9710e00640 1 --2- 172.21.15.102:0/1971448706 >> v2:172.21.15.193:3300/0 conn(0x7f970c073c30 0x7f970c0c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.261+0000 7f9710e00640 1 -- 172.21.15.102:0/1971448706 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c0c8fc0 con 0x7f970c13bc80 2024-04-20T21:43:00.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.262+0000 7f9710e00640 1 --2- 172.21.15.102:0/1971448706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c13bc80 0x7f970c0c84e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f970c0cc730 tx=0x7f96fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e38a85e5aafef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:00.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.262+0000 7f970b400640 1 -- 172.21.15.102:0/1971448706 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc035020 con 0x7f970c13bc80 2024-04-20T21:43:00.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.262+0000 7f970b400640 1 -- 172.21.15.102:0/1971448706 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc031870 con 0x7f970c13bc80 2024-04-20T21:43:00.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.262+0000 7f970b400640 1 -- 172.21.15.102:0/1971448706 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc031c40 con 0x7f970c13bc80 2024-04-20T21:43:00.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 -- 172.21.15.102:0/1971448706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c13bc80 msgr2=0x7f970c0c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:00.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 --2- 172.21.15.102:0/1971448706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c13bc80 0x7f970c0c84e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f970c0cc730 tx=0x7f96fc02b910 comp rx=0 tx=0).stop 2024-04-20T21:43:00.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 -- 172.21.15.102:0/1971448706 shutdown_connections 2024-04-20T21:43:00.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 --2- 172.21.15.102:0/1971448706 >> v2:172.21.15.193:3300/0 conn(0x7f970c073c30 0x7f970c0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 --2- 172.21.15.102:0/1971448706 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c0737f0 0x7f970c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 --2- 172.21.15.102:0/1971448706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c13bc80 0x7f970c0c84e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 -- 172.21.15.102:0/1971448706 >> 172.21.15.102:0/1971448706 conn(0x7f970c06ebf0 msgr2=0x7f970c06efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:00.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 -- 172.21.15.102:0/1971448706 shutdown_connections 2024-04-20T21:43:00.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.263+0000 7f9713800640 1 -- 172.21.15.102:0/1971448706 wait complete. 2024-04-20T21:43:00.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.265+0000 7f9713800640 1 Processor -- start 2024-04-20T21:43:00.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.265+0000 7f9713800640 1 -- start start 2024-04-20T21:43:00.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9713800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c0737f0 0x7f970c159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:00.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9713800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c073c30 0x7f970c0bf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:00.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9713800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f970c13bc80 0x7f970c0c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:00.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9713800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f970c0c66c0 con 0x7f970c0737f0 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9713800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f970c0c63e0 con 0x7f970c13bc80 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9713800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f970c0c8a20 con 0x7f970c073c30 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9710e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c0737f0 0x7f970c159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9710e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c0737f0 0x7f970c159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49178/0 (socket says 172.21.15.102:49178) 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9710e00640 1 -- 172.21.15.102:0/3052208576 learned_addr learned my addr 172.21.15.102:0/3052208576 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f970be00640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c073c30 0x7f970c0bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.266+0000 7f9711800640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f970c13bc80 0x7f970c0c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9710e00640 1 -- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c073c30 msgr2=0x7f970c0bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9710e00640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c073c30 0x7f970c0bf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9710e00640 1 -- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f970c13bc80 msgr2=0x7f970c0c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9710e00640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f970c13bc80 0x7f970c0c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9710e00640 1 -- 172.21.15.102:0/3052208576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c1f2150 con 0x7f970c0737f0 2024-04-20T21:43:00.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9710e00640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c0737f0 0x7f970c159110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f96fc02b950 tx=0x7f96fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:00.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9708c00640 1 -- 172.21.15.102:0/3052208576 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc040070 con 0x7f970c0737f0 2024-04-20T21:43:00.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9708c00640 1 -- 172.21.15.102:0/3052208576 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc031e10 con 0x7f970c0737f0 2024-04-20T21:43:00.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c1f1b80 con 0x7f970c0737f0 2024-04-20T21:43:00.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9708c00640 1 -- 172.21.15.102:0/3052208576 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc035040 con 0x7f970c0737f0 2024-04-20T21:43:00.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.267+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c0cdfe0 con 0x7f970c0737f0 2024-04-20T21:43:00.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.269+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c0bed10 con 0x7f970c0737f0 2024-04-20T21:43:00.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.269+0000 7f9708c00640 1 -- 172.21.15.102:0/3052208576 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96fc038020 con 0x7f970c0737f0 2024-04-20T21:43:00.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.270+0000 7f9708c00640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f96ec06c920 0x7f96ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:00.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.270+0000 7f9708c00640 1 -- 172.21.15.102:0/3052208576 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7f96fc0ae0c0 con 0x7f970c0737f0 2024-04-20T21:43:00.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.271+0000 7f970be00640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f96ec06c920 0x7f96ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:00.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.271+0000 7f970be00640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f96ec06c920 0x7f96ec06ede0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f970c13b560 tx=0x7f96f8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:00.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.275+0000 7f9708c00640 1 -- 172.21.15.102:0/3052208576 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f96fc07b4f0 con 0x7f970c0737f0 2024-04-20T21:43:00.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.513+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f970c06d930 con 0x7f970c0737f0 2024-04-20T21:43:00.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.514+0000 7f9708c00640 1 -- 172.21.15.102:0/3052208576 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5199 (secure 0 0 0) 0x7f96fc07f7f0 con 0x7f970c0737f0 2024-04-20T21:43:00.513 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:00.513 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:55.885106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:43:00.513 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 89 2024-04-20T21:43:00.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f96ec06c920 msgr2=0x7f96ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:00.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f96ec06c920 0x7f96ec06ede0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f970c13b560 tx=0x7f96f8009190 comp rx=0 tx=0).stop 2024-04-20T21:43:00.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c0737f0 msgr2=0x7f970c159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:00.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c0737f0 0x7f970c159110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f96fc02b950 tx=0x7f96fc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 shutdown_connections 2024-04-20T21:43:00.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f96ec06c920 0x7f96ec06ede0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f970c13bc80 0x7f970c0c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f970c073c30 0x7f970c0bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 --2- 172.21.15.102:0/3052208576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f970c0737f0 0x7f970c159110 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:00.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 >> 172.21.15.102:0/3052208576 conn(0x7f970c06ebf0 msgr2=0x7f970c0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:00.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 shutdown_connections 2024-04-20T21:43:00.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:00.516+0000 7f9713800640 1 -- 172.21.15.102:0/3052208576 wait complete. 2024-04-20T21:43:00.534 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:43:01.353 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:43:01.361 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:43:02.534 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:02.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.752+0000 7f90bb800640 1 Processor -- start 2024-04-20T21:43:02.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.752+0000 7f90bb800640 1 -- start start 2024-04-20T21:43:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.753+0000 7f90bb800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f90b4151c90 0x7f90b40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.753+0000 7f90bb800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b40c7290 0x7f90b4158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.753+0000 7f90bb800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b4158780 con 0x7f90b4071a70 2024-04-20T21:43:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.753+0000 7f90bb800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f90b40c9200 con 0x7f90b4151c90 2024-04-20T21:43:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.753+0000 7f90bb800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f90b4070df0 con 0x7f90b40c7290 2024-04-20T21:43:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b8e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f90b4151c90 0x7f90b40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b40c7290 0x7f90b4158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:02.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b8e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f90b4151c90 0x7f90b40c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49184/0 (socket says 172.21.15.102:49184) 2024-04-20T21:43:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b8e00640 1 -- 172.21.15.102:0/2736403892 learned_addr learned my addr 172.21.15.102:0/2736403892 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b9800640 1 --1- 172.21.15.102:0/2736403892 >> v1:172.21.15.102:6789/0 conn(0x7f90b4071a70 0x7f90b406d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38376/0 (socket says 172.21.15.102:38376) 2024-04-20T21:43:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b8e00640 1 -- 172.21.15.102:0/2736403892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b40c7290 msgr2=0x7f90b4158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b8e00640 1 --2- 172.21.15.102:0/2736403892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b40c7290 0x7f90b4158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b8e00640 1 -- 172.21.15.102:0/2736403892 >> v1:172.21.15.102:6789/0 conn(0x7f90b4071a70 legacy=0x7f90b406d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b8e00640 1 -- 172.21.15.102:0/2736403892 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b4071040 con 0x7f90b4151c90 2024-04-20T21:43:02.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.754+0000 7f90b3e00640 1 --2- 172.21.15.102:0/2736403892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b40c7290 0x7f90b4158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.755+0000 7f90b8e00640 1 --2- 172.21.15.102:0/2736403892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b4151c90 0x7f90b40c8cc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f90a800bb00 tx=0x7f90a802dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa08702ff7dbb97a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.755+0000 7f90b3400640 1 -- 172.21.15.102:0/2736403892 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8034070 con 0x7f90b4151c90 2024-04-20T21:43:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.755+0000 7f90b3400640 1 -- 172.21.15.102:0/2736403892 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a802ed70 con 0x7f90b4151c90 2024-04-20T21:43:02.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90b3400640 1 -- 172.21.15.102:0/2736403892 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8038620 con 0x7f90b4151c90 2024-04-20T21:43:02.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90bb800640 1 -- 172.21.15.102:0/2736403892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b4151c90 msgr2=0x7f90b40c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:02.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90bb800640 1 --2- 172.21.15.102:0/2736403892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b4151c90 0x7f90b40c8cc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f90a800bb00 tx=0x7f90a802dfc0 comp rx=0 tx=0).stop 2024-04-20T21:43:02.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90bb800640 1 -- 172.21.15.102:0/2736403892 shutdown_connections 2024-04-20T21:43:02.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90bb800640 1 --2- 172.21.15.102:0/2736403892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b40c7290 0x7f90b4158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:02.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90bb800640 1 --2- 172.21.15.102:0/2736403892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b4151c90 0x7f90b40c8cc0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:02.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90bb800640 1 -- 172.21.15.102:0/2736403892 >> 172.21.15.102:0/2736403892 conn(0x7f90b40c9390 msgr2=0x7f90b40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:02.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90bb800640 1 -- 172.21.15.102:0/2736403892 shutdown_connections 2024-04-20T21:43:02.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.756+0000 7f90bb800640 1 -- 172.21.15.102:0/2736403892 wait complete. 2024-04-20T21:43:02.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.758+0000 7f90bb800640 1 Processor -- start 2024-04-20T21:43:02.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.758+0000 7f90bb800640 1 -- start start 2024-04-20T21:43:02.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90bb800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b4071a70 0x7f90b414d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:02.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90bb800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b40c7290 0x7f90b4150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:02.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90bb800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4151c90 0x7f90b41543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:02.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90bb800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4071600 con 0x7f90b40c7290 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90bb800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4157db0 con 0x7f90b4151c90 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90bb800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f90b4157a40 con 0x7f90b4071a70 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90b8e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b4071a70 0x7f90b414d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b40c7290 0x7f90b4150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b40c7290 0x7f90b4150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49200/0 (socket says 172.21.15.102:49200) 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90b8e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b4071a70 0x7f90b414d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:45706/0 (socket says 172.21.15.102:45706) 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90b3e00640 1 -- 172.21.15.102:0/2536533929 learned_addr learned my addr 172.21.15.102:0/2536533929 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.759+0000 7f90b9800640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4151c90 0x7f90b41543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b3e00640 1 -- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b4071a70 msgr2=0x7f90b414d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b3e00640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b4071a70 0x7f90b414d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b3e00640 1 -- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4151c90 msgr2=0x7f90b41543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b3e00640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4151c90 0x7f90b41543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b3e00640 1 -- 172.21.15.102:0/2536533929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b41f0a20 con 0x7f90b40c7290 2024-04-20T21:43:02.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b3e00640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b40c7290 0x7f90b4150740 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f90a000ebb0 tx=0x7f90a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:02.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b0c00640 1 -- 172.21.15.102:0/2536533929 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0017070 con 0x7f90b40c7290 2024-04-20T21:43:02.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b0c00640 1 -- 172.21.15.102:0/2536533929 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a000cbd0 con 0x7f90b40c7290 2024-04-20T21:43:02.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b414e020 con 0x7f90b40c7290 2024-04-20T21:43:02.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90b0c00640 1 -- 172.21.15.102:0/2536533929 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0013470 con 0x7f90b40c7290 2024-04-20T21:43:02.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.760+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b41f0ca0 con 0x7f90b40c7290 2024-04-20T21:43:02.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.761+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b4074340 con 0x7f90b40c7290 2024-04-20T21:43:02.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.762+0000 7f90b0c00640 1 -- 172.21.15.102:0/2536533929 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90a001a070 con 0x7f90b40c7290 2024-04-20T21:43:02.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.763+0000 7f90b0c00640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f909406c8f0 0x7f909406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:02.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.763+0000 7f90b0c00640 1 -- 172.21.15.102:0/2536533929 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7f90a008c3a0 con 0x7f90b40c7290 2024-04-20T21:43:02.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.763+0000 7f90b8e00640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f909406c8f0 0x7f909406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:02.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.764+0000 7f90b8e00640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f909406c8f0 0x7f909406edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90a8039000 tx=0x7f90a8034000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:02.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:02.768+0000 7f90b0c00640 1 -- 172.21.15.102:0/2536533929 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f90b41f0a20 con 0x7f90b40c7290 2024-04-20T21:43:03.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~61s 2024-04-20T21:43:03.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.012+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90b414e300 con 0x7f90b40c7290 2024-04-20T21:43:03.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.013+0000 7f90b0c00640 1 -- 172.21.15.102:0/2536533929 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5199 (secure 0 0 0) 0x7f90a0059740 con 0x7f90b40c7290 2024-04-20T21:43:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:03.013 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:55.885106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:43:03.013 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 89 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f909406c8f0 msgr2=0x7f909406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f909406c8f0 0x7f909406edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90a8039000 tx=0x7f90a8034000 comp rx=0 tx=0).stop 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b40c7290 msgr2=0x7f90b4150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b40c7290 0x7f90b4150740 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f90a000ebb0 tx=0x7f90a000c520 comp rx=0 tx=0).stop 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 shutdown_connections 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f909406c8f0 0x7f909406edb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4151c90 0x7f90b41543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b40c7290 0x7f90b4150740 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 --2- 172.21.15.102:0/2536533929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90b4071a70 0x7f90b414d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.016+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 >> 172.21.15.102:0/2536533929 conn(0x7f90b40c9390 msgr2=0x7f90b4072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.017+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 shutdown_connections 2024-04-20T21:43:03.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:03.017+0000 7f90bb800640 1 -- 172.21.15.102:0/2536533929 wait complete. 2024-04-20T21:43:03.033 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:43:05.034 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:05.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.244+0000 7fc043e00640 1 Processor -- start 2024-04-20T21:43:05.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.244+0000 7fc043e00640 1 -- start start 2024-04-20T21:43:05.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc043e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:05.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc043e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc03c071a70 0x7fc03c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:05.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc043e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03c071040 con 0x7fc03c151c90 2024-04-20T21:43:05.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc043e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fc03c0c9200 con 0x7fc03c071a70 2024-04-20T21:43:05.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc043e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c070df0 con 0x7fc03c0c7290 2024-04-20T21:43:05.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc041e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc03c071a70 0x7fc03c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:05.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc040a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:05.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc040a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c0c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:45712/0 (socket says 172.21.15.102:45712) 2024-04-20T21:43:05.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.246+0000 7fc040a00640 1 -- 172.21.15.102:0/2079408910 learned_addr learned my addr 172.21.15.102:0/2079408910 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:05.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.247+0000 7fc040a00640 1 -- 172.21.15.102:0/2079408910 >> v2:172.21.15.102:3300/0 conn(0x7fc03c071a70 msgr2=0x7fc03c06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:05.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.247+0000 7fc040a00640 1 --2- 172.21.15.102:0/2079408910 >> v2:172.21.15.102:3300/0 conn(0x7fc03c071a70 0x7fc03c06d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.247+0000 7fc040a00640 1 -- 172.21.15.102:0/2079408910 >> v1:172.21.15.102:6789/0 conn(0x7fc03c151c90 legacy=0x7fc03c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:05.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.247+0000 7fc040a00640 1 -- 172.21.15.102:0/2079408910 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c071320 con 0x7fc03c0c7290 2024-04-20T21:43:05.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.248+0000 7fc040a00640 1 --2- 172.21.15.102:0/2079408910 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c0c8cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc028009870 tx=0x7fc02802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5716ff7ff1a1396 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:05.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.248+0000 7fc03be00640 1 -- 172.21.15.102:0/2079408910 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02802f830 con 0x7fc03c0c7290 2024-04-20T21:43:05.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.248+0000 7fc03be00640 1 -- 172.21.15.102:0/2079408910 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02802f9d0 con 0x7fc03c0c7290 2024-04-20T21:43:05.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.248+0000 7fc03be00640 1 -- 172.21.15.102:0/2079408910 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02802fda0 con 0x7fc03c0c7290 2024-04-20T21:43:05.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.249+0000 7fc043e00640 1 -- 172.21.15.102:0/2079408910 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 msgr2=0x7fc03c0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:05.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.249+0000 7fc043e00640 1 --2- 172.21.15.102:0/2079408910 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c0c8cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc028009870 tx=0x7fc02802ba10 comp rx=0 tx=0).stop 2024-04-20T21:43:05.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.249+0000 7fc043e00640 1 -- 172.21.15.102:0/2079408910 shutdown_connections 2024-04-20T21:43:05.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.249+0000 7fc043e00640 1 --2- 172.21.15.102:0/2079408910 >> v2:172.21.15.102:3300/0 conn(0x7fc03c071a70 0x7fc03c06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.249+0000 7fc043e00640 1 --2- 172.21.15.102:0/2079408910 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c0c8cc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.249+0000 7fc043e00640 1 -- 172.21.15.102:0/2079408910 >> 172.21.15.102:0/2079408910 conn(0x7fc03c0c9390 msgr2=0x7fc03c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:05.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.249+0000 7fc043e00640 1 -- 172.21.15.102:0/2079408910 shutdown_connections 2024-04-20T21:43:05.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.249+0000 7fc043e00640 1 -- 172.21.15.102:0/2079408910 wait complete. 2024-04-20T21:43:05.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.251+0000 7fc043e00640 1 Processor -- start 2024-04-20T21:43:05.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.251+0000 7fc043e00640 1 -- start start 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc043e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c071a70 0x7fc03c1f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc043e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c1f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc043e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc03c151c90 0x7fc03c1f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc043e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c157a40 con 0x7fc03c151c90 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc043e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c157db0 con 0x7fc03c071a70 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc043e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c0ca1f0 con 0x7fc03c0c7290 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc040a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c1f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc040a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c1f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:45722/0 (socket says 172.21.15.102:45722) 2024-04-20T21:43:05.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc040a00640 1 -- 172.21.15.102:0/319882486 learned_addr learned my addr 172.21.15.102:0/319882486 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:05.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc041400640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c071a70 0x7fc03c1f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:05.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.252+0000 7fc041e00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc03c151c90 0x7fc03c1f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc040a00640 1 -- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c071a70 msgr2=0x7fc03c1f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc040a00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c071a70 0x7fc03c1f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc040a00640 1 -- 172.21.15.102:0/319882486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc03c151c90 msgr2=0x7fc03c1f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc040a00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc03c151c90 0x7fc03c1f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc040a00640 1 -- 172.21.15.102:0/319882486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c156220 con 0x7fc03c0c7290 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc041e00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc03c151c90 0x7fc03c1f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc041400640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c071a70 0x7fc03c1f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc040a00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c1f4f50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc0280097c0 tx=0x7fc028004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc039600640 1 -- 172.21.15.102:0/319882486 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028047070 con 0x7fc03c0c7290 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc039600640 1 -- 172.21.15.102:0/319882486 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc028037c40 con 0x7fc03c0c7290 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c1564c0 con 0x7fc03c0c7290 2024-04-20T21:43:05.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc039600640 1 -- 172.21.15.102:0/319882486 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028042480 con 0x7fc03c0c7290 2024-04-20T21:43:05.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.253+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c156a60 con 0x7fc03c0c7290 2024-04-20T21:43:05.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.254+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c074340 con 0x7fc03c0c7290 2024-04-20T21:43:05.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.256+0000 7fc039600640 1 -- 172.21.15.102:0/319882486 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc02804a070 con 0x7fc03c0c7290 2024-04-20T21:43:05.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.256+0000 7fc039600640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc01c06c8f0 0x7fc01c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:05.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.257+0000 7fc039600640 1 -- 172.21.15.102:0/319882486 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7fc0280ae600 con 0x7fc03c0c7290 2024-04-20T21:43:05.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.257+0000 7fc041400640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc01c06c8f0 0x7fc01c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:05.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.257+0000 7fc041400640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc01c06c8f0 0x7fc01c06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc030001ad0 tx=0x7fc030009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:05.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.261+0000 7fc039600640 1 -- 172.21.15.102:0/319882486 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc028033030 con 0x7fc03c0c7290 2024-04-20T21:43:05.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.506+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc03c14ad60 con 0x7fc03c0c7290 2024-04-20T21:43:05.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.507+0000 7fc039600640 1 -- 172.21.15.102:0/319882486 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5199 (secure 0 0 0) 0x7fc02807bac0 con 0x7fc03c0c7290 2024-04-20T21:43:05.506 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:05.507 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:55.885106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:43:05.507 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 89 2024-04-20T21:43:05.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc01c06c8f0 msgr2=0x7fc01c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:05.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc01c06c8f0 0x7fc01c06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc030001ad0 tx=0x7fc030009040 comp rx=0 tx=0).stop 2024-04-20T21:43:05.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 msgr2=0x7fc03c1f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c1f4f50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc0280097c0 tx=0x7fc028004730 comp rx=0 tx=0).stop 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 shutdown_connections 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc03c151c90 0x7fc03c1f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc03c0c7290 0x7fc03c1f4f50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fc01c06c8f0 0x7fc01c06edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 --2- 172.21.15.102:0/319882486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c071a70 0x7fc03c1f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 >> 172.21.15.102:0/319882486 conn(0x7fc03c0c9390 msgr2=0x7fc03c06d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.510+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 shutdown_connections 2024-04-20T21:43:05.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:05.511+0000 7fc043e00640 1 -- 172.21.15.102:0/319882486 wait complete. 2024-04-20T21:43:05.529 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:43:07.530 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:07.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.751+0000 7f6586a00640 1 Processor -- start 2024-04-20T21:43:07.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.751+0000 7f6586a00640 1 -- start start 2024-04-20T21:43:07.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.753+0000 7f6586a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6580073a10 0x7f65800c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:07.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.753+0000 7f6586a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65801536d0 con 0x7f65800c9eb0 2024-04-20T21:43:07.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.753+0000 7f6586a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65800c76c0 con 0x7f658014fa50 2024-04-20T21:43:07.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.753+0000 7f6586a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f65800c7960 con 0x7f6580073a10 2024-04-20T21:43:07.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.753+0000 7f657fe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f658014fa50 0x7f65800c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:34968/0 (socket says 172.21.15.102:34968) 2024-04-20T21:43:07.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.753+0000 7f657fe00640 1 -- 172.21.15.102:0/831587458 learned_addr learned my addr 172.21.15.102:0/831587458 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:43:07.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.753+0000 7f6584a00640 1 --2- 172.21.15.102:0/831587458 >> v2:172.21.15.102:3300/0 conn(0x7f6580073a10 0x7f65800c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:07.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.754+0000 7f657f400640 1 -- 172.21.15.102:0/831587458 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2927605911 0 0) 0x7f65800c76c0 con 0x7f658014fa50 2024-04-20T21:43:07.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.754+0000 7f657f400640 1 -- 172.21.15.102:0/831587458 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6564003520 con 0x7f658014fa50 2024-04-20T21:43:07.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.754+0000 7f6584a00640 1 -- 172.21.15.102:0/831587458 >> v1:172.21.15.193:6789/0 conn(0x7f658014fa50 legacy=0x7f65800c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:07.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.754+0000 7f6584a00640 1 -- 172.21.15.102:0/831587458 >> v1:172.21.15.102:6789/0 conn(0x7f65800c9eb0 legacy=0x7f6580161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:07.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.754+0000 7f6584a00640 1 -- 172.21.15.102:0/831587458 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65800c3d90 con 0x7f6580073a10 2024-04-20T21:43:07.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.755+0000 7f6584a00640 1 --2- 172.21.15.102:0/831587458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6580073a10 0x7f65800c4830 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f65800c7890 tx=0x7f657802bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f110304777665493 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.755+0000 7f657f400640 1 -- 172.21.15.102:0/831587458 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578039070 con 0x7f6580073a10 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.755+0000 7f657f400640 1 -- 172.21.15.102:0/831587458 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657802cd70 con 0x7f6580073a10 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.755+0000 7f657f400640 1 -- 172.21.15.102:0/831587458 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578034660 con 0x7f6580073a10 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.756+0000 7f6586a00640 1 -- 172.21.15.102:0/831587458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6580073a10 msgr2=0x7f65800c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.756+0000 7f6586a00640 1 --2- 172.21.15.102:0/831587458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6580073a10 0x7f65800c4830 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f65800c7890 tx=0x7f657802bfc0 comp rx=0 tx=0).stop 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.756+0000 7f6586a00640 1 -- 172.21.15.102:0/831587458 shutdown_connections 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.756+0000 7f6586a00640 1 --2- 172.21.15.102:0/831587458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6580073a10 0x7f65800c4830 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.756+0000 7f6586a00640 1 -- 172.21.15.102:0/831587458 >> 172.21.15.102:0/831587458 conn(0x7f65800c9390 msgr2=0x7f65800c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:07.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.756+0000 7f6586a00640 1 -- 172.21.15.102:0/831587458 shutdown_connections 2024-04-20T21:43:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.756+0000 7f6586a00640 1 -- 172.21.15.102:0/831587458 wait complete. 2024-04-20T21:43:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.758+0000 7f6586a00640 1 Processor -- start 2024-04-20T21:43:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.758+0000 7f6586a00640 1 -- start start 2024-04-20T21:43:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6586a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6580073a10 0x7f658014a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6586a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65800c9eb0 0x7f65801f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6586a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658014fa50 0x7f6580072410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6586a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f65800c4030 con 0x7f65800c9eb0 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6586a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65800c7c00 con 0x7f658014fa50 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6586a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f65800c4310 con 0x7f6580073a10 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6584a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658014fa50 0x7f6580072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f657fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6580073a10 0x7f658014a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6584a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658014fa50 0x7f6580072410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:52840/0 (socket says 172.21.15.102:52840) 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6584a00640 1 -- 172.21.15.102:0/1340519473 learned_addr learned my addr 172.21.15.102:0/1340519473 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6577e00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65800c9eb0 0x7f65801f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6584a00640 1 -- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6580073a10 msgr2=0x7f658014a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6584a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6580073a10 0x7f658014a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6584a00640 1 -- 172.21.15.102:0/1340519473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65800c9eb0 msgr2=0x7f65801f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6584a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65800c9eb0 0x7f65801f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f6584a00640 1 -- 172.21.15.102:0/1340519473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658014aaf0 con 0x7f658014fa50 2024-04-20T21:43:07.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.759+0000 7f657fe00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6580073a10 0x7f658014a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:07.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.760+0000 7f6584a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658014fa50 0x7f6580072410 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f657802c5d0 tx=0x7f6578004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:07.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.760+0000 7f657cc00640 1 -- 172.21.15.102:0/1340519473 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578039050 con 0x7f658014fa50 2024-04-20T21:43:07.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.760+0000 7f657cc00640 1 -- 172.21.15.102:0/1340519473 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6578045070 con 0x7f658014fa50 2024-04-20T21:43:07.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.760+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6580071370 con 0x7f658014fa50 2024-04-20T21:43:07.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.760+0000 7f657cc00640 1 -- 172.21.15.102:0/1340519473 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578009450 con 0x7f658014fa50 2024-04-20T21:43:07.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.760+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65801f1670 con 0x7f658014fa50 2024-04-20T21:43:07.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.761+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658014ce80 con 0x7f658014fa50 2024-04-20T21:43:07.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.762+0000 7f657cc00640 1 -- 172.21.15.102:0/1340519473 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f657803e850 con 0x7f658014fa50 2024-04-20T21:43:07.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.763+0000 7f657cc00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f655c06c920 0x7f655c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:07.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.763+0000 7f657cc00640 1 -- 172.21.15.102:0/1340519473 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7f65780af1a0 con 0x7f658014fa50 2024-04-20T21:43:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.764+0000 7f657fe00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f655c06c920 0x7f655c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.764+0000 7f657fe00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f655c06c920 0x7f655c06ede0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65700011e0 tx=0x7f6570007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:07.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:07.768+0000 7f657cc00640 1 -- 172.21.15.102:0/1340519473 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f658014ce80 con 0x7f658014fa50 2024-04-20T21:43:08.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.016+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65801f1460 con 0x7f658014fa50 2024-04-20T21:43:08.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.017+0000 7f657cc00640 1 -- 172.21.15.102:0/1340519473 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5199 (secure 0 0 0) 0x7f657807c770 con 0x7f658014fa50 2024-04-20T21:43:08.016 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:08.016 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:42:55.885106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:43:08.017 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 89 2024-04-20T21:43:08.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f655c06c920 msgr2=0x7f655c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:08.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f655c06c920 0x7f655c06ede0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65700011e0 tx=0x7f6570007d30 comp rx=0 tx=0).stop 2024-04-20T21:43:08.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658014fa50 msgr2=0x7f6580072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658014fa50 0x7f6580072410 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f657802c5d0 tx=0x7f6578004a40 comp rx=0 tx=0).stop 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 shutdown_connections 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f655c06c920 0x7f655c06ede0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658014fa50 0x7f6580072410 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65800c9eb0 0x7f65801f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 --2- 172.21.15.102:0/1340519473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6580073a10 0x7f658014a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 >> 172.21.15.102:0/1340519473 conn(0x7f65800c9390 msgr2=0x7f658006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 shutdown_connections 2024-04-20T21:43:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:08.020+0000 7f6586a00640 1 -- 172.21.15.102:0/1340519473 wait complete. 2024-04-20T21:43:08.038 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:43:09.715 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~68s 2024-04-20T21:43:10.039 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:10.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.251+0000 7f5205a00640 1 Processor -- start 2024-04-20T21:43:10.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.251+0000 7f5205a00640 1 -- start start 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.252+0000 7f5205a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5200152170 0x7f52000c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.252+0000 7f5205a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200158770 0x7f5200070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.252+0000 7f5205a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5200071f70 con 0x7f5200158b50 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.252+0000 7f5205a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f52000c6dc0 con 0x7f5200152170 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.252+0000 7f5205a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5200072210 con 0x7f5200158770 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51fea00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5200152170 0x7f52000c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51f7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200158770 0x7f5200070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51fea00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5200152170 0x7f52000c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:36218/0 (socket says 172.21.15.102:36218) 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51fea00640 1 -- 172.21.15.102:0/4032339024 learned_addr learned my addr 172.21.15.102:0/4032339024 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:10.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51ff400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f5200158b50 0x7f520006c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:45376/0 (socket says 172.21.15.102:45376) 2024-04-20T21:43:10.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51fea00640 1 -- 172.21.15.102:0/4032339024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200158770 msgr2=0x7f5200070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:10.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51fea00640 1 --2- 172.21.15.102:0/4032339024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200158770 0x7f5200070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51fea00640 1 -- 172.21.15.102:0/4032339024 >> v1:172.21.15.193:6789/0 conn(0x7f5200158b50 legacy=0x7f520006c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51fea00640 1 -- 172.21.15.102:0/4032339024 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52000724b0 con 0x7f5200152170 2024-04-20T21:43:10.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51f7e00640 1 --2- 172.21.15.102:0/4032339024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200158770 0x7f5200070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:10.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.253+0000 7f51fe000640 1 -- 172.21.15.102:0/4032339024 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607895004 0 0) 0x7f5200071f70 con 0x7f5200158b50 2024-04-20T21:43:10.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.254+0000 7f51fea00640 1 --2- 172.21.15.102:0/4032339024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200152170 0x7f52000c8db0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5200072680 tx=0x7f51f002e1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb20c9207ea1d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.255+0000 7f51fe000640 1 -- 172.21.15.102:0/4032339024 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f003b070 con 0x7f5200152170 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.255+0000 7f51fe000640 1 -- 172.21.15.102:0/4032339024 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0036440 con 0x7f5200152170 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.255+0000 7f51fe000640 1 -- 172.21.15.102:0/4032339024 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0036810 con 0x7f5200152170 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.255+0000 7f5205a00640 1 -- 172.21.15.102:0/4032339024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200152170 msgr2=0x7f52000c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.255+0000 7f5205a00640 1 --2- 172.21.15.102:0/4032339024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200152170 0x7f52000c8db0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5200072680 tx=0x7f51f002e1d0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.256+0000 7f5205a00640 1 -- 172.21.15.102:0/4032339024 shutdown_connections 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.256+0000 7f5205a00640 1 --2- 172.21.15.102:0/4032339024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200158770 0x7f5200070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.256+0000 7f5205a00640 1 --2- 172.21.15.102:0/4032339024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200152170 0x7f52000c8db0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.256+0000 7f5205a00640 1 -- 172.21.15.102:0/4032339024 >> 172.21.15.102:0/4032339024 conn(0x7f52000c9390 msgr2=0x7f52000c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.256+0000 7f5205a00640 1 -- 172.21.15.102:0/4032339024 shutdown_connections 2024-04-20T21:43:10.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.256+0000 7f5205a00640 1 -- 172.21.15.102:0/4032339024 wait complete. 2024-04-20T21:43:10.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.257+0000 7f5205a00640 1 Processor -- start 2024-04-20T21:43:10.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.257+0000 7f5205a00640 1 -- start start 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.258+0000 7f5205a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200152170 0x7f5200150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.258+0000 7f5205a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200158770 0x7f52001544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.258+0000 7f5205a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200158b50 0x7f52001e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.258+0000 7f5205a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f52000c9f20 con 0x7f5200158770 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.258+0000 7f5205a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5200158250 con 0x7f5200158b50 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.258+0000 7f5205a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5200157f20 con 0x7f5200152170 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51ff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200158b50 0x7f52001e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200152170 0x7f5200150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200152170 0x7f5200150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:34766/0 (socket says 172.21.15.102:34766) 2024-04-20T21:43:10.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 -- 172.21.15.102:0/252259066 learned_addr learned my addr 172.21.15.102:0/252259066 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:10.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51f7e00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200158770 0x7f52001544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 -- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200158b50 msgr2=0x7f52001e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200158b50 0x7f52001e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 -- 172.21.15.102:0/252259066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200158770 msgr2=0x7f52001544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200158770 0x7f52001544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 -- 172.21.15.102:0/252259066 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52001512c0 con 0x7f5200152170 2024-04-20T21:43:10.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.259+0000 7f51fea00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200152170 0x7f5200150d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f51f000b2c0 tx=0x7f51f0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.260+0000 7f51f6a00640 1 -- 172.21.15.102:0/252259066 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0046070 con 0x7f5200152170 2024-04-20T21:43:10.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.260+0000 7f51f6a00640 1 -- 172.21.15.102:0/252259066 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0004230 con 0x7f5200152170 2024-04-20T21:43:10.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.260+0000 7f51f6a00640 1 -- 172.21.15.102:0/252259066 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f003b070 con 0x7f5200152170 2024-04-20T21:43:10.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.260+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52001e55f0 con 0x7f5200152170 2024-04-20T21:43:10.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.260+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001557d0 con 0x7f5200152170 2024-04-20T21:43:10.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.261+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5200074340 con 0x7f5200152170 2024-04-20T21:43:10.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.262+0000 7f51f6a00640 1 -- 172.21.15.102:0/252259066 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51f0049070 con 0x7f5200152170 2024-04-20T21:43:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.263+0000 7f51f6a00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.263+0000 7f51f6a00640 1 -- 172.21.15.102:0/252259066 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7f51f00ae5c0 con 0x7f5200152170 2024-04-20T21:43:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.263+0000 7f51f7e00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.264+0000 7f51f7e00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f51dc06c8f0 0x7f51dc06edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009560 tx=0x7f51e8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.268+0000 7f51f6a00640 1 -- 172.21.15.102:0/252259066 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f52001512c0 con 0x7f5200152170 2024-04-20T21:43:10.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.512+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5200153000 con 0x7f5200152170 2024-04-20T21:43:10.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.514+0000 7f51f6a00640 1 -- 172.21.15.102:0/252259066 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5204 (secure 0 0 0) 0x7f51f007b9f0 con 0x7f5200152170 2024-04-20T21:43:10.512 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:43:08.303377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:reconnect","state_seq":396,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:43:10.513 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 90 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f51dc06c8f0 msgr2=0x7f51dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f51dc06c8f0 0x7f51dc06edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009560 tx=0x7f51e8009190 comp rx=0 tx=0).stop 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200152170 msgr2=0x7f5200150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200152170 0x7f5200150d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f51f000b2c0 tx=0x7f51f0004730 comp rx=0 tx=0).stop 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 shutdown_connections 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5200158b50 0x7f52001e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200158770 0x7f52001544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 --2- 172.21.15.102:0/252259066 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5200152170 0x7f5200150d80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 >> 172.21.15.102:0/252259066 conn(0x7f52000c9390 msgr2=0x7f520006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.516+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 shutdown_connections 2024-04-20T21:43:10.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:10.517+0000 7f5205a00640 1 -- 172.21.15.102:0/252259066 wait complete. 2024-04-20T21:43:10.534 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 90, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 90, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4285": { "addr": "172.21.15.193:6839/2076039362", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6836", "nonce": 2076039362, "type": "v2" }, { "addr": "172.21.15.193:6839", "nonce": 2076039362, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4285, "incarnation": 89, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 396 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:43:08.303377+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4285 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:43:12.535 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:12.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.747+0000 7fa854800640 1 Processor -- start 2024-04-20T21:43:12.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.747+0000 7fa854800640 1 -- start start 2024-04-20T21:43:12.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.748+0000 7fa854800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa84c152170 0x7fa84c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:12.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.748+0000 7fa854800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84c071f70 con 0x7fa84c158770 2024-04-20T21:43:12.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.748+0000 7fa854800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84c072210 con 0x7fa84c158b80 2024-04-20T21:43:12.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.748+0000 7fa854800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa84c0c6dc0 con 0x7fa84c152170 2024-04-20T21:43:12.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa852800640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fa84c158b80 0x7fa84c1628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:45388/0 (socket says 172.21.15.102:45388) 2024-04-20T21:43:12.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa852800640 1 -- 172.21.15.102:0/491115399 learned_addr learned my addr 172.21.15.102:0/491115399 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:43:12.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa851e00640 1 --2- 172.21.15.102:0/491115399 >> v2:172.21.15.102:3300/0 conn(0x7fa84c152170 0x7fa84c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:12.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa850a00640 1 -- 172.21.15.102:0/491115399 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525777733 0 0) 0x7fa84c072210 con 0x7fa84c158b80 2024-04-20T21:43:12.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa850a00640 1 -- 172.21.15.102:0/491115399 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa834005730 con 0x7fa84c158b80 2024-04-20T21:43:12.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa851e00640 1 -- 172.21.15.102:0/491115399 >> v1:172.21.15.193:6789/0 conn(0x7fa84c158b80 legacy=0x7fa84c1628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:12.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa851e00640 1 -- 172.21.15.102:0/491115399 >> v1:172.21.15.102:6789/0 conn(0x7fa84c158770 legacy=0x7fa84c070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:12.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa850a00640 1 -- 172.21.15.102:0/491115399 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318471960 0 0) 0x7fa84c071f70 con 0x7fa84c158770 2024-04-20T21:43:12.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.749+0000 7fa851e00640 1 -- 172.21.15.102:0/491115399 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c0724b0 con 0x7fa84c152170 2024-04-20T21:43:12.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.750+0000 7fa851e00640 1 --2- 172.21.15.102:0/491115399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c152170 0x7fa84c0c8db0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097a0 tx=0x7fa84002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e4e1871391f031 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:12.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.750+0000 7fa850a00640 1 -- 172.21.15.102:0/491115399 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840034070 con 0x7fa84c152170 2024-04-20T21:43:12.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.750+0000 7fa850a00640 1 -- 172.21.15.102:0/491115399 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840030610 con 0x7fa84c152170 2024-04-20T21:43:12.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.751+0000 7fa850a00640 1 -- 172.21.15.102:0/491115399 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400309e0 con 0x7fa84c152170 2024-04-20T21:43:12.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.751+0000 7fa854800640 1 -- 172.21.15.102:0/491115399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c152170 msgr2=0x7fa84c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:12.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.751+0000 7fa854800640 1 --2- 172.21.15.102:0/491115399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c152170 0x7fa84c0c8db0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097a0 tx=0x7fa84002b960 comp rx=0 tx=0).stop 2024-04-20T21:43:12.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.751+0000 7fa854800640 1 -- 172.21.15.102:0/491115399 shutdown_connections 2024-04-20T21:43:12.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.751+0000 7fa854800640 1 --2- 172.21.15.102:0/491115399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c152170 0x7fa84c0c8db0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:12.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.751+0000 7fa854800640 1 -- 172.21.15.102:0/491115399 >> 172.21.15.102:0/491115399 conn(0x7fa84c0c9390 msgr2=0x7fa84c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:12.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.751+0000 7fa854800640 1 -- 172.21.15.102:0/491115399 shutdown_connections 2024-04-20T21:43:12.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.751+0000 7fa854800640 1 -- 172.21.15.102:0/491115399 wait complete. 2024-04-20T21:43:12.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.753+0000 7fa854800640 1 Processor -- start 2024-04-20T21:43:12.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.753+0000 7fa854800640 1 -- start start 2024-04-20T21:43:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa854800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa84c152170 0x7fa84c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa854800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c158770 0x7fa84c1f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa854800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa84c158b80 0x7fa84c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa854800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c0c9f20 con 0x7fa84c158770 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa854800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c158250 con 0x7fa84c158b80 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa854800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa84c157f20 con 0x7fa84c152170 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa851400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c158770 0x7fa84c1f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa851400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c158770 0x7fa84c1f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47326/0 (socket says 172.21.15.102:47326) 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa851400640 1 -- 172.21.15.102:0/1064914382 learned_addr learned my addr 172.21.15.102:0/1064914382 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.754+0000 7fa851e00640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa84c152170 0x7fa84c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa852800640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa84c158b80 0x7fa84c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa851400640 1 -- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa84c152170 msgr2=0x7fa84c1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:12.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa851400640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa84c152170 0x7fa84c1590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa851400640 1 -- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa84c158b80 msgr2=0x7fa84c152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa851400640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa84c158b80 0x7fa84c152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa851400640 1 -- 172.21.15.102:0/1064914382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c1f0b00 con 0x7fa84c158770 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa851400640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c158770 0x7fa84c1f1580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c520 tx=0x7fa83800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa846000640 1 -- 172.21.15.102:0/1064914382 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838017070 con 0x7fa84c158770 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa846000640 1 -- 172.21.15.102:0/1064914382 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8380052d0 con 0x7fa84c158770 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa846000640 1 -- 172.21.15.102:0/1064914382 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838012870 con 0x7fa84c158770 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.755+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c159770 con 0x7fa84c158770 2024-04-20T21:43:12.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.756+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c159d10 con 0x7fa84c158770 2024-04-20T21:43:12.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.757+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84c074340 con 0x7fa84c158770 2024-04-20T21:43:12.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.758+0000 7fa846000640 1 -- 172.21.15.102:0/1064914382 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa83801a070 con 0x7fa84c158770 2024-04-20T21:43:12.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.759+0000 7fa846000640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82c06c8f0 0x7fa82c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:12.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.759+0000 7fa846000640 1 -- 172.21.15.102:0/1064914382 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7fa83808b810 con 0x7fa84c158770 2024-04-20T21:43:12.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.759+0000 7fa851e00640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82c06c8f0 0x7fa82c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:12.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.760+0000 7fa851e00640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82c06c8f0 0x7fa82c06edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa840036000 tx=0x7fa84003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:12.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:12.764+0000 7fa846000640 1 -- 172.21.15.102:0/1064914382 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa84c1f0b00 con 0x7fa84c158770 2024-04-20T21:43:13.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.011+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa84c0c3460 con 0x7fa84c158770 2024-04-20T21:43:13.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.012+0000 7fa846000640 1 -- 172.21.15.102:0/1064914382 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5204 (secure 0 0 0) 0x7fa838058bc0 con 0x7fa84c158770 2024-04-20T21:43:13.010 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:43:08.303377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:reconnect","state_seq":396,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:43:13.011 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 90 2024-04-20T21:43:13.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82c06c8f0 msgr2=0x7fa82c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82c06c8f0 0x7fa82c06edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa840036000 tx=0x7fa84003b040 comp rx=0 tx=0).stop 2024-04-20T21:43:13.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c158770 msgr2=0x7fa84c1f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c158770 0x7fa84c1f1580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c520 tx=0x7fa83800c9e0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 shutdown_connections 2024-04-20T21:43:13.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa82c06c8f0 0x7fa82c06edb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa84c158b80 0x7fa84c152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c158770 0x7fa84c1f1580 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 --2- 172.21.15.102:0/1064914382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa84c152170 0x7fa84c1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 >> 172.21.15.102:0/1064914382 conn(0x7fa84c0c9390 msgr2=0x7fa84c070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 shutdown_connections 2024-04-20T21:43:13.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:13.015+0000 7fa854800640 1 -- 172.21.15.102:0/1064914382 wait complete. 2024-04-20T21:43:13.031 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:43:15.032 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:15.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.246+0000 7f3b60800640 1 Processor -- start 2024-04-20T21:43:15.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.246+0000 7f3b60800640 1 -- start start 2024-04-20T21:43:15.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b60800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3b58155560 0x7f3b5815be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b60800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3b580c5cc0 0x7f3b580ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b60800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b58073230 0x7f3b580736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b60800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f3b5815f0c0 con 0x7f3b580c5cc0 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b60800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f3b5815e7b0 con 0x7f3b58155560 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b60800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3b58155940 con 0x7f3b58073230 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b5de00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3b58155560 0x7f3b5815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b5e800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b58073230 0x7f3b580736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b5d400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3b580c5cc0 0x7f3b580ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b5de00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3b58155560 0x7f3b5815be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:36236/0 (socket says 172.21.15.102:36236) 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b5e800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b58073230 0x7f3b580736a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:34782/0 (socket says 172.21.15.102:34782) 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.248+0000 7f3b5de00640 1 -- 172.21.15.102:0/2701553042 learned_addr learned my addr 172.21.15.102:0/2701553042 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.249+0000 7f3b5de00640 1 -- 172.21.15.102:0/2701553042 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b58073230 msgr2=0x7f3b580736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.249+0000 7f3b5de00640 1 --2- 172.21.15.102:0/2701553042 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b58073230 0x7f3b580736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.249+0000 7f3b5de00640 1 -- 172.21.15.102:0/2701553042 >> v2:172.21.15.102:3300/0 conn(0x7f3b580c5cc0 msgr2=0x7f3b580ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.249+0000 7f3b5de00640 1 --2- 172.21.15.102:0/2701553042 >> v2:172.21.15.102:3300/0 conn(0x7f3b580c5cc0 0x7f3b580ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.249+0000 7f3b5de00640 1 -- 172.21.15.102:0/2701553042 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b580cbea0 con 0x7f3b58155560 2024-04-20T21:43:15.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.249+0000 7f3b5e800640 1 --2- 172.21.15.102:0/2701553042 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b58073230 0x7f3b580736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.250+0000 7f3b5de00640 1 --2- 172.21.15.102:0/2701553042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58155560 0x7f3b5815be30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3b580cb610 tx=0x7f3b4c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9231e8b3ccba60b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.250+0000 7f3b5ca00640 1 -- 172.21.15.102:0/2701553042 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c034070 con 0x7f3b58155560 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.250+0000 7f3b5ca00640 1 -- 172.21.15.102:0/2701553042 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4c030860 con 0x7f3b58155560 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.250+0000 7f3b5ca00640 1 -- 172.21.15.102:0/2701553042 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c030c30 con 0x7f3b58155560 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 -- 172.21.15.102:0/2701553042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58155560 msgr2=0x7f3b5815be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 --2- 172.21.15.102:0/2701553042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58155560 0x7f3b5815be30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3b580cb610 tx=0x7f3b4c02b9a0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 -- 172.21.15.102:0/2701553042 shutdown_connections 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 --2- 172.21.15.102:0/2701553042 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b58073230 0x7f3b580736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 --2- 172.21.15.102:0/2701553042 >> v2:172.21.15.102:3300/0 conn(0x7f3b580c5cc0 0x7f3b580ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 --2- 172.21.15.102:0/2701553042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58155560 0x7f3b5815be30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 -- 172.21.15.102:0/2701553042 >> 172.21.15.102:0/2701553042 conn(0x7f3b5806cd40 msgr2=0x7f3b5806d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 -- 172.21.15.102:0/2701553042 shutdown_connections 2024-04-20T21:43:15.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.251+0000 7f3b60800640 1 -- 172.21.15.102:0/2701553042 wait complete. 2024-04-20T21:43:15.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.253+0000 7f3b60800640 1 Processor -- start 2024-04-20T21:43:15.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.253+0000 7f3b60800640 1 -- start start 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.254+0000 7f3b60800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58073230 0x7f3b581e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.254+0000 7f3b60800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b580c5cc0 0x7f3b58149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.254+0000 7f3b60800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b58155560 0x7f3b5814a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.254+0000 7f3b60800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5815ed50 con 0x7f3b58155560 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.254+0000 7f3b60800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5815ea70 con 0x7f3b58073230 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.254+0000 7f3b60800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f3b580cbb60 con 0x7f3b580c5cc0 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58073230 0x7f3b581e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58073230 0x7f3b581e5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:36238/0 (socket says 172.21.15.102:36238) 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 -- 172.21.15.102:0/2611708385 learned_addr learned my addr 172.21.15.102:0/2611708385 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5d400640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b580c5cc0 0x7f3b58149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5e800640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b58155560 0x7f3b5814a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 -- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b580c5cc0 msgr2=0x7f3b58149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b580c5cc0 0x7f3b58149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 -- 172.21.15.102:0/2611708385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b58155560 msgr2=0x7f3b5814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b58155560 0x7f3b5814a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 -- 172.21.15.102:0/2611708385 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b58147760 con 0x7f3b58073230 2024-04-20T21:43:15.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.255+0000 7f3b5de00640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58073230 0x7f3b581e5cb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009960 tx=0x7f3b4c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.256+0000 7f3b52000640 1 -- 172.21.15.102:0/2611708385 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c048070 con 0x7f3b58073230 2024-04-20T21:43:15.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.256+0000 7f3b52000640 1 -- 172.21.15.102:0/2611708385 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4c030660 con 0x7f3b58073230 2024-04-20T21:43:15.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.256+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b58148400 con 0x7f3b58073230 2024-04-20T21:43:15.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.256+0000 7f3b52000640 1 -- 172.21.15.102:0/2611708385 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c034040 con 0x7f3b58073230 2024-04-20T21:43:15.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.256+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b58147480 con 0x7f3b58073230 2024-04-20T21:43:15.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.257+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b58074340 con 0x7f3b58073230 2024-04-20T21:43:15.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.258+0000 7f3b52000640 1 -- 172.21.15.102:0/2611708385 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b4c04b070 con 0x7f3b58073230 2024-04-20T21:43:15.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.259+0000 7f3b52000640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b3c06c8f0 0x7f3b3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.259+0000 7f3b52000640 1 -- 172.21.15.102:0/2611708385 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7f3b4c0ae190 con 0x7f3b58073230 2024-04-20T21:43:15.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.259+0000 7f3b5d400640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b3c06c8f0 0x7f3b3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.260+0000 7f3b5d400640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b3c06c8f0 0x7f3b3c06edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b58148330 tx=0x7f3b48009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.264+0000 7f3b52000640 1 -- 172.21.15.102:0/2611708385 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3b4c07b6f0 con 0x7f3b58073230 2024-04-20T21:43:15.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.505+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b58149c60 con 0x7f3b58073230 2024-04-20T21:43:15.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.506+0000 7f3b52000640 1 -- 172.21.15.102:0/2611708385 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5201 (secure 0 0 0) 0x7f3b4c07f9f0 con 0x7f3b58073230 2024-04-20T21:43:15.506 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:15.506 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:43:14.188606+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:rejoin","state_seq":398,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:43:15.506 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 91 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b3c06c8f0 msgr2=0x7f3b3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b3c06c8f0 0x7f3b3c06edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b58148330 tx=0x7f3b48009190 comp rx=0 tx=0).stop 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58073230 msgr2=0x7f3b581e5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58073230 0x7f3b581e5cb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009960 tx=0x7f3b4c004730 comp rx=0 tx=0).stop 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 shutdown_connections 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b58155560 0x7f3b5814a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b580c5cc0 0x7f3b58149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f3b3c06c8f0 0x7f3b3c06edb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 --2- 172.21.15.102:0/2611708385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b58073230 0x7f3b581e5cb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.509+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 >> 172.21.15.102:0/2611708385 conn(0x7f3b5806cd40 msgr2=0x7f3b580b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.510+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 shutdown_connections 2024-04-20T21:43:15.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:15.510+0000 7f3b60800640 1 -- 172.21.15.102:0/2611708385 wait complete. 2024-04-20T21:43:15.527 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:43:16.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~74s 2024-04-20T21:43:17.528 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:17.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.738+0000 7f447ac00640 1 Processor -- start 2024-04-20T21:43:17.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.738+0000 7f447ac00640 1 -- start start 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f447ac00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f44740c9eb0 0x7f44740c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f447ac00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4474073a10 0x7f4474158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f447ac00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44741536d0 con 0x7f447414fa50 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f447ac00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f4474159500 con 0x7f44740c9eb0 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f447ac00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f44740c76c0 con 0x7f4474073a10 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f446b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f44740c9eb0 0x7f44740c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f446b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f44740c9eb0 0x7f44740c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47374/0 (socket says 172.21.15.102:47374) 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f446b400640 1 -- 172.21.15.102:0/2652034867 learned_addr learned my addr 172.21.15.102:0/2652034867 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.740+0000 7f4478c00640 1 --2- 172.21.15.102:0/2652034867 >> v2:172.21.15.193:3300/0 conn(0x7f4474073a10 0x7f4474158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.741+0000 7f446b400640 1 -- 172.21.15.102:0/2652034867 >> v2:172.21.15.193:3300/0 conn(0x7f4474073a10 msgr2=0x7f4474158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.741+0000 7f446b400640 1 --2- 172.21.15.102:0/2652034867 >> v2:172.21.15.193:3300/0 conn(0x7f4474073a10 0x7f4474158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.741+0000 7f446b400640 1 -- 172.21.15.102:0/2652034867 >> v1:172.21.15.193:6789/0 conn(0x7f447414fa50 legacy=0x7f44740c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.741+0000 7f446b400640 1 -- 172.21.15.102:0/2652034867 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44740c7960 con 0x7f44740c9eb0 2024-04-20T21:43:17.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.742+0000 7f446b400640 1 --2- 172.21.15.102:0/2652034867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44740c9eb0 0x7f44740c4830 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4474156b00 tx=0x7f445c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6303da71981b377 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.742+0000 7f4473400640 1 -- 172.21.15.102:0/2652034867 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c02f850 con 0x7f44740c9eb0 2024-04-20T21:43:17.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.742+0000 7f4473400640 1 -- 172.21.15.102:0/2652034867 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445c02f9f0 con 0x7f44740c9eb0 2024-04-20T21:43:17.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.742+0000 7f4473400640 1 -- 172.21.15.102:0/2652034867 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c02fdc0 con 0x7f44740c9eb0 2024-04-20T21:43:17.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.743+0000 7f447ac00640 1 -- 172.21.15.102:0/2652034867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44740c9eb0 msgr2=0x7f44740c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.743+0000 7f447ac00640 1 --2- 172.21.15.102:0/2652034867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44740c9eb0 0x7f44740c4830 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4474156b00 tx=0x7f445c02ba30 comp rx=0 tx=0).stop 2024-04-20T21:43:17.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.743+0000 7f447ac00640 1 -- 172.21.15.102:0/2652034867 shutdown_connections 2024-04-20T21:43:17.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.743+0000 7f447ac00640 1 --2- 172.21.15.102:0/2652034867 >> v2:172.21.15.193:3300/0 conn(0x7f4474073a10 0x7f4474158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.743+0000 7f447ac00640 1 --2- 172.21.15.102:0/2652034867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44740c9eb0 0x7f44740c4830 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.743+0000 7f447ac00640 1 -- 172.21.15.102:0/2652034867 >> 172.21.15.102:0/2652034867 conn(0x7f44740c9390 msgr2=0x7f44740c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.743+0000 7f447ac00640 1 -- 172.21.15.102:0/2652034867 shutdown_connections 2024-04-20T21:43:17.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.743+0000 7f447ac00640 1 -- 172.21.15.102:0/2652034867 wait complete. 2024-04-20T21:43:17.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.745+0000 7f447ac00640 1 Processor -- start 2024-04-20T21:43:17.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.745+0000 7f447ac00640 1 -- start start 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f447ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4474073a10 0x7f447415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f447ac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44740c9eb0 0x7f4474145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f447ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f447414fa50 0x7f4474146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f447ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f44740c7c00 con 0x7f447414fa50 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f447ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44740c3d90 con 0x7f4474073a10 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f447ac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f44740c4030 con 0x7f44740c9eb0 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f4478c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f447414fa50 0x7f4474146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f4478c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f447414fa50 0x7f4474146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47376/0 (socket says 172.21.15.102:47376) 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f4478c00640 1 -- 172.21.15.102:0/3427263379 learned_addr learned my addr 172.21.15.102:0/3427263379 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f446b400640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44740c9eb0 0x7f4474145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.746+0000 7f4473e00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4474073a10 0x7f447415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f446b400640 1 -- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4474073a10 msgr2=0x7f447415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f446b400640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4474073a10 0x7f447415bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f446b400640 1 -- 172.21.15.102:0/3427263379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f447414fa50 msgr2=0x7f4474146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f446b400640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f447414fa50 0x7f4474146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f446b400640 1 -- 172.21.15.102:0/3427263379 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4474159190 con 0x7f44740c9eb0 2024-04-20T21:43:17.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f4478c00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f447414fa50 0x7f4474146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:17.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f446b400640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44740c9eb0 0x7f4474145d30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f445c033000 tx=0x7f445c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f4470c00640 1 -- 172.21.15.102:0/3427263379 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c037bc0 con 0x7f44740c9eb0 2024-04-20T21:43:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f4470c00640 1 -- 172.21.15.102:0/3427263379 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445c046070 con 0x7f44740c9eb0 2024-04-20T21:43:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f4470c00640 1 -- 172.21.15.102:0/3427263379 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c041400 con 0x7f44740c9eb0 2024-04-20T21:43:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.747+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447415a3d0 con 0x7f44740c9eb0 2024-04-20T21:43:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.748+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4474156c10 con 0x7f44740c9eb0 2024-04-20T21:43:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.749+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4474074340 con 0x7f44740c9eb0 2024-04-20T21:43:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.750+0000 7f4470c00640 1 -- 172.21.15.102:0/3427263379 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f445c040070 con 0x7f44740c9eb0 2024-04-20T21:43:17.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.750+0000 7f4470c00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.751+0000 7f4470c00640 1 -- 172.21.15.102:0/3427263379 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7f445c0ae9e0 con 0x7f44740c9eb0 2024-04-20T21:43:17.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.751+0000 7f4473e00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.751+0000 7f4473e00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f44640012e0 tx=0x7f4464009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.755+0000 7f4470c00640 1 -- 172.21.15.102:0/3427263379 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f445c033030 con 0x7f44740c9eb0 2024-04-20T21:43:17.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:17.999+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44740c7320 con 0x7f44740c9eb0 2024-04-20T21:43:17.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.000+0000 7f4470c00640 1 -- 172.21.15.102:0/3427263379 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5201 (secure 0 0 0) 0x7f445c07be20 con 0x7f44740c9eb0 2024-04-20T21:43:17.999 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:17.999 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:43:14.188606+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:rejoin","state_seq":398,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:43:17.999 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 91 2024-04-20T21:43:18.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 msgr2=0x7f445006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:18.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f44640012e0 tx=0x7f4464009040 comp rx=0 tx=0).stop 2024-04-20T21:43:18.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44740c9eb0 msgr2=0x7f4474145d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:18.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44740c9eb0 0x7f4474145d30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f445c033000 tx=0x7f445c004730 comp rx=0 tx=0).stop 2024-04-20T21:43:18.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 shutdown_connections 2024-04-20T21:43:18.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f447414fa50 0x7f4474146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:18.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44740c9eb0 0x7f4474145d30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:18.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f445006c920 0x7f445006ede0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:18.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 --2- 172.21.15.102:0/3427263379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4474073a10 0x7f447415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:18.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 >> 172.21.15.102:0/3427263379 conn(0x7f44740c9390 msgr2=0x7f44740c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:18.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 shutdown_connections 2024-04-20T21:43:18.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:18.003+0000 7f447ac00640 1 -- 172.21.15.102:0/3427263379 wait complete. 2024-04-20T21:43:18.021 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:43:20.022 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:20.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.230+0000 7f9504c00640 1 Processor -- start 2024-04-20T21:43:20.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.230+0000 7f9504c00640 1 -- start start 2024-04-20T21:43:20.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f9504c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f95001618a0 0x7f9500163c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:20.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f9504c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9500158e80 con 0x7f95000c8db0 2024-04-20T21:43:20.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f9504c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95000c91c0 con 0x7f9500152170 2024-04-20T21:43:20.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f9504c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f95000c6dc0 con 0x7f95001618a0 2024-04-20T21:43:20.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f94fe800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f95001618a0 0x7f9500163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:20.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f94f5400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f95000c8db0 0x7f9500070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49528/0 (socket says 172.21.15.102:49528) 2024-04-20T21:43:20.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f94f5400640 1 -- 172.21.15.102:0/1534437889 learned_addr learned my addr 172.21.15.102:0/1534437889 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:43:20.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f94fe800640 1 -- 172.21.15.102:0/1534437889 >> v1:172.21.15.193:6789/0 conn(0x7f9500152170 legacy=0x7f9500158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:20.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f94fe800640 1 -- 172.21.15.102:0/1534437889 >> v1:172.21.15.102:6789/0 conn(0x7f95000c8db0 legacy=0x7f9500070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:20.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f94fd400640 1 -- 172.21.15.102:0/1534437889 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551420745 0 0) 0x7f9500158e80 con 0x7f95000c8db0 2024-04-20T21:43:20.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.232+0000 7f94fe800640 1 -- 172.21.15.102:0/1534437889 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9500071f70 con 0x7f95001618a0 2024-04-20T21:43:20.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.234+0000 7f94fe800640 1 --2- 172.21.15.102:0/1534437889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 0x7f9500163c90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009d90 tx=0x7f94f802bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=589e11c680cb1860 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:20.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.234+0000 7f94fd400640 1 -- 172.21.15.102:0/1534437889 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8039070 con 0x7f95001618a0 2024-04-20T21:43:20.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.234+0000 7f94fd400640 1 -- 172.21.15.102:0/1534437889 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f802cae0 con 0x7f95001618a0 2024-04-20T21:43:20.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.234+0000 7f94fd400640 1 -- 172.21.15.102:0/1534437889 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8034440 con 0x7f95001618a0 2024-04-20T21:43:20.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.234+0000 7f9504c00640 1 -- 172.21.15.102:0/1534437889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 msgr2=0x7f9500163c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:20.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.234+0000 7f9504c00640 1 --2- 172.21.15.102:0/1534437889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 0x7f9500163c90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009d90 tx=0x7f94f802bdb0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.235+0000 7f9504c00640 1 -- 172.21.15.102:0/1534437889 shutdown_connections 2024-04-20T21:43:20.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.235+0000 7f9504c00640 1 --2- 172.21.15.102:0/1534437889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 0x7f9500163c90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.235+0000 7f9504c00640 1 -- 172.21.15.102:0/1534437889 >> 172.21.15.102:0/1534437889 conn(0x7f95000c9390 msgr2=0x7f95000c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:20.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.235+0000 7f9504c00640 1 -- 172.21.15.102:0/1534437889 shutdown_connections 2024-04-20T21:43:20.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.235+0000 7f9504c00640 1 -- 172.21.15.102:0/1534437889 wait complete. 2024-04-20T21:43:20.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.236+0000 7f9504c00640 1 Processor -- start 2024-04-20T21:43:20.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.236+0000 7f9504c00640 1 -- start start 2024-04-20T21:43:20.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.237+0000 7f9504c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95000c8db0 0x7f95001ef230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.237+0000 7f9504c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9500152170 0x7f95001f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.237+0000 7f9504c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 0x7f95000cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.237+0000 7f9504c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f95000724b0 con 0x7f95000c8db0 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.237+0000 7f9504c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9500072210 con 0x7f95001618a0 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.237+0000 7f9504c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9500158250 con 0x7f9500152170 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95000c8db0 0x7f95001ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95000c8db0 0x7f95001ef230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39504/0 (socket says 172.21.15.102:39504) 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 -- 172.21.15.102:0/2809307252 learned_addr learned my addr 172.21.15.102:0/2809307252 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fe800640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 0x7f95000cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94f5400640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9500152170 0x7f95001f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 -- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9500152170 msgr2=0x7f95001f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:20.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9500152170 0x7f95001f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 -- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 msgr2=0x7f95000cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:20.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 0x7f95000cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 -- 172.21.15.102:0/2809307252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95001526f0 con 0x7f95000c8db0 2024-04-20T21:43:20.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fe800640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 0x7f95000cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:20.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.238+0000 7f94fde00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95000c8db0 0x7f95001ef230 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f94f000c7e0 tx=0x7f94f000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:20.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.239+0000 7f94f6a00640 1 -- 172.21.15.102:0/2809307252 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0017070 con 0x7f95000c8db0 2024-04-20T21:43:20.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.239+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9500152990 con 0x7f95000c8db0 2024-04-20T21:43:20.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.239+0000 7f94f6a00640 1 -- 172.21.15.102:0/2809307252 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f0005270 con 0x7f95000c8db0 2024-04-20T21:43:20.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.239+0000 7f94f6a00640 1 -- 172.21.15.102:0/2809307252 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f000a640 con 0x7f95000c8db0 2024-04-20T21:43:20.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.239+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95001f17f0 con 0x7f95000c8db0 2024-04-20T21:43:20.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.241+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9500074340 con 0x7f95000c8db0 2024-04-20T21:43:20.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.242+0000 7f94f6a00640 1 -- 172.21.15.102:0/2809307252 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94f001a070 con 0x7f95000c8db0 2024-04-20T21:43:20.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.243+0000 7f94f6a00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f94dc06c920 0x7f94dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:20.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.243+0000 7f94f6a00640 1 -- 172.21.15.102:0/2809307252 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7f94f0056e50 con 0x7f95000c8db0 2024-04-20T21:43:20.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.243+0000 7f94f5400640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f94dc06c920 0x7f94dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:20.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.244+0000 7f94f5400640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f94dc06c920 0x7f94dc06ede0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95000cc820 tx=0x7f94e8007e40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:20.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.248+0000 7f94f6a00640 1 -- 172.21.15.102:0/2809307252 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f94f0058db0 con 0x7f95000c8db0 2024-04-20T21:43:20.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.482+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95000c3460 con 0x7f95000c8db0 2024-04-20T21:43:20.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.483+0000 7f94f6a00640 1 -- 172.21.15.102:0/2809307252 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5208 (secure 0 0 0) 0x7f94f005d0b0 con 0x7f95000c8db0 2024-04-20T21:43:20.482 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:20.482 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:43:20.240411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":400,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T21:43:20.483 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 92 2024-04-20T21:43:20.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.485+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f94dc06c920 msgr2=0x7f94dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.485+0000 7f9504c00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f94dc06c920 0x7f94dc06ede0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95000cc820 tx=0x7f94e8007e40 comp rx=0 tx=0).stop 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.485+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95000c8db0 msgr2=0x7f95001ef230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.485+0000 7f9504c00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95000c8db0 0x7f95001ef230 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f94f000c7e0 tx=0x7f94f000cca0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.485+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 shutdown_connections 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.485+0000 7f9504c00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f94dc06c920 0x7f94dc06ede0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.485+0000 7f9504c00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95001618a0 0x7f95000cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.486+0000 7f9504c00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9500152170 0x7f95001f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.486+0000 7f9504c00640 1 --2- 172.21.15.102:0/2809307252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95000c8db0 0x7f95001ef230 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.486+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 >> 172.21.15.102:0/2809307252 conn(0x7f95000c9390 msgr2=0x7f95000731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.486+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 shutdown_connections 2024-04-20T21:43:20.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:20.486+0000 7f9504c00640 1 -- 172.21.15.102:0/2809307252 wait complete. 2024-04-20T21:43:20.503 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=0, replacing gid=6738 2024-04-20T21:43:20.503 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.c 2024-04-20T21:43:23.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~81s 2024-04-20T21:43:24.504 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:43:24.504 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:43:24.504 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:43:24.512 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:43:24.512 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:24.660 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:43:24.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.693+0000 7f8ddde00640 1 Processor -- start 2024-04-20T21:43:24.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.694+0000 7f8ddde00640 1 -- start start 2024-04-20T21:43:24.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8ddde00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8dd8162e40 0x7f8dd806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:24.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8ddde00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163660 0x7f8dd80c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:24.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8ddde00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd806d2e0 con 0x7f8dd8163220 2024-04-20T21:43:24.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8ddde00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd80c7290 con 0x7f8dd8162e40 2024-04-20T21:43:24.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8ddde00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd80c7570 con 0x7f8dd8163660 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8dd6a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8dd8162e40 0x7f8dd806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8dd7400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163660 0x7f8dd80c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8dd6a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8dd8162e40 0x7f8dd806c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58100/0 (socket says 172.21.15.102:58100) 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.695+0000 7f8dd6a00640 1 -- 172.21.15.102:0/739416870 learned_addr learned my addr 172.21.15.102:0/739416870 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.696+0000 7f8dd6a00640 1 -- 172.21.15.102:0/739416870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163660 msgr2=0x7f8dd80c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.696+0000 7f8dd6a00640 1 --2- 172.21.15.102:0/739416870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163660 0x7f8dd80c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.696+0000 7f8dd6a00640 1 -- 172.21.15.102:0/739416870 >> v1:172.21.15.193:6789/0 conn(0x7f8dd8163220 legacy=0x7f8dd8159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.696+0000 7f8dd5600640 1 -- 172.21.15.102:0/739416870 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062361390 0 0) 0x7f8dd806d2e0 con 0x7f8dd8163220 2024-04-20T21:43:24.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.696+0000 7f8dd6a00640 1 -- 172.21.15.102:0/739416870 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd806d5c0 con 0x7f8dd8162e40 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.697+0000 7f8dd6a00640 1 --2- 172.21.15.102:0/739416870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8162e40 0x7f8dd806c9e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80c7410 tx=0x7f8dc802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47b3d523c04be43 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.697+0000 7f8dd5600640 1 -- 172.21.15.102:0/739416870 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8034070 con 0x7f8dd8162e40 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.697+0000 7f8dd5600640 1 -- 172.21.15.102:0/739416870 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc8030840 con 0x7f8dd8162e40 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.697+0000 7f8dd5600640 1 -- 172.21.15.102:0/739416870 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8030c10 con 0x7f8dd8162e40 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.698+0000 7f8ddde00640 1 -- 172.21.15.102:0/739416870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8162e40 msgr2=0x7f8dd806c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.698+0000 7f8ddde00640 1 --2- 172.21.15.102:0/739416870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8162e40 0x7f8dd806c9e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80c7410 tx=0x7f8dc802b980 comp rx=0 tx=0).stop 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.698+0000 7f8ddde00640 1 -- 172.21.15.102:0/739416870 shutdown_connections 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.698+0000 7f8ddde00640 1 --2- 172.21.15.102:0/739416870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163660 0x7f8dd80c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.698+0000 7f8ddde00640 1 --2- 172.21.15.102:0/739416870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8162e40 0x7f8dd806c9e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.698+0000 7f8ddde00640 1 -- 172.21.15.102:0/739416870 >> 172.21.15.102:0/739416870 conn(0x7f8dd80c9390 msgr2=0x7f8dd80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.698+0000 7f8ddde00640 1 -- 172.21.15.102:0/739416870 shutdown_connections 2024-04-20T21:43:24.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.698+0000 7f8ddde00640 1 -- 172.21.15.102:0/739416870 wait complete. 2024-04-20T21:43:24.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.700+0000 7f8ddde00640 1 Processor -- start 2024-04-20T21:43:24.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.700+0000 7f8ddde00640 1 -- start start 2024-04-20T21:43:24.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.700+0000 7f8ddde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dd8162e40 0x7f8dd81f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:24.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8ddde00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163220 0x7f8dd81622b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:24.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8ddde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8163660 0x7f8dd815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:24.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8ddde00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd80c9080 con 0x7f8dd8162e40 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8ddde00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd806db80 con 0x7f8dd8163660 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8ddde00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd806cf20 con 0x7f8dd8163220 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8dd6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dd8162e40 0x7f8dd81f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8dd7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8163660 0x7f8dd815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8dd6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dd8162e40 0x7f8dd81f8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39530/0 (socket says 172.21.15.102:39530) 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8dd6000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163220 0x7f8dd81622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8dd6a00640 1 -- 172.21.15.102:0/3413208116 learned_addr learned my addr 172.21.15.102:0/3413208116 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8dd7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8163660 0x7f8dd815da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58116/0 (socket says 172.21.15.102:58116) 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8dd7400640 1 -- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163220 msgr2=0x7f8dd81622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.701+0000 7f8dd7400640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163220 0x7f8dd81622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8dd7400640 1 -- 172.21.15.102:0/3413208116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dd8162e40 msgr2=0x7f8dd81f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8dd7400640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dd8162e40 0x7f8dd81f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8dd7400640 1 -- 172.21.15.102:0/3413208116 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd8159c00 con 0x7f8dd8163660 2024-04-20T21:43:24.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8dd7400640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8163660 0x7f8dd815da20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009d90 tx=0x7f8dd000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:24.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8dcf400640 1 -- 172.21.15.102:0/3413208116 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0018020 con 0x7f8dd8163660 2024-04-20T21:43:24.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8dcf400640 1 -- 172.21.15.102:0/3413208116 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd0014480 con 0x7f8dd8163660 2024-04-20T21:43:24.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8dcf400640 1 -- 172.21.15.102:0/3413208116 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0014870 con 0x7f8dd8163660 2024-04-20T21:43:24.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd8159ea0 con 0x7f8dd8163660 2024-04-20T21:43:24.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.702+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd815a460 con 0x7f8dd8163660 2024-04-20T21:43:24.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.703+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd8074340 con 0x7f8dd8163660 2024-04-20T21:43:24.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.705+0000 7f8dcf400640 1 -- 172.21.15.102:0/3413208116 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dd0005d10 con 0x7f8dd8163660 2024-04-20T21:43:24.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.705+0000 7f8dcf400640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:24.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.706+0000 7f8dcf400640 1 -- 172.21.15.102:0/3413208116 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7f8dd008d070 con 0x7f8dd8163660 2024-04-20T21:43:24.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.706+0000 7f8dd6a00640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:24.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.706+0000 7f8dd6a00640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8db806c8f0 0x7f8db806edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009630 tx=0x7f8dc803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:24.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.710+0000 7f8dcf400640 1 -- 172.21.15.102:0/3413208116 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8dd0022030 con 0x7f8dd8163660 2024-04-20T21:43:24.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.948+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dd81561c0 con 0x7f8dd8163660 2024-04-20T21:43:24.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.950+0000 7f8dcf400640 1 -- 172.21.15.102:0/3413208116 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5208 (secure 0 0 0) 0x7f8dd0010040 con 0x7f8dd8163660 2024-04-20T21:43:24.949 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:24.949 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:43:20.240411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":400,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T21:43:24.949 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 92 2024-04-20T21:43:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8db806c8f0 msgr2=0x7f8db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:24.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8db806c8f0 0x7f8db806edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009630 tx=0x7f8dc803c040 comp rx=0 tx=0).stop 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8163660 msgr2=0x7f8dd815da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8163660 0x7f8dd815da20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009d90 tx=0x7f8dd000c490 comp rx=0 tx=0).stop 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 shutdown_connections 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8db806c8f0 0x7f8db806edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dd8163660 0x7f8dd815da20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8dd8163220 0x7f8dd81622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 --2- 172.21.15.102:0/3413208116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dd8162e40 0x7f8dd81f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 >> 172.21.15.102:0/3413208116 conn(0x7f8dd80c9390 msgr2=0x7f8dd8156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.952+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 shutdown_connections 2024-04-20T21:43:24.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:24.953+0000 7f8ddde00640 1 -- 172.21.15.102:0/3413208116 wait complete. 2024-04-20T21:43:24.970 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-20T21:43:25.386 INFO:tasks.ceph.mds.b.smithi193.stderr:terminate called after throwing an instance of 'std::out_of_range' 2024-04-20T21:43:25.386 INFO:tasks.ceph.mds.b.smithi193.stderr: what(): map::at 2024-04-20T21:43:25.386 INFO:tasks.ceph.mds.b.smithi193.stderr:*** Caught signal (Aborted) ** 2024-04-20T21:43:25.386 INFO:tasks.ceph.mds.b.smithi193.stderr: in thread 7f50c2895640 thread_name:MR_Finisher 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f50ce23e6f0] 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f50ce28b94c] 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 3: raise() 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 4: abort() 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7f50ce6a1b21] 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7f50ce6ad52c] 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7f50ce6ad597] 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7f50ce6ad7f9] 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7f50ce6a4533] 2024-04-20T21:43:25.387 INFO:tasks.ceph.mds.b.smithi193.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x55f2dccb1d86] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x55f2dcdb8cf7] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x55f2dcdf998f] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x55f2dcd6bfef] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x706) [0x55f2dcd4da76] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x55f2dcd46ac6] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55f2dcd47411] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 17: (Locker::scatter_writebehind_finish(ScatterLock*, boost::intrusive_ptr&)+0xbf) [0x55f2dcd6b42f] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55f2dce4d2ac] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55f2dce4d6fc] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 20: (MDSLogContextBase::complete(int)+0x53) [0x55f2dce4d853] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 21: (Finisher::finisher_thread_entry()+0x175) [0x7f50cec30455] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 22: /lib64/libc.so.6(+0x89c02) [0x7f50ce289c02] 2024-04-20T21:43:25.388 INFO:tasks.ceph.mds.b.smithi193.stderr: 23: /lib64/libc.so.6(+0x10ec40) [0x7f50ce30ec40] 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr:2024-04-20T21:43:25.389+0000 7f50c2895640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: in thread 7f50c2895640 thread_name:MR_Finisher 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f50ce23e6f0] 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f50ce28b94c] 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 3: raise() 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 4: abort() 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7f50ce6a1b21] 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7f50ce6ad52c] 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7f50ce6ad597] 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7f50ce6ad7f9] 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7f50ce6a4533] 2024-04-20T21:43:25.389 INFO:tasks.ceph.mds.b.smithi193.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x55f2dccb1d86] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x55f2dcdb8cf7] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x55f2dcdf998f] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x55f2dcd6bfef] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x706) [0x55f2dcd4da76] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x55f2dcd46ac6] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55f2dcd47411] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 17: (Locker::scatter_writebehind_finish(ScatterLock*, boost::intrusive_ptr&)+0xbf) [0x55f2dcd6b42f] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55f2dce4d2ac] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55f2dce4d6fc] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 20: (MDSLogContextBase::complete(int)+0x53) [0x55f2dce4d853] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 21: (Finisher::finisher_thread_entry()+0x175) [0x7f50cec30455] 2024-04-20T21:43:25.390 INFO:tasks.ceph.mds.b.smithi193.stderr: 22: /lib64/libc.so.6(+0x89c02) [0x7f50ce289c02] 2024-04-20T21:43:25.391 INFO:tasks.ceph.mds.b.smithi193.stderr: 23: /lib64/libc.so.6(+0x10ec40) [0x7f50ce30ec40] 2024-04-20T21:43:25.391 INFO:tasks.ceph.mds.b.smithi193.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:43:25.391 INFO:tasks.ceph.mds.b.smithi193.stderr: 2024-04-20T21:43:25.406 INFO:tasks.ceph.mds.b.smithi193.stderr: 0> 2024-04-20T21:43:25.389+0000 7f50c2895640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:43:25.406 INFO:tasks.ceph.mds.b.smithi193.stderr: in thread 7f50c2895640 thread_name:MR_Finisher 2024-04-20T21:43:25.406 INFO:tasks.ceph.mds.b.smithi193.stderr: 2024-04-20T21:43:25.406 INFO:tasks.ceph.mds.b.smithi193.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f50ce23e6f0] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f50ce28b94c] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 3: raise() 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 4: abort() 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7f50ce6a1b21] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7f50ce6ad52c] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7f50ce6ad597] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7f50ce6ad7f9] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7f50ce6a4533] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x55f2dccb1d86] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x55f2dcdb8cf7] 2024-04-20T21:43:25.407 INFO:tasks.ceph.mds.b.smithi193.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x55f2dcdf998f] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x55f2dcd6bfef] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x706) [0x55f2dcd4da76] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x55f2dcd46ac6] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55f2dcd47411] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 17: (Locker::scatter_writebehind_finish(ScatterLock*, boost::intrusive_ptr&)+0xbf) [0x55f2dcd6b42f] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55f2dce4d2ac] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55f2dce4d6fc] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 20: (MDSLogContextBase::complete(int)+0x53) [0x55f2dce4d853] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 21: (Finisher::finisher_thread_entry()+0x175) [0x7f50cec30455] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 22: /lib64/libc.so.6(+0x89c02) [0x7f50ce289c02] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 23: /lib64/libc.so.6(+0x10ec40) [0x7f50ce30ec40] 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:43:25.408 INFO:tasks.ceph.mds.b.smithi193.stderr: 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: -54> 2024-04-20T21:43:25.389+0000 7f50c2895640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: in thread 7f50c2895640 thread_name:MR_Finisher 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f50ce23e6f0] 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f50ce28b94c] 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: 3: raise() 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: 4: abort() 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7f50ce6a1b21] 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7f50ce6ad52c] 2024-04-20T21:43:25.420 INFO:tasks.ceph.mds.b.smithi193.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7f50ce6ad597] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7f50ce6ad7f9] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7f50ce6a4533] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x55f2dccb1d86] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x55f2dcdb8cf7] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x55f2dcdf998f] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x55f2dcd6bfef] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x706) [0x55f2dcd4da76] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x55f2dcd46ac6] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55f2dcd47411] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 17: (Locker::scatter_writebehind_finish(ScatterLock*, boost::intrusive_ptr&)+0xbf) [0x55f2dcd6b42f] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55f2dce4d2ac] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55f2dce4d6fc] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 20: (MDSLogContextBase::complete(int)+0x53) [0x55f2dce4d853] 2024-04-20T21:43:25.421 INFO:tasks.ceph.mds.b.smithi193.stderr: 21: (Finisher::finisher_thread_entry()+0x175) [0x7f50cec30455] 2024-04-20T21:43:25.422 INFO:tasks.ceph.mds.b.smithi193.stderr: 22: /lib64/libc.so.6(+0x89c02) [0x7f50ce289c02] 2024-04-20T21:43:25.422 INFO:tasks.ceph.mds.b.smithi193.stderr: 23: /lib64/libc.so.6(+0x10ec40) [0x7f50ce30ec40] 2024-04-20T21:43:25.422 INFO:tasks.ceph.mds.b.smithi193.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:43:25.422 INFO:tasks.ceph.mds.b.smithi193.stderr: 2024-04-20T21:43:26.350 INFO:tasks.ceph.mds.b.smithi193.stderr:daemon-helper: command crashed with signal 6 2024-04-20T21:43:26.971 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:27.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.184+0000 7fefd8a00640 1 Processor -- start 2024-04-20T21:43:27.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.184+0000 7fefd8a00640 1 -- start start 2024-04-20T21:43:27.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.185+0000 7fefd8a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fefd4153180 0x7fefd4158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:27.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.185+0000 7fefd8a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd4153560 con 0x7fefd4159380 2024-04-20T21:43:27.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd8a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd40c7a70 con 0x7fefd40c75a0 2024-04-20T21:43:27.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd8a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fefd40c7d30 con 0x7fefd4153180 2024-04-20T21:43:27.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fefd4153180 0x7fefd4158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:27.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd2a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fefd4159380 0x7fefd40c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49534/0 (socket says 172.21.15.102:49534) 2024-04-20T21:43:27.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd2a00640 1 -- 172.21.15.102:0/1763733808 learned_addr learned my addr 172.21.15.102:0/1763733808 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:43:27.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd3400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fefd4153180 0x7fefd4158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39538/0 (socket says 172.21.15.102:39538) 2024-04-20T21:43:27.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd3400640 1 -- 172.21.15.102:0/1763733808 >> v1:172.21.15.193:6789/0 conn(0x7fefd40c75a0 legacy=0x7fefd41637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:27.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd3400640 1 -- 172.21.15.102:0/1763733808 >> v1:172.21.15.102:6789/0 conn(0x7fefd4159380 legacy=0x7fefd40c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:27.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.186+0000 7fefd2000640 1 -- 172.21.15.102:0/1763733808 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310463019 0 0) 0x7fefd40c7a70 con 0x7fefd40c75a0 2024-04-20T21:43:27.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.187+0000 7fefd3400640 1 -- 172.21.15.102:0/1763733808 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd415c4d0 con 0x7fefd4153180 2024-04-20T21:43:27.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.188+0000 7fefd3400640 1 --2- 172.21.15.102:0/1763733808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd4158e40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2ebc54ede45e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:27.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.188+0000 7fefd2000640 1 -- 172.21.15.102:0/1763733808 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4039070 con 0x7fefd4153180 2024-04-20T21:43:27.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.188+0000 7fefd2000640 1 -- 172.21.15.102:0/1763733808 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc402cab0 con 0x7fefd4153180 2024-04-20T21:43:27.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.188+0000 7fefd2000640 1 -- 172.21.15.102:0/1763733808 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4034440 con 0x7fefd4153180 2024-04-20T21:43:27.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.189+0000 7fefd8a00640 1 -- 172.21.15.102:0/1763733808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 msgr2=0x7fefd4158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:27.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.189+0000 7fefd8a00640 1 --2- 172.21.15.102:0/1763733808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd4158e40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc402bd80 comp rx=0 tx=0).stop 2024-04-20T21:43:27.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.189+0000 7fefd8a00640 1 -- 172.21.15.102:0/1763733808 shutdown_connections 2024-04-20T21:43:27.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.189+0000 7fefd8a00640 1 --2- 172.21.15.102:0/1763733808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd4158e40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:27.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.189+0000 7fefd8a00640 1 -- 172.21.15.102:0/1763733808 >> 172.21.15.102:0/1763733808 conn(0x7fefd40cc820 msgr2=0x7fefd40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:27.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.189+0000 7fefd8a00640 1 -- 172.21.15.102:0/1763733808 shutdown_connections 2024-04-20T21:43:27.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.189+0000 7fefd8a00640 1 -- 172.21.15.102:0/1763733808 wait complete. 2024-04-20T21:43:27.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.191+0000 7fefd8a00640 1 Processor -- start 2024-04-20T21:43:27.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.191+0000 7fefd8a00640 1 -- start start 2024-04-20T21:43:27.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd8a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefd40c75a0 0x7fefd40cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:27.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd41582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:27.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefd4159380 0x7fefd41f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:27.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fefd415d290 con 0x7fefd4153180 2024-04-20T21:43:27.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefd415ca90 con 0x7fefd4159380 2024-04-20T21:43:27.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd8a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fefd40c6be0 con 0x7fefd40c75a0 2024-04-20T21:43:27.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd41582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd41582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39546/0 (socket says 172.21.15.102:39546) 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd2a00640 1 -- 172.21.15.102:0/4232423173 learned_addr learned my addr 172.21.15.102:0/4232423173 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd3e00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefd4159380 0x7fefd41f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.192+0000 7fefd3400640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefd40c75a0 0x7fefd40cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefd2a00640 1 -- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefd40c75a0 msgr2=0x7fefd40cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefd2a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefd40c75a0 0x7fefd40cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefd2a00640 1 -- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefd4159380 msgr2=0x7fefd41f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefd2a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefd4159380 0x7fefd41f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefd2a00640 1 -- 172.21.15.102:0/4232423173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd4158830 con 0x7fefd4153180 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefd2a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd41582f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c7b0 tx=0x7fefbc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefcb400640 1 -- 172.21.15.102:0/4232423173 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc018070 con 0x7fefd4153180 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefcb400640 1 -- 172.21.15.102:0/4232423173 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc0072d0 con 0x7fefd4153180 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd40c6510 con 0x7fefd4153180 2024-04-20T21:43:27.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefcb400640 1 -- 172.21.15.102:0/4232423173 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0047e0 con 0x7fefd4153180 2024-04-20T21:43:27.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.193+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd41f48f0 con 0x7fefd4153180 2024-04-20T21:43:27.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.195+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd404fd60 con 0x7fefd4153180 2024-04-20T21:43:27.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.196+0000 7fefcb400640 1 -- 172.21.15.102:0/4232423173 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefbc01b070 con 0x7fefd4153180 2024-04-20T21:43:27.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.196+0000 7fefcb400640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefb006c920 0x7fefb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:27.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.197+0000 7fefcb400640 1 -- 172.21.15.102:0/4232423173 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 8025+0+0 (secure 0 0 0) 0x7fefbc08b920 con 0x7fefd4153180 2024-04-20T21:43:27.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.197+0000 7fefd3400640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefb006c920 0x7fefb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:27.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.198+0000 7fefd3400640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefb006c920 0x7fefb006ede0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fefc4036000 tx=0x7fefc4030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:27.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.201+0000 7fefcb400640 1 -- 172.21.15.102:0/4232423173 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fefbc058cc0 con 0x7fefd4153180 2024-04-20T21:43:27.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.441+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefd40c3690 con 0x7fefd4153180 2024-04-20T21:43:27.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.442+0000 7fefcb400640 1 -- 172.21.15.102:0/4232423173 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+6036 (secure 0 0 0) 0x7fefbc05cfc0 con 0x7fefd4153180 2024-04-20T21:43:27.441 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:43:27.442 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2976263880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2976263880},{"type":"v1","addr":"172.21.15.102:6833","nonce":2976263880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:43:20.240411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":400,"addr":"172.21.15.193:6839/2076039362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2076039362},{"type":"v1","addr":"172.21.15.193:6839","nonce":2076039362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T21:43:27.442 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 93 2024-04-20T21:43:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefb006c920 msgr2=0x7fefb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefb006c920 0x7fefb006ede0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fefc4036000 tx=0x7fefc4030000 comp rx=0 tx=0).stop 2024-04-20T21:43:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 msgr2=0x7fefd41582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd41582f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c7b0 tx=0x7fefbc00cc70 comp rx=0 tx=0).stop 2024-04-20T21:43:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 shutdown_connections 2024-04-20T21:43:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fefb006c920 0x7fefb006ede0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefd4159380 0x7fefd41f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:27.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefd4153180 0x7fefd41582f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:27.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 --2- 172.21.15.102:0/4232423173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefd40c75a0 0x7fefd40cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:27.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 >> 172.21.15.102:0/4232423173 conn(0x7fefd40cc820 msgr2=0x7fefd4150bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:27.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 shutdown_connections 2024-04-20T21:43:27.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:43:27.445+0000 7fefd8a00640 1 -- 172.21.15.102:0/4232423173 wait complete. 2024-04-20T21:43:27.462 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:43:27.462 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 88 secs before thrashing 2024-04-20T21:43:29.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~88s 2024-04-20T21:43:29.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-20T21:43:31.477 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:43:31.490 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:43:36.570 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~95s 2024-04-20T21:43:36.570 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2024-04-20T21:43:43.283 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~101s 2024-04-20T21:43:43.283 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2024-04-20T21:43:49.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~108s 2024-04-20T21:43:49.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~20s 2024-04-20T21:43:56.710 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~115s 2024-04-20T21:43:56.711 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~27s 2024-04-20T21:44:01.603 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:44:01.611 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:44:03.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~121s 2024-04-20T21:44:03.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~34s 2024-04-20T21:44:10.139 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~128s 2024-04-20T21:44:10.139 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~40s 2024-04-20T21:44:10.279 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:44:10.280+0000 7f125e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:44:13.877 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 25% (137/537) Updating files: 26% (140/537) Updating files: 27% (145/537) Updating files: 28% (151/537) Updating files: 29% (156/537) Updating files: 30% (162/537) Updating files: 31% (167/537) Updating files: 32% (172/537) Updating files: 33% (178/537) Updating files: 34% (183/537) Updating files: 35% (188/537) Updating files: 36% (194/537) Updating files: 37% (199/537) Updating files: 38% (205/537) Updating files: 39% (210/537) Updating files: 40% (215/537) Updating files: 41% (221/537) Updating files: 42% (226/537) Updating files: 43% (231/537) Updating files: 43% (233/537) Updating files: 43% (235/537) Updating files: 43% (236/537) Updating files: 44% (237/537) Updating files: 45% (242/537) Updating files: 46% (248/537) Updating files: 46% (251/537) Updating files: 47% (253/537) Updating files: 48% (258/537) Updating files: 49% (264/537) Updating files: 50% (269/537) Updating files: 51% (274/537) Updating files: 52% (280/537) Updating files: 53% (285/537) Updating files: 54% (290/537) Updating files: 55% (296/537) Updating files: 56% (301/537) Updating files: 57% (307/537) Updating files: 57% (308/537) Updating files: 58% (312/537) Updating files: 59% (317/537) Updating files: 60% (323/537) Updating files: 61% (328/537) Updating files: 62% (333/537) Updating files: 62% (337/537) Updating files: 63% (339/537) Updating files: 64% (344/537) Updating files: 65% (350/537) Updating files: 66% (355/537) Updating files: 67% (360/537) Updating files: 68% (366/537) Updating files: 69% (371/537) Updating files: 70% (376/537) Updating files: 71% (382/537) Updating files: 72% (387/537) Updating files: 73% (393/537) Updating files: 74% (398/537) Updating files: 75% (403/537) Updating files: 75% (407/537) Updating files: 76% (409/537) Updating files: 77% (414/537) Updating files: 78% (419/537) Updating files: 79% (425/537) Updating files: 80% (430/537) Updating files: 81% (435/537) Updating files: 82% (441/537) Updating files: 83% (446/537) Updating files: 84% (452/537) Updating files: 85% (457/537) Updating files: 86% (462/537) Updating files: 87% (468/537) Updating files: 88% (473/537) Updating files: 89% (478/537) Updating files: 90% (484/537) Updating files: 91% (489/537) Updating files: 92% (495/537) Updating files: 93% (500/537) Updating files: 94% (505/537) Updating files: 95% (511/537) Updating files: 96% (516/537) Updating files: 96% (520/537) Updating files: 97% (521/537) Updating files: 98% (527/537) Updating files: 99% (532/537) Updating files: 100% (537/537) Updating files: 100% (537/537), done. 2024-04-20T21:44:14.814 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 090436f56ac v0.21 2024-04-20T21:44:14.825 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.21 2024-04-20T21:44:14.878 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:44:14.878 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 22 -eq 48 ']' 2024-04-20T21:44:14.878 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.22 2024-04-20T21:44:14.879 INFO:tasks.workunit.client.0.smithi102.stdout:v0.22 2024-04-20T21:44:14.879 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.22 2024-04-20T21:44:14.879 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.22 2024-04-20T21:44:16.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~135s 2024-04-20T21:44:16.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~47s 2024-04-20T21:44:20.693 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 27% (57/204) Updating files: 28% (58/204) Updating files: 29% (60/204) Updating files: 30% (62/204) Updating files: 31% (64/204) Updating files: 32% (66/204) Updating files: 33% (68/204) Updating files: 34% (70/204) Updating files: 35% (72/204) Updating files: 36% (74/204) Updating files: 37% (76/204) Updating files: 38% (78/204) Updating files: 39% (80/204) Updating files: 40% (82/204) Updating files: 41% (84/204) Updating files: 42% (86/204) Updating files: 43% (88/204) Updating files: 44% (90/204) Updating files: 45% (92/204) Updating files: 46% (94/204) Updating files: 47% (96/204) Updating files: 48% (98/204) Updating files: 49% (100/204) Updating files: 50% (102/204) Updating files: 51% (105/204) Updating files: 52% (107/204) Updating files: 53% (109/204) Updating files: 54% (111/204) Updating files: 55% (113/204) Updating files: 56% (115/204) Updating files: 57% (117/204) Updating files: 58% (119/204) Updating files: 59% (121/204) Updating files: 60% (123/204) Updating files: 60% (124/204) Updating files: 61% (125/204) Updating files: 62% (127/204) Updating files: 63% (129/204) Updating files: 64% (131/204) Updating files: 65% (133/204) Updating files: 66% (135/204) Updating files: 67% (137/204) Updating files: 68% (139/204) Updating files: 69% (141/204) Updating files: 70% (143/204) Updating files: 71% (145/204) Updating files: 72% (147/204) Updating files: 73% (149/204) Updating files: 74% (151/204) Updating files: 75% (153/204) Updating files: 76% (156/204) Updating files: 77% (158/204) Updating files: 78% (160/204) Updating files: 79% (162/204) Updating files: 80% (164/204) Updating files: 81% (166/204) Updating files: 82% (168/204) Updating files: 83% (170/204) Updating files: 84% (172/204) Updating files: 85% (174/204) Updating files: 86% (176/204) Updating files: 87% (178/204) Updating files: 88% (180/204) Updating files: 89% (182/204) Updating files: 90% (184/204) Updating files: 91% (186/204) Updating files: 92% (188/204) Updating files: 93% (190/204) Updating files: 93% (191/204) Updating files: 94% (192/204) Updating files: 95% (194/204) Updating files: 96% (196/204) Updating files: 97% (198/204) Updating files: 98% (200/204) Updating files: 99% (202/204) Updating files: 100% (204/204) Updating files: 100% (204/204), done. 2024-04-20T21:44:20.758 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 8a7c95f60ad v0.22 2024-04-20T21:44:20.769 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.22 2024-04-20T21:44:21.152 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:44:21.152 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 23 -eq 48 ']' 2024-04-20T21:44:21.152 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.23 2024-04-20T21:44:21.153 INFO:tasks.workunit.client.0.smithi102.stdout:v0.23 2024-04-20T21:44:21.153 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.23 2024-04-20T21:44:21.153 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.23 2024-04-20T21:44:23.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~142s 2024-04-20T21:44:23.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~54s 2024-04-20T21:44:26.046 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 22% (37/163) Updating files: 23% (38/163) Updating files: 24% (40/163) Updating files: 25% (41/163) Updating files: 26% (43/163) Updating files: 27% (45/163) Updating files: 28% (46/163) Updating files: 29% (48/163) Updating files: 30% (49/163) Updating files: 31% (51/163) Updating files: 32% (53/163) Updating files: 33% (54/163) Updating files: 34% (56/163) Updating files: 35% (58/163) Updating files: 36% (59/163) Updating files: 37% (61/163) Updating files: 38% (62/163) Updating files: 39% (64/163) Updating files: 40% (66/163) Updating files: 41% (67/163) Updating files: 42% (69/163) Updating files: 43% (71/163) Updating files: 44% (72/163) Updating files: 45% (74/163) Updating files: 46% (75/163) Updating files: 47% (77/163) Updating files: 48% (79/163) Updating files: 49% (80/163) Updating files: 50% (82/163) Updating files: 51% (84/163) Updating files: 52% (85/163) Updating files: 53% (87/163) Updating files: 54% (89/163) Updating files: 55% (90/163) Updating files: 56% (92/163) Updating files: 57% (93/163) Updating files: 58% (95/163) Updating files: 59% (97/163) Updating files: 60% (98/163) Updating files: 61% (100/163) Updating files: 62% (102/163) Updating files: 63% (103/163) Updating files: 64% (105/163) Updating files: 65% (106/163) Updating files: 66% (108/163) Updating files: 67% (110/163) Updating files: 68% (111/163) Updating files: 69% (113/163) Updating files: 70% (115/163) Updating files: 71% (116/163) Updating files: 72% (118/163) Updating files: 73% (119/163) Updating files: 74% (121/163) Updating files: 75% (123/163) Updating files: 76% (124/163) Updating files: 77% (126/163) Updating files: 78% (128/163) Updating files: 79% (129/163) Updating files: 80% (131/163) Updating files: 81% (133/163) Updating files: 82% (134/163) Updating files: 83% (136/163) Updating files: 84% (137/163) Updating files: 85% (139/163) Updating files: 86% (141/163) Updating files: 87% (142/163) Updating files: 88% (144/163) Updating files: 89% (146/163) Updating files: 90% (147/163) Updating files: 91% (149/163) Updating files: 92% (150/163) Updating files: 93% (152/163) Updating files: 94% (154/163) Updating files: 95% (155/163) Updating files: 96% (157/163) Updating files: 97% (159/163) Updating files: 98% (160/163) Updating files: 99% (162/163) Updating files: 100% (163/163) Updating files: 100% (163/163), done. 2024-04-20T21:44:26.077 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 5d1d8d0c460 v0.23 2024-04-20T21:44:26.083 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.23 2024-04-20T21:44:26.215 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:44:26.215 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 24 -eq 48 ']' 2024-04-20T21:44:26.215 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.24 2024-04-20T21:44:26.216 INFO:tasks.workunit.client.0.smithi102.stdout:v0.24 2024-04-20T21:44:26.216 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.24 2024-04-20T21:44:26.216 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.24 2024-04-20T21:44:28.300 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 47% (87/185) Updating files: 48% (89/185) Updating files: 49% (91/185) Updating files: 50% (93/185) Updating files: 51% (95/185) Updating files: 52% (97/185) Updating files: 53% (99/185) Updating files: 54% (100/185) Updating files: 55% (102/185) Updating files: 56% (104/185) Updating files: 57% (106/185) Updating files: 58% (108/185) Updating files: 59% (110/185) Updating files: 60% (111/185) Updating files: 61% (113/185) Updating files: 62% (115/185) Updating files: 63% (117/185) Updating files: 64% (119/185) Updating files: 65% (121/185) Updating files: 66% (123/185) Updating files: 67% (124/185) Updating files: 68% (126/185) Updating files: 69% (128/185) Updating files: 70% (130/185) Updating files: 71% (132/185) Updating files: 72% (134/185) Updating files: 73% (136/185) Updating files: 74% (137/185) Updating files: 75% (139/185) Updating files: 76% (141/185) Updating files: 77% (143/185) Updating files: 78% (145/185) Updating files: 79% (147/185) Updating files: 80% (148/185) Updating files: 81% (150/185) Updating files: 82% (152/185) Updating files: 83% (154/185) Updating files: 84% (156/185) Updating files: 85% (158/185) Updating files: 86% (160/185) Updating files: 87% (161/185) Updating files: 88% (163/185) Updating files: 89% (165/185) Updating files: 90% (167/185) Updating files: 91% (169/185) Updating files: 92% (171/185) Updating files: 93% (173/185) Updating files: 94% (174/185) Updating files: 95% (176/185) Updating files: 95% (177/185) Updating files: 96% (178/185) Updating files: 97% (180/185) Updating files: 98% (182/185) Updating files: 99% (184/185) Updating files: 100% (185/185) Updating files: 100% (185/185), done. 2024-04-20T21:44:28.364 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 180a4176035 v0.24 2024-04-20T21:44:28.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.24 2024-04-20T21:44:28.462 INFO:tasks.workunit.client.0.smithi102.stdout:v0.25 2024-04-20T21:44:28.463 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:44:28.463 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 25 -eq 48 ']' 2024-04-20T21:44:28.463 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.25 2024-04-20T21:44:28.463 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.25 2024-04-20T21:44:28.463 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.25 2024-04-20T21:44:30.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~148s 2024-04-20T21:44:30.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~60s 2024-04-20T21:44:31.771 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:44:31.778 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:44:36.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~155s 2024-04-20T21:44:36.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~67s 2024-04-20T21:44:42.014 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 18% (118/622) Updating files: 19% (119/622) Updating files: 20% (125/622) Updating files: 21% (131/622) Updating files: 22% (137/622) Updating files: 23% (144/622) Updating files: 24% (150/622) Updating files: 25% (156/622) Updating files: 26% (162/622) Updating files: 27% (168/622) Updating files: 28% (175/622) Updating files: 29% (181/622) Updating files: 30% (187/622) Updating files: 30% (188/622) Updating files: 31% (193/622) Updating files: 32% (200/622) Updating files: 33% (206/622) Updating files: 34% (212/622) Updating files: 35% (218/622) Updating files: 36% (224/622) Updating files: 37% (231/622) Updating files: 38% (237/622) Updating files: 39% (243/622) Updating files: 40% (249/622) Updating files: 41% (256/622) Updating files: 42% (262/622) Updating files: 43% (268/622) Updating files: 44% (274/622) Updating files: 45% (280/622) Updating files: 46% (287/622) Updating files: 47% (293/622) Updating files: 48% (299/622) Updating files: 49% (305/622) Updating files: 50% (311/622) Updating files: 51% (318/622) Updating files: 52% (324/622) Updating files: 53% (330/622) Updating files: 54% (336/622) Updating files: 55% (343/622) Updating files: 55% (348/622) Updating files: 56% (349/622) Updating files: 57% (355/622) Updating files: 58% (361/622) Updating files: 59% (367/622) Updating files: 60% (374/622) Updating files: 61% (380/622) Updating files: 62% (386/622) Updating files: 63% (392/622) Updating files: 64% (399/622) Updating files: 65% (405/622) Updating files: 66% (411/622) Updating files: 66% (414/622) Updating files: 67% (417/622) Updating files: 68% (423/622) Updating files: 69% (430/622) Updating files: 70% (436/622) Updating files: 71% (442/622) Updating files: 72% (448/622) Updating files: 73% (455/622) Updating files: 74% (461/622) Updating files: 75% (467/622) Updating files: 76% (473/622) Updating files: 77% (479/622) Updating files: 77% (483/622) Updating files: 78% (486/622) Updating files: 79% (492/622) Updating files: 80% (498/622) Updating files: 81% (504/622) Updating files: 82% (511/622) Updating files: 83% (517/622) Updating files: 84% (523/622) Updating files: 84% (527/622) Updating files: 85% (529/622) Updating files: 86% (535/622) Updating files: 87% (542/622) Updating files: 88% (548/622) Updating files: 89% (554/622) Updating files: 90% (560/622) Updating files: 91% (567/622) Updating files: 92% (573/622) Updating files: 93% (579/622) Updating files: 94% (585/622) Updating files: 94% (588/622) Updating files: 95% (591/622) Updating files: 96% (598/622) Updating files: 97% (604/622) Updating files: 98% (610/622) Updating files: 99% (616/622) Updating files: 100% (622/622) Updating files: 100% (622/622), done. 2024-04-20T21:44:42.071 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 3fa121bf0a8 v0.25 2024-04-20T21:44:42.086 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.25 2024-04-20T21:44:42.225 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:44:42.232 INFO:tasks.workunit.client.0.smithi102.stdout:v0.26 2024-04-20T21:44:42.232 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 26 -eq 48 ']' 2024-04-20T21:44:42.232 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.26 2024-04-20T21:44:42.232 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.26 2024-04-20T21:44:42.232 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.26 2024-04-20T21:44:43.698 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~162s 2024-04-20T21:44:43.698 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~74s 2024-04-20T21:44:48.103 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 42% (101/239) Updating files: 43% (103/239) Updating files: 44% (106/239) Updating files: 45% (108/239) Updating files: 46% (110/239) Updating files: 47% (113/239) Updating files: 48% (115/239) Updating files: 49% (118/239) Updating files: 50% (120/239) Updating files: 51% (122/239) Updating files: 52% (125/239) Updating files: 53% (127/239) Updating files: 54% (130/239) Updating files: 55% (132/239) Updating files: 56% (134/239) Updating files: 57% (137/239) Updating files: 58% (139/239) Updating files: 59% (142/239) Updating files: 59% (143/239) Updating files: 60% (144/239) Updating files: 61% (146/239) Updating files: 62% (149/239) Updating files: 63% (151/239) Updating files: 64% (153/239) Updating files: 65% (156/239) Updating files: 66% (158/239) Updating files: 67% (161/239) Updating files: 68% (163/239) Updating files: 69% (165/239) Updating files: 70% (168/239) Updating files: 71% (170/239) Updating files: 72% (173/239) Updating files: 73% (175/239) Updating files: 74% (177/239) Updating files: 75% (180/239) Updating files: 76% (182/239) Updating files: 77% (185/239) Updating files: 78% (187/239) Updating files: 79% (189/239) Updating files: 80% (192/239) Updating files: 81% (194/239) Updating files: 82% (196/239) Updating files: 83% (199/239) Updating files: 84% (201/239) Updating files: 85% (204/239) Updating files: 86% (206/239) Updating files: 87% (208/239) Updating files: 88% (211/239) Updating files: 89% (213/239) Updating files: 90% (216/239) Updating files: 91% (218/239) Updating files: 92% (220/239) Updating files: 93% (223/239) Updating files: 94% (225/239) Updating files: 95% (228/239) Updating files: 96% (230/239) Updating files: 97% (232/239) Updating files: 98% (235/239) Updating files: 99% (237/239) Updating files: 100% (239/239) Updating files: 100% (239/239), done. 2024-04-20T21:44:48.182 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 9981ff90968 v0.26 2024-04-20T21:44:48.199 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.26 2024-04-20T21:44:48.366 INFO:tasks.workunit.client.0.smithi102.stdout:v0.27 2024-04-20T21:44:48.367 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:44:48.367 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 27 -eq 48 ']' 2024-04-20T21:44:48.367 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.27 2024-04-20T21:44:48.367 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.27 2024-04-20T21:44:48.367 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.27 2024-04-20T21:44:50.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~168s 2024-04-20T21:44:50.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~81s 2024-04-20T21:44:55.463 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:44:55.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.677+0000 7fee63000640 1 Processor -- start 2024-04-20T21:44:55.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.677+0000 7fee63000640 1 -- start start 2024-04-20T21:44:55.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.679+0000 7fee63000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:55.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.679+0000 7fee63000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fee5c0ce310 0x7fee5c072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:55.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.679+0000 7fee63000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5c15f940 con 0x7fee5c0cd940 2024-04-20T21:44:55.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.679+0000 7fee63000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fee5c15eb60 con 0x7fee5c0ce310 2024-04-20T21:44:55.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.679+0000 7fee63000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c15ee00 con 0x7fee5c0c7900 2024-04-20T21:44:55.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.679+0000 7fee61000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fee5c0ce310 0x7fee5c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:55.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.679+0000 7fee5be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:55.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.680+0000 7fee61000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fee5c0ce310 0x7fee5c072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:53048/0 (socket says 172.21.15.102:53048) 2024-04-20T21:44:55.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.680+0000 7fee61000640 1 -- 172.21.15.102:0/3620549446 learned_addr learned my addr 172.21.15.102:0/3620549446 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:44:55.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.680+0000 7fee5b400640 1 --1- 172.21.15.102:0/3620549446 >> v1:172.21.15.193:6789/0 conn(0x7fee5c0cd940 0x7fee5c074ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:50876/0 (socket says 172.21.15.102:50876) 2024-04-20T21:44:55.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.680+0000 7fee61000640 1 -- 172.21.15.102:0/3620549446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 msgr2=0x7fee5c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:55.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.680+0000 7fee61000640 1 --2- 172.21.15.102:0/3620549446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.680+0000 7fee61000640 1 -- 172.21.15.102:0/3620549446 >> v1:172.21.15.193:6789/0 conn(0x7fee5c0cd940 legacy=0x7fee5c074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:55.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.680+0000 7fee61000640 1 -- 172.21.15.102:0/3620549446 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c15f0a0 con 0x7fee5c0ce310 2024-04-20T21:44:55.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.680+0000 7fee5aa00640 1 -- 172.21.15.102:0/3620549446 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1812547937 0 0) 0x7fee5c15f940 con 0x7fee5c0cd940 2024-04-20T21:44:55.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.681+0000 7fee61000640 1 --2- 172.21.15.102:0/3620549446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0ce310 0x7fee5c072230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fee54009e50 tx=0x7fee5402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b41ec3d4cf1d6085 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:55.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.681+0000 7fee5aa00640 1 -- 172.21.15.102:0/3620549446 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5402cb00 con 0x7fee5c0ce310 2024-04-20T21:44:55.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee5aa00640 1 -- 172.21.15.102:0/3620549446 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5402cca0 con 0x7fee5c0ce310 2024-04-20T21:44:55.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee5aa00640 1 -- 172.21.15.102:0/3620549446 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54034610 con 0x7fee5c0ce310 2024-04-20T21:44:55.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee63000640 1 -- 172.21.15.102:0/3620549446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0ce310 msgr2=0x7fee5c072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:55.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee63000640 1 --2- 172.21.15.102:0/3620549446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0ce310 0x7fee5c072230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fee54009e50 tx=0x7fee5402bf60 comp rx=0 tx=0).stop 2024-04-20T21:44:55.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee63000640 1 -- 172.21.15.102:0/3620549446 shutdown_connections 2024-04-20T21:44:55.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee63000640 1 --2- 172.21.15.102:0/3620549446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0ce310 0x7fee5c072230 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee63000640 1 --2- 172.21.15.102:0/3620549446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee63000640 1 -- 172.21.15.102:0/3620549446 >> 172.21.15.102:0/3620549446 conn(0x7fee5c06dad0 msgr2=0x7fee5c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:55.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee63000640 1 -- 172.21.15.102:0/3620549446 shutdown_connections 2024-04-20T21:44:55.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.682+0000 7fee63000640 1 -- 172.21.15.102:0/3620549446 wait complete. 2024-04-20T21:44:55.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.684+0000 7fee63000640 1 Processor -- start 2024-04-20T21:44:55.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.684+0000 7fee63000640 1 -- start start 2024-04-20T21:44:55.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee63000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:55.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee63000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0cd940 0x7fee5c1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:55.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee63000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee5c0ce310 0x7fee5c1502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:55.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee63000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c0755b0 con 0x7fee5c0ce310 2024-04-20T21:44:55.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee63000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c15f660 con 0x7fee5c0cd940 2024-04-20T21:44:55.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee63000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c075730 con 0x7fee5c0c7900 2024-04-20T21:44:55.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee61000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee5c0ce310 0x7fee5c1502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:55.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee61000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee5c0ce310 0x7fee5c1502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60084/0 (socket says 172.21.15.102:60084) 2024-04-20T21:44:55.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee61000640 1 -- 172.21.15.102:0/1138615339 learned_addr learned my addr 172.21.15.102:0/1138615339 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:44:55.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee5b400640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0cd940 0x7fee5c1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:55.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.685+0000 7fee5be00640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee5be00640 1 -- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0cd940 msgr2=0x7fee5c1f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee5be00640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0cd940 0x7fee5c1f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee5be00640 1 -- 172.21.15.102:0/1138615339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee5c0ce310 msgr2=0x7fee5c1502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee5be00640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee5c0ce310 0x7fee5c1502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee5be00640 1 -- 172.21.15.102:0/1138615339 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c1592b0 con 0x7fee5c0c7900 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee5be00640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15ae50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e880 tx=0x7fee4c00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee53e00640 1 -- 172.21.15.102:0/1138615339 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c017070 con 0x7fee5c0c7900 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee53e00640 1 -- 172.21.15.102:0/1138615339 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4c00c940 con 0x7fee5c0c7900 2024-04-20T21:44:55.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c15bcc0 con 0x7fee5c0c7900 2024-04-20T21:44:55.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee53e00640 1 -- 172.21.15.102:0/1138615339 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c00cd30 con 0x7fee5c0c7900 2024-04-20T21:44:55.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.686+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c1f0eb0 con 0x7fee5c0c7900 2024-04-20T21:44:55.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.687+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5c074340 con 0x7fee5c0c7900 2024-04-20T21:44:55.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.688+0000 7fee53e00640 1 -- 172.21.15.102:0/1138615339 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee4c01a070 con 0x7fee5c0c7900 2024-04-20T21:44:55.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.689+0000 7fee53e00640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fee3806c920 0x7fee3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:55.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.689+0000 7fee53e00640 1 -- 172.21.15.102:0/1138615339 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 8111+0+0 (secure 0 0 0) 0x7fee4c08c720 con 0x7fee5c0c7900 2024-04-20T21:44:55.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.690+0000 7fee5b400640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fee3806c920 0x7fee3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:55.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.690+0000 7fee5b400640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fee3806c920 0x7fee3806ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fee5c155020 tx=0x7fee44007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:55.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.694+0000 7fee53e00640 1 -- 172.21.15.102:0/1138615339 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee4c059b00 con 0x7fee5c0c7900 2024-04-20T21:44:55.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.955+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee5c0c8540 con 0x7fee5c0c7900 2024-04-20T21:44:55.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.956+0000 7fee53e00640 1 -- 172.21.15.102:0/1138615339 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5207 (secure 0 0 0) 0x7fee4c05de00 con 0x7fee5c0c7900 2024-04-20T21:44:55.955 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:44:55.955 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:44:04.387547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6833/2976263880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2976263880},{"type":"v1","addr":"172.21.15.102:6833","nonce":2976263880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6873,"qdb_cluster":[6873]},"id":1}]} 2024-04-20T21:44:55.956 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 97 2024-04-20T21:44:55.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fee3806c920 msgr2=0x7fee3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fee3806c920 0x7fee3806ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fee5c155020 tx=0x7fee44007d50 comp rx=0 tx=0).stop 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 msgr2=0x7fee5c15ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15ae50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e880 tx=0x7fee4c00ed40 comp rx=0 tx=0).stop 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 shutdown_connections 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee5c0ce310 0x7fee5c1502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fee3806c920 0x7fee3806ede0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee5c0cd940 0x7fee5c1f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 --2- 172.21.15.102:0/1138615339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fee5c0c7900 0x7fee5c15ae50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 >> 172.21.15.102:0/1138615339 conn(0x7fee5c06dad0 msgr2=0x7fee5c074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 shutdown_connections 2024-04-20T21:44:55.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:55.959+0000 7fee63000640 1 -- 172.21.15.102:0/1138615339 wait complete. 2024-04-20T21:44:55.976 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-04-20T21:44:55.976 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:44:55.976 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:44:56.010 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:44:56.010 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:44:56.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.250+0000 7f75a5c00640 1 Processor -- start 2024-04-20T21:44:56.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.251+0000 7f75a5c00640 1 -- start start 2024-04-20T21:44:56.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.252+0000 7f75a5c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f75a0155530 0x7f75a015e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:56.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.252+0000 7f75a5c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cc470 0x7f75a00cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:56.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.252+0000 7f75a5c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75a015ef60 con 0x7f75a00cdfc0 2024-04-20T21:44:56.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.252+0000 7f75a5c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f75a00c5ca0 con 0x7f75a0155530 2024-04-20T21:44:56.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.252+0000 7f75a5c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f75a00cd5f0 con 0x7f75a00cc470 2024-04-20T21:44:56.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f7597400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cc470 0x7f75a00cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:56.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f759fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f75a0155530 0x7f75a015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:56.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f75a5200640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f75a00cdfc0 0x7f75a00cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:49048/0 (socket says 172.21.15.102:49048) 2024-04-20T21:44:56.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f75a5200640 1 -- 172.21.15.102:0/3181434640 learned_addr learned my addr 172.21.15.102:0/3181434640 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:44:56.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f759fe00640 1 -- 172.21.15.102:0/3181434640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cc470 msgr2=0x7f75a00cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:56.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f759fe00640 1 --2- 172.21.15.102:0/3181434640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cc470 0x7f75a00cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f759fe00640 1 -- 172.21.15.102:0/3181434640 >> v1:172.21.15.102:6789/0 conn(0x7f75a00cdfc0 legacy=0x7f75a00cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:56.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f759fe00640 1 -- 172.21.15.102:0/3181434640 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a015f240 con 0x7f75a0155530 2024-04-20T21:44:56.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.253+0000 7f759f400640 1 -- 172.21.15.102:0/3181434640 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 230289873 0 0) 0x7f75a015ef60 con 0x7f75a00cdfc0 2024-04-20T21:44:56.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.255+0000 7f759fe00640 1 --2- 172.21.15.102:0/3181434640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a0155530 0x7f75a015e640 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7590009d90 tx=0x7f759002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e6a8418ac7d313 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:56.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.258+0000 7f759f400640 1 -- 172.21.15.102:0/3181434640 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590039070 con 0x7f75a0155530 2024-04-20T21:44:56.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f759f400640 1 -- 172.21.15.102:0/3181434640 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f759002cc50 con 0x7f75a0155530 2024-04-20T21:44:56.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f759f400640 1 -- 172.21.15.102:0/3181434640 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590034440 con 0x7f75a0155530 2024-04-20T21:44:56.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f75a5c00640 1 -- 172.21.15.102:0/3181434640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a0155530 msgr2=0x7f75a015e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:56.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3181434640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a0155530 0x7f75a015e640 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7590009d90 tx=0x7f759002bea0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f75a5c00640 1 -- 172.21.15.102:0/3181434640 shutdown_connections 2024-04-20T21:44:56.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3181434640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cc470 0x7f75a00cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3181434640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a0155530 0x7f75a015e640 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f75a5c00640 1 -- 172.21.15.102:0/3181434640 >> 172.21.15.102:0/3181434640 conn(0x7f75a0075080 msgr2=0x7f75a0075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:56.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.259+0000 7f75a5c00640 1 -- 172.21.15.102:0/3181434640 shutdown_connections 2024-04-20T21:44:56.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.260+0000 7f75a5c00640 1 -- 172.21.15.102:0/3181434640 wait complete. 2024-04-20T21:44:56.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.261+0000 7f75a5c00640 1 Processor -- start 2024-04-20T21:44:56.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.261+0000 7f75a5c00640 1 -- start start 2024-04-20T21:44:56.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.262+0000 7f75a5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a00cc470 0x7f75a01e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:56.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.262+0000 7f75a5c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cdfc0 0x7f75a015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:56.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.262+0000 7f75a5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a015c230 0x7f75a015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:56.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.262+0000 7f75a5c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f75a00cd280 con 0x7f75a00cc470 2024-04-20T21:44:56.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.262+0000 7f75a5c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75a00ccfa0 con 0x7f75a015c230 2024-04-20T21:44:56.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.262+0000 7f75a5c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f75a00ccbe0 con 0x7f75a00cdfc0 2024-04-20T21:44:56.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a00cc470 0x7f75a01e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:56.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a00cc470 0x7f75a01e6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60090/0 (socket says 172.21.15.102:60090) 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 -- 172.21.15.102:0/3577724106 learned_addr learned my addr 172.21.15.102:0/3577724106 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f7597400640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cdfc0 0x7f75a015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f75a5200640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a015c230 0x7f75a015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 -- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cdfc0 msgr2=0x7f75a015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cdfc0 0x7f75a015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 -- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a015c230 msgr2=0x7f75a015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a015c230 0x7f75a015dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 -- 172.21.15.102:0/3577724106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a00b63b0 con 0x7f75a00cc470 2024-04-20T21:44:56.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.263+0000 7f759fe00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a00cc470 0x7f75a01e6b80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7590037040 tx=0x7f7590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:56.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.264+0000 7f759cc00640 1 -- 172.21.15.102:0/3577724106 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590039050 con 0x7f75a00cc470 2024-04-20T21:44:56.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.264+0000 7f759cc00640 1 -- 172.21.15.102:0/3577724106 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7590045070 con 0x7f75a00cc470 2024-04-20T21:44:56.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.264+0000 7f759cc00640 1 -- 172.21.15.102:0/3577724106 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759003e610 con 0x7f75a00cc470 2024-04-20T21:44:56.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.264+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a00b6690 con 0x7f75a00cc470 2024-04-20T21:44:56.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.264+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a015d160 con 0x7f75a00cc470 2024-04-20T21:44:56.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.265+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a004fd60 con 0x7f75a00cc470 2024-04-20T21:44:56.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.266+0000 7f759cc00640 1 -- 172.21.15.102:0/3577724106 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f759003e7b0 con 0x7f75a00cc470 2024-04-20T21:44:56.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.267+0000 7f759cc00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f757c06c920 0x7f757c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:56.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.267+0000 7f759cc00640 1 -- 172.21.15.102:0/3577724106 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 8111+0+0 (secure 0 0 0) 0x7f75900aedf0 con 0x7f75a00cc470 2024-04-20T21:44:56.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.267+0000 7f7597400640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f757c06c920 0x7f757c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:56.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.268+0000 7f7597400640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f757c06c920 0x7f757c06ede0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f75a00c7960 tx=0x7f7588009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:56.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.272+0000 7f759cc00640 1 -- 172.21.15.102:0/3577724106 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f759007c1d0 con 0x7f75a00cc470 2024-04-20T21:44:56.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.513+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75a00cc9c0 con 0x7f75a00cc470 2024-04-20T21:44:56.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.515+0000 7f759cc00640 1 -- 172.21.15.102:0/3577724106 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5207 (secure 0 0 0) 0x7f75900804d0 con 0x7f75a00cc470 2024-04-20T21:44:56.514 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:44:56.514 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:44:04.387547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6833/2976263880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2976263880},{"type":"v1","addr":"172.21.15.102:6833","nonce":2976263880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6873,"qdb_cluster":[6873]},"id":1}]} 2024-04-20T21:44:56.514 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 97 2024-04-20T21:44:56.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.517+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f757c06c920 msgr2=0x7f757c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:56.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.517+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f757c06c920 0x7f757c06ede0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f75a00c7960 tx=0x7f7588009190 comp rx=0 tx=0).stop 2024-04-20T21:44:56.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.517+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a00cc470 msgr2=0x7f75a01e6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:56.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.517+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a00cc470 0x7f75a01e6b80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7590037040 tx=0x7f7590004a40 comp rx=0 tx=0).stop 2024-04-20T21:44:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.517+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 shutdown_connections 2024-04-20T21:44:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.517+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f757c06c920 0x7f757c06ede0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.517+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a015c230 0x7f75a015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.517+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75a00cdfc0 0x7f75a015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.518+0000 7f75a5c00640 1 --2- 172.21.15.102:0/3577724106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a00cc470 0x7f75a01e6b80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.518+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 >> 172.21.15.102:0/3577724106 conn(0x7f75a0075080 msgr2=0x7f75a014b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.518+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 shutdown_connections 2024-04-20T21:44:56.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:56.518+0000 7f75a5c00640 1 -- 172.21.15.102:0/3577724106 wait complete. 2024-04-20T21:44:56.535 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:44:57.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~175s 2024-04-20T21:44:57.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~87s 2024-04-20T21:44:58.536 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:44:58.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.747+0000 7f53c9a00640 1 Processor -- start 2024-04-20T21:44:58.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.747+0000 7f53c9a00640 1 -- start start 2024-04-20T21:44:58.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.748+0000 7f53c9a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f53c414fa50 0x7f53c40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:58.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.748+0000 7f53c9a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f53c40c9eb0 0x7f53c4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:58.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53c9a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c41536d0 con 0x7f53c4073a10 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53c9a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f53c4159500 con 0x7f53c40c9eb0 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53c9a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f53c40c76c0 con 0x7f53c414fa50 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53c2a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f53c414fa50 0x7f53c40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53ba000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f53c40c9eb0 0x7f53c4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53c3400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f53c4073a10 0x7f53c40c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55014/0 (socket says 172.21.15.102:55014) 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53c3400640 1 -- 172.21.15.102:0/2283391965 learned_addr learned my addr 172.21.15.102:0/2283391965 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53ba000640 1 -- 172.21.15.102:0/2283391965 >> v2:172.21.15.193:3300/0 conn(0x7f53c414fa50 msgr2=0x7f53c40c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53ba000640 1 --2- 172.21.15.102:0/2283391965 >> v2:172.21.15.193:3300/0 conn(0x7f53c414fa50 0x7f53c40c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53ba000640 1 -- 172.21.15.102:0/2283391965 >> v1:172.21.15.102:6789/0 conn(0x7f53c4073a10 legacy=0x7f53c40c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53ba000640 1 -- 172.21.15.102:0/2283391965 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c40c7960 con 0x7f53c40c9eb0 2024-04-20T21:44:58.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.749+0000 7f53c2000640 1 -- 172.21.15.102:0/2283391965 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4107904607 0 0) 0x7f53bc001490 con 0x7f53c4073a10 2024-04-20T21:44:58.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.751+0000 7f53ba000640 1 --2- 172.21.15.102:0/2283391965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c40c9eb0 0x7f53c4158fc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f53c40706a0 tx=0x7f53ac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8814c027f4565ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:58.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.751+0000 7f53c2000640 1 -- 172.21.15.102:0/2283391965 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac034070 con 0x7f53c40c9eb0 2024-04-20T21:44:58.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.751+0000 7f53c2000640 1 -- 172.21.15.102:0/2283391965 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ac030610 con 0x7f53c40c9eb0 2024-04-20T21:44:58.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.751+0000 7f53c2000640 1 -- 172.21.15.102:0/2283391965 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0309e0 con 0x7f53c40c9eb0 2024-04-20T21:44:58.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.751+0000 7f53c9a00640 1 -- 172.21.15.102:0/2283391965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c40c9eb0 msgr2=0x7f53c4158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:58.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.751+0000 7f53c9a00640 1 --2- 172.21.15.102:0/2283391965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c40c9eb0 0x7f53c4158fc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f53c40706a0 tx=0x7f53ac02b960 comp rx=0 tx=0).stop 2024-04-20T21:44:58.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.752+0000 7f53c9a00640 1 -- 172.21.15.102:0/2283391965 shutdown_connections 2024-04-20T21:44:58.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.752+0000 7f53c9a00640 1 --2- 172.21.15.102:0/2283391965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c40c9eb0 0x7f53c4158fc0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:58.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.752+0000 7f53c9a00640 1 --2- 172.21.15.102:0/2283391965 >> v2:172.21.15.193:3300/0 conn(0x7f53c414fa50 0x7f53c40c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:58.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.752+0000 7f53c9a00640 1 -- 172.21.15.102:0/2283391965 >> 172.21.15.102:0/2283391965 conn(0x7f53c40c9390 msgr2=0x7f53c40c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:58.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.752+0000 7f53c9a00640 1 -- 172.21.15.102:0/2283391965 shutdown_connections 2024-04-20T21:44:58.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.752+0000 7f53c9a00640 1 -- 172.21.15.102:0/2283391965 wait complete. 2024-04-20T21:44:58.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.753+0000 7f53c9a00640 1 Processor -- start 2024-04-20T21:44:58.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.753+0000 7f53c9a00640 1 -- start start 2024-04-20T21:44:58.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.754+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53c4073a10 0x7f53c4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:58.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.754+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53c40c9eb0 0x7f53c4142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:58.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.754+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c414fa50 0x7f53c41f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:58.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.754+0000 7f53c9a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53c40c7c00 con 0x7f53c414fa50 2024-04-20T21:44:58.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.754+0000 7f53c9a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53c40c3d90 con 0x7f53c4073a10 2024-04-20T21:44:58.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.754+0000 7f53c9a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f53c40c4030 con 0x7f53c40c9eb0 2024-04-20T21:44:58.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53c2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53c4073a10 0x7f53c4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53ba000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53c40c9eb0 0x7f53c4142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53ba000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53c40c9eb0 0x7f53c4142220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:60568/0 (socket says 172.21.15.102:60568) 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53c2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53c4073a10 0x7f53c4156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:35164/0 (socket says 172.21.15.102:35164) 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53ba000640 1 -- 172.21.15.102:0/310438239 learned_addr learned my addr 172.21.15.102:0/310438239 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53c3400640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c414fa50 0x7f53c41f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53c2a00640 1 -- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53c40c9eb0 msgr2=0x7f53c4142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53c2a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53c40c9eb0 0x7f53c4142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53c2a00640 1 -- 172.21.15.102:0/310438239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c414fa50 msgr2=0x7f53c41f0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53c2a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c414fa50 0x7f53c41f0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.755+0000 7f53c2a00640 1 -- 172.21.15.102:0/310438239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c41f5200 con 0x7f53c4073a10 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.756+0000 7f53c2a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53c4073a10 0x7f53c4156de0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f53b400e8a0 tx=0x7f53b400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.756+0000 7f53bb400640 1 -- 172.21.15.102:0/310438239 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4017070 con 0x7f53c4073a10 2024-04-20T21:44:58.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.756+0000 7f53bb400640 1 -- 172.21.15.102:0/310438239 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b400c8e0 con 0x7f53c4073a10 2024-04-20T21:44:58.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.756+0000 7f53bb400640 1 -- 172.21.15.102:0/310438239 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4009e60 con 0x7f53c4073a10 2024-04-20T21:44:58.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.756+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c41f4480 con 0x7f53c4073a10 2024-04-20T21:44:58.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.756+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c41f4720 con 0x7f53c4073a10 2024-04-20T21:44:58.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.757+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c4074340 con 0x7f53c4073a10 2024-04-20T21:44:58.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.758+0000 7f53bb400640 1 -- 172.21.15.102:0/310438239 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53b401a070 con 0x7f53c4073a10 2024-04-20T21:44:58.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.759+0000 7f53bb400640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53a006c8f0 0x7f53a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:58.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.759+0000 7f53bb400640 1 -- 172.21.15.102:0/310438239 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 8111+0+0 (secure 0 0 0) 0x7f53b408c4b0 con 0x7f53c4073a10 2024-04-20T21:44:58.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.760+0000 7f53ba000640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53a006c8f0 0x7f53a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:58.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.760+0000 7f53ba000640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53a006c8f0 0x7f53a006edb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f53c4148580 tx=0x7f53ac03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:58.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:58.764+0000 7f53bb400640 1 -- 172.21.15.102:0/310438239 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f53b40599c0 con 0x7f53c4073a10 2024-04-20T21:44:59.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.017+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53c40c4b50 con 0x7f53c4073a10 2024-04-20T21:44:59.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.018+0000 7f53bb400640 1 -- 172.21.15.102:0/310438239 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5207 (secure 0 0 0) 0x7f53b405dcc0 con 0x7f53c4073a10 2024-04-20T21:44:59.017 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:44:59.017 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:44:04.387547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6833/2976263880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2976263880},{"type":"v1","addr":"172.21.15.102:6833","nonce":2976263880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6873,"qdb_cluster":[6873]},"id":1}]} 2024-04-20T21:44:59.018 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 97 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53a006c8f0 msgr2=0x7f53a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53a006c8f0 0x7f53a006edb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f53c4148580 tx=0x7f53ac03b040 comp rx=0 tx=0).stop 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53c4073a10 msgr2=0x7f53c4156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53c4073a10 0x7f53c4156de0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f53b400e8a0 tx=0x7f53b400ed60 comp rx=0 tx=0).stop 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 shutdown_connections 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53c414fa50 0x7f53c41f0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53c40c9eb0 0x7f53c4142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f53a006c8f0 0x7f53a006edb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 --2- 172.21.15.102:0/310438239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53c4073a10 0x7f53c4156de0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 >> 172.21.15.102:0/310438239 conn(0x7f53c40c9390 msgr2=0x7f53c406e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:59.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 shutdown_connections 2024-04-20T21:44:59.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:44:59.021+0000 7f53c9a00640 1 -- 172.21.15.102:0/310438239 wait complete. 2024-04-20T21:44:59.039 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:45:01.040 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:01.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.248+0000 7f8c6ca00640 1 Processor -- start 2024-04-20T21:45:01.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.248+0000 7f8c6ca00640 1 -- start start 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.249+0000 7f8c6ca00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c68155530 0x7f8c6815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.249+0000 7f8c6ca00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c680cc470 0x7f8c680cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c6ca00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c6815ef60 con 0x7f8c680cdfc0 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c6ca00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8c680c5ca0 con 0x7f8c680cc470 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c6ca00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8c680cd5f0 con 0x7f8c68155530 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c5de00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c680cc470 0x7f8c680cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c66800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c68155530 0x7f8c6815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c67200640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f8c680cdfc0 0x7f8c680cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:54302/0 (socket says 172.21.15.102:54302) 2024-04-20T21:45:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c67200640 1 -- 172.21.15.102:0/2215736494 learned_addr learned my addr 172.21.15.102:0/2215736494 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c5de00640 1 -- 172.21.15.102:0/2215736494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c68155530 msgr2=0x7f8c6815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c5de00640 1 --2- 172.21.15.102:0/2215736494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c68155530 0x7f8c6815e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.250+0000 7f8c5de00640 1 -- 172.21.15.102:0/2215736494 >> v1:172.21.15.193:6789/0 conn(0x7f8c680cdfc0 legacy=0x7f8c680cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.251+0000 7f8c5de00640 1 -- 172.21.15.102:0/2215736494 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6815f240 con 0x7f8c680cc470 2024-04-20T21:45:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.251+0000 7f8c66800640 1 --2- 172.21.15.102:0/2215736494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c68155530 0x7f8c6815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.251+0000 7f8c65e00640 1 -- 172.21.15.102:0/2215736494 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120940097 0 0) 0x7f8c6815ef60 con 0x7f8c680cdfc0 2024-04-20T21:45:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.252+0000 7f8c5de00640 1 --2- 172.21.15.102:0/2215736494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c680cc470 0x7f8c680cda80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8c500097a0 tx=0x7f8c5002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a63f35326680ae2b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.252+0000 7f8c65e00640 1 -- 172.21.15.102:0/2215736494 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50034070 con 0x7f8c680cc470 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.252+0000 7f8c65e00640 1 -- 172.21.15.102:0/2215736494 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c50030800 con 0x7f8c680cc470 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.252+0000 7f8c65e00640 1 -- 172.21.15.102:0/2215736494 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50030bd0 con 0x7f8c680cc470 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.253+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2215736494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c680cc470 msgr2=0x7f8c680cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.253+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2215736494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c680cc470 0x7f8c680cda80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8c500097a0 tx=0x7f8c5002b940 comp rx=0 tx=0).stop 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.253+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2215736494 shutdown_connections 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.253+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2215736494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c680cc470 0x7f8c680cda80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.253+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2215736494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c68155530 0x7f8c6815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.253+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2215736494 >> 172.21.15.102:0/2215736494 conn(0x7f8c68075080 msgr2=0x7f8c68075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.253+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2215736494 shutdown_connections 2024-04-20T21:45:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.253+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2215736494 wait complete. 2024-04-20T21:45:01.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.255+0000 7f8c6ca00640 1 Processor -- start 2024-04-20T21:45:01.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.255+0000 7f8c6ca00640 1 -- start start 2024-04-20T21:45:01.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.256+0000 7f8c6ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c680cc470 0x7f8c6815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.256+0000 7f8c6ca00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c680cdfc0 0x7f8c68152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.256+0000 7f8c6ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c68155530 0x7f8c681442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.256+0000 7f8c6ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680cd280 con 0x7f8c680cc470 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.256+0000 7f8c6ca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680ccfa0 con 0x7f8c68155530 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.256+0000 7f8c6ca00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8c680ccbe0 con 0x7f8c680cdfc0 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c5de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c680cdfc0 0x7f8c68152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c67200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c68155530 0x7f8c681442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c5de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c680cdfc0 0x7f8c68152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:60574/0 (socket says 172.21.15.102:60574) 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c5de00640 1 -- 172.21.15.102:0/2392476053 learned_addr learned my addr 172.21.15.102:0/2392476053 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c66800640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c680cc470 0x7f8c6815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c67200640 1 -- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c680cdfc0 msgr2=0x7f8c68152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c67200640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c680cdfc0 0x7f8c68152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c67200640 1 -- 172.21.15.102:0/2392476053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c680cc470 msgr2=0x7f8c6815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:01.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c67200640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c680cc470 0x7f8c6815cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c67200640 1 -- 172.21.15.102:0/2392476053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c681f5390 con 0x7f8c68155530 2024-04-20T21:45:01.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.257+0000 7f8c67200640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c68155530 0x7f8c681442d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8c60007fd0 tx=0x7f8c6000c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:01.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.258+0000 7f8c5f400640 1 -- 172.21.15.102:0/2392476053 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600074e0 con 0x7f8c68155530 2024-04-20T21:45:01.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.258+0000 7f8c5f400640 1 -- 172.21.15.102:0/2392476053 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c60007680 con 0x7f8c68155530 2024-04-20T21:45:01.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.258+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c681f5670 con 0x7f8c68155530 2024-04-20T21:45:01.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.258+0000 7f8c5f400640 1 -- 172.21.15.102:0/2392476053 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60004a20 con 0x7f8c68155530 2024-04-20T21:45:01.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.258+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c680c9210 con 0x7f8c68155530 2024-04-20T21:45:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.259+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c6804fd60 con 0x7f8c68155530 2024-04-20T21:45:01.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.260+0000 7f8c5f400640 1 -- 172.21.15.102:0/2392476053 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c60011070 con 0x7f8c68155530 2024-04-20T21:45:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.261+0000 7f8c5f400640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c4406c920 0x7f8c4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.261+0000 7f8c5f400640 1 -- 172.21.15.102:0/2392476053 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 8111+0+0 (secure 0 0 0) 0x7f8c6008bf10 con 0x7f8c68155530 2024-04-20T21:45:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.261+0000 7f8c66800640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c4406c920 0x7f8c4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:01.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.262+0000 7f8c66800640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c4406c920 0x7f8c4406ede0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009bd0 tx=0x7f8c5800b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:01.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.265+0000 7f8c5f400640 1 -- 172.21.15.102:0/2392476053 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c600592f0 con 0x7f8c68155530 2024-04-20T21:45:01.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.504+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c680cc9c0 con 0x7f8c68155530 2024-04-20T21:45:01.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.505+0000 7f8c5f400640 1 -- 172.21.15.102:0/2392476053 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5207 (secure 0 0 0) 0x7f8c6005d5f0 con 0x7f8c68155530 2024-04-20T21:45:01.505 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:01.505 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:44:04.387547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6833/2976263880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2976263880},{"type":"v1","addr":"172.21.15.102:6833","nonce":2976263880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6873,"qdb_cluster":[6873]},"id":1}]} 2024-04-20T21:45:01.505 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 97 2024-04-20T21:45:01.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c4406c920 msgr2=0x7f8c4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c4406c920 0x7f8c4406ede0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009bd0 tx=0x7f8c5800b040 comp rx=0 tx=0).stop 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c68155530 msgr2=0x7f8c681442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c68155530 0x7f8c681442d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8c60007fd0 tx=0x7f8c6000c870 comp rx=0 tx=0).stop 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 shutdown_connections 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8c4406c920 0x7f8c4406ede0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c68155530 0x7f8c681442d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c680cdfc0 0x7f8c68152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 --2- 172.21.15.102:0/2392476053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c680cc470 0x7f8c6815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 >> 172.21.15.102:0/2392476053 conn(0x7f8c68075080 msgr2=0x7f8c68075460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 shutdown_connections 2024-04-20T21:45:01.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:01.508+0000 7f8c6ca00640 1 -- 172.21.15.102:0/2392476053 wait complete. 2024-04-20T21:45:01.527 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:45:01.939 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:45:01.947 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:45:03.528 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:03.634 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~182s 2024-04-20T21:45:03.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~94s 2024-04-20T21:45:03.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.736+0000 7fe5e6a00640 1 Processor -- start 2024-04-20T21:45:03.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.736+0000 7fe5e6a00640 1 -- start start 2024-04-20T21:45:03.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.737+0000 7fe5e6a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe5e0152170 0x7fe5e00c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.737+0000 7fe5e6a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e01628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.737+0000 7fe5e6a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5e0071f70 con 0x7fe5e0158770 2024-04-20T21:45:03.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.737+0000 7fe5e6a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fe5e00c6dc0 con 0x7fe5e0152170 2024-04-20T21:45:03.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.737+0000 7fe5e6a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e0072210 con 0x7fe5e0158b80 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5e4a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e01628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5dfe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe5e0152170 0x7fe5e00c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5dfe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe5e0152170 0x7fe5e00c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:35196/0 (socket says 172.21.15.102:35196) 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5e4a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e01628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:60590/0 (socket says 172.21.15.102:60590) 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5dfe00640 1 -- 172.21.15.102:0/669434746 learned_addr learned my addr 172.21.15.102:0/669434746 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5df400640 1 --1- 172.21.15.102:0/669434746 >> v1:172.21.15.193:6789/0 conn(0x7fe5e0158770 0x7fe5e0070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:54310/0 (socket says 172.21.15.102:54310) 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5dfe00640 1 -- 172.21.15.102:0/669434746 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 msgr2=0x7fe5e01628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5dfe00640 1 --2- 172.21.15.102:0/669434746 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e01628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5dfe00640 1 -- 172.21.15.102:0/669434746 >> v1:172.21.15.193:6789/0 conn(0x7fe5e0158770 legacy=0x7fe5e0070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.738+0000 7fe5dfe00640 1 -- 172.21.15.102:0/669434746 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e00724b0 con 0x7fe5e0152170 2024-04-20T21:45:03.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.739+0000 7fe5dfe00640 1 --2- 172.21.15.102:0/669434746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0152170 0x7fe5e00c8db0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0072680 tx=0x7fe5d002e1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51b91230d3cfaf97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:03.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5dea00640 1 -- 172.21.15.102:0/669434746 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d003b070 con 0x7fe5e0152170 2024-04-20T21:45:03.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5dea00640 1 -- 172.21.15.102:0/669434746 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d0036440 con 0x7fe5e0152170 2024-04-20T21:45:03.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5dea00640 1 -- 172.21.15.102:0/669434746 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0036810 con 0x7fe5e0152170 2024-04-20T21:45:03.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5e6a00640 1 -- 172.21.15.102:0/669434746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0152170 msgr2=0x7fe5e00c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/669434746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0152170 0x7fe5e00c8db0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0072680 tx=0x7fe5d002e1d0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5e6a00640 1 -- 172.21.15.102:0/669434746 shutdown_connections 2024-04-20T21:45:03.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/669434746 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e01628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/669434746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0152170 0x7fe5e00c8db0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5e6a00640 1 -- 172.21.15.102:0/669434746 >> 172.21.15.102:0/669434746 conn(0x7fe5e00c9390 msgr2=0x7fe5e00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:03.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5e6a00640 1 -- 172.21.15.102:0/669434746 shutdown_connections 2024-04-20T21:45:03.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.740+0000 7fe5e6a00640 1 -- 172.21.15.102:0/669434746 wait complete. 2024-04-20T21:45:03.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.742+0000 7fe5e6a00640 1 Processor -- start 2024-04-20T21:45:03.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.742+0000 7fe5e6a00640 1 -- start start 2024-04-20T21:45:03.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.743+0000 7fe5e6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e0152170 0x7fe5e00c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.743+0000 7fe5e6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0158770 0x7fe5e0155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.743+0000 7fe5e6a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e0148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.743+0000 7fe5e6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e00c9f20 con 0x7fe5e0152170 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.743+0000 7fe5e6a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0158250 con 0x7fe5e0158770 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.743+0000 7fe5e6a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e0157f20 con 0x7fe5e0158b80 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5df400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0158770 0x7fe5e0155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5e4a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e0148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e0152170 0x7fe5e00c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5e4a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e0148390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:60594/0 (socket says 172.21.15.102:60594) 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5e4a00640 1 -- 172.21.15.102:0/3228662994 learned_addr learned my addr 172.21.15.102:0/3228662994 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5df400640 1 -- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 msgr2=0x7fe5e0148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5df400640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e0148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5df400640 1 -- 172.21.15.102:0/3228662994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e0152170 msgr2=0x7fe5e00c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5df400640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e0152170 0x7fe5e00c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5df400640 1 -- 172.21.15.102:0/3228662994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e01f0cb0 con 0x7fe5e0158770 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5e4a00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e0148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.744+0000 7fe5df400640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0158770 0x7fe5e0155060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800c490 tx=0x7fe5c800c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.745+0000 7fe5d7e00640 1 -- 172.21.15.102:0/3228662994 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80074e0 con 0x7fe5e0158770 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.745+0000 7fe5d7e00640 1 -- 172.21.15.102:0/3228662994 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c8007680 con 0x7fe5e0158770 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.745+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e015b430 con 0x7fe5e0158770 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.745+0000 7fe5d7e00640 1 -- 172.21.15.102:0/3228662994 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80049e0 con 0x7fe5e0158770 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.745+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01f0f70 con 0x7fe5e0158770 2024-04-20T21:45:03.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.746+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e0074340 con 0x7fe5e0158770 2024-04-20T21:45:03.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.747+0000 7fe5d7e00640 1 -- 172.21.15.102:0/3228662994 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5c801a020 con 0x7fe5e0158770 2024-04-20T21:45:03.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.748+0000 7fe5d7e00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe5bc06c8f0 0x7fe5bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.748+0000 7fe5d7e00640 1 -- 172.21.15.102:0/3228662994 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 8111+0+0 (secure 0 0 0) 0x7fe5c808bb80 con 0x7fe5e0158770 2024-04-20T21:45:03.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.748+0000 7fe5dfe00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe5bc06c8f0 0x7fe5bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.749+0000 7fe5dfe00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe5bc06c8f0 0x7fe5bc06edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0038000 tx=0x7fe5d0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:03.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.752+0000 7fe5d7e00640 1 -- 172.21.15.102:0/3228662994 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe5c8058f60 con 0x7fe5e0158770 2024-04-20T21:45:03.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.991+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5e00c3460 con 0x7fe5e0158770 2024-04-20T21:45:03.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.993+0000 7fe5d7e00640 1 -- 172.21.15.102:0/3228662994 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5207 (secure 0 0 0) 0x7fe5c805d260 con 0x7fe5e0158770 2024-04-20T21:45:03.992 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:03.992 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:44:04.387547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6833/2976263880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2976263880},{"type":"v1","addr":"172.21.15.102:6833","nonce":2976263880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6873,"qdb_cluster":[6873]},"id":1}]} 2024-04-20T21:45:03.992 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 97 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe5bc06c8f0 msgr2=0x7fe5bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe5bc06c8f0 0x7fe5bc06edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0038000 tx=0x7fe5d0008040 comp rx=0 tx=0).stop 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0158770 msgr2=0x7fe5e0155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0158770 0x7fe5e0155060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800c490 tx=0x7fe5c800c950 comp rx=0 tx=0).stop 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 shutdown_connections 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe5e0158b80 0x7fe5e0148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe5bc06c8f0 0x7fe5bc06edb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e0158770 0x7fe5e0155060 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 --2- 172.21.15.102:0/3228662994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe5e0152170 0x7fe5e00c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.995+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 >> 172.21.15.102:0/3228662994 conn(0x7fe5e00c9390 msgr2=0x7fe5e01573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:03.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.996+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 shutdown_connections 2024-04-20T21:45:03.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:03.996+0000 7fe5e6a00640 1 -- 172.21.15.102:0/3228662994 wait complete. 2024-04-20T21:45:04.014 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:45:06.015 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:06.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.224+0000 7f4948a00640 1 Processor -- start 2024-04-20T21:45:06.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.224+0000 7f4948a00640 1 -- start start 2024-04-20T21:45:06.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4948a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f494414fa50 0x7f49440c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4948a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f49440c9eb0 0x7f4944158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4948a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49441536d0 con 0x7f4944073a10 2024-04-20T21:45:06.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4948a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f4944159500 con 0x7f494414fa50 2024-04-20T21:45:06.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4948a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f49440c76c0 con 0x7f49440c9eb0 2024-04-20T21:45:06.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4941e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f494414fa50 0x7f49440c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4942800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4944073a10 0x7f49440c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55020/0 (socket says 172.21.15.102:55020) 2024-04-20T21:45:06.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4942800640 1 -- 172.21.15.102:0/2195636319 learned_addr learned my addr 172.21.15.102:0/2195636319 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:06.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.226+0000 7f4939400640 1 --2- 172.21.15.102:0/2195636319 >> v2:172.21.15.193:3300/0 conn(0x7f49440c9eb0 0x7f4944158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.227+0000 7f4941e00640 1 -- 172.21.15.102:0/2195636319 >> v2:172.21.15.193:3300/0 conn(0x7f49440c9eb0 msgr2=0x7f4944158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:06.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.227+0000 7f4941e00640 1 --2- 172.21.15.102:0/2195636319 >> v2:172.21.15.193:3300/0 conn(0x7f49440c9eb0 0x7f4944158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.227+0000 7f4941400640 1 -- 172.21.15.102:0/2195636319 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 755322375 0 0) 0x7f493c001490 con 0x7f4944073a10 2024-04-20T21:45:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.227+0000 7f4941e00640 1 -- 172.21.15.102:0/2195636319 >> v1:172.21.15.102:6789/0 conn(0x7f4944073a10 legacy=0x7f49440c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.227+0000 7f4941e00640 1 -- 172.21.15.102:0/2195636319 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49440c7960 con 0x7f494414fa50 2024-04-20T21:45:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.228+0000 7f4941e00640 1 --2- 172.21.15.102:0/2195636319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494414fa50 0x7f49440c4830 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f494406a220 tx=0x7f493402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae88a438df8b63b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.228+0000 7f4941400640 1 -- 172.21.15.102:0/2195636319 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934034070 con 0x7f494414fa50 2024-04-20T21:45:06.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.228+0000 7f4941400640 1 -- 172.21.15.102:0/2195636319 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4934030610 con 0x7f494414fa50 2024-04-20T21:45:06.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.228+0000 7f4941400640 1 -- 172.21.15.102:0/2195636319 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49340309e0 con 0x7f494414fa50 2024-04-20T21:45:06.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.229+0000 7f4948a00640 1 -- 172.21.15.102:0/2195636319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494414fa50 msgr2=0x7f49440c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.229+0000 7f4948a00640 1 --2- 172.21.15.102:0/2195636319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494414fa50 0x7f49440c4830 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f494406a220 tx=0x7f493402b960 comp rx=0 tx=0).stop 2024-04-20T21:45:06.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.229+0000 7f4948a00640 1 -- 172.21.15.102:0/2195636319 shutdown_connections 2024-04-20T21:45:06.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.229+0000 7f4948a00640 1 --2- 172.21.15.102:0/2195636319 >> v2:172.21.15.193:3300/0 conn(0x7f49440c9eb0 0x7f4944158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.229+0000 7f4948a00640 1 --2- 172.21.15.102:0/2195636319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494414fa50 0x7f49440c4830 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.229+0000 7f4948a00640 1 -- 172.21.15.102:0/2195636319 >> 172.21.15.102:0/2195636319 conn(0x7f49440c9390 msgr2=0x7f49440c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:06.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.229+0000 7f4948a00640 1 -- 172.21.15.102:0/2195636319 shutdown_connections 2024-04-20T21:45:06.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.229+0000 7f4948a00640 1 -- 172.21.15.102:0/2195636319 wait complete. 2024-04-20T21:45:06.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.231+0000 7f4948a00640 1 Processor -- start 2024-04-20T21:45:06.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.231+0000 7f4948a00640 1 -- start start 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4948a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4944073a10 0x7f4944152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4948a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49440c9eb0 0x7f49440cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4948a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f494414fa50 0x7f4944149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4948a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f49440c7c00 con 0x7f4944073a10 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4948a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49440c3d90 con 0x7f494414fa50 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4948a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f49440c4030 con 0x7f49440c9eb0 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4941e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4944073a10 0x7f4944152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4941e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4944073a10 0x7f4944152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34480/0 (socket says 172.21.15.102:34480) 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.232+0000 7f4941e00640 1 -- 172.21.15.102:0/3714934348 learned_addr learned my addr 172.21.15.102:0/3714934348 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4942800640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f494414fa50 0x7f4944149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4939400640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49440c9eb0 0x7f49440cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4941e00640 1 -- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49440c9eb0 msgr2=0x7f49440cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4941e00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49440c9eb0 0x7f49440cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4941e00640 1 -- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f494414fa50 msgr2=0x7f4944149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4941e00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f494414fa50 0x7f4944149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4941e00640 1 -- 172.21.15.102:0/3714934348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4944156dc0 con 0x7f4944073a10 2024-04-20T21:45:06.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4939400640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49440c9eb0 0x7f49440cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:45:06.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4942800640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f494414fa50 0x7f4944149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:45:06.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4941e00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4944073a10 0x7f4944152850 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4934009890 tx=0x7f49340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f493aa00640 1 -- 172.21.15.102:0/3714934348 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934034040 con 0x7f4944073a10 2024-04-20T21:45:06.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f493aa00640 1 -- 172.21.15.102:0/3714934348 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f493403aaa0 con 0x7f4944073a10 2024-04-20T21:45:06.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f493aa00640 1 -- 172.21.15.102:0/3714934348 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49340443f0 con 0x7f4944073a10 2024-04-20T21:45:06.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.233+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49441570a0 con 0x7f4944073a10 2024-04-20T21:45:06.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.234+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4944157660 con 0x7f4944073a10 2024-04-20T21:45:06.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.235+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4944074340 con 0x7f4944073a10 2024-04-20T21:45:06.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.236+0000 7f493aa00640 1 -- 172.21.15.102:0/3714934348 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4934037070 con 0x7f4944073a10 2024-04-20T21:45:06.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.237+0000 7f493aa00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f492006c8f0 0x7f492006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.237+0000 7f493aa00640 1 -- 172.21.15.102:0/3714934348 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 8111+0+0 (secure 0 0 0) 0x7f49340ae390 con 0x7f4944073a10 2024-04-20T21:45:06.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.237+0000 7f4939400640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f492006c8f0 0x7f492006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.238+0000 7f4939400640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f492006c8f0 0x7f492006edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4944156c20 tx=0x7f492c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.242+0000 7f493aa00640 1 -- 172.21.15.102:0/3714934348 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f493403f070 con 0x7f4944073a10 2024-04-20T21:45:06.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.481+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49440c7320 con 0x7f4944073a10 2024-04-20T21:45:06.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.482+0000 7f493aa00640 1 -- 172.21.15.102:0/3714934348 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5207 (secure 0 0 0) 0x7f493407bee0 con 0x7f4944073a10 2024-04-20T21:45:06.481 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:06.481 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:44:04.387547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6833/2976263880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2976263880},{"type":"v1","addr":"172.21.15.102:6833","nonce":2976263880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6873,"qdb_cluster":[6873]},"id":1}]} 2024-04-20T21:45:06.482 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 97 2024-04-20T21:45:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f492006c8f0 msgr2=0x7f492006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f492006c8f0 0x7f492006edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4944156c20 tx=0x7f492c008040 comp rx=0 tx=0).stop 2024-04-20T21:45:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4944073a10 msgr2=0x7f4944152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4944073a10 0x7f4944152850 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4934009890 tx=0x7f49340046c0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 shutdown_connections 2024-04-20T21:45:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f492006c8f0 0x7f492006edb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f494414fa50 0x7f4944149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49440c9eb0 0x7f49440cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 --2- 172.21.15.102:0/3714934348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4944073a10 0x7f4944152850 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 >> 172.21.15.102:0/3714934348 conn(0x7f49440c9390 msgr2=0x7f494406e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 shutdown_connections 2024-04-20T21:45:06.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:06.485+0000 7f4948a00640 1 -- 172.21.15.102:0/3714934348 wait complete. 2024-04-20T21:45:06.503 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:45:08.504 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:08.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.716+0000 7fcf78c00640 1 Processor -- start 2024-04-20T21:45:08.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.716+0000 7fcf78c00640 1 -- start start 2024-04-20T21:45:08.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf78c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcf740c9eb0 0x7fcf740c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf78c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf741536d0 con 0x7fcf74073a10 2024-04-20T21:45:08.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf78c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf740c76c0 con 0x7fcf7414fa50 2024-04-20T21:45:08.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf78c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fcf740c7960 con 0x7fcf740c9eb0 2024-04-20T21:45:08.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf71400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcf740c9eb0 0x7fcf740c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf72800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcf74073a10 0x7fcf74163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60690/0 (socket says 172.21.15.102:60690) 2024-04-20T21:45:08.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf72800640 1 -- 172.21.15.102:0/847996939 learned_addr learned my addr 172.21.15.102:0/847996939 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:08.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf71400640 1 -- 172.21.15.102:0/847996939 >> v1:172.21.15.193:6789/0 conn(0x7fcf7414fa50 legacy=0x7fcf740c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf71400640 1 -- 172.21.15.102:0/847996939 >> v1:172.21.15.102:6789/0 conn(0x7fcf74073a10 legacy=0x7fcf74163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.718+0000 7fcf71400640 1 -- 172.21.15.102:0/847996939 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf740c3d90 con 0x7fcf740c9eb0 2024-04-20T21:45:08.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.719+0000 7fcf71400640 1 --2- 172.21.15.102:0/847996939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf740c9eb0 0x7fcf740c4830 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcf740c7890 tx=0x7fcf6002b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea9ef81d2b12fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:08.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf70a00640 1 -- 172.21.15.102:0/847996939 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60034070 con 0x7fcf740c9eb0 2024-04-20T21:45:08.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf70a00640 1 -- 172.21.15.102:0/847996939 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf60030890 con 0x7fcf740c9eb0 2024-04-20T21:45:08.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf70a00640 1 -- 172.21.15.102:0/847996939 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60030c60 con 0x7fcf740c9eb0 2024-04-20T21:45:08.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf78c00640 1 -- 172.21.15.102:0/847996939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf740c9eb0 msgr2=0x7fcf740c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf78c00640 1 --2- 172.21.15.102:0/847996939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf740c9eb0 0x7fcf740c4830 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcf740c7890 tx=0x7fcf6002b9d0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf78c00640 1 -- 172.21.15.102:0/847996939 shutdown_connections 2024-04-20T21:45:08.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf78c00640 1 --2- 172.21.15.102:0/847996939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf740c9eb0 0x7fcf740c4830 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf78c00640 1 -- 172.21.15.102:0/847996939 >> 172.21.15.102:0/847996939 conn(0x7fcf740c9390 msgr2=0x7fcf740c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:08.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.720+0000 7fcf78c00640 1 -- 172.21.15.102:0/847996939 shutdown_connections 2024-04-20T21:45:08.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.721+0000 7fcf78c00640 1 -- 172.21.15.102:0/847996939 wait complete. 2024-04-20T21:45:08.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.722+0000 7fcf78c00640 1 Processor -- start 2024-04-20T21:45:08.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.722+0000 7fcf78c00640 1 -- start start 2024-04-20T21:45:08.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf78c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf74073a10 0x7fcf74158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf78c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf740c9eb0 0x7fcf741f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf78c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf7414fa50 0x7fcf7414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf78c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcf740c4030 con 0x7fcf74073a10 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf78c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf740c7c00 con 0x7fcf740c9eb0 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf78c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fcf740c4310 con 0x7fcf7414fa50 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf71e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf74073a10 0x7fcf74158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf71e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf74073a10 0x7fcf74158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54584/0 (socket says 172.21.15.102:54584) 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.723+0000 7fcf71e00640 1 -- 172.21.15.102:0/52097276 learned_addr learned my addr 172.21.15.102:0/52097276 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf71400640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf740c9eb0 0x7fcf741f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf72800640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf7414fa50 0x7fcf7414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf71e00640 1 -- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf7414fa50 msgr2=0x7fcf7414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf71e00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf7414fa50 0x7fcf7414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf71e00640 1 -- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf740c9eb0 msgr2=0x7fcf741f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf71e00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf740c9eb0 0x7fcf741f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf71e00640 1 -- 172.21.15.102:0/52097276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf74153a50 con 0x7fcf74073a10 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf71e00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf74073a10 0x7fcf74158c10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c430 tx=0x7fcf6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf6a000640 1 -- 172.21.15.102:0/52097276 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64017070 con 0x7fcf74073a10 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf6a000640 1 -- 172.21.15.102:0/52097276 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf640052d0 con 0x7fcf74073a10 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.724+0000 7fcf6a000640 1 -- 172.21.15.102:0/52097276 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64003830 con 0x7fcf74073a10 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.725+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf741efe20 con 0x7fcf74073a10 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.725+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf741f00c0 con 0x7fcf74073a10 2024-04-20T21:45:08.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.726+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7414e8d0 con 0x7fcf74073a10 2024-04-20T21:45:08.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.727+0000 7fcf6a000640 1 -- 172.21.15.102:0/52097276 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf6401a070 con 0x7fcf74073a10 2024-04-20T21:45:08.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.727+0000 7fcf6a000640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcf5406c8f0 0x7fcf5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.728+0000 7fcf6a000640 1 -- 172.21.15.102:0/52097276 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 8111+0+0 (secure 0 0 0) 0x7fcf6408b860 con 0x7fcf74073a10 2024-04-20T21:45:08.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.728+0000 7fcf71400640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcf5406c8f0 0x7fcf5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.728+0000 7fcf71400640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcf5406c8f0 0x7fcf5406edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcf60036000 tx=0x7fcf6003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:08.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.732+0000 7fcf6a000640 1 -- 172.21.15.102:0/52097276 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcf7414e8d0 con 0x7fcf74073a10 2024-04-20T21:45:08.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.972+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf74074340 con 0x7fcf74073a10 2024-04-20T21:45:08.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.973+0000 7fcf6a000640 1 -- 172.21.15.102:0/52097276 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5207 (secure 0 0 0) 0x7fcf64058bb0 con 0x7fcf74073a10 2024-04-20T21:45:08.972 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:44:04.387547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":13,"addr":"172.21.15.102:6833/2976263880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2976263880},{"type":"v1","addr":"172.21.15.102:6833","nonce":2976263880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6873,"qdb_cluster":[6873]},"id":1}]} 2024-04-20T21:45:08.972 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 97 2024-04-20T21:45:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcf5406c8f0 msgr2=0x7fcf5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcf5406c8f0 0x7fcf5406edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcf60036000 tx=0x7fcf6003b040 comp rx=0 tx=0).stop 2024-04-20T21:45:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf74073a10 msgr2=0x7fcf74158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf74073a10 0x7fcf74158c10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c430 tx=0x7fcf6400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 shutdown_connections 2024-04-20T21:45:08.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf7414fa50 0x7fcf7414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fcf5406c8f0 0x7fcf5406edb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf740c9eb0 0x7fcf741f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 --2- 172.21.15.102:0/52097276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf74073a10 0x7fcf74158c10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 >> 172.21.15.102:0/52097276 conn(0x7fcf740c9390 msgr2=0x7fcf7414dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 shutdown_connections 2024-04-20T21:45:08.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:08.976+0000 7fcf78c00640 1 -- 172.21.15.102:0/52097276 wait complete. 2024-04-20T21:45:08.995 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T21:45:10.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~188s 2024-04-20T21:45:10.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~100s 2024-04-20T21:45:10.996 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.205+0000 7f85a2c00640 1 Processor -- start 2024-04-20T21:45:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.205+0000 7f85a2c00640 1 -- start start 2024-04-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.207+0000 7f85a2c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f859c0cd940 0x7f859c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.207+0000 7f85a2c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0ce310 0x7f859c072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.207+0000 7f85a2c00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f859c15f940 con 0x7f859c0c7900 2024-04-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.207+0000 7f85a2c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f859c15eb60 con 0x7f859c0cd940 2024-04-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.207+0000 7f85a2c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f859c15ee00 con 0x7f859c0ce310 2024-04-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f859b400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f859c0cd940 0x7f859c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.207+0000 7f859be00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f859c0c7900 0x7f859c074ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60692/0 (socket says 172.21.15.102:60692) 2024-04-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f859be00640 1 -- 172.21.15.102:0/3156590257 learned_addr learned my addr 172.21.15.102:0/3156590257 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f85a0c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0ce310 0x7f859c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f859b400640 1 -- 172.21.15.102:0/3156590257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0ce310 msgr2=0x7f859c072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f859b400640 1 --2- 172.21.15.102:0/3156590257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0ce310 0x7f859c072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f859b400640 1 -- 172.21.15.102:0/3156590257 >> v1:172.21.15.102:6789/0 conn(0x7f859c0c7900 legacy=0x7f859c074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f859b400640 1 -- 172.21.15.102:0/3156590257 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c15f0a0 con 0x7f859c0cd940 2024-04-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f85a0c00640 1 --2- 172.21.15.102:0/3156590257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0ce310 0x7f859c072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.208+0000 7f859aa00640 1 -- 172.21.15.102:0/3156590257 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1963945186 0 0) 0x7f859c15f940 con 0x7f859c0c7900 2024-04-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.209+0000 7f859b400640 1 --2- 172.21.15.102:0/3156590257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0cd940 0x7f859c15bee0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f85880097a0 tx=0x7f858802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=322ce04af7bfed72 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.209+0000 7f859aa00640 1 -- 172.21.15.102:0/3156590257 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588035020 con 0x7f859c0cd940 2024-04-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.209+0000 7f859aa00640 1 -- 172.21.15.102:0/3156590257 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8588031810 con 0x7f859c0cd940 2024-04-20T21:45:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.209+0000 7f859aa00640 1 -- 172.21.15.102:0/3156590257 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588031be0 con 0x7f859c0cd940 2024-04-20T21:45:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.210+0000 7f85a2c00640 1 -- 172.21.15.102:0/3156590257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0cd940 msgr2=0x7f859c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.210+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3156590257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0cd940 0x7f859c15bee0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f85880097a0 tx=0x7f858802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.210+0000 7f85a2c00640 1 -- 172.21.15.102:0/3156590257 shutdown_connections 2024-04-20T21:45:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.210+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3156590257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0ce310 0x7f859c072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.210+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3156590257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0cd940 0x7f859c15bee0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.210+0000 7f85a2c00640 1 -- 172.21.15.102:0/3156590257 >> 172.21.15.102:0/3156590257 conn(0x7f859c06dad0 msgr2=0x7f859c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.210+0000 7f85a2c00640 1 -- 172.21.15.102:0/3156590257 shutdown_connections 2024-04-20T21:45:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.210+0000 7f85a2c00640 1 -- 172.21.15.102:0/3156590257 wait complete. 2024-04-20T21:45:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.212+0000 7f85a2c00640 1 Processor -- start 2024-04-20T21:45:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.212+0000 7f85a2c00640 1 -- start start 2024-04-20T21:45:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f85a2c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0c7900 0x7f859c15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f85a2c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c0cd940 0x7f859c1e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f85a2c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0ce310 0x7f859c1e7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f85a2c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f859c0755b0 con 0x7f859c0cd940 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f85a2c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f859c15f660 con 0x7f859c0ce310 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f85a2c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f859c075730 con 0x7f859c0c7900 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f859b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c0cd940 0x7f859c1e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f859b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c0cd940 0x7f859c1e4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54590/0 (socket says 172.21.15.102:54590) 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.213+0000 7f859b400640 1 -- 172.21.15.102:0/3579806066 learned_addr learned my addr 172.21.15.102:0/3579806066 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f85a0c00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0ce310 0x7f859c1e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f859be00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0c7900 0x7f859c15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f859b400640 1 -- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0c7900 msgr2=0x7f859c15a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f859b400640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0c7900 0x7f859c15a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f859b400640 1 -- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0ce310 msgr2=0x7f859c1e7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f859b400640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0ce310 0x7f859c1e7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f859b400640 1 -- 172.21.15.102:0/3579806066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c1f4e90 con 0x7f859c0cd940 2024-04-20T21:45:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f859b400640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c0cd940 0x7f859c1e4650 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f85880097a0 tx=0x7f85880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:11.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.214+0000 7f8593e00640 1 -- 172.21.15.102:0/3579806066 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588035050 con 0x7f859c0cd940 2024-04-20T21:45:11.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.215+0000 7f8593e00640 1 -- 172.21.15.102:0/3579806066 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858803bcf0 con 0x7f859c0cd940 2024-04-20T21:45:11.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.215+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c1e5880 con 0x7f859c0cd940 2024-04-20T21:45:11.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.215+0000 7f8593e00640 1 -- 172.21.15.102:0/3579806066 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588006e10 con 0x7f859c0cd940 2024-04-20T21:45:11.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.215+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c1e55a0 con 0x7f859c0cd940 2024-04-20T21:45:11.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.216+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859c074340 con 0x7f859c0cd940 2024-04-20T21:45:11.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.217+0000 7f8593e00640 1 -- 172.21.15.102:0/3579806066 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8588038020 con 0x7f859c0cd940 2024-04-20T21:45:11.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.218+0000 7f8593e00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857c06c8f0 0x7f857c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.218+0000 7f8593e00640 1 -- 172.21.15.102:0/3579806066 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f85880ae400 con 0x7f859c0cd940 2024-04-20T21:45:11.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.218+0000 7f859be00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857c06c8f0 0x7f857c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.219+0000 7f859be00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857c06c8f0 0x7f857c06edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f858c007fd0 tx=0x7f858c007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:11.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.222+0000 7f8593e00640 1 -- 172.21.15.102:0/3579806066 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8588040070 con 0x7f859c0cd940 2024-04-20T21:45:11.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.464+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f859c0c55f0 con 0x7f859c0cd940 2024-04-20T21:45:11.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.465+0000 7f8593e00640 1 -- 172.21.15.102:0/3579806066 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+4372 (secure 0 0 0) 0x7f858807bdc0 con 0x7f859c0cd940 2024-04-20T21:45:11.464 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:10.928580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:11.465 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 98 2024-04-20T21:45:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857c06c8f0 msgr2=0x7f857c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857c06c8f0 0x7f857c06edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f858c007fd0 tx=0x7f858c007de0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c0cd940 msgr2=0x7f859c1e4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c0cd940 0x7f859c1e4650 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f85880097a0 tx=0x7f85880046c0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 shutdown_connections 2024-04-20T21:45:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f857c06c8f0 0x7f857c06edb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f859c0ce310 0x7f859c1e7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f859c0cd940 0x7f859c1e4650 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 --2- 172.21.15.102:0/3579806066 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f859c0c7900 0x7f859c15a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 >> 172.21.15.102:0/3579806066 conn(0x7f859c06dad0 msgr2=0x7f859c074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:11.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 shutdown_connections 2024-04-20T21:45:11.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.468+0000 7f85a2c00640 1 -- 172.21.15.102:0/3579806066 wait complete. 2024-04-20T21:45:11.486 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T21:45:11.486 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:45:11.487 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:11.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.680+0000 7f1f29600640 1 Processor -- start 2024-04-20T21:45:11.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.681+0000 7f1f29600640 1 -- start start 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.682+0000 7f1f29600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1f2414fa50 0x7f1f240c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.682+0000 7f1f29600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1f240c9eb0 0x7f1f24158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.682+0000 7f1f29600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f241536d0 con 0x7f1f24073a10 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.682+0000 7f1f29600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f1f24159500 con 0x7f1f240c9eb0 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.682+0000 7f1f29600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1f240c76c0 con 0x7f1f2414fa50 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.683+0000 7f1f22a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1f2414fa50 0x7f1f240c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.683+0000 7f1f23400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f1f24073a10 0x7f1f240c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49930/0 (socket says 172.21.15.102:49930) 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.683+0000 7f1f23400640 1 -- 172.21.15.102:0/4194089547 learned_addr learned my addr 172.21.15.102:0/4194089547 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.683+0000 7f1f22000640 1 --2- 172.21.15.102:0/4194089547 >> v2:172.21.15.102:3300/0 conn(0x7f1f240c9eb0 0x7f1f24158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.683+0000 7f1f22a00640 1 -- 172.21.15.102:0/4194089547 >> v2:172.21.15.102:3300/0 conn(0x7f1f240c9eb0 msgr2=0x7f1f24158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.683+0000 7f1f22a00640 1 --2- 172.21.15.102:0/4194089547 >> v2:172.21.15.102:3300/0 conn(0x7f1f240c9eb0 0x7f1f24158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.683+0000 7f1f22a00640 1 -- 172.21.15.102:0/4194089547 >> v1:172.21.15.193:6789/0 conn(0x7f1f24073a10 legacy=0x7f1f240c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.683+0000 7f1f22a00640 1 -- 172.21.15.102:0/4194089547 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f240c7960 con 0x7f1f2414fa50 2024-04-20T21:45:11.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.684+0000 7f1f22a00640 1 --2- 172.21.15.102:0/4194089547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f2414fa50 0x7f1f240c4830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1f240706a0 tx=0x7f1f1402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24d52662b842290d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:11.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.684+0000 7f1f21600640 1 -- 172.21.15.102:0/4194089547 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14034070 con 0x7f1f2414fa50 2024-04-20T21:45:11.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.684+0000 7f1f21600640 1 -- 172.21.15.102:0/4194089547 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14030610 con 0x7f1f2414fa50 2024-04-20T21:45:11.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.684+0000 7f1f21600640 1 -- 172.21.15.102:0/4194089547 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f140309e0 con 0x7f1f2414fa50 2024-04-20T21:45:11.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.685+0000 7f1f29600640 1 -- 172.21.15.102:0/4194089547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f2414fa50 msgr2=0x7f1f240c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.685+0000 7f1f29600640 1 --2- 172.21.15.102:0/4194089547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f2414fa50 0x7f1f240c4830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1f240706a0 tx=0x7f1f1402b960 comp rx=0 tx=0).stop 2024-04-20T21:45:11.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.685+0000 7f1f29600640 1 -- 172.21.15.102:0/4194089547 shutdown_connections 2024-04-20T21:45:11.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.685+0000 7f1f29600640 1 --2- 172.21.15.102:0/4194089547 >> v2:172.21.15.102:3300/0 conn(0x7f1f240c9eb0 0x7f1f24158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.685+0000 7f1f29600640 1 --2- 172.21.15.102:0/4194089547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f2414fa50 0x7f1f240c4830 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.685+0000 7f1f29600640 1 -- 172.21.15.102:0/4194089547 >> 172.21.15.102:0/4194089547 conn(0x7f1f240c9390 msgr2=0x7f1f240c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:11.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.685+0000 7f1f29600640 1 -- 172.21.15.102:0/4194089547 shutdown_connections 2024-04-20T21:45:11.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.685+0000 7f1f29600640 1 -- 172.21.15.102:0/4194089547 wait complete. 2024-04-20T21:45:11.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.687+0000 7f1f29600640 1 Processor -- start 2024-04-20T21:45:11.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.687+0000 7f1f29600640 1 -- start start 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.688+0000 7f1f29600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f24073a10 0x7f1f241f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.688+0000 7f1f29600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f240c9eb0 0x7f1f24157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.688+0000 7f1f29600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f2414fa50 0x7f1f2414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.688+0000 7f1f29600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1f240c7c00 con 0x7f1f24073a10 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.688+0000 7f1f29600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1f240c3d90 con 0x7f1f240c9eb0 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.688+0000 7f1f29600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1f240c4030 con 0x7f1f2414fa50 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f24073a10 0x7f1f241f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f240c9eb0 0x7f1f24157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f24073a10 0x7f1f241f00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54608/0 (socket says 172.21.15.102:54608) 2024-04-20T21:45:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22a00640 1 -- 172.21.15.102:0/304247157 learned_addr learned my addr 172.21.15.102:0/304247157 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f23400640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f2414fa50 0x7f1f2414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22000640 1 -- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f2414fa50 msgr2=0x7f1f2414a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22000640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f2414fa50 0x7f1f2414a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22000640 1 -- 172.21.15.102:0/304247157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f24073a10 msgr2=0x7f1f241f00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22000640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f24073a10 0x7f1f241f00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22000640 1 -- 172.21.15.102:0/304247157 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f241ef6f0 con 0x7f1f240c9eb0 2024-04-20T21:45:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.689+0000 7f1f22000640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f240c9eb0 0x7f1f24157e80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e8b0 tx=0x7f1f1000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.690+0000 7f1f1aa00640 1 -- 172.21.15.102:0/304247157 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10017070 con 0x7f1f240c9eb0 2024-04-20T21:45:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.690+0000 7f1f1aa00640 1 -- 172.21.15.102:0/304247157 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1000c940 con 0x7f1f240c9eb0 2024-04-20T21:45:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.690+0000 7f1f1aa00640 1 -- 172.21.15.102:0/304247157 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000cd30 con 0x7f1f240c9eb0 2024-04-20T21:45:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.690+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f241ef9b0 con 0x7f1f240c9eb0 2024-04-20T21:45:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.690+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f241e3f90 con 0x7f1f240c9eb0 2024-04-20T21:45:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.691+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f240c56b0 con 0x7f1f240c9eb0 2024-04-20T21:45:11.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.692+0000 7f1f1aa00640 1 -- 172.21.15.102:0/304247157 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f1001a070 con 0x7f1f240c9eb0 2024-04-20T21:45:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.693+0000 7f1f1aa00640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f0406c8f0 0x7f1f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.693+0000 7f1f1aa00640 1 -- 172.21.15.102:0/304247157 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f1f1008c880 con 0x7f1f240c9eb0 2024-04-20T21:45:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.693+0000 7f1f22a00640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f0406c8f0 0x7f1f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.694+0000 7f1f22a00640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f0406c8f0 0x7f1f0406edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009560 tx=0x7f1f1403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:11.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.697+0000 7f1f1aa00640 1 -- 172.21.15.102:0/304247157 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1f240c56b0 con 0x7f1f240c9eb0 2024-04-20T21:45:11.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.939+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f24074340 con 0x7f1f240c9eb0 2024-04-20T21:45:11.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.940+0000 7f1f1aa00640 1 -- 172.21.15.102:0/304247157 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+4372 (secure 0 0 0) 0x7f1f10059b60 con 0x7f1f240c9eb0 2024-04-20T21:45:11.939 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:10.928580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:11.939 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 98 2024-04-20T21:45:11.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f0406c8f0 msgr2=0x7f1f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f0406c8f0 0x7f1f0406edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009560 tx=0x7f1f1403b040 comp rx=0 tx=0).stop 2024-04-20T21:45:11.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f240c9eb0 msgr2=0x7f1f24157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:11.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f240c9eb0 0x7f1f24157e80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e8b0 tx=0x7f1f1000ed70 comp rx=0 tx=0).stop 2024-04-20T21:45:11.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 shutdown_connections 2024-04-20T21:45:11.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f2414fa50 0x7f1f2414a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f1f0406c8f0 0x7f1f0406edb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f240c9eb0 0x7f1f24157e80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 --2- 172.21.15.102:0/304247157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f24073a10 0x7f1f241f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:11.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 >> 172.21.15.102:0/304247157 conn(0x7f1f240c9390 msgr2=0x7f1f2414dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:11.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 shutdown_connections 2024-04-20T21:45:11.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:11.943+0000 7f1f29600640 1 -- 172.21.15.102:0/304247157 wait complete. 2024-04-20T21:45:11.961 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 98, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 98, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4282": { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 98, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:45:10.928580+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4282 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:45:13.962 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.173+0000 7ff8b2400640 1 Processor -- start 2024-04-20T21:45:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.173+0000 7ff8b2400640 1 -- start start 2024-04-20T21:45:14.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8b2400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff8ac0cc470 0x7ff8ac0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:14.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8b2400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff8ac0cdfc0 0x7ff8ac0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:14.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8b2400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ac15ef60 con 0x7ff8ac155530 2024-04-20T21:45:14.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8b2400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff8ac0c5ca0 con 0x7ff8ac0cc470 2024-04-20T21:45:14.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8b2400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff8ac0cd5f0 con 0x7ff8ac0cdfc0 2024-04-20T21:45:14.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8abe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff8ac0cc470 0x7ff8ac0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:14.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8b1000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff8ac155530 0x7ff8ac15e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49932/0 (socket says 172.21.15.102:49932) 2024-04-20T21:45:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8b1000640 1 -- 172.21.15.102:0/3802667295 learned_addr learned my addr 172.21.15.102:0/3802667295 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.175+0000 7ff8b1a00640 1 --2- 172.21.15.102:0/3802667295 >> v2:172.21.15.193:3300/0 conn(0x7ff8ac0cdfc0 0x7ff8ac0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.176+0000 7ff8abe00640 1 -- 172.21.15.102:0/3802667295 >> v2:172.21.15.193:3300/0 conn(0x7ff8ac0cdfc0 msgr2=0x7ff8ac0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.176+0000 7ff8abe00640 1 --2- 172.21.15.102:0/3802667295 >> v2:172.21.15.193:3300/0 conn(0x7ff8ac0cdfc0 0x7ff8ac0cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.176+0000 7ff8abe00640 1 -- 172.21.15.102:0/3802667295 >> v1:172.21.15.193:6789/0 conn(0x7ff8ac155530 legacy=0x7ff8ac15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:14.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.176+0000 7ff8ab400640 1 -- 172.21.15.102:0/3802667295 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699538939 0 0) 0x7ff8ac15ef60 con 0x7ff8ac155530 2024-04-20T21:45:14.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.176+0000 7ff8abe00640 1 -- 172.21.15.102:0/3802667295 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac15f240 con 0x7ff8ac0cc470 2024-04-20T21:45:14.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.177+0000 7ff8abe00640 1 --2- 172.21.15.102:0/3802667295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac0cda80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac08e290 tx=0x7ff89402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624c750609fd92cf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.177+0000 7ff8ab400640 1 -- 172.21.15.102:0/3802667295 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894035020 con 0x7ff8ac0cc470 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.177+0000 7ff8ab400640 1 -- 172.21.15.102:0/3802667295 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff894031830 con 0x7ff8ac0cc470 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.177+0000 7ff8ab400640 1 -- 172.21.15.102:0/3802667295 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894031c00 con 0x7ff8ac0cc470 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.178+0000 7ff8b2400640 1 -- 172.21.15.102:0/3802667295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 msgr2=0x7ff8ac0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.178+0000 7ff8b2400640 1 --2- 172.21.15.102:0/3802667295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac0cda80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac08e290 tx=0x7ff89402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.178+0000 7ff8b2400640 1 -- 172.21.15.102:0/3802667295 shutdown_connections 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.178+0000 7ff8b2400640 1 --2- 172.21.15.102:0/3802667295 >> v2:172.21.15.193:3300/0 conn(0x7ff8ac0cdfc0 0x7ff8ac0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.178+0000 7ff8b2400640 1 --2- 172.21.15.102:0/3802667295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac0cda80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.178+0000 7ff8b2400640 1 -- 172.21.15.102:0/3802667295 >> 172.21.15.102:0/3802667295 conn(0x7ff8ac075080 msgr2=0x7ff8ac075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.178+0000 7ff8b2400640 1 -- 172.21.15.102:0/3802667295 shutdown_connections 2024-04-20T21:45:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.178+0000 7ff8b2400640 1 -- 172.21.15.102:0/3802667295 wait complete. 2024-04-20T21:45:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.180+0000 7ff8b2400640 1 Processor -- start 2024-04-20T21:45:14.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.180+0000 7ff8b2400640 1 -- start start 2024-04-20T21:45:14.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.180+0000 7ff8b2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac0cdfc0 0x7ff8ac1e4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b2400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8ac155530 0x7ff8ac1e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac0cd280 con 0x7ff8ac0cc470 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b2400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac0ccfa0 con 0x7ff8ac0cdfc0 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b2400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ac0ccbe0 con 0x7ff8ac155530 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac143d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54640/0 (socket says 172.21.15.102:54640) 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1000640 1 -- 172.21.15.102:0/206190666 learned_addr learned my addr 172.21.15.102:0/206190666 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1a00640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8ac155530 0x7ff8ac1e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:14.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8abe00640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac0cdfc0 0x7ff8ac1e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1000640 1 -- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8ac155530 msgr2=0x7ff8ac1e49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1000640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8ac155530 0x7ff8ac1e49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1000640 1 -- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac0cdfc0 msgr2=0x7ff8ac1e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1000640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac0cdfc0 0x7ff8ac1e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.181+0000 7ff8b1000640 1 -- 172.21.15.102:0/206190666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac1f11d0 con 0x7ff8ac0cc470 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.182+0000 7ff8b1000640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac143d80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c460 tx=0x7ff89c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.182+0000 7ff8a8c00640 1 -- 172.21.15.102:0/206190666 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c018070 con 0x7ff8ac0cc470 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.182+0000 7ff8a8c00640 1 -- 172.21.15.102:0/206190666 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89c007500 con 0x7ff8ac0cc470 2024-04-20T21:45:14.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.182+0000 7ff8a8c00640 1 -- 172.21.15.102:0/206190666 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c004a20 con 0x7ff8ac0cc470 2024-04-20T21:45:14.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.182+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac1f1470 con 0x7ff8ac0cc470 2024-04-20T21:45:14.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.182+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac1f1a10 con 0x7ff8ac0cc470 2024-04-20T21:45:14.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.183+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac04fd60 con 0x7ff8ac0cc470 2024-04-20T21:45:14.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.184+0000 7ff8a8c00640 1 -- 172.21.15.102:0/206190666 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff89c01b070 con 0x7ff8ac0cc470 2024-04-20T21:45:14.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.185+0000 7ff8a8c00640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff88806c920 0x7ff88806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:14.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.185+0000 7ff8a8c00640 1 -- 172.21.15.102:0/206190666 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7ff89c08b7e0 con 0x7ff8ac0cc470 2024-04-20T21:45:14.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.186+0000 7ff8abe00640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff88806c920 0x7ff88806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:14.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.186+0000 7ff8abe00640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff88806c920 0x7ff88806ede0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff894036000 tx=0x7ff89403d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:14.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.190+0000 7ff8a8c00640 1 -- 172.21.15.102:0/206190666 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff89c058ac0 con 0x7ff8ac0cc470 2024-04-20T21:45:14.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.435+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8ac0cc1a0 con 0x7ff8ac0cc470 2024-04-20T21:45:14.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.436+0000 7ff8a8c00640 1 -- 172.21.15.102:0/206190666 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+4372 (secure 0 0 0) 0x7ff89c05cdc0 con 0x7ff8ac0cc470 2024-04-20T21:45:14.435 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:10.928580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:14.436 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 98 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff88806c920 msgr2=0x7ff88806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff88806c920 0x7ff88806ede0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff894036000 tx=0x7ff89403d040 comp rx=0 tx=0).stop 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 msgr2=0x7ff8ac143d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac143d80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c460 tx=0x7ff89c00c920 comp rx=0 tx=0).stop 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 shutdown_connections 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8ac155530 0x7ff8ac1e49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff88806c920 0x7ff88806ede0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac0cdfc0 0x7ff8ac1e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 --2- 172.21.15.102:0/206190666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac0cc470 0x7ff8ac143d80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.439+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 >> 172.21.15.102:0/206190666 conn(0x7ff8ac075080 msgr2=0x7ff8ac0cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.440+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 shutdown_connections 2024-04-20T21:45:14.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:14.440+0000 7ff8b2400640 1 -- 172.21.15.102:0/206190666 wait complete. 2024-04-20T21:45:14.458 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:16.459 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:16.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.672+0000 7f4870000640 1 Processor -- start 2024-04-20T21:45:16.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.672+0000 7f4870000640 1 -- start start 2024-04-20T21:45:16.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f4870000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f48680c9eb0 0x7f48680c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:16.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f4870000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4868073a10 0x7f4868158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:16.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f4870000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48681536d0 con 0x7f486814fa50 2024-04-20T21:45:16.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f4870000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f4868159500 con 0x7f4868073a10 2024-04-20T21:45:16.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f4870000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f48680c76c0 con 0x7f48680c9eb0 2024-04-20T21:45:16.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f486cc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f48680c9eb0 0x7f48680c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:16.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f486e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4868073a10 0x7f4868158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f486d600640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f486814fa50 0x7f48680c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:49944/0 (socket says 172.21.15.102:49944) 2024-04-20T21:45:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.674+0000 7f486d600640 1 -- 172.21.15.102:0/605572890 learned_addr learned my addr 172.21.15.102:0/605572890 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.675+0000 7f486cc00640 1 -- 172.21.15.102:0/605572890 >> v2:172.21.15.102:3300/0 conn(0x7f4868073a10 msgr2=0x7f4868158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.675+0000 7f486cc00640 1 --2- 172.21.15.102:0/605572890 >> v2:172.21.15.102:3300/0 conn(0x7f4868073a10 0x7f4868158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.675+0000 7f4867e00640 1 -- 172.21.15.102:0/605572890 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037113797 0 0) 0x7f485c001110 con 0x7f486814fa50 2024-04-20T21:45:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.675+0000 7f486cc00640 1 -- 172.21.15.102:0/605572890 >> v1:172.21.15.193:6789/0 conn(0x7f486814fa50 legacy=0x7f48680c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.675+0000 7f486cc00640 1 -- 172.21.15.102:0/605572890 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48680c7960 con 0x7f48680c9eb0 2024-04-20T21:45:16.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.676+0000 7f486cc00640 1 --2- 172.21.15.102:0/605572890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f48680c9eb0 0x7f48680c4830 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f48540097a0 tx=0x7f485402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7301a7fb9771ff69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:16.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.676+0000 7f4867e00640 1 -- 172.21.15.102:0/605572890 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854034070 con 0x7f48680c9eb0 2024-04-20T21:45:16.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.676+0000 7f4867e00640 1 -- 172.21.15.102:0/605572890 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4854030610 con 0x7f48680c9eb0 2024-04-20T21:45:16.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.676+0000 7f4867e00640 1 -- 172.21.15.102:0/605572890 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48540309e0 con 0x7f48680c9eb0 2024-04-20T21:45:16.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.677+0000 7f4870000640 1 -- 172.21.15.102:0/605572890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f48680c9eb0 msgr2=0x7f48680c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:16.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.677+0000 7f4870000640 1 --2- 172.21.15.102:0/605572890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f48680c9eb0 0x7f48680c4830 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f48540097a0 tx=0x7f485402b960 comp rx=0 tx=0).stop 2024-04-20T21:45:16.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.677+0000 7f4870000640 1 -- 172.21.15.102:0/605572890 shutdown_connections 2024-04-20T21:45:16.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.677+0000 7f4870000640 1 --2- 172.21.15.102:0/605572890 >> v2:172.21.15.102:3300/0 conn(0x7f4868073a10 0x7f4868158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.677+0000 7f4870000640 1 --2- 172.21.15.102:0/605572890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f48680c9eb0 0x7f48680c4830 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.677+0000 7f4870000640 1 -- 172.21.15.102:0/605572890 >> 172.21.15.102:0/605572890 conn(0x7f48680c9390 msgr2=0x7f48680c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:16.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.677+0000 7f4870000640 1 -- 172.21.15.102:0/605572890 shutdown_connections 2024-04-20T21:45:16.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.677+0000 7f4870000640 1 -- 172.21.15.102:0/605572890 wait complete. 2024-04-20T21:45:16.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.679+0000 7f4870000640 1 Processor -- start 2024-04-20T21:45:16.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.679+0000 7f4870000640 1 -- start start 2024-04-20T21:45:16.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f4870000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4868073a10 0x7f486814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:16.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f4870000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48680c9eb0 0x7f4868158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:16.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f4870000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f486814fa50 0x7f48681ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:16.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f4870000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f48680c7c00 con 0x7f48680c9eb0 2024-04-20T21:45:16.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f4870000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48680c3d90 con 0x7f486814fa50 2024-04-20T21:45:16.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f4870000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f48680c4030 con 0x7f4868073a10 2024-04-20T21:45:16.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f486d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4868073a10 0x7f486814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:16.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f486d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4868073a10 0x7f486814a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:37894/0 (socket says 172.21.15.102:37894) 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f486d600640 1 -- 172.21.15.102:0/321729071 learned_addr learned my addr 172.21.15.102:0/321729071 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f486cc00640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48680c9eb0 0x7f4868158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.680+0000 7f486e000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f486814fa50 0x7f48681ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f486e000640 1 -- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4868073a10 msgr2=0x7f486814a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f486e000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4868073a10 0x7f486814a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f486e000640 1 -- 172.21.15.102:0/321729071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48680c9eb0 msgr2=0x7f4868158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f486e000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48680c9eb0 0x7f4868158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f486e000640 1 -- 172.21.15.102:0/321729071 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486814aa80 con 0x7f486814fa50 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f486d600640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4868073a10 0x7f486814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f486e000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f486814fa50 0x7f48681ecf50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4860009d90 tx=0x7f486000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f4865600640 1 -- 172.21.15.102:0/321729071 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860018020 con 0x7f486814fa50 2024-04-20T21:45:16.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f4865600640 1 -- 172.21.15.102:0/321729071 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4860014480 con 0x7f486814fa50 2024-04-20T21:45:16.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f4865600640 1 -- 172.21.15.102:0/321729071 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860014870 con 0x7f486814fa50 2024-04-20T21:45:16.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48680c3330 con 0x7f486814fa50 2024-04-20T21:45:16.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.681+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48680c38d0 con 0x7f486814fa50 2024-04-20T21:45:16.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.682+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4868074340 con 0x7f486814fa50 2024-04-20T21:45:16.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.683+0000 7f4865600640 1 -- 172.21.15.102:0/321729071 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f486001b020 con 0x7f486814fa50 2024-04-20T21:45:16.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.684+0000 7f4865600640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f484806c8f0 0x7f484806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:16.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.684+0000 7f4865600640 1 -- 172.21.15.102:0/321729071 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f486008d2d0 con 0x7f486814fa50 2024-04-20T21:45:16.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.684+0000 7f486d600640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f484806c8f0 0x7f484806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:16.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.685+0000 7f486d600640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f484806c8f0 0x7f484806edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f485c000f60 tx=0x7f485c007ea0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:16.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.689+0000 7f4865600640 1 -- 172.21.15.102:0/321729071 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4860022020 con 0x7f486814fa50 2024-04-20T21:45:16.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~195s 2024-04-20T21:45:16.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~107s 2024-04-20T21:45:16.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.930+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4868073140 con 0x7f486814fa50 2024-04-20T21:45:16.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.931+0000 7f4865600640 1 -- 172.21.15.102:0/321729071 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+4372 (secure 0 0 0) 0x7f486814aa80 con 0x7f486814fa50 2024-04-20T21:45:16.930 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:16.930 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:10.928580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:16.930 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 98 2024-04-20T21:45:16.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.933+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f484806c8f0 msgr2=0x7f484806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:16.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.933+0000 7f4870000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f484806c8f0 0x7f484806edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f485c000f60 tx=0x7f485c007ea0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f486814fa50 msgr2=0x7f48681ecf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:16.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f486814fa50 0x7f48681ecf50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4860009d90 tx=0x7f486000c490 comp rx=0 tx=0).stop 2024-04-20T21:45:16.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 shutdown_connections 2024-04-20T21:45:16.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f484806c8f0 0x7f484806edb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f486814fa50 0x7f48681ecf50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48680c9eb0 0x7f4868158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 --2- 172.21.15.102:0/321729071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4868073a10 0x7f486814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:16.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 >> 172.21.15.102:0/321729071 conn(0x7f48680c9390 msgr2=0x7f486814b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:16.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 shutdown_connections 2024-04-20T21:45:16.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:16.934+0000 7f4870000640 1 -- 172.21.15.102:0/321729071 wait complete. 2024-04-20T21:45:16.952 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:18.952 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:19.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.159+0000 7fae7c200640 1 Processor -- start 2024-04-20T21:45:19.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.159+0000 7fae7c200640 1 -- start start 2024-04-20T21:45:19.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.161+0000 7fae7c200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fae740c7900 0x7fae7415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:19.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.161+0000 7fae7c200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7415f940 con 0x7fae740ce310 2024-04-20T21:45:19.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.161+0000 7fae7c200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7415eb60 con 0x7fae740cd940 2024-04-20T21:45:19.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.161+0000 7fae7c200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fae7415ee00 con 0x7fae740c7900 2024-04-20T21:45:19.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.161+0000 7fae78e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fae740cd940 0x7fae74074ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:41478/0 (socket says 172.21.15.102:41478) 2024-04-20T21:45:19.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.161+0000 7fae78e00640 1 -- 172.21.15.102:0/4039150517 learned_addr learned my addr 172.21.15.102:0/4039150517 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:19.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.161+0000 7fae79800640 1 --2- 172.21.15.102:0/4039150517 >> v2:172.21.15.102:3300/0 conn(0x7fae740c7900 0x7fae7415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:19.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.162+0000 7fae73e00640 1 -- 172.21.15.102:0/4039150517 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2212189582 0 0) 0x7fae7415eb60 con 0x7fae740cd940 2024-04-20T21:45:19.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.162+0000 7fae79800640 1 -- 172.21.15.102:0/4039150517 >> v1:172.21.15.193:6789/0 conn(0x7fae740cd940 legacy=0x7fae74074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:19.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.162+0000 7fae79800640 1 -- 172.21.15.102:0/4039150517 >> v1:172.21.15.102:6789/0 conn(0x7fae740ce310 legacy=0x7fae74072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:19.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.162+0000 7fae79800640 1 -- 172.21.15.102:0/4039150517 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7415f0a0 con 0x7fae740c7900 2024-04-20T21:45:19.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.163+0000 7fae79800640 1 --2- 172.21.15.102:0/4039150517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740c7900 0x7fae7415bee0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fae64009d90 tx=0x7fae6402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bfe5d76a16be29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:19.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.163+0000 7fae73e00640 1 -- 172.21.15.102:0/4039150517 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64039070 con 0x7fae740c7900 2024-04-20T21:45:19.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.163+0000 7fae73e00640 1 -- 172.21.15.102:0/4039150517 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6402cab0 con 0x7fae740c7900 2024-04-20T21:45:19.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.163+0000 7fae73e00640 1 -- 172.21.15.102:0/4039150517 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64034440 con 0x7fae740c7900 2024-04-20T21:45:19.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.164+0000 7fae7c200640 1 -- 172.21.15.102:0/4039150517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740c7900 msgr2=0x7fae7415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:19.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.164+0000 7fae7c200640 1 --2- 172.21.15.102:0/4039150517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740c7900 0x7fae7415bee0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fae64009d90 tx=0x7fae6402bd80 comp rx=0 tx=0).stop 2024-04-20T21:45:19.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.164+0000 7fae7c200640 1 -- 172.21.15.102:0/4039150517 shutdown_connections 2024-04-20T21:45:19.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.164+0000 7fae7c200640 1 --2- 172.21.15.102:0/4039150517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740c7900 0x7fae7415bee0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:19.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.164+0000 7fae7c200640 1 -- 172.21.15.102:0/4039150517 >> 172.21.15.102:0/4039150517 conn(0x7fae7406dad0 msgr2=0x7fae7406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:19.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.164+0000 7fae7c200640 1 -- 172.21.15.102:0/4039150517 shutdown_connections 2024-04-20T21:45:19.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.164+0000 7fae7c200640 1 -- 172.21.15.102:0/4039150517 wait complete. 2024-04-20T21:45:19.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.165+0000 7fae7c200640 1 Processor -- start 2024-04-20T21:45:19.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.166+0000 7fae7c200640 1 -- start start 2024-04-20T21:45:19.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.166+0000 7fae7c200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae740c7900 0x7fae741f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.166+0000 7fae7c200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae740cd940 0x7fae740ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.166+0000 7fae7c200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740ce310 0x7fae7414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.166+0000 7fae7c200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae740755b0 con 0x7fae740ce310 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.166+0000 7fae7c200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae7415f660 con 0x7fae740cd940 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.166+0000 7fae7c200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fae74075730 con 0x7fae740c7900 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae740c7900 0x7fae741f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae78e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae740cd940 0x7fae740ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae740c7900 0x7fae741f41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56704/0 (socket says 172.21.15.102:56704) 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 -- 172.21.15.102:0/3130902748 learned_addr learned my addr 172.21.15.102:0/3130902748 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae7a200640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740ce310 0x7fae7414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 -- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae740cd940 msgr2=0x7fae740ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:19.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae740cd940 0x7fae740ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 -- 172.21.15.102:0/3130902748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740ce310 msgr2=0x7fae7414fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740ce310 0x7fae7414fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 -- 172.21.15.102:0/3130902748 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7415e780 con 0x7fae740c7900 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae78e00640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae740cd940 0x7fae740ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.167+0000 7fae79800640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae740c7900 0x7fae741f41f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae64009d90 tx=0x7fae64004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.168+0000 7fae71600640 1 -- 172.21.15.102:0/3130902748 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64044020 con 0x7fae740c7900 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.168+0000 7fae71600640 1 -- 172.21.15.102:0/3130902748 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6403f670 con 0x7fae740c7900 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.168+0000 7fae71600640 1 -- 172.21.15.102:0/3130902748 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64039040 con 0x7fae740c7900 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.168+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae741f0c00 con 0x7fae740c7900 2024-04-20T21:45:19.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.168+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae741f0ec0 con 0x7fae740c7900 2024-04-20T21:45:19.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.169+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae740c7ce0 con 0x7fae740c7900 2024-04-20T21:45:19.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.170+0000 7fae71600640 1 -- 172.21.15.102:0/3130902748 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae64047020 con 0x7fae740c7900 2024-04-20T21:45:19.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.171+0000 7fae71600640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae5406c920 0x7fae5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:19.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.171+0000 7fae71600640 1 -- 172.21.15.102:0/3130902748 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7fae640aee10 con 0x7fae740c7900 2024-04-20T21:45:19.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.171+0000 7fae78e00640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae5406c920 0x7fae5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:19.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.172+0000 7fae78e00640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae5406c920 0x7fae5406ede0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae740cb050 tx=0x7fae60007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:19.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.179+0000 7fae71600640 1 -- 172.21.15.102:0/3130902748 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fae740c7ce0 con 0x7fae740c7900 2024-04-20T21:45:19.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.419+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae74074340 con 0x7fae740c7900 2024-04-20T21:45:19.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.420+0000 7fae71600640 1 -- 172.21.15.102:0/3130902748 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+4372 (secure 0 0 0) 0x7fae640b2030 con 0x7fae740c7900 2024-04-20T21:45:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:19.419 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:10.928580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:19.419 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 98 2024-04-20T21:45:19.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae5406c920 msgr2=0x7fae5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:19.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae5406c920 0x7fae5406ede0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae740cb050 tx=0x7fae60007d50 comp rx=0 tx=0).stop 2024-04-20T21:45:19.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae740c7900 msgr2=0x7fae741f41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:19.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae740c7900 0x7fae741f41f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae64009d90 tx=0x7fae64004a40 comp rx=0 tx=0).stop 2024-04-20T21:45:19.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 shutdown_connections 2024-04-20T21:45:19.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae740ce310 0x7fae7414fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:19.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fae5406c920 0x7fae5406ede0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:19.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae740cd940 0x7fae740ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:19.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 --2- 172.21.15.102:0/3130902748 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae740c7900 0x7fae741f41f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:19.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 >> 172.21.15.102:0/3130902748 conn(0x7fae7406dad0 msgr2=0x7fae74074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:19.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 shutdown_connections 2024-04-20T21:45:19.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:19.424+0000 7fae7c200640 1 -- 172.21.15.102:0/3130902748 wait complete. 2024-04-20T21:45:19.443 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:21.443 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:21.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.651+0000 7f751c200640 1 Processor -- start 2024-04-20T21:45:21.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.651+0000 7f751c200640 1 -- start start 2024-04-20T21:45:21.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.653+0000 7f751c200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7514152170 0x7f75140c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:21.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.653+0000 7f751c200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f75141628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:21.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.653+0000 7f751c200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7514071f70 con 0x7f7514158770 2024-04-20T21:45:21.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.653+0000 7f751c200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f75140c6dc0 con 0x7f7514152170 2024-04-20T21:45:21.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.653+0000 7f751c200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7514072210 con 0x7f7514158b80 2024-04-20T21:45:21.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.653+0000 7f7518e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7514158770 0x7f7514070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53156/0 (socket says 172.21.15.102:53156) 2024-04-20T21:45:21.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.653+0000 7f7518e00640 1 -- 172.21.15.102:0/2030064343 learned_addr learned my addr 172.21.15.102:0/2030064343 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f7519800640 1 --2- 172.21.15.102:0/2030064343 >> v2:172.21.15.193:3300/0 conn(0x7f7514152170 0x7f75140c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f751a200640 1 --2- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f75141628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f7513e00640 1 -- 172.21.15.102:0/2030064343 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300518612 0 0) 0x7f7514071f70 con 0x7f7514158770 2024-04-20T21:45:21.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f7513e00640 1 -- 172.21.15.102:0/2030064343 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74fc005730 con 0x7f7514158770 2024-04-20T21:45:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f7519800640 1 -- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 msgr2=0x7f75141628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f7519800640 1 --2- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f75141628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f7513e00640 1 -- 172.21.15.102:0/2030064343 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1208083806 0 0) 0x7f74fc005730 con 0x7f7514158770 2024-04-20T21:45:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f7519800640 1 -- 172.21.15.102:0/2030064343 >> v1:172.21.15.102:6789/0 conn(0x7f7514158770 legacy=0x7f7514070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f7519800640 1 -- 172.21.15.102:0/2030064343 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75140724b0 con 0x7f7514152170 2024-04-20T21:45:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.654+0000 7f751a200640 1 --2- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f75141628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:21.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.655+0000 7f7519800640 1 --2- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140c8db0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7514072680 tx=0x7f750802b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05b44bf8e95201 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:21.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.656+0000 7f7513e00640 1 -- 172.21.15.102:0/2030064343 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508037070 con 0x7f7514152170 2024-04-20T21:45:21.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.656+0000 7f7513e00640 1 -- 172.21.15.102:0/2030064343 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7508030510 con 0x7f7514152170 2024-04-20T21:45:21.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.656+0000 7f7513e00640 1 -- 172.21.15.102:0/2030064343 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500001670 con 0x7f7514152170 2024-04-20T21:45:21.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.656+0000 7f751c200640 1 -- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 msgr2=0x7f75140c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:21.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.656+0000 7f751c200640 1 --2- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140c8db0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7514072680 tx=0x7f750802b800 comp rx=0 tx=0).stop 2024-04-20T21:45:21.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.657+0000 7f751c200640 1 -- 172.21.15.102:0/2030064343 shutdown_connections 2024-04-20T21:45:21.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.657+0000 7f751c200640 1 --2- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f75141628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.657+0000 7f751c200640 1 --2- 172.21.15.102:0/2030064343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140c8db0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.657+0000 7f751c200640 1 -- 172.21.15.102:0/2030064343 >> 172.21.15.102:0/2030064343 conn(0x7f75140c9390 msgr2=0x7f75140c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:21.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.657+0000 7f751c200640 1 -- 172.21.15.102:0/2030064343 shutdown_connections 2024-04-20T21:45:21.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.657+0000 7f751c200640 1 -- 172.21.15.102:0/2030064343 wait complete. 2024-04-20T21:45:21.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.658+0000 7f751c200640 1 Processor -- start 2024-04-20T21:45:21.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.659+0000 7f751c200640 1 -- start start 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.659+0000 7f751c200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f751c200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7514158770 0x7f75140c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f751c200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f751415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f751c200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f75140c9f20 con 0x7f7514158770 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f751c200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7514158250 con 0x7f7514152170 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f751c200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f7514157f20 con 0x7f7514158b80 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7519800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f751a200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f751415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7519800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:39798/0 (socket says 172.21.15.102:39798) 2024-04-20T21:45:21.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7518e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7514158770 0x7f75140c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7519800640 1 -- 172.21.15.102:0/128182598 learned_addr learned my addr 172.21.15.102:0/128182598 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7519800640 1 -- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 msgr2=0x7f751415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7519800640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f751415a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7519800640 1 -- 172.21.15.102:0/128182598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7514158770 msgr2=0x7f75140c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7519800640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7514158770 0x7f75140c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f7519800640 1 -- 172.21.15.102:0/128182598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75141ecf60 con 0x7f7514152170 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.660+0000 7f751a200640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f751415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.661+0000 7f7519800640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140cbe80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f750802be10 tx=0x7f75080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:21.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.661+0000 7f7511600640 1 -- 172.21.15.102:0/128182598 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508040020 con 0x7f7514152170 2024-04-20T21:45:21.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.661+0000 7f7511600640 1 -- 172.21.15.102:0/128182598 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7508032700 con 0x7f7514152170 2024-04-20T21:45:21.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.661+0000 7f7511600640 1 -- 172.21.15.102:0/128182598 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508037040 con 0x7f7514152170 2024-04-20T21:45:21.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.661+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75140cc4c0 con 0x7f7514152170 2024-04-20T21:45:21.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.661+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75141ed500 con 0x7f7514152170 2024-04-20T21:45:21.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.662+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7514070310 con 0x7f7514152170 2024-04-20T21:45:21.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.663+0000 7f7511600640 1 -- 172.21.15.102:0/128182598 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7508043020 con 0x7f7514152170 2024-04-20T21:45:21.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.664+0000 7f7511600640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f74f406c8f0 0x7f74f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:21.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.664+0000 7f7511600640 1 -- 172.21.15.102:0/128182598 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f75080ad9a0 con 0x7f7514152170 2024-04-20T21:45:21.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.664+0000 7f7518e00640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f74f406c8f0 0x7f74f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:21.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.665+0000 7f7518e00640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f74f406c8f0 0x7f74f406edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75140c4780 tx=0x7f75000016d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:21.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.668+0000 7f7511600640 1 -- 172.21.15.102:0/128182598 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7514070310 con 0x7f7514152170 2024-04-20T21:45:21.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.917+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7514156d20 con 0x7f7514152170 2024-04-20T21:45:21.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.918+0000 7f7511600640 1 -- 172.21.15.102:0/128182598 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+4372 (secure 0 0 0) 0x7f7514156d20 con 0x7f7514152170 2024-04-20T21:45:21.917 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:21.917 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:10.928580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:21.917 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 98 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.920+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f74f406c8f0 msgr2=0x7f74f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.920+0000 7f751c200640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f74f406c8f0 0x7f74f406edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75140c4780 tx=0x7f75000016d0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.920+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 msgr2=0x7f75140cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.920+0000 7f751c200640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140cbe80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f750802be10 tx=0x7f75080046c0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.921+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 shutdown_connections 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.921+0000 7f751c200640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7514158b80 0x7f751415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.921+0000 7f751c200640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7514158770 0x7f75140c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.921+0000 7f751c200640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f74f406c8f0 0x7f74f406edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.921+0000 7f751c200640 1 --2- 172.21.15.102:0/128182598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514152170 0x7f75140cbe80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.921+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 >> 172.21.15.102:0/128182598 conn(0x7f75140c9390 msgr2=0x7f7514070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.921+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 shutdown_connections 2024-04-20T21:45:21.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:21.921+0000 7f751c200640 1 -- 172.21.15.102:0/128182598 wait complete. 2024-04-20T21:45:21.939 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:23.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~202s 2024-04-20T21:45:23.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~114s 2024-04-20T21:45:23.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:24.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.152+0000 7f5026800640 1 Processor -- start 2024-04-20T21:45:24.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.152+0000 7f5026800640 1 -- start start 2024-04-20T21:45:24.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.153+0000 7f5026800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5020153180 0x7f5020158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:24.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.153+0000 7f5026800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5020159380 0x7f50200c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:24.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5026800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5020153560 con 0x7f50200c75a0 2024-04-20T21:45:24.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5026800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f50200c7a70 con 0x7f5020153180 2024-04-20T21:45:24.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5026800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f50200c7d30 con 0x7f5020159380 2024-04-20T21:45:24.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5024a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5020159380 0x7f50200c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:24.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5024a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5020159380 0x7f50200c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:56746/0 (socket says 172.21.15.102:56746) 2024-04-20T21:45:24.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5024a00640 1 -- 172.21.15.102:0/1110271250 learned_addr learned my addr 172.21.15.102:0/1110271250 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:24.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5025e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f50200c75a0 0x7f5020070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:41482/0 (socket says 172.21.15.102:41482) 2024-04-20T21:45:24.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5025400640 1 --2- 172.21.15.102:0/1110271250 >> v2:172.21.15.102:3300/0 conn(0x7f5020153180 0x7f5020158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:24.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5024a00640 1 -- 172.21.15.102:0/1110271250 >> v2:172.21.15.102:3300/0 conn(0x7f5020153180 msgr2=0x7f5020158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:24.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5024a00640 1 --2- 172.21.15.102:0/1110271250 >> v2:172.21.15.102:3300/0 conn(0x7f5020153180 0x7f5020158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5024a00640 1 -- 172.21.15.102:0/1110271250 >> v1:172.21.15.193:6789/0 conn(0x7f50200c75a0 legacy=0x7f5020070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:24.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.154+0000 7f5024a00640 1 -- 172.21.15.102:0/1110271250 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502015c4d0 con 0x7f5020159380 2024-04-20T21:45:24.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.155+0000 7f5024a00640 1 --2- 172.21.15.102:0/1110271250 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5020159380 0x7f50200c5f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f500c009870 tx=0x7f500c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b84a68d35c5a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.156+0000 7f501fe00640 1 -- 172.21.15.102:0/1110271250 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c034070 con 0x7f5020159380 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.156+0000 7f501fe00640 1 -- 172.21.15.102:0/1110271250 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c030840 con 0x7f5020159380 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.156+0000 7f501fe00640 1 -- 172.21.15.102:0/1110271250 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c030c10 con 0x7f5020159380 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.156+0000 7f5026800640 1 -- 172.21.15.102:0/1110271250 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5020159380 msgr2=0x7f50200c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.156+0000 7f5026800640 1 --2- 172.21.15.102:0/1110271250 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5020159380 0x7f50200c5f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f500c009870 tx=0x7f500c02b980 comp rx=0 tx=0).stop 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.156+0000 7f5026800640 1 -- 172.21.15.102:0/1110271250 shutdown_connections 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.156+0000 7f5026800640 1 --2- 172.21.15.102:0/1110271250 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5020159380 0x7f50200c5f30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.156+0000 7f5026800640 1 --2- 172.21.15.102:0/1110271250 >> v2:172.21.15.102:3300/0 conn(0x7f5020153180 0x7f5020158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.157+0000 7f5026800640 1 -- 172.21.15.102:0/1110271250 >> 172.21.15.102:0/1110271250 conn(0x7f50200cc820 msgr2=0x7f50200ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.157+0000 7f5026800640 1 -- 172.21.15.102:0/1110271250 shutdown_connections 2024-04-20T21:45:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.157+0000 7f5026800640 1 -- 172.21.15.102:0/1110271250 wait complete. 2024-04-20T21:45:24.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.158+0000 7f5026800640 1 Processor -- start 2024-04-20T21:45:24.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.158+0000 7f5026800640 1 -- start start 2024-04-20T21:45:24.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.159+0000 7f5026800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50200c75a0 0x7f50200cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.159+0000 7f5026800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5020153180 0x7f50200c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.159+0000 7f5026800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5020159380 0x7f50200ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.159+0000 7f5026800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f502015d290 con 0x7f5020153180 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.159+0000 7f5026800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f502015ca90 con 0x7f5020159380 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.159+0000 7f5026800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f50200c6be0 con 0x7f50200c75a0 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5024a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5020153180 0x7f50200c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5025400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50200c75a0 0x7f50200cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5024a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5020153180 0x7f50200c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43760/0 (socket says 172.21.15.102:43760) 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5024a00640 1 -- 172.21.15.102:0/3064086019 learned_addr learned my addr 172.21.15.102:0/3064086019 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5025e00640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5020159380 0x7f50200ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:24.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5025400640 1 -- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5020159380 msgr2=0x7f50200ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:24.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5025400640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5020159380 0x7f50200ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5025400640 1 -- 172.21.15.102:0/3064086019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5020153180 msgr2=0x7f50200c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5025400640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5020153180 0x7f50200c27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5025400640 1 -- 172.21.15.102:0/3064086019 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50200cc350 con 0x7f50200c75a0 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.160+0000 7f5025400640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50200c75a0 0x7f50200cbe10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f501400d550 tx=0x7f501400da10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.161+0000 7f501d600640 1 -- 172.21.15.102:0/3064086019 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140125f0 con 0x7f50200c75a0 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.161+0000 7f501d600640 1 -- 172.21.15.102:0/3064086019 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5014019070 con 0x7f50200c75a0 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.161+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50201f1420 con 0x7f50200c75a0 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.161+0000 7f501d600640 1 -- 172.21.15.102:0/3064086019 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140129b0 con 0x7f50200c75a0 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.161+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f50201f19a0 con 0x7f50200c75a0 2024-04-20T21:45:24.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.162+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5020070820 con 0x7f50200c75a0 2024-04-20T21:45:24.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.163+0000 7f501d600640 1 -- 172.21.15.102:0/3064086019 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5014012b50 con 0x7f50200c75a0 2024-04-20T21:45:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.164+0000 7f501d600640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f500006c8f0 0x7f500006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.164+0000 7f501d600640 1 -- 172.21.15.102:0/3064086019 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f501408c850 con 0x7f50200c75a0 2024-04-20T21:45:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.164+0000 7f5024a00640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f500006c8f0 0x7f500006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:24.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.165+0000 7f5024a00640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f500006c8f0 0x7f500006edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f500c009870 tx=0x7f500c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:24.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.169+0000 7f501d600640 1 -- 172.21.15.102:0/3064086019 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5020070820 con 0x7f50200c75a0 2024-04-20T21:45:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.415+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f502004fd60 con 0x7f50200c75a0 2024-04-20T21:45:24.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.416+0000 7f501d600640 1 -- 172.21.15.102:0/3064086019 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+4377 (secure 0 0 0) 0x7f5014059b30 con 0x7f50200c75a0 2024-04-20T21:45:24.415 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:24.415 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:22.627351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":430,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:24.416 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 99 2024-04-20T21:45:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f500006c8f0 msgr2=0x7f500006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f500006c8f0 0x7f500006edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f500c009870 tx=0x7f500c008030 comp rx=0 tx=0).stop 2024-04-20T21:45:24.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50200c75a0 msgr2=0x7f50200cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50200c75a0 0x7f50200cbe10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f501400d550 tx=0x7f501400da10 comp rx=0 tx=0).stop 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 shutdown_connections 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f500006c8f0 0x7f500006edb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5020159380 0x7f50200ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5020153180 0x7f50200c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 --2- 172.21.15.102:0/3064086019 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50200c75a0 0x7f50200cbe10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 >> 172.21.15.102:0/3064086019 conn(0x7f50200cc820 msgr2=0x7f50200c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 shutdown_connections 2024-04-20T21:45:24.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:24.419+0000 7f5026800640 1 -- 172.21.15.102:0/3064086019 wait complete. 2024-04-20T21:45:24.438 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 99, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 99, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4282": { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 98, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:reconnect", "state_seq": 430 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:45:22.627351+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4282 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:45:26.439 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:26.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.653+0000 7ff61d600640 1 Processor -- start 2024-04-20T21:45:26.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.653+0000 7ff61d600640 1 -- start start 2024-04-20T21:45:26.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.654+0000 7ff61d600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff618155530 0x7ff61815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:26.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.654+0000 7ff61d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 0x7ff6180cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:26.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.654+0000 7ff61d600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff6180cdfc0 0x7ff6180cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:26.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61d600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff6180c5ca0 con 0x7ff6180cdfc0 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61d600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff6180cd5f0 con 0x7ff618155530 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61d600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff61815ef60 con 0x7ff6180cc470 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61cc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff6180cdfc0 0x7ff6180cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff617e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff618155530 0x7ff61815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61cc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff6180cdfc0 0x7ff6180cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43772/0 (socket says 172.21.15.102:43772) 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61cc00640 1 -- 172.21.15.102:0/1864080397 learned_addr learned my addr 172.21.15.102:0/1864080397 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff617400640 1 --2- 172.21.15.102:0/1864080397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 0x7ff6180cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61cc00640 1 -- 172.21.15.102:0/1864080397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 msgr2=0x7ff6180cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61cc00640 1 --2- 172.21.15.102:0/1864080397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 0x7ff6180cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61cc00640 1 -- 172.21.15.102:0/1864080397 >> v2:172.21.15.193:3300/0 conn(0x7ff618155530 msgr2=0x7ff61815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61cc00640 1 --2- 172.21.15.102:0/1864080397 >> v2:172.21.15.193:3300/0 conn(0x7ff618155530 0x7ff61815e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff61cc00640 1 -- 172.21.15.102:0/1864080397 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61815f240 con 0x7ff6180cdfc0 2024-04-20T21:45:26.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.655+0000 7ff617e00640 1 --2- 172.21.15.102:0/1864080397 >> v2:172.21.15.193:3300/0 conn(0x7ff618155530 0x7ff61815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:26.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.657+0000 7ff61cc00640 1 --2- 172.21.15.102:0/1864080397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180cb6a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff61806a050 tx=0x7ff61002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931f015564a785ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:26.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.657+0000 7ff616a00640 1 -- 172.21.15.102:0/1864080397 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61002ca80 con 0x7ff6180cdfc0 2024-04-20T21:45:26.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.657+0000 7ff616a00640 1 -- 172.21.15.102:0/1864080397 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61002cc20 con 0x7ff6180cdfc0 2024-04-20T21:45:26.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.657+0000 7ff616a00640 1 -- 172.21.15.102:0/1864080397 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100343f0 con 0x7ff6180cdfc0 2024-04-20T21:45:26.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 -- 172.21.15.102:0/1864080397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 msgr2=0x7ff6180cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:26.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 --2- 172.21.15.102:0/1864080397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180cb6a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff61806a050 tx=0x7ff61002bee0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 -- 172.21.15.102:0/1864080397 shutdown_connections 2024-04-20T21:45:26.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 --2- 172.21.15.102:0/1864080397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180cb6a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 --2- 172.21.15.102:0/1864080397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 0x7ff6180cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 --2- 172.21.15.102:0/1864080397 >> v2:172.21.15.193:3300/0 conn(0x7ff618155530 0x7ff61815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 -- 172.21.15.102:0/1864080397 >> 172.21.15.102:0/1864080397 conn(0x7ff618075080 msgr2=0x7ff618075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:26.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 -- 172.21.15.102:0/1864080397 shutdown_connections 2024-04-20T21:45:26.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.658+0000 7ff61d600640 1 -- 172.21.15.102:0/1864080397 wait complete. 2024-04-20T21:45:26.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.660+0000 7ff61d600640 1 Processor -- start 2024-04-20T21:45:26.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.660+0000 7ff61d600640 1 -- start start 2024-04-20T21:45:26.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff61d600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 0x7ff61815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:26.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff61d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:26.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff61d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff618155530 0x7ff6180cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:26.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff61d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff6180cd280 con 0x7ff6180cdfc0 2024-04-20T21:45:26.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff61d600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff6180ccfa0 con 0x7ff618155530 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff61d600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff6180ccbe0 con 0x7ff6180cc470 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff617400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff617e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 0x7ff61815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff617400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43776/0 (socket says 172.21.15.102:43776) 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff617400640 1 -- 172.21.15.102:0/3663208719 learned_addr learned my addr 172.21.15.102:0/3663208719 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff61cc00640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff618155530 0x7ff6180cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff617400640 1 -- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 msgr2=0x7ff61815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff617400640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 0x7ff61815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.661+0000 7ff617400640 1 -- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff618155530 msgr2=0x7ff6180cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.662+0000 7ff617400640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff618155530 0x7ff6180cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.662+0000 7ff617400640 1 -- 172.21.15.102:0/3663208719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61815d920 con 0x7ff6180cdfc0 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.662+0000 7ff617400640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180ceec0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff60000e890 tx=0x7ff60000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.662+0000 7ff60fe00640 1 -- 172.21.15.102:0/3663208719 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600017070 con 0x7ff6180cdfc0 2024-04-20T21:45:26.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.662+0000 7ff60fe00640 1 -- 172.21.15.102:0/3663208719 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60000c940 con 0x7ff6180cdfc0 2024-04-20T21:45:26.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.662+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61815dc00 con 0x7ff6180cdfc0 2024-04-20T21:45:26.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.662+0000 7ff60fe00640 1 -- 172.21.15.102:0/3663208719 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60000cd10 con 0x7ff6180cdfc0 2024-04-20T21:45:26.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.662+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61815e1c0 con 0x7ff6180cdfc0 2024-04-20T21:45:26.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.664+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618069340 con 0x7ff6180cdfc0 2024-04-20T21:45:26.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.664+0000 7ff60fe00640 1 -- 172.21.15.102:0/3663208719 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff60001a070 con 0x7ff6180cdfc0 2024-04-20T21:45:26.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.665+0000 7ff60fe00640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff5f406c8f0 0x7ff5f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:26.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.665+0000 7ff60fe00640 1 -- 172.21.15.102:0/3663208719 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7ff60008c700 con 0x7ff6180cdfc0 2024-04-20T21:45:26.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.666+0000 7ff617e00640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff5f406c8f0 0x7ff5f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:26.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.667+0000 7ff617e00640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff5f406c8f0 0x7ff5f406edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff608009560 tx=0x7ff608009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:26.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.671+0000 7ff60fe00640 1 -- 172.21.15.102:0/3663208719 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff618069340 con 0x7ff6180cdfc0 2024-04-20T21:45:26.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.912+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff61804fd60 con 0x7ff6180cdfc0 2024-04-20T21:45:26.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.913+0000 7ff60fe00640 1 -- 172.21.15.102:0/3663208719 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+4377 (secure 0 0 0) 0x7ff600059b60 con 0x7ff6180cdfc0 2024-04-20T21:45:26.912 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:26.912 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:22.627351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":430,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:26.913 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 99 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.915+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff5f406c8f0 msgr2=0x7ff5f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.915+0000 7ff61d600640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff5f406c8f0 0x7ff5f406edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff608009560 tx=0x7ff608009190 comp rx=0 tx=0).stop 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 msgr2=0x7ff6180ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180ceec0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff60000e890 tx=0x7ff60000ed50 comp rx=0 tx=0).stop 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 shutdown_connections 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff5f406c8f0 0x7ff5f406edb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff618155530 0x7ff6180cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6180cdfc0 0x7ff6180ceec0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 --2- 172.21.15.102:0/3663208719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6180cc470 0x7ff61815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:26.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 >> 172.21.15.102:0/3663208719 conn(0x7ff618075080 msgr2=0x7ff6181546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:26.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 shutdown_connections 2024-04-20T21:45:26.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:26.916+0000 7ff61d600640 1 -- 172.21.15.102:0/3663208719 wait complete. 2024-04-20T21:45:26.934 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:28.935 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.142+0000 7efcd6200640 1 Processor -- start 2024-04-20T21:45:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.142+0000 7efcd6200640 1 -- start start 2024-04-20T21:45:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcd6200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd00c75a0 0x7efcd00c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:29.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcd6200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efcd0073c70 0x7efcd00c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:29.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcd6200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd0153560 con 0x7efcd0153180 2024-04-20T21:45:29.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcd6200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7efcd00c7a70 con 0x7efcd0073c70 2024-04-20T21:45:29.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcd6200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efcd00c7d30 con 0x7efcd00c75a0 2024-04-20T21:45:29.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd00c75a0 0x7efcd00c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:29.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcd4e00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7efcd0153180 0x7efcd0158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:58636/0 (socket says 172.21.15.102:58636) 2024-04-20T21:45:29.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcd4e00640 1 -- 172.21.15.102:0/3217523024 learned_addr learned my addr 172.21.15.102:0/3217523024 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efcd5800640 1 --2- 172.21.15.102:0/3217523024 >> v2:172.21.15.102:3300/0 conn(0x7efcd0073c70 0x7efcd00c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efccfe00640 1 -- 172.21.15.102:0/3217523024 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3208825269 0 0) 0x7efcd0153560 con 0x7efcd0153180 2024-04-20T21:45:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.144+0000 7efccfe00640 1 -- 172.21.15.102:0/3217523024 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcb4003520 con 0x7efcd0153180 2024-04-20T21:45:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.145+0000 7efcc7e00640 1 -- 172.21.15.102:0/3217523024 >> v2:172.21.15.102:3300/0 conn(0x7efcd0073c70 msgr2=0x7efcd00c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.145+0000 7efcc7e00640 1 --2- 172.21.15.102:0/3217523024 >> v2:172.21.15.102:3300/0 conn(0x7efcd0073c70 0x7efcd00c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.145+0000 7efcc7e00640 1 -- 172.21.15.102:0/3217523024 >> v1:172.21.15.193:6789/0 conn(0x7efcd0153180 legacy=0x7efcd0158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.145+0000 7efcc7e00640 1 -- 172.21.15.102:0/3217523024 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd015c4d0 con 0x7efcd00c75a0 2024-04-20T21:45:29.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.145+0000 7efcd5800640 1 --2- 172.21.15.102:0/3217523024 >> v2:172.21.15.102:3300/0 conn(0x7efcd0073c70 0x7efcd00c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:29.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.146+0000 7efcc7e00640 1 --2- 172.21.15.102:0/3217523024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd00c75a0 0x7efcd00c5f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efcb80097a0 tx=0x7efcb802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cefc0ec2d06d722c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:29.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.146+0000 7efccfe00640 1 -- 172.21.15.102:0/3217523024 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8035020 con 0x7efcd00c75a0 2024-04-20T21:45:29.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.146+0000 7efccfe00640 1 -- 172.21.15.102:0/3217523024 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb8031810 con 0x7efcd00c75a0 2024-04-20T21:45:29.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.146+0000 7efccfe00640 1 -- 172.21.15.102:0/3217523024 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8031be0 con 0x7efcd00c75a0 2024-04-20T21:45:29.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.147+0000 7efcd6200640 1 -- 172.21.15.102:0/3217523024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd00c75a0 msgr2=0x7efcd00c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:29.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.147+0000 7efcd6200640 1 --2- 172.21.15.102:0/3217523024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd00c75a0 0x7efcd00c5f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efcb80097a0 tx=0x7efcb802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.147+0000 7efcd6200640 1 -- 172.21.15.102:0/3217523024 shutdown_connections 2024-04-20T21:45:29.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.147+0000 7efcd6200640 1 --2- 172.21.15.102:0/3217523024 >> v2:172.21.15.102:3300/0 conn(0x7efcd0073c70 0x7efcd00c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.147+0000 7efcd6200640 1 --2- 172.21.15.102:0/3217523024 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd00c75a0 0x7efcd00c5f30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.147+0000 7efcd6200640 1 -- 172.21.15.102:0/3217523024 >> 172.21.15.102:0/3217523024 conn(0x7efcd00cc820 msgr2=0x7efcd00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:29.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.147+0000 7efcd6200640 1 -- 172.21.15.102:0/3217523024 shutdown_connections 2024-04-20T21:45:29.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.147+0000 7efcd6200640 1 -- 172.21.15.102:0/3217523024 wait complete. 2024-04-20T21:45:29.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.149+0000 7efcd6200640 1 Processor -- start 2024-04-20T21:45:29.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.149+0000 7efcd6200640 1 -- start start 2024-04-20T21:45:29.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcd6200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd0073c70 0x7efcd00c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:29.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcd6200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd00c75a0 0x7efcd0151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:29.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcd6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd0153180 0x7efcd01f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:29.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcd6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0159550 con 0x7efcd0153180 2024-04-20T21:45:29.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcd6200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efcd015ca90 con 0x7efcd00c75a0 2024-04-20T21:45:29.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcd6200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efcd015d290 con 0x7efcd0073c70 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcd5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd0153180 0x7efcd01f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd00c75a0 0x7efcd0151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcd4e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd0073c70 0x7efcd00c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd00c75a0 0x7efcd0151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58904/0 (socket says 172.21.15.102:58904) 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.150+0000 7efcc7e00640 1 -- 172.21.15.102:0/1949483688 learned_addr learned my addr 172.21.15.102:0/1949483688 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcd5800640 1 -- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd0073c70 msgr2=0x7efcd00c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcd5800640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd0073c70 0x7efcd00c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcd5800640 1 -- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd00c75a0 msgr2=0x7efcd0151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcd5800640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd00c75a0 0x7efcd0151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcd5800640 1 -- 172.21.15.102:0/1949483688 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd0158e70 con 0x7efcd0153180 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcd4e00640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd0073c70 0x7efcd00c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:29.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcc7e00640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd00c75a0 0x7efcd0151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:45:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcd5800640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd0153180 0x7efcd01f88e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7efcc800efc0 tx=0x7efcc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efccd600640 1 -- 172.21.15.102:0/1949483688 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8018030 con 0x7efcd0153180 2024-04-20T21:45:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efccd600640 1 -- 172.21.15.102:0/1949483688 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc800cd30 con 0x7efcd0153180 2024-04-20T21:45:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efccd600640 1 -- 172.21.15.102:0/1949483688 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8003bb0 con 0x7efcd0153180 2024-04-20T21:45:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.151+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd0159150 con 0x7efcd0153180 2024-04-20T21:45:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.152+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd00c6040 con 0x7efcd0153180 2024-04-20T21:45:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.153+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd004fd60 con 0x7efcd0153180 2024-04-20T21:45:29.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.154+0000 7efccd600640 1 -- 172.21.15.102:0/1949483688 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcc801b070 con 0x7efcd0153180 2024-04-20T21:45:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.155+0000 7efccd600640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efcac06c8f0 0x7efcac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.155+0000 7efccd600640 1 -- 172.21.15.102:0/1949483688 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7efcc808cda0 con 0x7efcd0153180 2024-04-20T21:45:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.155+0000 7efcd4e00640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efcac06c8f0 0x7efcac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.156+0000 7efcd4e00640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efcac06c8f0 0x7efcac06edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efcc00011e0 tx=0x7efcc0007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.159+0000 7efccd600640 1 -- 172.21.15.102:0/1949483688 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efcc8010040 con 0x7efcd0153180 2024-04-20T21:45:29.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.390+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcd00c3690 con 0x7efcd0153180 2024-04-20T21:45:29.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.391+0000 7efccd600640 1 -- 172.21.15.102:0/1949483688 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4376 (secure 0 0 0) 0x7efcc805a7f0 con 0x7efcd0153180 2024-04-20T21:45:29.390 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:29.391 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:28.386616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:rejoin","state_seq":432,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:29.391 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 100 2024-04-20T21:45:29.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efcac06c8f0 msgr2=0x7efcac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:29.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efcac06c8f0 0x7efcac06edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efcc00011e0 tx=0x7efcc0007d30 comp rx=0 tx=0).stop 2024-04-20T21:45:29.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd0153180 msgr2=0x7efcd01f88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:29.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd0153180 0x7efcd01f88e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7efcc800efc0 tx=0x7efcc800c490 comp rx=0 tx=0).stop 2024-04-20T21:45:29.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 shutdown_connections 2024-04-20T21:45:29.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd0153180 0x7efcd01f88e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efcac06c8f0 0x7efcac06edb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd00c75a0 0x7efcd0151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 --2- 172.21.15.102:0/1949483688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcd0073c70 0x7efcd00c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:29.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 >> 172.21.15.102:0/1949483688 conn(0x7efcd00cc820 msgr2=0x7efcd0148310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:29.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 shutdown_connections 2024-04-20T21:45:29.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:29.394+0000 7efcd6200640 1 -- 172.21.15.102:0/1949483688 wait complete. 2024-04-20T21:45:29.411 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:30.084 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~208s 2024-04-20T21:45:30.084 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~120s 2024-04-20T21:45:31.412 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:31.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.621+0000 7fe959200640 1 Processor -- start 2024-04-20T21:45:31.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.622+0000 7fe959200640 1 -- start start 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.623+0000 7fe959200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415f2b0 0x7fe95406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.623+0000 7fe959200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9540c8f20 con 0x7fe95415ee00 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.623+0000 7fe959200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe95406cb80 con 0x7fe95415ea20 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.623+0000 7fe959200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe95406ce60 con 0x7fe95415f2b0 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.623+0000 7fe949400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe95415ee00 0x7fe954154e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35456/0 (socket says 172.21.15.102:35456) 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.623+0000 7fe949400640 1 -- 172.21.15.102:0/773127632 learned_addr learned my addr 172.21.15.102:0/773127632 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.623+0000 7fe952800640 1 --2- 172.21.15.102:0/773127632 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415f2b0 0x7fe95406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.624+0000 7fe951400640 1 -- 172.21.15.102:0/773127632 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2497971112 0 0) 0x7fe95406cb80 con 0x7fe95415ea20 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.624+0000 7fe951400640 1 -- 172.21.15.102:0/773127632 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe938003520 con 0x7fe95415ea20 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.624+0000 7fe951400640 1 -- 172.21.15.102:0/773127632 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599224984 0 0) 0x7fe9540c8f20 con 0x7fe95415ee00 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.624+0000 7fe951400640 1 -- 172.21.15.102:0/773127632 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe95406cb80 con 0x7fe95415ee00 2024-04-20T21:45:31.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.624+0000 7fe952800640 1 -- 172.21.15.102:0/773127632 >> v1:172.21.15.193:6789/0 conn(0x7fe95415ea20 legacy=0x7fe9540c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:31.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.624+0000 7fe952800640 1 -- 172.21.15.102:0/773127632 >> v1:172.21.15.102:6789/0 conn(0x7fe95415ee00 legacy=0x7fe954154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:31.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.624+0000 7fe952800640 1 -- 172.21.15.102:0/773127632 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95406d140 con 0x7fe95415f2b0 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.625+0000 7fe952800640 1 --2- 172.21.15.102:0/773127632 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415f2b0 0x7fe95406d940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0097d0 tx=0x7fe94c02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6dda724689d5a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.625+0000 7fe951400640 1 -- 172.21.15.102:0/773127632 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c034070 con 0x7fe95415f2b0 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.625+0000 7fe951400640 1 -- 172.21.15.102:0/773127632 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c030880 con 0x7fe95415f2b0 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.625+0000 7fe951400640 1 -- 172.21.15.102:0/773127632 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c030c50 con 0x7fe95415f2b0 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.626+0000 7fe959200640 1 -- 172.21.15.102:0/773127632 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415f2b0 msgr2=0x7fe95406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.626+0000 7fe959200640 1 --2- 172.21.15.102:0/773127632 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415f2b0 0x7fe95406d940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0097d0 tx=0x7fe94c02b9c0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.626+0000 7fe959200640 1 -- 172.21.15.102:0/773127632 shutdown_connections 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.626+0000 7fe959200640 1 --2- 172.21.15.102:0/773127632 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415f2b0 0x7fe95406d940 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.626+0000 7fe959200640 1 -- 172.21.15.102:0/773127632 >> 172.21.15.102:0/773127632 conn(0x7fe9540c9390 msgr2=0x7fe9540c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.626+0000 7fe959200640 1 -- 172.21.15.102:0/773127632 shutdown_connections 2024-04-20T21:45:31.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.626+0000 7fe959200640 1 -- 172.21.15.102:0/773127632 wait complete. 2024-04-20T21:45:31.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.628+0000 7fe959200640 1 Processor -- start 2024-04-20T21:45:31.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.628+0000 7fe959200640 1 -- start start 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe959200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415ea20 0x7fe95414f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe959200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95415ee00 0x7fe95414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe959200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe95415f2b0 0x7fe954157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe959200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9540c8c80 con 0x7fe95415f2b0 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe959200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9540c8b00 con 0x7fe95415ee00 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe959200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe954155590 con 0x7fe95415ea20 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe952800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe95415f2b0 0x7fe954157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe951e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415ea20 0x7fe95414f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe952800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe95415f2b0 0x7fe954157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51632/0 (socket says 172.21.15.102:51632) 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe949400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95415ee00 0x7fe95414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.629+0000 7fe952800640 1 -- 172.21.15.102:0/1497260979 learned_addr learned my addr 172.21.15.102:0/1497260979 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe952800640 1 -- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415ea20 msgr2=0x7fe95414f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:31.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe952800640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415ea20 0x7fe95414f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe952800640 1 -- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95415ee00 msgr2=0x7fe95414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:31.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe952800640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95415ee00 0x7fe95414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe952800640 1 -- 172.21.15.102:0/1497260979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9541f18b0 con 0x7fe95415f2b0 2024-04-20T21:45:31.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe949400640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95415ee00 0x7fe95414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:31.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe952800640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe95415f2b0 0x7fe954157ab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0098f0 tx=0x7fe94c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:31.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe94aa00640 1 -- 172.21.15.102:0/1497260979 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c034040 con 0x7fe95415f2b0 2024-04-20T21:45:31.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe94aa00640 1 -- 172.21.15.102:0/1497260979 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c03ad20 con 0x7fe95415f2b0 2024-04-20T21:45:31.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe94aa00640 1 -- 172.21.15.102:0/1497260979 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c006e10 con 0x7fe95415f2b0 2024-04-20T21:45:31.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.630+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9541f1b70 con 0x7fe95415f2b0 2024-04-20T21:45:31.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.631+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9540cae30 con 0x7fe95415f2b0 2024-04-20T21:45:31.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.632+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe954074340 con 0x7fe95415f2b0 2024-04-20T21:45:31.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.633+0000 7fe94aa00640 1 -- 172.21.15.102:0/1497260979 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe94c037070 con 0x7fe95415f2b0 2024-04-20T21:45:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.634+0000 7fe94aa00640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe93006c8f0 0x7fe93006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:31.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.634+0000 7fe94aa00640 1 -- 172.21.15.102:0/1497260979 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7fe94c0ae530 con 0x7fe95415f2b0 2024-04-20T21:45:31.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.634+0000 7fe951e00640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe93006c8f0 0x7fe93006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:31.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.635+0000 7fe951e00640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe93006c8f0 0x7fe93006edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe944007fd0 tx=0x7fe944009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:31.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.639+0000 7fe94aa00640 1 -- 172.21.15.102:0/1497260979 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe94c03f070 con 0x7fe95415f2b0 2024-04-20T21:45:31.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.879+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe95406ea70 con 0x7fe95415f2b0 2024-04-20T21:45:31.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.880+0000 7fe94aa00640 1 -- 172.21.15.102:0/1497260979 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4376 (secure 0 0 0) 0x7fe94c07ba20 con 0x7fe95415f2b0 2024-04-20T21:45:31.879 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:31.879 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:28.386616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:rejoin","state_seq":432,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:31.879 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 100 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.882+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe93006c8f0 msgr2=0x7fe93006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.882+0000 7fe959200640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe93006c8f0 0x7fe93006edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe944007fd0 tx=0x7fe944009040 comp rx=0 tx=0).stop 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.882+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe95415f2b0 msgr2=0x7fe954157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.882+0000 7fe959200640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe95415f2b0 0x7fe954157ab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0098f0 tx=0x7fe94c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.883+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 shutdown_connections 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.883+0000 7fe959200640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe95415f2b0 0x7fe954157ab0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.883+0000 7fe959200640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe93006c8f0 0x7fe93006edb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.883+0000 7fe959200640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95415ee00 0x7fe95414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.883+0000 7fe959200640 1 --2- 172.21.15.102:0/1497260979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe95415ea20 0x7fe95414f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.883+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 >> 172.21.15.102:0/1497260979 conn(0x7fe9540c9390 msgr2=0x7fe9541496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:31.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.883+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 shutdown_connections 2024-04-20T21:45:31.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:31.883+0000 7fe959200640 1 -- 172.21.15.102:0/1497260979 wait complete. 2024-04-20T21:45:31.901 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:32.049 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:45:32.057 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:45:33.902 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.113+0000 7f63a8800640 1 Processor -- start 2024-04-20T21:45:34.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.113+0000 7f63a8800640 1 -- start start 2024-04-20T21:45:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.115+0000 7f63a8800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f63a0163220 0x7f63a0159440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.115+0000 7f63a8800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a006d2e0 con 0x7f63a0163660 2024-04-20T21:45:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.115+0000 7f63a8800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a006d5c0 con 0x7f63a0162e40 2024-04-20T21:45:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.115+0000 7f63a8800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f63a00c7290 con 0x7f63a0163220 2024-04-20T21:45:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.115+0000 7f63a6800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f63a0163660 0x7f63a00c7a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35466/0 (socket says 172.21.15.102:35466) 2024-04-20T21:45:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.115+0000 7f63a6800640 1 -- 172.21.15.102:0/2877789346 learned_addr learned my addr 172.21.15.102:0/2877789346 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.115+0000 7f63a5e00640 1 --1- 172.21.15.102:0/2877789346 >> v1:172.21.15.193:6789/0 conn(0x7f63a0162e40 0x7f63a006c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:58656/0 (socket says 172.21.15.102:58656) 2024-04-20T21:45:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.115+0000 7f63a5400640 1 --2- 172.21.15.102:0/2877789346 >> v2:172.21.15.193:3300/0 conn(0x7f63a0163220 0x7f63a0159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:34.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.116+0000 7f63a4a00640 1 -- 172.21.15.102:0/2877789346 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559744547 0 0) 0x7f63a006d5c0 con 0x7f63a0162e40 2024-04-20T21:45:34.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.116+0000 7f63a4a00640 1 -- 172.21.15.102:0/2877789346 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6388003520 con 0x7f63a0162e40 2024-04-20T21:45:34.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.116+0000 7f63a5400640 1 -- 172.21.15.102:0/2877789346 >> v1:172.21.15.193:6789/0 conn(0x7f63a0162e40 legacy=0x7f63a006c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:34.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.116+0000 7f63a5400640 1 -- 172.21.15.102:0/2877789346 >> v1:172.21.15.102:6789/0 conn(0x7f63a0163660 legacy=0x7f63a00c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:34.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.116+0000 7f63a4a00640 1 -- 172.21.15.102:0/2877789346 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3855341719 0 0) 0x7f63a006d2e0 con 0x7f63a0163660 2024-04-20T21:45:34.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.116+0000 7f63a5400640 1 -- 172.21.15.102:0/2877789346 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a006d8a0 con 0x7f63a0163220 2024-04-20T21:45:34.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.117+0000 7f63a5400640 1 --2- 172.21.15.102:0/2877789346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 0x7f63a0159440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f63900097a0 tx=0x7f639002b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478d5a46040f3a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:34.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.117+0000 7f63a4a00640 1 -- 172.21.15.102:0/2877789346 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639002f810 con 0x7f63a0163220 2024-04-20T21:45:34.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.117+0000 7f63a4a00640 1 -- 172.21.15.102:0/2877789346 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639002f9b0 con 0x7f63a0163220 2024-04-20T21:45:34.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.117+0000 7f63a4a00640 1 -- 172.21.15.102:0/2877789346 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639002fd80 con 0x7f63a0163220 2024-04-20T21:45:34.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.118+0000 7f63a8800640 1 -- 172.21.15.102:0/2877789346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 msgr2=0x7f63a0159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:34.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.118+0000 7f63a8800640 1 --2- 172.21.15.102:0/2877789346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 0x7f63a0159440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f63900097a0 tx=0x7f639002b9f0 comp rx=0 tx=0).stop 2024-04-20T21:45:34.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.118+0000 7f63a8800640 1 -- 172.21.15.102:0/2877789346 shutdown_connections 2024-04-20T21:45:34.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.118+0000 7f63a8800640 1 --2- 172.21.15.102:0/2877789346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 0x7f63a0159440 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:34.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.118+0000 7f63a8800640 1 -- 172.21.15.102:0/2877789346 >> 172.21.15.102:0/2877789346 conn(0x7f63a00c9390 msgr2=0x7f63a00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:34.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.118+0000 7f63a8800640 1 -- 172.21.15.102:0/2877789346 shutdown_connections 2024-04-20T21:45:34.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.118+0000 7f63a8800640 1 -- 172.21.15.102:0/2877789346 wait complete. 2024-04-20T21:45:34.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.119+0000 7f63a8800640 1 Processor -- start 2024-04-20T21:45:34.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.119+0000 7f63a8800640 1 -- start start 2024-04-20T21:45:34.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.120+0000 7f63a8800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63a0162e40 0x7f63a0153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.120+0000 7f63a8800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 0x7f63a0162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.120+0000 7f63a8800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a0163660 0x7f63a014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.120+0000 7f63a8800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63a00c9080 con 0x7f63a0163660 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.120+0000 7f63a8800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63a00c7570 con 0x7f63a0163220 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.120+0000 7f63a8800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f63a00c8110 con 0x7f63a0162e40 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63a0162e40 0x7f63a0153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a0163660 0x7f63a014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 0x7f63a0162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63a0162e40 0x7f63a0153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:34528/0 (socket says 172.21.15.102:34528) 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 -- 172.21.15.102:0/3660443230 learned_addr learned my addr 172.21.15.102:0/3660443230 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 -- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 msgr2=0x7f63a0162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:34.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 0x7f63a0162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 -- 172.21.15.102:0/3660443230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a0163660 msgr2=0x7f63a014cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a0163660 0x7f63a014cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 -- 172.21.15.102:0/3660443230 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a00c4000 con 0x7f63a0162e40 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.121+0000 7f63a5e00640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63a0162e40 0x7f63a0153180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f639400c460 tx=0x7f639400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.122+0000 7f639a000640 1 -- 172.21.15.102:0/3660443230 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394018070 con 0x7f63a0162e40 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.122+0000 7f639a000640 1 -- 172.21.15.102:0/3660443230 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63940074a0 con 0x7f63a0162e40 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.122+0000 7f639a000640 1 -- 172.21.15.102:0/3660443230 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394004a20 con 0x7f63a0162e40 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.122+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a00c42e0 con 0x7f63a0162e40 2024-04-20T21:45:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.122+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a00c48a0 con 0x7f63a0162e40 2024-04-20T21:45:34.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.123+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a0074340 con 0x7f63a0162e40 2024-04-20T21:45:34.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.124+0000 7f639a000640 1 -- 172.21.15.102:0/3660443230 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f639401b070 con 0x7f63a0162e40 2024-04-20T21:45:34.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.125+0000 7f639a000640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f638406c920 0x7f638406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:34.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.125+0000 7f639a000640 1 -- 172.21.15.102:0/3660443230 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f639408b6f0 con 0x7f63a0162e40 2024-04-20T21:45:34.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.125+0000 7f63a5400640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f638406c920 0x7f638406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:34.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.126+0000 7f63a5400640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f638406c920 0x7f638406ede0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6390036000 tx=0x7f6390008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:34.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.129+0000 7f639a000640 1 -- 172.21.15.102:0/3660443230 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6394007650 con 0x7f63a0162e40 2024-04-20T21:45:34.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.373+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63a00c60c0 con 0x7f63a0162e40 2024-04-20T21:45:34.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.374+0000 7f639a000640 1 -- 172.21.15.102:0/3660443230 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4376 (secure 0 0 0) 0x7f6394058bb0 con 0x7f63a0162e40 2024-04-20T21:45:34.373 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:34.373 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:28.386616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:rejoin","state_seq":432,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:34.373 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 100 2024-04-20T21:45:34.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.376+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f638406c920 msgr2=0x7f638406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:34.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.376+0000 7f63a8800640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f638406c920 0x7f638406ede0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6390036000 tx=0x7f6390008030 comp rx=0 tx=0).stop 2024-04-20T21:45:34.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63a0162e40 msgr2=0x7f63a0153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:34.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63a0162e40 0x7f63a0153180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f639400c460 tx=0x7f639400c920 comp rx=0 tx=0).stop 2024-04-20T21:45:34.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 shutdown_connections 2024-04-20T21:45:34.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a0163660 0x7f63a014cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:34.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f638406c920 0x7f638406ede0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:34.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0163220 0x7f63a0162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:34.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 --2- 172.21.15.102:0/3660443230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63a0162e40 0x7f63a0153180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:34.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 >> 172.21.15.102:0/3660443230 conn(0x7f63a00c9390 msgr2=0x7f63a0157490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:34.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 shutdown_connections 2024-04-20T21:45:34.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:34.377+0000 7f63a8800640 1 -- 172.21.15.102:0/3660443230 wait complete. 2024-04-20T21:45:34.395 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:36.396 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:36.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.606+0000 7f0154a00640 1 Processor -- start 2024-04-20T21:45:36.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.606+0000 7f0154a00640 1 -- start start 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.607+0000 7f0154a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0150152290 0x7f01500cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.607+0000 7f0154a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01500c3700 0x7f015006e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.607+0000 7f0154a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015015c220 con 0x7f015006eb70 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.607+0000 7f0154a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f01500c9180 con 0x7f01500c3700 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.607+0000 7f0154a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f01500c4840 con 0x7f0150152290 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.608+0000 7f0147e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01500c3700 0x7f015006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.608+0000 7f014fe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f015006eb70 0x7f01500c8330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35480/0 (socket says 172.21.15.102:35480) 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.608+0000 7f014fe00640 1 -- 172.21.15.102:0/3771415706 learned_addr learned my addr 172.21.15.102:0/3771415706 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:36.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.608+0000 7f014f400640 1 --2- 172.21.15.102:0/3771415706 >> v2:172.21.15.193:3300/0 conn(0x7f0150152290 0x7f01500cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.608+0000 7f0147e00640 1 -- 172.21.15.102:0/3771415706 >> v2:172.21.15.193:3300/0 conn(0x7f0150152290 msgr2=0x7f01500cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.608+0000 7f0147e00640 1 --2- 172.21.15.102:0/3771415706 >> v2:172.21.15.193:3300/0 conn(0x7f0150152290 0x7f01500cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.608+0000 7f0147e00640 1 -- 172.21.15.102:0/3771415706 >> v1:172.21.15.102:6789/0 conn(0x7f015006eb70 legacy=0x7f01500c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.608+0000 7f0147e00640 1 -- 172.21.15.102:0/3771415706 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01500c4560 con 0x7f01500c3700 2024-04-20T21:45:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.609+0000 7f0147e00640 1 --2- 172.21.15.102:0/3771415706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f015006e630 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f01500c3eb0 tx=0x7f013802db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65475934c96ed4b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f014ea00640 1 -- 172.21.15.102:0/3771415706 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013803b070 con 0x7f01500c3700 2024-04-20T21:45:36.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f014ea00640 1 -- 172.21.15.102:0/3771415706 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f013802e920 con 0x7f01500c3700 2024-04-20T21:45:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f014ea00640 1 -- 172.21.15.102:0/3771415706 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013802ecf0 con 0x7f01500c3700 2024-04-20T21:45:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f0154a00640 1 -- 172.21.15.102:0/3771415706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 msgr2=0x7f015006e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f0154a00640 1 --2- 172.21.15.102:0/3771415706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f015006e630 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f01500c3eb0 tx=0x7f013802db70 comp rx=0 tx=0).stop 2024-04-20T21:45:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f0154a00640 1 -- 172.21.15.102:0/3771415706 shutdown_connections 2024-04-20T21:45:36.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f0154a00640 1 --2- 172.21.15.102:0/3771415706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f015006e630 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f0154a00640 1 --2- 172.21.15.102:0/3771415706 >> v2:172.21.15.193:3300/0 conn(0x7f0150152290 0x7f01500cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f0154a00640 1 -- 172.21.15.102:0/3771415706 >> 172.21.15.102:0/3771415706 conn(0x7f01500cc820 msgr2=0x7f01500ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f0154a00640 1 -- 172.21.15.102:0/3771415706 shutdown_connections 2024-04-20T21:45:36.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.610+0000 7f0154a00640 1 -- 172.21.15.102:0/3771415706 wait complete. 2024-04-20T21:45:36.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.612+0000 7f0154a00640 1 Processor -- start 2024-04-20T21:45:36.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.612+0000 7f0154a00640 1 -- start start 2024-04-20T21:45:36.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0154a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f015006eb70 0x7f01500cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:36.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0154a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f0150155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:36.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0154a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0150152290 0x7f0150157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:36.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0154a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f015015ba20 con 0x7f01500c3700 2024-04-20T21:45:36.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0154a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f015015b740 con 0x7f0150152290 2024-04-20T21:45:36.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0154a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f015015bd00 con 0x7f015006eb70 2024-04-20T21:45:36.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0147e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f0150155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:36.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f014f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f015006eb70 0x7f01500cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:36.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f014fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0150152290 0x7f0150157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:36.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0147e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f0150155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51648/0 (socket says 172.21.15.102:51648) 2024-04-20T21:45:36.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.613+0000 7f0147e00640 1 -- 172.21.15.102:0/755656911 learned_addr learned my addr 172.21.15.102:0/755656911 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147e00640 1 -- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f015006eb70 msgr2=0x7f01500cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147e00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f015006eb70 0x7f01500cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147e00640 1 -- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0150152290 msgr2=0x7f0150157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147e00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0150152290 0x7f0150157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147e00640 1 -- 172.21.15.102:0/755656911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01501f0570 con 0x7f01500c3700 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f014f400640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f015006eb70 0x7f01500cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147e00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f0150155990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0138000c00 tx=0x7f01380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147400640 1 -- 172.21.15.102:0/755656911 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013803b040 con 0x7f01500c3700 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147400640 1 -- 172.21.15.102:0/755656911 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0138046070 con 0x7f01500c3700 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0147400640 1 -- 172.21.15.102:0/755656911 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138036ce0 con 0x7f01500c3700 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.614+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01501f0850 con 0x7f01500c3700 2024-04-20T21:45:36.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.615+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01501f0e10 con 0x7f01500c3700 2024-04-20T21:45:36.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.616+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015004fd60 con 0x7f01500c3700 2024-04-20T21:45:36.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.617+0000 7f0147400640 1 -- 172.21.15.102:0/755656911 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f013804b460 con 0x7f01500c3700 2024-04-20T21:45:36.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.617+0000 7f0147400640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f012c06c920 0x7f012c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:36.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.618+0000 7f0147400640 1 -- 172.21.15.102:0/755656911 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f01380b0490 con 0x7f01500c3700 2024-04-20T21:45:36.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.618+0000 7f014f400640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f012c06c920 0x7f012c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:36.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.619+0000 7f014f400640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f012c06c920 0x7f012c06ede0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0140009f30 tx=0x7f0140009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:36.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.623+0000 7f0147400640 1 -- 172.21.15.102:0/755656911 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f013807d800 con 0x7f01500c3700 2024-04-20T21:45:36.697 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~215s 2024-04-20T21:45:36.697 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~127s 2024-04-20T21:45:36.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.863+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01500c32d0 con 0x7f01500c3700 2024-04-20T21:45:36.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.864+0000 7f0147400640 1 -- 172.21.15.102:0/755656911 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+4382 (secure 0 0 0) 0x7f0138081b00 con 0x7f01500c3700 2024-04-20T21:45:36.863 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:36.863 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:34.031388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:clientreplay","state_seq":434,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:36.863 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 101 2024-04-20T21:45:36.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.866+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f012c06c920 msgr2=0x7f012c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.866+0000 7f0154a00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f012c06c920 0x7f012c06ede0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0140009f30 tx=0x7f0140009190 comp rx=0 tx=0).stop 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.866+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 msgr2=0x7f0150155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.866+0000 7f0154a00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f0150155990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0138000c00 tx=0x7f01380046c0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.867+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 shutdown_connections 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.867+0000 7f0154a00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f012c06c920 0x7f012c06ede0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.867+0000 7f0154a00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0150152290 0x7f0150157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.867+0000 7f0154a00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01500c3700 0x7f0150155990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.867+0000 7f0154a00640 1 --2- 172.21.15.102:0/755656911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f015006eb70 0x7f01500cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:36.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.867+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 >> 172.21.15.102:0/755656911 conn(0x7f01500cc820 msgr2=0x7f01501470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:36.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.867+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 shutdown_connections 2024-04-20T21:45:36.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:36.867+0000 7f0154a00640 1 -- 172.21.15.102:0/755656911 wait complete. 2024-04-20T21:45:36.884 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 101, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 101, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4282": { "addr": "172.21.15.193:6838/2467124512", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6835", "nonce": 2467124512, "type": "v2" }, { "addr": "172.21.15.193:6838", "nonce": 2467124512, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4282, "incarnation": 98, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:clientreplay", "state_seq": 434 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:45:34.031388+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4282 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:45:38.885 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:39.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.094+0000 7fbf7ea00640 1 Processor -- start 2024-04-20T21:45:39.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.094+0000 7fbf7ea00640 1 -- start start 2024-04-20T21:45:39.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf78155530 0x7fbf7815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:39.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbf780cb6a0 0x7fbf780cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:39.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7ea00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7815ef60 con 0x7fbf780cc470 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7ea00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fbf780c5ca0 con 0x7fbf780cb6a0 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7ea00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbf780cd5f0 con 0x7fbf78155530 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7cc00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbf780cc470 0x7fbf780cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59264/0 (socket says 172.21.15.102:59264) 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7cc00640 1 -- 172.21.15.102:0/911807936 learned_addr learned my addr 172.21.15.102:0/911807936 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7e000640 1 --2- 172.21.15.102:0/911807936 >> v2:172.21.15.102:3300/0 conn(0x7fbf780cb6a0 0x7fbf780cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf7d600640 1 --2- 172.21.15.102:0/911807936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf78155530 0x7fbf7815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf77e00640 1 -- 172.21.15.102:0/911807936 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066720786 0 0) 0x7fbf7815ef60 con 0x7fbf780cc470 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.095+0000 7fbf77e00640 1 -- 172.21.15.102:0/911807936 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf60003520 con 0x7fbf780cc470 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.096+0000 7fbf7d600640 1 -- 172.21.15.102:0/911807936 >> v2:172.21.15.102:3300/0 conn(0x7fbf780cb6a0 msgr2=0x7fbf780cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.096+0000 7fbf7d600640 1 --2- 172.21.15.102:0/911807936 >> v2:172.21.15.102:3300/0 conn(0x7fbf780cb6a0 0x7fbf780cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.096+0000 7fbf7d600640 1 -- 172.21.15.102:0/911807936 >> v1:172.21.15.102:6789/0 conn(0x7fbf780cc470 legacy=0x7fbf780cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.096+0000 7fbf7d600640 1 -- 172.21.15.102:0/911807936 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7815f240 con 0x7fbf78155530 2024-04-20T21:45:39.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.097+0000 7fbf7d600640 1 --2- 172.21.15.102:0/911807936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf78155530 0x7fbf7815e640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0097a0 tx=0x7fbf6c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec333f66004549e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:39.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.097+0000 7fbf77e00640 1 -- 172.21.15.102:0/911807936 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c034070 con 0x7fbf78155530 2024-04-20T21:45:39.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.097+0000 7fbf77e00640 1 -- 172.21.15.102:0/911807936 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6c030800 con 0x7fbf78155530 2024-04-20T21:45:39.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.097+0000 7fbf77e00640 1 -- 172.21.15.102:0/911807936 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c030bb0 con 0x7fbf78155530 2024-04-20T21:45:39.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.098+0000 7fbf7ea00640 1 -- 172.21.15.102:0/911807936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf78155530 msgr2=0x7fbf7815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:39.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.098+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/911807936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf78155530 0x7fbf7815e640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0097a0 tx=0x7fbf6c02b940 comp rx=0 tx=0).stop 2024-04-20T21:45:39.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.098+0000 7fbf7ea00640 1 -- 172.21.15.102:0/911807936 shutdown_connections 2024-04-20T21:45:39.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.098+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/911807936 >> v2:172.21.15.102:3300/0 conn(0x7fbf780cb6a0 0x7fbf780cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.098+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/911807936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf78155530 0x7fbf7815e640 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.098+0000 7fbf7ea00640 1 -- 172.21.15.102:0/911807936 >> 172.21.15.102:0/911807936 conn(0x7fbf78075080 msgr2=0x7fbf78075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:39.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.098+0000 7fbf7ea00640 1 -- 172.21.15.102:0/911807936 shutdown_connections 2024-04-20T21:45:39.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.098+0000 7fbf7ea00640 1 -- 172.21.15.102:0/911807936 wait complete. 2024-04-20T21:45:39.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.100+0000 7fbf7ea00640 1 Processor -- start 2024-04-20T21:45:39.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.100+0000 7fbf7ea00640 1 -- start start 2024-04-20T21:45:39.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.101+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf780cb6a0 0x7fbf7814ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.101+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf780cc470 0x7fbf781f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.101+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf78155530 0x7fbf7815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.101+0000 7fbf7ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf780cd280 con 0x7fbf78155530 2024-04-20T21:45:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.101+0000 7fbf7ea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf780ccfa0 con 0x7fbf780cb6a0 2024-04-20T21:45:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.101+0000 7fbf7ea00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbf780ccbe0 con 0x7fbf780cc470 2024-04-20T21:45:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf780cc470 0x7fbf781f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf780cb6a0 0x7fbf7814ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf78155530 0x7fbf7815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf780cc470 0x7fbf781f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:41338/0 (socket says 172.21.15.102:41338) 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 -- 172.21.15.102:0/2971776647 learned_addr learned my addr 172.21.15.102:0/2971776647 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf780cb6a0 0x7fbf7814ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54290/0 (socket says 172.21.15.102:54290) 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 -- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf780cb6a0 msgr2=0x7fbf7814ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf780cb6a0 0x7fbf7814ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 -- 172.21.15.102:0/2971776647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf78155530 msgr2=0x7fbf7815c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf78155530 0x7fbf7815c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 -- 172.21.15.102:0/2971776647 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf781f1350 con 0x7fbf780cc470 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.102+0000 7fbf7cc00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf780cc470 0x7fbf781f4db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c460 tx=0x7fbf6400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.103+0000 7fbf75600640 1 -- 172.21.15.102:0/2971776647 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64017070 con 0x7fbf780cc470 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.103+0000 7fbf75600640 1 -- 172.21.15.102:0/2971776647 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64005ce0 con 0x7fbf780cc470 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.103+0000 7fbf75600640 1 -- 172.21.15.102:0/2971776647 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64007290 con 0x7fbf780cc470 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.103+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf781f1610 con 0x7fbf780cc470 2024-04-20T21:45:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.103+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf781f1bd0 con 0x7fbf780cc470 2024-04-20T21:45:39.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.104+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf78069340 con 0x7fbf780cc470 2024-04-20T21:45:39.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.105+0000 7fbf75600640 1 -- 172.21.15.102:0/2971776647 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf6401a070 con 0x7fbf780cc470 2024-04-20T21:45:39.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.106+0000 7fbf75600640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf5806c920 0x7fbf5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:39.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.106+0000 7fbf75600640 1 -- 172.21.15.102:0/2971776647 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7fbf6408b8d0 con 0x7fbf780cc470 2024-04-20T21:45:39.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.106+0000 7fbf7d600640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf5806c920 0x7fbf5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:39.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.107+0000 7fbf7d600640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf5806c920 0x7fbf5806ede0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c036000 tx=0x7fbf6c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:39.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.110+0000 7fbf75600640 1 -- 172.21.15.102:0/2971776647 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbf78069340 con 0x7fbf780cc470 2024-04-20T21:45:39.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.351+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf7804fd60 con 0x7fbf780cc470 2024-04-20T21:45:39.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.352+0000 7fbf75600640 1 -- 172.21.15.102:0/2971776647 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+4382 (secure 0 0 0) 0x7fbf64058cc0 con 0x7fbf780cc470 2024-04-20T21:45:39.350 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:39.350 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:34.031388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:clientreplay","state_seq":434,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:45:39.351 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 101 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.353+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf5806c920 msgr2=0x7fbf5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.353+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf5806c920 0x7fbf5806ede0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c036000 tx=0x7fbf6c03c040 comp rx=0 tx=0).stop 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.353+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf780cc470 msgr2=0x7fbf781f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.353+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf780cc470 0x7fbf781f4db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c460 tx=0x7fbf6400c920 comp rx=0 tx=0).stop 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.353+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 shutdown_connections 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.353+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf78155530 0x7fbf7815c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.353+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf780cc470 0x7fbf781f4db0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.354+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbf5806c920 0x7fbf5806ede0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.354+0000 7fbf7ea00640 1 --2- 172.21.15.102:0/2971776647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf780cb6a0 0x7fbf7814ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:39.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.354+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 >> 172.21.15.102:0/2971776647 conn(0x7fbf78075080 msgr2=0x7fbf78150ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:39.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.354+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 shutdown_connections 2024-04-20T21:45:39.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:39.354+0000 7fbf7ea00640 1 -- 172.21.15.102:0/2971776647 wait complete. 2024-04-20T21:45:39.374 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:45:41.374 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:41.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.588+0000 7f8d27600640 1 Processor -- start 2024-04-20T21:45:41.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.588+0000 7f8d27600640 1 -- start start 2024-04-20T21:45:41.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d27600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8d200c8db0 0x7f8d20070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:41.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d27600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20071330 0x7f8d201628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d27600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d20158e80 con 0x7f8d20152170 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d27600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8d200c6dc0 con 0x7f8d200c8db0 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d27600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8d20071f70 con 0x7f8d20071330 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d1fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8d200c8db0 0x7f8d20070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d25600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20071330 0x7f8d201628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d24c00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f8d20152170 0x7f8d20158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:33962/0 (socket says 172.21.15.102:33962) 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d24c00640 1 -- 172.21.15.102:0/1131539184 learned_addr learned my addr 172.21.15.102:0/1131539184 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d1fe00640 1 -- 172.21.15.102:0/1131539184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20071330 msgr2=0x7f8d201628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:41.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d1fe00640 1 --2- 172.21.15.102:0/1131539184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20071330 0x7f8d201628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.590+0000 7f8d1fe00640 1 -- 172.21.15.102:0/1131539184 >> v1:172.21.15.193:6789/0 conn(0x7f8d20152170 legacy=0x7f8d20158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:41.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.591+0000 7f8d1fe00640 1 -- 172.21.15.102:0/1131539184 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d200c91c0 con 0x7f8d200c8db0 2024-04-20T21:45:41.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.592+0000 7f8d1fe00640 1 --2- 172.21.15.102:0/1131539184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d200c8db0 0x7f8d20070df0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0097a0 tx=0x7f8d0c02b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ec5d0cb5f0e2d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.592+0000 7f8d1f400640 1 -- 172.21.15.102:0/1131539184 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c037070 con 0x7f8d200c8db0 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.592+0000 7f8d1f400640 1 -- 172.21.15.102:0/1131539184 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0c030710 con 0x7f8d200c8db0 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.592+0000 7f8d1f400640 1 -- 172.21.15.102:0/1131539184 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c030ae0 con 0x7f8d200c8db0 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.592+0000 7f8d27600640 1 -- 172.21.15.102:0/1131539184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d200c8db0 msgr2=0x7f8d20070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.592+0000 7f8d27600640 1 --2- 172.21.15.102:0/1131539184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d200c8db0 0x7f8d20070df0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0097a0 tx=0x7f8d0c02b710 comp rx=0 tx=0).stop 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.593+0000 7f8d27600640 1 -- 172.21.15.102:0/1131539184 shutdown_connections 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.593+0000 7f8d27600640 1 --2- 172.21.15.102:0/1131539184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20071330 0x7f8d201628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.593+0000 7f8d27600640 1 --2- 172.21.15.102:0/1131539184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d200c8db0 0x7f8d20070df0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.593+0000 7f8d27600640 1 -- 172.21.15.102:0/1131539184 >> 172.21.15.102:0/1131539184 conn(0x7f8d200c9390 msgr2=0x7f8d200c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:41.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.593+0000 7f8d27600640 1 -- 172.21.15.102:0/1131539184 shutdown_connections 2024-04-20T21:45:41.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.593+0000 7f8d27600640 1 -- 172.21.15.102:0/1131539184 wait complete. 2024-04-20T21:45:41.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.594+0000 7f8d27600640 1 Processor -- start 2024-04-20T21:45:41.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.594+0000 7f8d27600640 1 -- start start 2024-04-20T21:45:41.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.595+0000 7f8d27600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d20071330 0x7f8d201590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.595+0000 7f8d27600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d200c8db0 0x7f8d201f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.595+0000 7f8d27600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20152170 0x7f8d200cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.595+0000 7f8d27600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8d200724b0 con 0x7f8d200c8db0 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.595+0000 7f8d27600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20072210 con 0x7f8d20071330 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.595+0000 7f8d27600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8d20158250 con 0x7f8d20152170 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d200c8db0 0x7f8d201f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d200c8db0 0x7f8d201f1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50286/0 (socket says 172.21.15.102:50286) 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 -- 172.21.15.102:0/2558542824 learned_addr learned my addr 172.21.15.102:0/2558542824 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d24c00640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d20071330 0x7f8d201590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d25600640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20152170 0x7f8d200cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 -- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20152170 msgr2=0x7f8d200cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20152170 0x7f8d200cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 -- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d20071330 msgr2=0x7f8d201590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d20071330 0x7f8d201590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 -- 172.21.15.102:0/2558542824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2015b0b0 con 0x7f8d200c8db0 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d24c00640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d20071330 0x7f8d201590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.596+0000 7f8d1fe00640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d200c8db0 0x7f8d201f1ab0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c035040 tx=0x7f8d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:41.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.597+0000 7f8d1cc00640 1 -- 172.21.15.102:0/2558542824 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c041030 con 0x7f8d200c8db0 2024-04-20T21:45:41.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.597+0000 7f8d1cc00640 1 -- 172.21.15.102:0/2558542824 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0c03c670 con 0x7f8d200c8db0 2024-04-20T21:45:41.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.597+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d201f0ff0 con 0x7f8d200c8db0 2024-04-20T21:45:41.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.597+0000 7f8d1cc00640 1 -- 172.21.15.102:0/2558542824 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c037040 con 0x7f8d200c8db0 2024-04-20T21:45:41.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.597+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d201f12b0 con 0x7f8d200c8db0 2024-04-20T21:45:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.598+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20074340 con 0x7f8d200c8db0 2024-04-20T21:45:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.599+0000 7f8d1cc00640 1 -- 172.21.15.102:0/2558542824 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d0c044070 con 0x7f8d200c8db0 2024-04-20T21:45:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.600+0000 7f8d1cc00640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d0006c8f0 0x7f8d0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:41.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.600+0000 7f8d1cc00640 1 -- 172.21.15.102:0/2558542824 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f8d0c0addf0 con 0x7f8d200c8db0 2024-04-20T21:45:41.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.600+0000 7f8d24c00640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d0006c8f0 0x7f8d0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:41.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.601+0000 7f8d24c00640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d0006c8f0 0x7f8d0006edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8d14000f60 tx=0x7f8d14007e00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:41.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.605+0000 7f8d1cc00640 1 -- 172.21.15.102:0/2558542824 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8d0c07b1f0 con 0x7f8d200c8db0 2024-04-20T21:45:41.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.853+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d200c6320 con 0x7f8d200c8db0 2024-04-20T21:45:41.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.854+0000 7f8d1cc00640 1 -- 172.21.15.102:0/2558542824 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+4383 (secure 0 0 0) 0x7f8d0c07f4f0 con 0x7f8d200c8db0 2024-04-20T21:45:41.852 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:41.853 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":102,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:45:41.853 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 102 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.857+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d0006c8f0 msgr2=0x7f8d0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.857+0000 7f8d27600640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d0006c8f0 0x7f8d0006edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8d14000f60 tx=0x7f8d14007e00 comp rx=0 tx=0).stop 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.857+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d200c8db0 msgr2=0x7f8d201f1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.857+0000 7f8d27600640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d200c8db0 0x7f8d201f1ab0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c035040 tx=0x7f8d0c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.858+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 shutdown_connections 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.858+0000 7f8d27600640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d20152170 0x7f8d200cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.858+0000 7f8d27600640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d200c8db0 0x7f8d201f1ab0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.858+0000 7f8d27600640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d0006c8f0 0x7f8d0006edb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.858+0000 7f8d27600640 1 --2- 172.21.15.102:0/2558542824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d20071330 0x7f8d201590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.858+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 >> 172.21.15.102:0/2558542824 conn(0x7f8d200c9390 msgr2=0x7f8d200c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.858+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 shutdown_connections 2024-04-20T21:45:41.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:41.858+0000 7f8d27600640 1 -- 172.21.15.102:0/2558542824 wait complete. 2024-04-20T21:45:41.876 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=6873 2024-04-20T21:45:41.876 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.c 2024-04-20T21:45:43.310 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~221s 2024-04-20T21:45:43.310 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~134s 2024-04-20T21:45:47.877 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T21:45:47.877 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:45:47.877 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:45:47.886 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:45:47.886 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:48.040 INFO:tasks.ceph.mds.c.smithi102.stdout:starting mds.c at 2024-04-20T21:45:48.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.067+0000 7fb05dc00640 1 Processor -- start 2024-04-20T21:45:48.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.067+0000 7fb05dc00640 1 -- start start 2024-04-20T21:45:48.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.068+0000 7fb05dc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb058152170 0x7fb0580c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:48.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.068+0000 7fb05dc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb058071f70 con 0x7fb058158770 2024-04-20T21:45:48.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.068+0000 7fb05dc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb058072210 con 0x7fb058158b80 2024-04-20T21:45:48.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.068+0000 7fb05dc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb0580c6dc0 con 0x7fb058152170 2024-04-20T21:45:48.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.068+0000 7fb056a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb058152170 0x7fb0580c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:48.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.068+0000 7fb057400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fb058158b80 0x7fb0581628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:53698/0 (socket says 172.21.15.102:53698) 2024-04-20T21:45:48.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.068+0000 7fb057400640 1 -- 172.21.15.102:0/4280848227 learned_addr learned my addr 172.21.15.102:0/4280848227 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:48.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.069+0000 7fb056a00640 1 -- 172.21.15.102:0/4280848227 >> v1:172.21.15.193:6789/0 conn(0x7fb058158b80 legacy=0x7fb0581628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:48.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.069+0000 7fb056a00640 1 -- 172.21.15.102:0/4280848227 >> v1:172.21.15.102:6789/0 conn(0x7fb058158770 legacy=0x7fb058070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:48.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.069+0000 7fb056a00640 1 -- 172.21.15.102:0/4280848227 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0580724b0 con 0x7fb058152170 2024-04-20T21:45:48.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.069+0000 7fb055600640 1 -- 172.21.15.102:0/4280848227 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2589281148 0 0) 0x7fb058072210 con 0x7fb058158b80 2024-04-20T21:45:48.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.070+0000 7fb056a00640 1 --2- 172.21.15.102:0/4280848227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058152170 0x7fb0580c8db0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb0480097a0 tx=0x7fb04802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757e31a889e554f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:48.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.070+0000 7fb055600640 1 -- 172.21.15.102:0/4280848227 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04802f810 con 0x7fb058152170 2024-04-20T21:45:48.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.070+0000 7fb055600640 1 -- 172.21.15.102:0/4280848227 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04802f9b0 con 0x7fb058152170 2024-04-20T21:45:48.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.070+0000 7fb055600640 1 -- 172.21.15.102:0/4280848227 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04802fd80 con 0x7fb058152170 2024-04-20T21:45:48.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.071+0000 7fb05dc00640 1 -- 172.21.15.102:0/4280848227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058152170 msgr2=0x7fb0580c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:48.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.071+0000 7fb05dc00640 1 --2- 172.21.15.102:0/4280848227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058152170 0x7fb0580c8db0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb0480097a0 tx=0x7fb04802b9f0 comp rx=0 tx=0).stop 2024-04-20T21:45:48.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.071+0000 7fb05dc00640 1 -- 172.21.15.102:0/4280848227 shutdown_connections 2024-04-20T21:45:48.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.071+0000 7fb05dc00640 1 --2- 172.21.15.102:0/4280848227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058152170 0x7fb0580c8db0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:48.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.071+0000 7fb05dc00640 1 -- 172.21.15.102:0/4280848227 >> 172.21.15.102:0/4280848227 conn(0x7fb0580c9390 msgr2=0x7fb0580c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:48.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.071+0000 7fb05dc00640 1 -- 172.21.15.102:0/4280848227 shutdown_connections 2024-04-20T21:45:48.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.071+0000 7fb05dc00640 1 -- 172.21.15.102:0/4280848227 wait complete. 2024-04-20T21:45:48.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.073+0000 7fb05dc00640 1 Processor -- start 2024-04-20T21:45:48.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.073+0000 7fb05dc00640 1 -- start start 2024-04-20T21:45:48.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb05dc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb058152170 0x7fb0581571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:48.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb05dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058158770 0x7fb0581e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:48.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb05dc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058158b80 0x7fb0581e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:48.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb05dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb0580c9f20 con 0x7fb058158770 2024-04-20T21:45:48.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb05dc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb058158250 con 0x7fb058158b80 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb05dc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb058157f20 con 0x7fb058152170 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058158770 0x7fb0581e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb057400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058158b80 0x7fb0581e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058158770 0x7fb0581e44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34340/0 (socket says 172.21.15.102:34340) 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056000640 1 -- 172.21.15.102:0/3786245335 learned_addr learned my addr 172.21.15.102:0/3786245335 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056a00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb058152170 0x7fb0581571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056a00640 1 -- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058158b80 msgr2=0x7fb0581e4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056a00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058158b80 0x7fb0581e4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056a00640 1 -- 172.21.15.102:0/3786245335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058158770 msgr2=0x7fb0581e44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056a00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058158770 0x7fb0581e44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:48.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.074+0000 7fb056a00640 1 -- 172.21.15.102:0/3786245335 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb058150230 con 0x7fb058152170 2024-04-20T21:45:48.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.075+0000 7fb056a00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb058152170 0x7fb0581571b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb048033000 tx=0x7fb048004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:48.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.075+0000 7fb04ea00640 1 -- 172.21.15.102:0/3786245335 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048046070 con 0x7fb058152170 2024-04-20T21:45:48.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.075+0000 7fb04ea00640 1 -- 172.21.15.102:0/3786245335 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb048037c80 con 0x7fb058152170 2024-04-20T21:45:48.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.075+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0581e3700 con 0x7fb058152170 2024-04-20T21:45:48.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.075+0000 7fb04ea00640 1 -- 172.21.15.102:0/3786245335 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04802fe60 con 0x7fb058152170 2024-04-20T21:45:48.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.075+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0581e3cc0 con 0x7fb058152170 2024-04-20T21:45:48.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.076+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb058074340 con 0x7fb058152170 2024-04-20T21:45:48.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.077+0000 7fb04ea00640 1 -- 172.21.15.102:0/3786245335 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb048049070 con 0x7fb058152170 2024-04-20T21:45:48.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.078+0000 7fb04ea00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c8f0 0x7fb03806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:48.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.078+0000 7fb04ea00640 1 -- 172.21.15.102:0/3786245335 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7fb0480ae190 con 0x7fb058152170 2024-04-20T21:45:48.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.078+0000 7fb056000640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c8f0 0x7fb03806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:48.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.079+0000 7fb056000640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c8f0 0x7fb03806edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb05814fdc0 tx=0x7fb044009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:48.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.083+0000 7fb04ea00640 1 -- 172.21.15.102:0/3786245335 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb048033030 con 0x7fb058152170 2024-04-20T21:45:48.170 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 9% (23/253) Updating files: 10% (26/253) Updating files: 11% (28/253) Updating files: 12% (31/253) Updating files: 13% (33/253) Updating files: 14% (36/253) Updating files: 15% (38/253) Updating files: 16% (41/253) Updating files: 17% (44/253) Updating files: 18% (46/253) Updating files: 19% (49/253) Updating files: 20% (51/253) Updating files: 21% (54/253) Updating files: 22% (56/253) Updating files: 23% (59/253) Updating files: 24% (61/253) Updating files: 25% (64/253) Updating files: 26% (66/253) Updating files: 27% (69/253) Updating files: 28% (71/253) Updating files: 29% (74/253) Updating files: 30% (76/253) Updating files: 31% (79/253) Updating files: 32% (81/253) Updating files: 33% (84/253) Updating files: 34% (87/253) Updating files: 35% (89/253) Updating files: 36% (92/253) Updating files: 37% (94/253) Updating files: 38% (97/253) Updating files: 39% (99/253) Updating files: 40% (102/253) Updating files: 41% (104/253) Updating files: 42% (107/253) Updating files: 43% (109/253) Updating files: 44% (112/253) Updating files: 45% (114/253) Updating files: 46% (117/253) Updating files: 47% (119/253) Updating files: 48% (122/253) Updating files: 49% (124/253) Updating files: 49% (125/253) Updating files: 50% (127/253) Updating files: 51% (130/253) Updating files: 52% (132/253) Updating files: 53% (135/253) Updating files: 54% (137/253) Updating files: 54% (138/253) Updating files: 55% (140/253) Updating files: 56% (142/253) Updating files: 57% (145/253) Updating files: 58% (147/253) Updating files: 59% (150/253) Updating files: 60% (152/253) Updating files: 61% (155/253) Updating files: 62% (157/253) Updating files: 63% (160/253) Updating files: 64% (162/253) Updating files: 65% (165/253) Updating files: 66% (167/253) Updating files: 67% (170/253) Updating files: 68% (173/253) Updating files: 69% (175/253) Updating files: 70% (178/253) Updating files: 70% (179/253) Updating files: 71% (180/253) Updating files: 72% (183/253) Updating files: 73% (185/253) Updating files: 74% (188/253) Updating files: 75% (190/253) Updating files: 76% (193/253) Updating files: 77% (195/253) Updating files: 78% (198/253) Updating files: 79% (200/253) Updating files: 80% (203/253) Updating files: 81% (205/253) Updating files: 82% (208/253) Updating files: 83% (210/253) Updating files: 84% (213/253) Updating files: 85% (216/253) Updating files: 86% (218/253) Updating files: 87% (221/253) Updating files: 88% (223/253) Updating files: 89% (226/253) Updating files: 90% (228/253) Updating files: 91% (231/253) Updating files: 92% (233/253) Updating files: 93% (236/253) Updating files: 94% (238/253) Updating files: 95% (241/253) Updating files: 96% (243/253) Updating files: 97% (246/253) Updating files: 98% (248/253) Updating files: 99% (251/253) Updating files: 100% (253/253) Updating files: 100% (253/253), done. 2024-04-20T21:45:48.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.343+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb058071560 con 0x7fb058152170 2024-04-20T21:45:48.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.343+0000 7fb04ea00640 1 -- 172.21.15.102:0/3786245335 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+4383 (secure 0 0 0) 0x7fb04807b470 con 0x7fb058152170 2024-04-20T21:45:48.342 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:48.342 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":102,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:45:48.343 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 102 2024-04-20T21:45:48.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.346+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c8f0 msgr2=0x7fb03806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.346+0000 7fb05dc00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c8f0 0x7fb03806edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb05814fdc0 tx=0x7fb044009040 comp rx=0 tx=0).stop 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.346+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb058152170 msgr2=0x7fb0581571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.346+0000 7fb05dc00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb058152170 0x7fb0581571b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb048033000 tx=0x7fb048004730 comp rx=0 tx=0).stop 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.346+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 shutdown_connections 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.347+0000 7fb05dc00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb03806c8f0 0x7fb03806edb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.347+0000 7fb05dc00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb058158b80 0x7fb0581e4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.347+0000 7fb05dc00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058158770 0x7fb0581e44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.347+0000 7fb05dc00640 1 --2- 172.21.15.102:0/3786245335 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb058152170 0x7fb0581571b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.347+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 >> 172.21.15.102:0/3786245335 conn(0x7fb0580c9390 msgr2=0x7fb05815b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.347+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 shutdown_connections 2024-04-20T21:45:48.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:48.347+0000 7fb05dc00640 1 -- 172.21.15.102:0/3786245335 wait complete. 2024-04-20T21:45:48.365 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-20T21:45:49.165 INFO:tasks.workunit.client.0.smithi102.stdout:HEAD is now at 793034c62c8 v0.27 2024-04-20T21:45:49.179 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir .snap/v0.27 2024-04-20T21:45:49.227 INFO:tasks.workunit.client.0.smithi102.stderr:+ for v in $versions 2024-04-20T21:45:49.228 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 28 -eq 48 ']' 2024-04-20T21:45:49.228 INFO:tasks.workunit.client.0.smithi102.stderr:+ ver=v0.28 2024-04-20T21:45:49.228 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo v0.28 2024-04-20T21:45:49.228 INFO:tasks.workunit.client.0.smithi102.stdout:v0.28 2024-04-20T21:45:49.229 INFO:tasks.workunit.client.0.smithi102.stderr:+ git reset --hard v0.28 2024-04-20T21:45:49.812 INFO:tasks.ceph.mds.f.smithi193.stderr:2024-04-20T21:45:49.814+0000 7fb1956aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:45:50.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~228s 2024-04-20T21:45:50.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~140s 2024-04-20T21:45:50.366 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:50.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.583+0000 7fb64e200640 1 Processor -- start 2024-04-20T21:45:50.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.583+0000 7fb64e200640 1 -- start start 2024-04-20T21:45:50.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.584+0000 7fb64e200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb64814fa50 0x7fb6480c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:50.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.584+0000 7fb64e200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb6480c9eb0 0x7fb648158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:50.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.584+0000 7fb64e200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb6480c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:50.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.584+0000 7fb64e200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fb648159500 con 0x7fb6480c9eb0 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.584+0000 7fb64e200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fb6481536d0 con 0x7fb64814fa50 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.584+0000 7fb64e200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb6480c76c0 con 0x7fb648073a10 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb63ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb6480c9eb0 0x7fb648158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb6480c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb64814fa50 0x7fb6480c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb64814fa50 0x7fb6480c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54738/0 (socket says 172.21.15.102:54738) 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb6480c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35442/0 (socket says 172.21.15.102:35442) 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb63ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb6480c9eb0 0x7fb648158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34350/0 (socket says 172.21.15.102:34350) 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647400640 1 -- 172.21.15.102:0/4003136437 learned_addr learned my addr 172.21.15.102:0/4003136437 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:50.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647400640 1 -- 172.21.15.102:0/4003136437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 msgr2=0x7fb6480c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:50.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647400640 1 --2- 172.21.15.102:0/4003136437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb6480c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647400640 1 -- 172.21.15.102:0/4003136437 >> v2:172.21.15.102:3300/0 conn(0x7fb6480c9eb0 msgr2=0x7fb648158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:50.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647400640 1 --2- 172.21.15.102:0/4003136437 >> v2:172.21.15.102:3300/0 conn(0x7fb6480c9eb0 0x7fb648158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.585+0000 7fb647400640 1 -- 172.21.15.102:0/4003136437 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6480c7960 con 0x7fb64814fa50 2024-04-20T21:45:50.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.586+0000 7fb647400640 1 --2- 172.21.15.102:0/4003136437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 0x7fb6480c4830 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb6480c7b30 tx=0x7fb63802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d96b82c6dd0c4d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:50.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb646a00640 1 -- 172.21.15.102:0/4003136437 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638034070 con 0x7fb64814fa50 2024-04-20T21:45:50.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb646a00640 1 -- 172.21.15.102:0/4003136437 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb638030840 con 0x7fb64814fa50 2024-04-20T21:45:50.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb646a00640 1 -- 172.21.15.102:0/4003136437 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638030c10 con 0x7fb64814fa50 2024-04-20T21:45:50.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 -- 172.21.15.102:0/4003136437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 msgr2=0x7fb6480c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:50.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 --2- 172.21.15.102:0/4003136437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 0x7fb6480c4830 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb6480c7b30 tx=0x7fb63802b980 comp rx=0 tx=0).stop 2024-04-20T21:45:50.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 -- 172.21.15.102:0/4003136437 shutdown_connections 2024-04-20T21:45:50.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 --2- 172.21.15.102:0/4003136437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb6480c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 --2- 172.21.15.102:0/4003136437 >> v2:172.21.15.102:3300/0 conn(0x7fb6480c9eb0 0x7fb648158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 --2- 172.21.15.102:0/4003136437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 0x7fb6480c4830 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 -- 172.21.15.102:0/4003136437 >> 172.21.15.102:0/4003136437 conn(0x7fb6480c9390 msgr2=0x7fb6480c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:50.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 -- 172.21.15.102:0/4003136437 shutdown_connections 2024-04-20T21:45:50.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.587+0000 7fb64e200640 1 -- 172.21.15.102:0/4003136437 wait complete. 2024-04-20T21:45:50.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.589+0000 7fb64e200640 1 Processor -- start 2024-04-20T21:45:50.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.589+0000 7fb64e200640 1 -- start start 2024-04-20T21:45:50.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.590+0000 7fb64e200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb648155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.590+0000 7fb64e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6480c9eb0 0x7fb64814c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.590+0000 7fb64e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 0x7fb64814cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb64e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb6480c7c00 con 0x7fb6480c9eb0 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb64e200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6480c3d90 con 0x7fb64814fa50 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb64e200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fb6480c4030 con 0x7fb648073a10 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb63ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6480c9eb0 0x7fb64814c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb63ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6480c9eb0 0x7fb64814c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34358/0 (socket says 172.21.15.102:34358) 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb63ea00640 1 -- 172.21.15.102:0/2582990051 learned_addr learned my addr 172.21.15.102:0/2582990051 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb647e00640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 0x7fb64814cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:50.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb647400640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb648155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:50.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb647400640 1 -- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 msgr2=0x7fb64814cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:50.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb647400640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 0x7fb64814cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb647400640 1 -- 172.21.15.102:0/2582990051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6480c9eb0 msgr2=0x7fb64814c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:50.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb647400640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6480c9eb0 0x7fb64814c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb647400640 1 -- 172.21.15.102:0/2582990051 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb648156110 con 0x7fb648073a10 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.591+0000 7fb63ea00640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6480c9eb0 0x7fb64814c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.592+0000 7fb647400640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb648155770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb638033000 tx=0x7fb6380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.592+0000 7fb63fe00640 1 -- 172.21.15.102:0/2582990051 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638034040 con 0x7fb648073a10 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.592+0000 7fb63fe00640 1 -- 172.21.15.102:0/2582990051 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb638030e20 con 0x7fb648073a10 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.592+0000 7fb63fe00640 1 -- 172.21.15.102:0/2582990051 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6380433f0 con 0x7fb648073a10 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.592+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb648154950 con 0x7fb648073a10 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.592+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb648072c80 con 0x7fb648073a10 2024-04-20T21:45:50.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.593+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb648074340 con 0x7fb648073a10 2024-04-20T21:45:50.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.594+0000 7fb63fe00640 1 -- 172.21.15.102:0/2582990051 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb638037070 con 0x7fb648073a10 2024-04-20T21:45:50.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.595+0000 7fb63fe00640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb62406c920 0x7fb62406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:50.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.595+0000 7fb63fe00640 1 -- 172.21.15.102:0/2582990051 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7fb6380ae320 con 0x7fb648073a10 2024-04-20T21:45:50.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.595+0000 7fb63ea00640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb62406c920 0x7fb62406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:50.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.596+0000 7fb63ea00640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb62406c920 0x7fb62406ede0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb64815c6b0 tx=0x7fb630009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:50.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.600+0000 7fb63fe00640 1 -- 172.21.15.102:0/2582990051 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb63803e070 con 0x7fb648073a10 2024-04-20T21:45:50.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.845+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6480c7320 con 0x7fb648073a10 2024-04-20T21:45:50.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.846+0000 7fb63fe00640 1 -- 172.21.15.102:0/2582990051 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7fb63807b720 con 0x7fb648073a10 2024-04-20T21:45:50.845 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:50.845 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:45:50.845 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.848+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb62406c920 msgr2=0x7fb62406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.848+0000 7fb64e200640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb62406c920 0x7fb62406ede0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb64815c6b0 tx=0x7fb630009190 comp rx=0 tx=0).stop 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.848+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 msgr2=0x7fb648155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.848+0000 7fb64e200640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb648155770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb638033000 tx=0x7fb6380046c0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.849+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 shutdown_connections 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.849+0000 7fb64e200640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fb62406c920 0x7fb62406ede0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.849+0000 7fb64e200640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb64814fa50 0x7fb64814cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.849+0000 7fb64e200640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6480c9eb0 0x7fb64814c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.849+0000 7fb64e200640 1 --2- 172.21.15.102:0/2582990051 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb648073a10 0x7fb648155770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.849+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 >> 172.21.15.102:0/2582990051 conn(0x7fb6480c9390 msgr2=0x7fb6480c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:50.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.849+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 shutdown_connections 2024-04-20T21:45:50.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:50.849+0000 7fb64e200640 1 -- 172.21.15.102:0/2582990051 wait complete. 2024-04-20T21:45:50.867 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T21:45:50.867 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before thrashing 2024-04-20T21:45:51.867 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:52.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.075+0000 7f38f3e00640 1 Processor -- start 2024-04-20T21:45:52.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.075+0000 7f38f3e00640 1 -- start start 2024-04-20T21:45:52.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f3e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f38f4152290 0x7f38f40cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f3e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f38f40c8330 0x7f38f40c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f3e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f415c220 con 0x7f38f40c3700 2024-04-20T21:45:52.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f3e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f38f40c9180 con 0x7f38f4152290 2024-04-20T21:45:52.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f3e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f38f40c4840 con 0x7f38f40c8330 2024-04-20T21:45:52.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f2000640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f38f40c3700 0x7f38f406e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56092/0 (socket says 172.21.15.102:56092) 2024-04-20T21:45:52.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f2a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f38f4152290 0x7f38f40cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f2000640 1 -- 172.21.15.102:0/501979373 learned_addr learned my addr 172.21.15.102:0/501979373 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:52.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.077+0000 7f38f3400640 1 --2- 172.21.15.102:0/501979373 >> v2:172.21.15.193:3300/0 conn(0x7f38f40c8330 0x7f38f40c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.078+0000 7f38f3400640 1 -- 172.21.15.102:0/501979373 >> v2:172.21.15.102:3300/0 conn(0x7f38f4152290 msgr2=0x7f38f40cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:52.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.078+0000 7f38f3400640 1 --2- 172.21.15.102:0/501979373 >> v2:172.21.15.102:3300/0 conn(0x7f38f4152290 0x7f38f40cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.078+0000 7f38f3400640 1 -- 172.21.15.102:0/501979373 >> v1:172.21.15.102:6789/0 conn(0x7f38f40c3700 legacy=0x7f38f406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.078+0000 7f38f3400640 1 -- 172.21.15.102:0/501979373 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f40c4560 con 0x7f38f40c8330 2024-04-20T21:45:52.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.078+0000 7f38f1600640 1 -- 172.21.15.102:0/501979373 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3511833986 0 0) 0x7f38f415c220 con 0x7f38f40c3700 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.079+0000 7f38f3400640 1 --2- 172.21.15.102:0/501979373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c8330 0x7f38f40c8740 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f38f406e1b0 tx=0x7f38ec02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b41185bf049e544 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.079+0000 7f38f1600640 1 -- 172.21.15.102:0/501979373 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec039070 con 0x7f38f40c8330 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.079+0000 7f38f1600640 1 -- 172.21.15.102:0/501979373 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ec02cd00 con 0x7f38f40c8330 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.079+0000 7f38f1600640 1 -- 172.21.15.102:0/501979373 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec034660 con 0x7f38f40c8330 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.080+0000 7f38f3e00640 1 -- 172.21.15.102:0/501979373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c8330 msgr2=0x7f38f40c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.080+0000 7f38f3e00640 1 --2- 172.21.15.102:0/501979373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c8330 0x7f38f40c8740 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f38f406e1b0 tx=0x7f38ec02bf50 comp rx=0 tx=0).stop 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.080+0000 7f38f3e00640 1 -- 172.21.15.102:0/501979373 shutdown_connections 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.080+0000 7f38f3e00640 1 --2- 172.21.15.102:0/501979373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c8330 0x7f38f40c8740 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.080+0000 7f38f3e00640 1 --2- 172.21.15.102:0/501979373 >> v2:172.21.15.102:3300/0 conn(0x7f38f4152290 0x7f38f40cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.080+0000 7f38f3e00640 1 -- 172.21.15.102:0/501979373 >> 172.21.15.102:0/501979373 conn(0x7f38f40cc820 msgr2=0x7f38f40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:52.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.080+0000 7f38f3e00640 1 -- 172.21.15.102:0/501979373 shutdown_connections 2024-04-20T21:45:52.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.080+0000 7f38f3e00640 1 -- 172.21.15.102:0/501979373 wait complete. 2024-04-20T21:45:52.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.082+0000 7f38f3e00640 1 Processor -- start 2024-04-20T21:45:52.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.082+0000 7f38f3e00640 1 -- start start 2024-04-20T21:45:52.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c3700 0x7f38f41477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f40c8330 0x7f38f4147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38f4152290 0x7f38f413ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f38f415ba20 con 0x7f38f40c8330 2024-04-20T21:45:52.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f38f415b740 con 0x7f38f40c3700 2024-04-20T21:45:52.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f3e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f38f415bd00 con 0x7f38f4152290 2024-04-20T21:45:52.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c3700 0x7f38f41477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c3700 0x7f38f41477d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54772/0 (socket says 172.21.15.102:54772) 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f40c8330 0x7f38f4147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f2a00640 1 -- 172.21.15.102:0/2775517935 learned_addr learned my addr 172.21.15.102:0/2775517935 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.083+0000 7f38f3400640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38f4152290 0x7f38f413ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38f2a00640 1 -- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38f4152290 msgr2=0x7f38f413ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38f2a00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38f4152290 0x7f38f413ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38f2a00640 1 -- 172.21.15.102:0/2775517935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f40c8330 msgr2=0x7f38f4147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38f2a00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f40c8330 0x7f38f4147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38f2a00640 1 -- 172.21.15.102:0/2775517935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f41efba0 con 0x7f38f40c3700 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38f2a00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c3700 0x7f38f41477d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f38e400e9b0 tx=0x7f38e400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:52.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38eaa00640 1 -- 172.21.15.102:0/2775517935 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4017070 con 0x7f38f40c3700 2024-04-20T21:45:52.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38eaa00640 1 -- 172.21.15.102:0/2775517935 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e400c9a0 con 0x7f38f40c3700 2024-04-20T21:45:52.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38eaa00640 1 -- 172.21.15.102:0/2775517935 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e400cd90 con 0x7f38f40c3700 2024-04-20T21:45:52.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f4146770 con 0x7f38f40c3700 2024-04-20T21:45:52.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.084+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f4146cf0 con 0x7f38f40c3700 2024-04-20T21:45:52.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.086+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f404fd60 con 0x7f38f40c3700 2024-04-20T21:45:52.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.086+0000 7f38eaa00640 1 -- 172.21.15.102:0/2775517935 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38e401a070 con 0x7f38f40c3700 2024-04-20T21:45:52.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.087+0000 7f38eaa00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38d406c920 0x7f38d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.087+0000 7f38eaa00640 1 -- 172.21.15.102:0/2775517935 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f38e408c4b0 con 0x7f38f40c3700 2024-04-20T21:45:52.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.087+0000 7f38f2000640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38d406c920 0x7f38d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.088+0000 7f38f2000640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38d406c920 0x7f38d406ede0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f38f4158d10 tx=0x7f38e0007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:52.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.092+0000 7f38eaa00640 1 -- 172.21.15.102:0/2775517935 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f38e4059820 con 0x7f38f40c3700 2024-04-20T21:45:52.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.361+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38f406dcb0 con 0x7f38f40c3700 2024-04-20T21:45:52.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.362+0000 7f38eaa00640 1 -- 172.21.15.102:0/2775517935 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7f38e405db20 con 0x7f38f40c3700 2024-04-20T21:45:52.361 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:52.361 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:45:52.362 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38d406c920 msgr2=0x7f38d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38d406c920 0x7f38d406ede0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f38f4158d10 tx=0x7f38e0007de0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c3700 msgr2=0x7f38f41477d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c3700 0x7f38f41477d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f38e400e9b0 tx=0x7f38e400ee70 comp rx=0 tx=0).stop 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 shutdown_connections 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38f4152290 0x7f38f413ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f40c8330 0x7f38f4147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f38d406c920 0x7f38d406ede0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 --2- 172.21.15.102:0/2775517935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38f40c3700 0x7f38f41477d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 >> 172.21.15.102:0/2775517935 conn(0x7f38f40cc820 msgr2=0x7f38f40c6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:52.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 shutdown_connections 2024-04-20T21:45:52.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.365+0000 7f38f3e00640 1 -- 172.21.15.102:0/2775517935 wait complete. 2024-04-20T21:45:52.383 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2024-04-20T21:45:52.383 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-20T21:45:52.383 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:45:52.396 INFO:tasks.ceph.mds.f:Stopped 2024-04-20T21:45:52.396 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:52.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.605+0000 7f8e32000640 1 Processor -- start 2024-04-20T21:45:52.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.605+0000 7f8e32000640 1 -- start start 2024-04-20T21:45:52.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.606+0000 7f8e32000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8e2c153180 0x7f8e2c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.606+0000 7f8e32000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.606+0000 7f8e32000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8e2c0c75a0 0x7f8e2c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.606+0000 7f8e32000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8e2c153560 con 0x7f8e2c0c75a0 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.606+0000 7f8e32000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f8e2c0c7a70 con 0x7f8e2c153180 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.606+0000 7f8e32000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2c0c7d30 con 0x7f8e2c159380 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e23e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e30c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8e2c153180 0x7f8e2c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e31600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8e2c0c75a0 0x7f8e2c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e30c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8e2c153180 0x7f8e2c158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:54788/0 (socket says 172.21.15.102:54788) 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e23e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c0c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:35482/0 (socket says 172.21.15.102:35482) 2024-04-20T21:45:52.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e30c00640 1 -- 172.21.15.102:0/555120055 learned_addr learned my addr 172.21.15.102:0/555120055 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:52.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e30c00640 1 -- 172.21.15.102:0/555120055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 msgr2=0x7f8e2c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:52.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e30c00640 1 --2- 172.21.15.102:0/555120055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c0c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e30c00640 1 -- 172.21.15.102:0/555120055 >> v2:172.21.15.102:3300/0 conn(0x7f8e2c0c75a0 msgr2=0x7f8e2c0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:52.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e30c00640 1 --2- 172.21.15.102:0/555120055 >> v2:172.21.15.102:3300/0 conn(0x7f8e2c0c75a0 0x7f8e2c0c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e30c00640 1 -- 172.21.15.102:0/555120055 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c15c4d0 con 0x7f8e2c153180 2024-04-20T21:45:52.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.607+0000 7f8e23e00640 1 --2- 172.21.15.102:0/555120055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c0c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:52.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.609+0000 7f8e30c00640 1 --2- 172.21.15.102:0/555120055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 0x7f8e2c158e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0cd860 tx=0x7f8e1c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20e30e33b2256b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:52.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.609+0000 7f8e2be00640 1 -- 172.21.15.102:0/555120055 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c035020 con 0x7f8e2c153180 2024-04-20T21:45:52.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.609+0000 7f8e2be00640 1 -- 172.21.15.102:0/555120055 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c031870 con 0x7f8e2c153180 2024-04-20T21:45:52.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.609+0000 7f8e2be00640 1 -- 172.21.15.102:0/555120055 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c031c40 con 0x7f8e2c153180 2024-04-20T21:45:52.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 -- 172.21.15.102:0/555120055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 msgr2=0x7f8e2c158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 --2- 172.21.15.102:0/555120055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 0x7f8e2c158e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0cd860 tx=0x7f8e1c02b910 comp rx=0 tx=0).stop 2024-04-20T21:45:52.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 -- 172.21.15.102:0/555120055 shutdown_connections 2024-04-20T21:45:52.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 --2- 172.21.15.102:0/555120055 >> v2:172.21.15.102:3300/0 conn(0x7f8e2c0c75a0 0x7f8e2c0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 --2- 172.21.15.102:0/555120055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c0c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 --2- 172.21.15.102:0/555120055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 0x7f8e2c158e40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 -- 172.21.15.102:0/555120055 >> 172.21.15.102:0/555120055 conn(0x7f8e2c0cc820 msgr2=0x7f8e2c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 -- 172.21.15.102:0/555120055 shutdown_connections 2024-04-20T21:45:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.610+0000 7f8e32000640 1 -- 172.21.15.102:0/555120055 wait complete. 2024-04-20T21:45:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.612+0000 7f8e32000640 1 Processor -- start 2024-04-20T21:45:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.612+0000 7f8e32000640 1 -- start start 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e32000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e2c0c75a0 0x7f8e2c1f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e32000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 0x7f8e2c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e32000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c1e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e32000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c15d290 con 0x7f8e2c0c75a0 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e32000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c15ca90 con 0x7f8e2c153180 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e32000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2c0c6be0 con 0x7f8e2c159380 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e30c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e2c0c75a0 0x7f8e2c1f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e30c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e2c0c75a0 0x7f8e2c1f15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34394/0 (socket says 172.21.15.102:34394) 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e30c00640 1 -- 172.21.15.102:0/869882876 learned_addr learned my addr 172.21.15.102:0/869882876 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:52.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e23e00640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 0x7f8e2c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e31600640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c1e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e23e00640 1 -- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 msgr2=0x7f8e2c1e23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e23e00640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c1e23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e23e00640 1 -- 172.21.15.102:0/869882876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e2c0c75a0 msgr2=0x7f8e2c1f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e23e00640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e2c0c75a0 0x7f8e2c1f15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e23e00640 1 -- 172.21.15.102:0/869882876 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c156fe0 con 0x7f8e2c153180 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.613+0000 7f8e30c00640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e2c0c75a0 0x7f8e2c1f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.614+0000 7f8e23e00640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 0x7f8e2c1494c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400e8d0 tx=0x7f8e1400ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.614+0000 7f8e29600640 1 -- 172.21.15.102:0/869882876 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14017070 con 0x7f8e2c153180 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.614+0000 7f8e29600640 1 -- 172.21.15.102:0/869882876 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1400c940 con 0x7f8e2c153180 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.614+0000 7f8e29600640 1 -- 172.21.15.102:0/869882876 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1400cd30 con 0x7f8e2c153180 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.614+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c157280 con 0x7f8e2c153180 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.614+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2c1e4f10 con 0x7f8e2c153180 2024-04-20T21:45:52.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.615+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e2c04fd60 con 0x7f8e2c153180 2024-04-20T21:45:52.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.616+0000 7f8e29600640 1 -- 172.21.15.102:0/869882876 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e1401a070 con 0x7f8e2c153180 2024-04-20T21:45:52.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.617+0000 7f8e29600640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8e0806c8f0 0x7f8e0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:52.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.617+0000 7f8e29600640 1 -- 172.21.15.102:0/869882876 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f8e1408c730 con 0x7f8e2c153180 2024-04-20T21:45:52.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.617+0000 7f8e30c00640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8e0806c8f0 0x7f8e0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:52.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.618+0000 7f8e30c00640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8e0806c8f0 0x7f8e0806edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009630 tx=0x7f8e1c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:52.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.621+0000 7f8e29600640 1 -- 172.21.15.102:0/869882876 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8e14059aa0 con 0x7f8e2c153180 2024-04-20T21:45:52.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.858+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e2c0c3690 con 0x7f8e2c153180 2024-04-20T21:45:52.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.859+0000 7f8e29600640 1 -- 172.21.15.102:0/869882876 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7f8e1405dda0 con 0x7f8e2c153180 2024-04-20T21:45:52.859 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:52.859 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:45:52.859 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:45:52.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8e0806c8f0 msgr2=0x7f8e0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8e0806c8f0 0x7f8e0806edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009630 tx=0x7f8e1c03c040 comp rx=0 tx=0).stop 2024-04-20T21:45:52.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 msgr2=0x7f8e2c1494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:52.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 0x7f8e2c1494c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400e8d0 tx=0x7f8e1400ed90 comp rx=0 tx=0).stop 2024-04-20T21:45:52.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 shutdown_connections 2024-04-20T21:45:52.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e2c159380 0x7f8e2c1e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8e0806c8f0 0x7f8e0806edb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c153180 0x7f8e2c1494c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 --2- 172.21.15.102:0/869882876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e2c0c75a0 0x7f8e2c1f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:52.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 >> 172.21.15.102:0/869882876 conn(0x7f8e2c0cc820 msgr2=0x7f8e2c073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:52.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.862+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 shutdown_connections 2024-04-20T21:45:52.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:52.863+0000 7f8e32000640 1 -- 172.21.15.102:0/869882876 wait complete. 2024-04-20T21:45:52.879 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T21:45:54.880 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:55.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.087+0000 7f641fe00640 1 Processor -- start 2024-04-20T21:45:55.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.087+0000 7f641fe00640 1 -- start start 2024-04-20T21:45:55.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.088+0000 7f641fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6420152290 0x7f64200cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:55.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64200c8330 0x7f64200c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:55.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641fe00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f642015c220 con 0x7f64200c3700 2024-04-20T21:45:55.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641fe00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f64200c9180 con 0x7f6420152290 2024-04-20T21:45:55.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641fe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f64200c4840 con 0x7f64200c8330 2024-04-20T21:45:55.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6420152290 0x7f64200cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6420152290 0x7f64200cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34410/0 (socket says 172.21.15.102:34410) 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641ea00640 1 -- 172.21.15.102:0/882342993 learned_addr learned my addr 172.21.15.102:0/882342993 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641f400640 1 --2- 172.21.15.102:0/882342993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64200c8330 0x7f64200c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641e000640 1 -- 172.21.15.102:0/882342993 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3530262757 0 0) 0x7f642015c220 con 0x7f64200c3700 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641e000640 1 -- 172.21.15.102:0/882342993 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6404003520 con 0x7f64200c3700 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641ea00640 1 -- 172.21.15.102:0/882342993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64200c8330 msgr2=0x7f64200c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.089+0000 7f641ea00640 1 --2- 172.21.15.102:0/882342993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64200c8330 0x7f64200c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.090+0000 7f641ea00640 1 -- 172.21.15.102:0/882342993 >> v1:172.21.15.193:6789/0 conn(0x7f64200c3700 legacy=0x7f642006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:55.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.090+0000 7f641ea00640 1 -- 172.21.15.102:0/882342993 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64200c4560 con 0x7f6420152290 2024-04-20T21:45:55.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.091+0000 7f641ea00640 1 --2- 172.21.15.102:0/882342993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6420152290 0x7f64200cd310 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f641000bb10 tx=0x7f641002e1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d770a1c6e96e835 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.091+0000 7f641e000640 1 -- 172.21.15.102:0/882342993 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641003b070 con 0x7f6420152290 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.091+0000 7f641e000640 1 -- 172.21.15.102:0/882342993 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6410036440 con 0x7f6420152290 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.091+0000 7f641e000640 1 -- 172.21.15.102:0/882342993 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410036810 con 0x7f6420152290 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.091+0000 7f641fe00640 1 -- 172.21.15.102:0/882342993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6420152290 msgr2=0x7f64200cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.091+0000 7f641fe00640 1 --2- 172.21.15.102:0/882342993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6420152290 0x7f64200cd310 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f641000bb10 tx=0x7f641002e1f0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.092+0000 7f641fe00640 1 -- 172.21.15.102:0/882342993 shutdown_connections 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.092+0000 7f641fe00640 1 --2- 172.21.15.102:0/882342993 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f64200c8330 0x7f64200c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.092+0000 7f641fe00640 1 --2- 172.21.15.102:0/882342993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6420152290 0x7f64200cd310 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.092+0000 7f641fe00640 1 -- 172.21.15.102:0/882342993 >> 172.21.15.102:0/882342993 conn(0x7f64200cc820 msgr2=0x7f64200ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.092+0000 7f641fe00640 1 -- 172.21.15.102:0/882342993 shutdown_connections 2024-04-20T21:45:55.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.092+0000 7f641fe00640 1 -- 172.21.15.102:0/882342993 wait complete. 2024-04-20T21:45:55.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.093+0000 7f641fe00640 1 Processor -- start 2024-04-20T21:45:55.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.094+0000 7f641fe00640 1 -- start start 2024-04-20T21:45:55.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.094+0000 7f641fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64200c3700 0x7f64201f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:55.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.094+0000 7f641fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64200c8330 0x7f642014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:55.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.094+0000 7f641fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6420152290 0x7f64200cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:55.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.094+0000 7f641fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f642015ba20 con 0x7f64200c3700 2024-04-20T21:45:55.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.094+0000 7f641fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f642015b740 con 0x7f64200c8330 2024-04-20T21:45:55.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.094+0000 7f641fe00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f642015bd00 con 0x7f6420152290 2024-04-20T21:45:55.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64200c3700 0x7f64201f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:55.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64200c3700 0x7f64201f4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34426/0 (socket says 172.21.15.102:34426) 2024-04-20T21:45:55.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641ea00640 1 -- 172.21.15.102:0/2149812181 learned_addr learned my addr 172.21.15.102:0/2149812181 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:55.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f6416000640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64200c8330 0x7f642014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:55.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641f400640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6420152290 0x7f64200cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:55.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641ea00640 1 -- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6420152290 msgr2=0x7f64200cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:55.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641ea00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6420152290 0x7f64200cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641ea00640 1 -- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64200c8330 msgr2=0x7f642014d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:55.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641ea00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64200c8330 0x7f642014d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.095+0000 7f641ea00640 1 -- 172.21.15.102:0/2149812181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6420073c90 con 0x7f64200c3700 2024-04-20T21:45:55.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.096+0000 7f641ea00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64200c3700 0x7f64201f4e20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f641002e230 tx=0x7f64100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:55.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.096+0000 7f6417400640 1 -- 172.21.15.102:0/2149812181 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410046070 con 0x7f64200c3700 2024-04-20T21:45:55.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.096+0000 7f6417400640 1 -- 172.21.15.102:0/2149812181 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641003d050 con 0x7f64200c3700 2024-04-20T21:45:55.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.096+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642015abe0 con 0x7f64200c3700 2024-04-20T21:45:55.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.096+0000 7f6417400640 1 -- 172.21.15.102:0/2149812181 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641003b030 con 0x7f64200c3700 2024-04-20T21:45:55.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.096+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642015b1a0 con 0x7f64200c3700 2024-04-20T21:45:55.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.097+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642004fd60 con 0x7f64200c3700 2024-04-20T21:45:55.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.098+0000 7f6417400640 1 -- 172.21.15.102:0/2149812181 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6410049070 con 0x7f64200c3700 2024-04-20T21:45:55.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.099+0000 7f6417400640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:55.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.099+0000 7f6417400640 1 -- 172.21.15.102:0/2149812181 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f64100add50 con 0x7f64200c3700 2024-04-20T21:45:55.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.099+0000 7f6416000640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:55.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.100+0000 7f6416000640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f63fc06c8f0 0x7f63fc06edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6408007fd0 tx=0x7f6408009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:55.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.104+0000 7f6417400640 1 -- 172.21.15.102:0/2149812181 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f64100b1030 con 0x7f64200c3700 2024-04-20T21:45:55.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.344+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f642006dcb0 con 0x7f64200c3700 2024-04-20T21:45:55.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.345+0000 7f6417400640 1 -- 172.21.15.102:0/2149812181 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7f641007b1b0 con 0x7f64200c3700 2024-04-20T21:45:55.345 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:55.345 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:45:55.345 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:45:55.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f63fc06c8f0 msgr2=0x7f63fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f63fc06c8f0 0x7f63fc06edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6408007fd0 tx=0x7f6408009040 comp rx=0 tx=0).stop 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64200c3700 msgr2=0x7f64201f4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64200c3700 0x7f64201f4e20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f641002e230 tx=0x7f64100046c0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 shutdown_connections 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6420152290 0x7f64200cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f63fc06c8f0 0x7f63fc06edb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64200c8330 0x7f642014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 --2- 172.21.15.102:0/2149812181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64200c3700 0x7f64201f4e20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 >> 172.21.15.102:0/2149812181 conn(0x7f64200cc820 msgr2=0x7f64200c7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.348+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 shutdown_connections 2024-04-20T21:45:55.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:55.349+0000 7f641fe00640 1 -- 172.21.15.102:0/2149812181 wait complete. 2024-04-20T21:45:55.368 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T21:45:56.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~235s 2024-04-20T21:45:56.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~147s 2024-04-20T21:45:57.369 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:57.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.583+0000 7f22edc00640 1 Processor -- start 2024-04-20T21:45:57.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.583+0000 7f22edc00640 1 -- start start 2024-04-20T21:45:57.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.584+0000 7f22edc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f22e815bd10 0x7f22e80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:57.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.584+0000 7f22edc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22e815c150 con 0x7f22e8155690 2024-04-20T21:45:57.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.584+0000 7f22edc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22e80d0b30 con 0x7f22e8075710 2024-04-20T21:45:57.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.584+0000 7f22edc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f22e80cc2f0 con 0x7f22e815bd10 2024-04-20T21:45:57.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e6a00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f22e8155690 0x7f22e80cb3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56106/0 (socket says 172.21.15.102:56106) 2024-04-20T21:45:57.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e6a00640 1 -- 172.21.15.102:0/2294699087 learned_addr learned my addr 172.21.15.102:0/2294699087 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:45:57.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e6000640 1 --2- 172.21.15.102:0/2294699087 >> v2:172.21.15.193:3300/0 conn(0x7f22e815bd10 0x7f22e80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:57.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e5600640 1 -- 172.21.15.102:0/2294699087 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2674466130 0 0) 0x7f22d8001110 con 0x7f22e8155690 2024-04-20T21:45:57.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e5600640 1 -- 172.21.15.102:0/2294699087 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22cc003520 con 0x7f22e8155690 2024-04-20T21:45:57.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e5600640 1 -- 172.21.15.102:0/2294699087 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340248350 0 0) 0x7f22e80d0b30 con 0x7f22e8075710 2024-04-20T21:45:57.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e5600640 1 -- 172.21.15.102:0/2294699087 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22d8001110 con 0x7f22e8075710 2024-04-20T21:45:57.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e6000640 1 -- 172.21.15.102:0/2294699087 >> v1:172.21.15.193:6789/0 conn(0x7f22e8075710 legacy=0x7f22e80cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:57.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e6000640 1 -- 172.21.15.102:0/2294699087 >> v1:172.21.15.102:6789/0 conn(0x7f22e8155690 legacy=0x7f22e80cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:57.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.585+0000 7f22e6000640 1 -- 172.21.15.102:0/2294699087 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e80d0e10 con 0x7f22e815bd10 2024-04-20T21:45:57.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22e6000640 1 --2- 172.21.15.102:0/2294699087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e815bd10 0x7f22e80751d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f22d40097a0 tx=0x7f22d402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3dd1984a226087 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:57.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22e5600640 1 -- 172.21.15.102:0/2294699087 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d402f810 con 0x7f22e815bd10 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22e5600640 1 -- 172.21.15.102:0/2294699087 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d402f9b0 con 0x7f22e815bd10 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22e5600640 1 -- 172.21.15.102:0/2294699087 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d402fd80 con 0x7f22e815bd10 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22edc00640 1 -- 172.21.15.102:0/2294699087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e815bd10 msgr2=0x7f22e80751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22edc00640 1 --2- 172.21.15.102:0/2294699087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e815bd10 0x7f22e80751d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f22d40097a0 tx=0x7f22d402b9f0 comp rx=0 tx=0).stop 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22edc00640 1 -- 172.21.15.102:0/2294699087 shutdown_connections 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22edc00640 1 --2- 172.21.15.102:0/2294699087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e815bd10 0x7f22e80751d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.587+0000 7f22edc00640 1 -- 172.21.15.102:0/2294699087 >> 172.21.15.102:0/2294699087 conn(0x7f22e80685f0 msgr2=0x7f22e806ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.588+0000 7f22edc00640 1 -- 172.21.15.102:0/2294699087 shutdown_connections 2024-04-20T21:45:57.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.588+0000 7f22edc00640 1 -- 172.21.15.102:0/2294699087 wait complete. 2024-04-20T21:45:57.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.589+0000 7f22edc00640 1 Processor -- start 2024-04-20T21:45:57.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.589+0000 7f22edc00640 1 -- start start 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.590+0000 7f22edc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e8075710 0x7f22e80b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.590+0000 7f22edc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22e8155690 0x7f22e81f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.590+0000 7f22edc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e815bd10 0x7f22e815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.590+0000 7f22edc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22e80d16b0 con 0x7f22e815bd10 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.590+0000 7f22edc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22e80d13d0 con 0x7f22e8075710 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.590+0000 7f22edc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f22e80cce90 con 0x7f22e8155690 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e815bd10 0x7f22e815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e815bd10 0x7f22e815a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34440/0 (socket says 172.21.15.102:34440) 2024-04-20T21:45:57.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 -- 172.21.15.102:0/847846035 learned_addr learned my addr 172.21.15.102:0/847846035 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:45:57.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e6000640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22e8155690 0x7f22e81f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:57.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e6a00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e8075710 0x7f22e80b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:57.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 -- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22e8155690 msgr2=0x7f22e81f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:57.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22e8155690 0x7f22e81f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:57.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 -- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e8075710 msgr2=0x7f22e80b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:57.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e8075710 0x7f22e80b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:57.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 -- 172.21.15.102:0/847846035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e815ad90 con 0x7f22e815bd10 2024-04-20T21:45:57.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.591+0000 7f22e7400640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e815bd10 0x7f22e815a3c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c7c0 tx=0x7f22e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:57.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.592+0000 7f22dea00640 1 -- 172.21.15.102:0/847846035 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0003b70 con 0x7f22e815bd10 2024-04-20T21:45:57.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.592+0000 7f22dea00640 1 -- 172.21.15.102:0/847846035 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e0003d10 con 0x7f22e815bd10 2024-04-20T21:45:57.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.592+0000 7f22dea00640 1 -- 172.21.15.102:0/847846035 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e000a9f0 con 0x7f22e815bd10 2024-04-20T21:45:57.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.592+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e815b030 con 0x7f22e815bd10 2024-04-20T21:45:57.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.592+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e81f4b90 con 0x7f22e815bd10 2024-04-20T21:45:57.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.593+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e81f46a0 con 0x7f22e815bd10 2024-04-20T21:45:57.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.594+0000 7f22dea00640 1 -- 172.21.15.102:0/847846035 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22e000ab90 con 0x7f22e815bd10 2024-04-20T21:45:57.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.595+0000 7f22dea00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f22c806c8f0 0x7f22c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:57.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.595+0000 7f22dea00640 1 -- 172.21.15.102:0/847846035 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f22e008c3f0 con 0x7f22e815bd10 2024-04-20T21:45:57.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.596+0000 7f22e6a00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f22c806c8f0 0x7f22c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:57.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.596+0000 7f22e6a00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f22c806c8f0 0x7f22c806edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f22d80013b0 tx=0x7f22d8007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:57.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.600+0000 7f22dea00640 1 -- 172.21.15.102:0/847846035 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f22e0059760 con 0x7f22e815bd10 2024-04-20T21:45:57.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.842+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22e80cae90 con 0x7f22e815bd10 2024-04-20T21:45:57.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.844+0000 7f22dea00640 1 -- 172.21.15.102:0/847846035 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7f22e005da60 con 0x7f22e815bd10 2024-04-20T21:45:57.843 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:45:57.843 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:45:57.843 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:45:57.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f22c806c8f0 msgr2=0x7f22c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:57.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f22c806c8f0 0x7f22c806edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f22d80013b0 tx=0x7f22d8007d30 comp rx=0 tx=0).stop 2024-04-20T21:45:57.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e815bd10 msgr2=0x7f22e815a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:57.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e815bd10 0x7f22e815a3c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c7c0 tx=0x7f22e000cc80 comp rx=0 tx=0).stop 2024-04-20T21:45:57.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 shutdown_connections 2024-04-20T21:45:57.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e815bd10 0x7f22e815a3c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:57.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22e8155690 0x7f22e81f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:57.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f22c806c8f0 0x7f22c806edb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:57.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 --2- 172.21.15.102:0/847846035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e8075710 0x7f22e80b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:57.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 >> 172.21.15.102:0/847846035 conn(0x7f22e80685f0 msgr2=0x7f22e8073130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:57.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 shutdown_connections 2024-04-20T21:45:57.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:45:57.847+0000 7f22edc00640 1 -- 172.21.15.102:0/847846035 wait complete. 2024-04-20T21:45:57.864 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T21:45:59.865 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:00.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.074+0000 7fd89e400640 1 Processor -- start 2024-04-20T21:46:00.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.074+0000 7fd89e400640 1 -- start start 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.075+0000 7fd89e400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd898070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.075+0000 7fd89e400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd898071330 0x7fd8981628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.075+0000 7fd89e400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd898158e80 con 0x7fd898152170 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.075+0000 7fd89e400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fd8980c6dc0 con 0x7fd898071330 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.075+0000 7fd89e400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd898071f70 con 0x7fd8980c8db0 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd897400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd898152170 0x7fd898158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59462/0 (socket says 172.21.15.102:59462) 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd897400640 1 -- 172.21.15.102:0/3449262758 learned_addr learned my addr 172.21.15.102:0/3449262758 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd897e00640 1 --2- 172.21.15.102:0/3449262758 >> v2:172.21.15.193:3300/0 conn(0x7fd898071330 0x7fd8981628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:00.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd896a00640 1 --2- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd898070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:00.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd897e00640 1 -- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 msgr2=0x7fd898070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:00.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd896000640 1 -- 172.21.15.102:0/3449262758 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751560635 0 0) 0x7fd898158e80 con 0x7fd898152170 2024-04-20T21:46:00.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd897e00640 1 --2- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd898070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd897e00640 1 -- 172.21.15.102:0/3449262758 >> v1:172.21.15.102:6789/0 conn(0x7fd898152170 legacy=0x7fd898158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:00.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd897e00640 1 -- 172.21.15.102:0/3449262758 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd898072210 con 0x7fd898071330 2024-04-20T21:46:00.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.076+0000 7fd896a00640 1 --2- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd898070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:00.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.077+0000 7fd897e00640 1 --2- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 0x7fd8981628a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd890009d90 tx=0x7fd89002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe4137d2de9accd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:00.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd896000640 1 -- 172.21.15.102:0/3449262758 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890039070 con 0x7fd898071330 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd896000640 1 -- 172.21.15.102:0/3449262758 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89002cc50 con 0x7fd898071330 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd896000640 1 -- 172.21.15.102:0/3449262758 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890034440 con 0x7fd898071330 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd89e400640 1 -- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 msgr2=0x7fd8981628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd89e400640 1 --2- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 0x7fd8981628a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd890009d90 tx=0x7fd89002bea0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd89e400640 1 -- 172.21.15.102:0/3449262758 shutdown_connections 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd89e400640 1 --2- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 0x7fd8981628a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd89e400640 1 --2- 172.21.15.102:0/3449262758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd898070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd89e400640 1 -- 172.21.15.102:0/3449262758 >> 172.21.15.102:0/3449262758 conn(0x7fd8980c9390 msgr2=0x7fd8980c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd89e400640 1 -- 172.21.15.102:0/3449262758 shutdown_connections 2024-04-20T21:46:00.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.078+0000 7fd89e400640 1 -- 172.21.15.102:0/3449262758 wait complete. 2024-04-20T21:46:00.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.080+0000 7fd89e400640 1 Processor -- start 2024-04-20T21:46:00.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.080+0000 7fd89e400640 1 -- start start 2024-04-20T21:46:00.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd89e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 0x7fd89815b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd89e400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd89806ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd89e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd898152170 0x7fd8981534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd89e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8980c9190 con 0x7fd898152170 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd89e400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd8980724b0 con 0x7fd898071330 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd89e400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd898158250 con 0x7fd8980c8db0 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd897e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd898152170 0x7fd8981534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd897e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd898152170 0x7fd8981534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43410/0 (socket says 172.21.15.102:43410) 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.081+0000 7fd897e00640 1 -- 172.21.15.102:0/1130983124 learned_addr learned my addr 172.21.15.102:0/1130983124 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd897400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 0x7fd89815b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:00.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd896a00640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd89806ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:00.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd897e00640 1 -- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 msgr2=0x7fd89806ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:00.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd897e00640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd89806ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd897e00640 1 -- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 msgr2=0x7fd89815b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:00.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd897e00640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 0x7fd89815b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd897e00640 1 -- 172.21.15.102:0/1130983124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89815aab0 con 0x7fd898152170 2024-04-20T21:46:00.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd897e00640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd898152170 0x7fd8981534b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd890037040 tx=0x7fd890004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:00.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.082+0000 7fd88f400640 1 -- 172.21.15.102:0/1130983124 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890044020 con 0x7fd898152170 2024-04-20T21:46:00.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.083+0000 7fd88f400640 1 -- 172.21.15.102:0/1130983124 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89003a040 con 0x7fd898152170 2024-04-20T21:46:00.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.083+0000 7fd88f400640 1 -- 172.21.15.102:0/1130983124 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890039050 con 0x7fd898152170 2024-04-20T21:46:00.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.083+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89815ad90 con 0x7fd898152170 2024-04-20T21:46:00.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.083+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89815b350 con 0x7fd898152170 2024-04-20T21:46:00.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.084+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd898074340 con 0x7fd898152170 2024-04-20T21:46:00.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.085+0000 7fd88f400640 1 -- 172.21.15.102:0/1130983124 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd890003990 con 0x7fd898152170 2024-04-20T21:46:00.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.086+0000 7fd88f400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd87806c8f0 0x7fd87806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:00.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.086+0000 7fd88f400640 1 -- 172.21.15.102:0/1130983124 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7fd8900ae7a0 con 0x7fd898152170 2024-04-20T21:46:00.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.086+0000 7fd897400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd87806c8f0 0x7fd87806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:00.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.087+0000 7fd897400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd87806c8f0 0x7fd87806edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd888007fd0 tx=0x7fd888007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:00.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.090+0000 7fd88f400640 1 -- 172.21.15.102:0/1130983124 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd89004e060 con 0x7fd898152170 2024-04-20T21:46:00.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.324+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8980c3460 con 0x7fd898152170 2024-04-20T21:46:00.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.325+0000 7fd88f400640 1 -- 172.21.15.102:0/1130983124 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7fd89007bc00 con 0x7fd898152170 2024-04-20T21:46:00.324 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:00.325 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:46:00.325 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd87806c8f0 msgr2=0x7fd87806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd87806c8f0 0x7fd87806edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd888007fd0 tx=0x7fd888007de0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd898152170 msgr2=0x7fd8981534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd898152170 0x7fd8981534b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd890037040 tx=0x7fd890004a40 comp rx=0 tx=0).stop 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 shutdown_connections 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd898152170 0x7fd8981534b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8980c8db0 0x7fd89806ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd87806c8f0 0x7fd87806edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 --2- 172.21.15.102:0/1130983124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd898071330 0x7fd89815b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:00.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 >> 172.21.15.102:0/1130983124 conn(0x7fd8980c9390 msgr2=0x7fd8980c76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:00.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 shutdown_connections 2024-04-20T21:46:00.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:00.328+0000 7fd89e400640 1 -- 172.21.15.102:0/1130983124 wait complete. 2024-04-20T21:46:00.344 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T21:46:02.211 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:46:02.219 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:46:02.345 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.554+0000 7f063a200640 1 Processor -- start 2024-04-20T21:46:02.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.554+0000 7f063a200640 1 -- start start 2024-04-20T21:46:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f063a200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0634155110 0x7f063406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f063a200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f063406f3d0 0x7f063415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f063a200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063415f3c0 con 0x7f06340c4040 2024-04-20T21:46:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f063a200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f06340c7c30 con 0x7f0634155110 2024-04-20T21:46:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f063a200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f063406ea00 con 0x7f063406f3d0 2024-04-20T21:46:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f0638e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0634155110 0x7f063406ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f0638e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0634155110 0x7f063406ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43424/0 (socket says 172.21.15.102:43424) 2024-04-20T21:46:02.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f0638e00640 1 -- 172.21.15.102:0/2848915327 learned_addr learned my addr 172.21.15.102:0/2848915327 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f0639800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f06340c4040 0x7f063415eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59474/0 (socket says 172.21.15.102:59474) 2024-04-20T21:46:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.556+0000 7f0633e00640 1 --2- 172.21.15.102:0/2848915327 >> v2:172.21.15.193:3300/0 conn(0x7f063406f3d0 0x7f063415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.557+0000 7f0638e00640 1 -- 172.21.15.102:0/2848915327 >> v2:172.21.15.193:3300/0 conn(0x7f063406f3d0 msgr2=0x7f063415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.557+0000 7f0638e00640 1 --2- 172.21.15.102:0/2848915327 >> v2:172.21.15.193:3300/0 conn(0x7f063406f3d0 0x7f063415e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.557+0000 7f0638e00640 1 -- 172.21.15.102:0/2848915327 >> v1:172.21.15.102:6789/0 conn(0x7f06340c4040 legacy=0x7f063415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:02.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.557+0000 7f0638e00640 1 -- 172.21.15.102:0/2848915327 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063406e6d0 con 0x7f0634155110 2024-04-20T21:46:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.557+0000 7f0633400640 1 -- 172.21.15.102:0/2848915327 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3134951004 0 0) 0x7f063415f3c0 con 0x7f06340c4040 2024-04-20T21:46:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.558+0000 7f0638e00640 1 --2- 172.21.15.102:0/2848915327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0634155110 0x7f063406ee90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f06340c7db0 tx=0x7f062802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ddae88fa66df5b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.558+0000 7f0633400640 1 -- 172.21.15.102:0/2848915327 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062802f850 con 0x7f0634155110 2024-04-20T21:46:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.558+0000 7f0633400640 1 -- 172.21.15.102:0/2848915327 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062802f9f0 con 0x7f0634155110 2024-04-20T21:46:02.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.558+0000 7f0633400640 1 -- 172.21.15.102:0/2848915327 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062802fdc0 con 0x7f0634155110 2024-04-20T21:46:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.559+0000 7f063a200640 1 -- 172.21.15.102:0/2848915327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0634155110 msgr2=0x7f063406ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.559+0000 7f063a200640 1 --2- 172.21.15.102:0/2848915327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0634155110 0x7f063406ee90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f06340c7db0 tx=0x7f062802ba30 comp rx=0 tx=0).stop 2024-04-20T21:46:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.559+0000 7f063a200640 1 -- 172.21.15.102:0/2848915327 shutdown_connections 2024-04-20T21:46:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.559+0000 7f063a200640 1 --2- 172.21.15.102:0/2848915327 >> v2:172.21.15.193:3300/0 conn(0x7f063406f3d0 0x7f063415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.559+0000 7f063a200640 1 --2- 172.21.15.102:0/2848915327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0634155110 0x7f063406ee90 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.559+0000 7f063a200640 1 -- 172.21.15.102:0/2848915327 >> 172.21.15.102:0/2848915327 conn(0x7f06340cc820 msgr2=0x7f06340ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:02.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.559+0000 7f063a200640 1 -- 172.21.15.102:0/2848915327 shutdown_connections 2024-04-20T21:46:02.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.559+0000 7f063a200640 1 -- 172.21.15.102:0/2848915327 wait complete. 2024-04-20T21:46:02.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.561+0000 7f063a200640 1 Processor -- start 2024-04-20T21:46:02.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.561+0000 7f063a200640 1 -- start start 2024-04-20T21:46:02.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.562+0000 7f063a200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f063406f3d0 0x7f06341f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:02.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.562+0000 7f063a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06340c4040 0x7f0634159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:02.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.562+0000 7f063a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0634155110 0x7f06341f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:02.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.562+0000 7f063a200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06341554f0 con 0x7f06340c4040 2024-04-20T21:46:02.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.562+0000 7f063a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06340c8160 con 0x7f0634155110 2024-04-20T21:46:02.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.562+0000 7f063a200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0634155670 con 0x7f063406f3d0 2024-04-20T21:46:02.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.562+0000 7f0633e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06340c4040 0x7f0634159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:02.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0633e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06340c4040 0x7f0634159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43430/0 (socket says 172.21.15.102:43430) 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0633e00640 1 -- 172.21.15.102:0/2641670343 learned_addr learned my addr 172.21.15.102:0/2641670343 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0639800640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0634155110 0x7f06341f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0638e00640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f063406f3d0 0x7f06341f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0633e00640 1 -- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f063406f3d0 msgr2=0x7f06341f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0633e00640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f063406f3d0 0x7f06341f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0633e00640 1 -- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0634155110 msgr2=0x7f06341f0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0633e00640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0634155110 0x7f06341f0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0633e00640 1 -- 172.21.15.102:0/2641670343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341f43a0 con 0x7f06340c4040 2024-04-20T21:46:02.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.563+0000 7f0633e00640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06340c4040 0x7f0634159d20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f062000e9e0 tx=0x7f062000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:02.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.564+0000 7f0630c00640 1 -- 172.21.15.102:0/2641670343 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620017070 con 0x7f06340c4040 2024-04-20T21:46:02.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.564+0000 7f0630c00640 1 -- 172.21.15.102:0/2641670343 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062000c9d0 con 0x7f06340c4040 2024-04-20T21:46:02.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.564+0000 7f0630c00640 1 -- 172.21.15.102:0/2641670343 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062000cdc0 con 0x7f06340c4040 2024-04-20T21:46:02.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.564+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063415bae0 con 0x7f06340c4040 2024-04-20T21:46:02.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.564+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06340cc330 con 0x7f06340c4040 2024-04-20T21:46:02.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.565+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06340c69a0 con 0x7f06340c4040 2024-04-20T21:46:02.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.566+0000 7f0630c00640 1 -- 172.21.15.102:0/2641670343 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f062001a070 con 0x7f06340c4040 2024-04-20T21:46:02.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.567+0000 7f0630c00640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c920 0x7f061406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:02.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.568+0000 7f0630c00640 1 -- 172.21.15.102:0/2641670343 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f062008c950 con 0x7f06340c4040 2024-04-20T21:46:02.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.568+0000 7f0638e00640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c920 0x7f061406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:02.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.568+0000 7f0638e00640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c920 0x7f061406ede0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0628036000 tx=0x7f0628008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:02.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.572+0000 7f0630c00640 1 -- 172.21.15.102:0/2641670343 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f06340c69a0 con 0x7f06340c4040 2024-04-20T21:46:02.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.815+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f063404fd60 con 0x7f06340c4040 2024-04-20T21:46:02.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.817+0000 7f0630c00640 1 -- 172.21.15.102:0/2641670343 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7f0620059c40 con 0x7f06340c4040 2024-04-20T21:46:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:02.816 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:46:02.816 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:46:02.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.819+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c920 msgr2=0x7f061406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.819+0000 7f063a200640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c920 0x7f061406ede0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0628036000 tx=0x7f0628008030 comp rx=0 tx=0).stop 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.819+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06340c4040 msgr2=0x7f0634159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.819+0000 7f063a200640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06340c4040 0x7f0634159d20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f062000e9e0 tx=0x7f062000eea0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.820+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 shutdown_connections 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.820+0000 7f063a200640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f061406c920 0x7f061406ede0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.820+0000 7f063a200640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0634155110 0x7f06341f0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.820+0000 7f063a200640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06340c4040 0x7f0634159d20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.820+0000 7f063a200640 1 --2- 172.21.15.102:0/2641670343 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f063406f3d0 0x7f06341f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.820+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 >> 172.21.15.102:0/2641670343 conn(0x7f06340cc820 msgr2=0x7f06340c9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.820+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 shutdown_connections 2024-04-20T21:46:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:02.820+0000 7f063a200640 1 -- 172.21.15.102:0/2641670343 wait complete. 2024-04-20T21:46:02.837 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T21:46:03.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~241s 2024-04-20T21:46:03.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~153s 2024-04-20T21:46:04.838 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:05.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.046+0000 7fd8e2c00640 1 Processor -- start 2024-04-20T21:46:05.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.047+0000 7fd8e2c00640 1 -- start start 2024-04-20T21:46:05.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.048+0000 7fd8e2c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 0x7fd8dc070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:05.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.048+0000 7fd8e2c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd8dc071330 0x7fd8dc1628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:05.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.048+0000 7fd8e2c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8dc158e80 con 0x7fd8dc152170 2024-04-20T21:46:05.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.048+0000 7fd8e2c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fd8dc0c6dc0 con 0x7fd8dc071330 2024-04-20T21:46:05.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.048+0000 7fd8e2c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc071f70 con 0x7fd8dc0c8db0 2024-04-20T21:46:05.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8e0c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd8dc071330 0x7fd8dc1628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:05.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8dbe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fd8dc152170 0x7fd8dc158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:52304/0 (socket says 172.21.15.102:52304) 2024-04-20T21:46:05.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8dbe00640 1 -- 172.21.15.102:0/699231496 learned_addr learned my addr 172.21.15.102:0/699231496 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:05.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8d3e00640 1 --2- 172.21.15.102:0/699231496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 0x7fd8dc070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:05.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8e0c00640 1 -- 172.21.15.102:0/699231496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 msgr2=0x7fd8dc070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:05.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8e0c00640 1 --2- 172.21.15.102:0/699231496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 0x7fd8dc070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8db400640 1 -- 172.21.15.102:0/699231496 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190755451 0 0) 0x7fd8dc158e80 con 0x7fd8dc152170 2024-04-20T21:46:05.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8e0c00640 1 -- 172.21.15.102:0/699231496 >> v1:172.21.15.193:6789/0 conn(0x7fd8dc152170 legacy=0x7fd8dc158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:05.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.049+0000 7fd8e0c00640 1 -- 172.21.15.102:0/699231496 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc072210 con 0x7fd8dc071330 2024-04-20T21:46:05.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.050+0000 7fd8e0c00640 1 --2- 172.21.15.102:0/699231496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1628a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009d90 tx=0x7fd8d402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5de9f471ddebafa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:05.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8db400640 1 -- 172.21.15.102:0/699231496 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4039070 con 0x7fd8dc071330 2024-04-20T21:46:05.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8db400640 1 -- 172.21.15.102:0/699231496 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d402cc50 con 0x7fd8dc071330 2024-04-20T21:46:05.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8db400640 1 -- 172.21.15.102:0/699231496 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4034440 con 0x7fd8dc071330 2024-04-20T21:46:05.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8e2c00640 1 -- 172.21.15.102:0/699231496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 msgr2=0x7fd8dc1628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:05.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/699231496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1628a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009d90 tx=0x7fd8d402bea0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8e2c00640 1 -- 172.21.15.102:0/699231496 shutdown_connections 2024-04-20T21:46:05.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/699231496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1628a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/699231496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 0x7fd8dc070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8e2c00640 1 -- 172.21.15.102:0/699231496 >> 172.21.15.102:0/699231496 conn(0x7fd8dc0c9390 msgr2=0x7fd8dc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:05.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.051+0000 7fd8e2c00640 1 -- 172.21.15.102:0/699231496 shutdown_connections 2024-04-20T21:46:05.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.052+0000 7fd8e2c00640 1 -- 172.21.15.102:0/699231496 wait complete. 2024-04-20T21:46:05.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.054+0000 7fd8e2c00640 1 Processor -- start 2024-04-20T21:46:05.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.054+0000 7fd8e2c00640 1 -- start start 2024-04-20T21:46:05.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8e2c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:05.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8e2c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 0x7fd8dc150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:05.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8e2c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8dc152170 0x7fd8dc0c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:05.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8e2c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc0c9190 con 0x7fd8dc071330 2024-04-20T21:46:05.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8e2c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc0724b0 con 0x7fd8dc152170 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8e2c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc158250 con 0x7fd8dc0c8db0 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43446/0 (socket says 172.21.15.102:43446) 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8dbe00640 1 -- 172.21.15.102:0/98716556 learned_addr learned my addr 172.21.15.102:0/98716556 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.055+0000 7fd8e0c00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8dc152170 0x7fd8dc0c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8d3e00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 0x7fd8dc150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8dbe00640 1 -- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 msgr2=0x7fd8dc150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8dbe00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 0x7fd8dc150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8dbe00640 1 -- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8dc152170 msgr2=0x7fd8dc0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8dbe00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8dc152170 0x7fd8dc0c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8dbe00640 1 -- 172.21.15.102:0/98716556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc159de0 con 0x7fd8dc071330 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8dbe00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1598a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00c460 tx=0x7fd8cc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8d8c00640 1 -- 172.21.15.102:0/98716556 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc018070 con 0x7fd8dc071330 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8d8c00640 1 -- 172.21.15.102:0/98716556 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc007500 con 0x7fd8dc071330 2024-04-20T21:46:05.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8d8c00640 1 -- 172.21.15.102:0/98716556 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc004a20 con 0x7fd8dc071330 2024-04-20T21:46:05.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.056+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8dc1590e0 con 0x7fd8dc071330 2024-04-20T21:46:05.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.057+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8dc1528b0 con 0x7fd8dc071330 2024-04-20T21:46:05.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.058+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8dc074340 con 0x7fd8dc071330 2024-04-20T21:46:05.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.059+0000 7fd8d8c00640 1 -- 172.21.15.102:0/98716556 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8cc01b070 con 0x7fd8dc071330 2024-04-20T21:46:05.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.060+0000 7fd8d8c00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8b806c920 0x7fd8b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:05.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.060+0000 7fd8d8c00640 1 -- 172.21.15.102:0/98716556 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7fd8cc08b6f0 con 0x7fd8dc071330 2024-04-20T21:46:05.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.060+0000 7fd8d3e00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8b806c920 0x7fd8b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:05.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.061+0000 7fd8d3e00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8b806c920 0x7fd8b806ede0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc15bb30 tx=0x7fd8c4009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:05.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.064+0000 7fd8d8c00640 1 -- 172.21.15.102:0/98716556 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd8dc159de0 con 0x7fd8dc071330 2024-04-20T21:46:05.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.300+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8dc0c3460 con 0x7fd8dc071330 2024-04-20T21:46:05.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.302+0000 7fd8d8c00640 1 -- 172.21.15.102:0/98716556 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7fd8cc058af0 con 0x7fd8dc071330 2024-04-20T21:46:05.301 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:05.301 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:46:05.301 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.304+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8b806c920 msgr2=0x7fd8b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.304+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8b806c920 0x7fd8b806ede0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc15bb30 tx=0x7fd8c4009190 comp rx=0 tx=0).stop 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.304+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 msgr2=0x7fd8dc1598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.304+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1598a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00c460 tx=0x7fd8cc00c920 comp rx=0 tx=0).stop 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.305+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 shutdown_connections 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.305+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fd8b806c920 0x7fd8b806ede0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.305+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8dc152170 0x7fd8dc0c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.305+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8dc0c8db0 0x7fd8dc150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.305+0000 7fd8e2c00640 1 --2- 172.21.15.102:0/98716556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8dc071330 0x7fd8dc1598a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.305+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 >> 172.21.15.102:0/98716556 conn(0x7fd8dc0c9390 msgr2=0x7fd8dc06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.305+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 shutdown_connections 2024-04-20T21:46:05.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:05.305+0000 7fd8e2c00640 1 -- 172.21.15.102:0/98716556 wait complete. 2024-04-20T21:46:05.320 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T21:46:07.321 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:07.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.530+0000 7f6da6600640 1 Processor -- start 2024-04-20T21:46:07.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.530+0000 7f6da6600640 1 -- start start 2024-04-20T21:46:07.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6da6600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da0155610 0x7f6da015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6da6600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6da015bd60 0x7f6da015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6da6600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6da015f6f0 con 0x7f6da0073950 2024-04-20T21:46:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6da6600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6da015b8d0 con 0x7f6da015bd60 2024-04-20T21:46:07.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6da6600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6da015c180 con 0x7f6da0155610 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6d9fe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6da0073950 0x7f6da00ceb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59486/0 (socket says 172.21.15.102:59486) 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6d9fe00640 1 -- 172.21.15.102:0/3620792126 learned_addr learned my addr 172.21.15.102:0/3620792126 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6d9ea00640 1 --2- 172.21.15.102:0/3620792126 >> v2:172.21.15.193:3300/0 conn(0x7f6da015bd60 0x7f6da015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.532+0000 7f6d9f400640 1 --2- 172.21.15.102:0/3620792126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da0155610 0x7f6da015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.533+0000 7f6d9e000640 1 -- 172.21.15.102:0/3620792126 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170996863 0 0) 0x7f6da015f6f0 con 0x7f6da0073950 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.533+0000 7f6d9ea00640 1 -- 172.21.15.102:0/3620792126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da0155610 msgr2=0x7f6da015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.533+0000 7f6d9ea00640 1 --2- 172.21.15.102:0/3620792126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da0155610 0x7f6da015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.533+0000 7f6d9ea00640 1 -- 172.21.15.102:0/3620792126 >> v1:172.21.15.102:6789/0 conn(0x7f6da0073950 legacy=0x7f6da00ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:07.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.533+0000 7f6d9ea00640 1 -- 172.21.15.102:0/3620792126 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da015b2c0 con 0x7f6da015bd60 2024-04-20T21:46:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.534+0000 7f6d9ea00640 1 --2- 172.21.15.102:0/3620792126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da015bd60 0x7f6da015f1b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0097a0 tx=0x7f6d8c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5678a36cbab94b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.534+0000 7f6d9e000640 1 -- 172.21.15.102:0/3620792126 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c034070 con 0x7f6da015bd60 2024-04-20T21:46:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.534+0000 7f6d9e000640 1 -- 172.21.15.102:0/3620792126 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8c030800 con 0x7f6da015bd60 2024-04-20T21:46:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.534+0000 7f6d9e000640 1 -- 172.21.15.102:0/3620792126 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c030bd0 con 0x7f6da015bd60 2024-04-20T21:46:07.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.535+0000 7f6da6600640 1 -- 172.21.15.102:0/3620792126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da015bd60 msgr2=0x7f6da015f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:07.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.535+0000 7f6da6600640 1 --2- 172.21.15.102:0/3620792126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da015bd60 0x7f6da015f1b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0097a0 tx=0x7f6d8c02b940 comp rx=0 tx=0).stop 2024-04-20T21:46:07.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.535+0000 7f6da6600640 1 -- 172.21.15.102:0/3620792126 shutdown_connections 2024-04-20T21:46:07.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.535+0000 7f6da6600640 1 --2- 172.21.15.102:0/3620792126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da015bd60 0x7f6da015f1b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.535+0000 7f6da6600640 1 --2- 172.21.15.102:0/3620792126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da0155610 0x7f6da015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.535+0000 7f6da6600640 1 -- 172.21.15.102:0/3620792126 >> 172.21.15.102:0/3620792126 conn(0x7f6da006ebf0 msgr2=0x7f6da006efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:07.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.535+0000 7f6da6600640 1 -- 172.21.15.102:0/3620792126 shutdown_connections 2024-04-20T21:46:07.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.535+0000 7f6da6600640 1 -- 172.21.15.102:0/3620792126 wait complete. 2024-04-20T21:46:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.537+0000 7f6da6600640 1 Processor -- start 2024-04-20T21:46:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.537+0000 7f6da6600640 1 -- start start 2024-04-20T21:46:07.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.538+0000 7f6da6600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da0073950 0x7f6da01f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:07.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.538+0000 7f6da6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da0155610 0x7f6da015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:07.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.538+0000 7f6da6600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da015bd60 0x7f6da0155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:07.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.538+0000 7f6da6600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6da015b560 con 0x7f6da0155610 2024-04-20T21:46:07.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.538+0000 7f6da6600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6da00c9d50 con 0x7f6da0073950 2024-04-20T21:46:07.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.538+0000 7f6da6600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6da00ca030 con 0x7f6da015bd60 2024-04-20T21:46:07.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.538+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da0155610 0x7f6da015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:07.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.538+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da0155610 0x7f6da015cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43448/0 (socket says 172.21.15.102:43448) 2024-04-20T21:46:07.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9ea00640 1 -- 172.21.15.102:0/4032510853 learned_addr learned my addr 172.21.15.102:0/4032510853 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:07.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9fe00640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da015bd60 0x7f6da0155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:07.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9f400640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da0073950 0x7f6da01f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:07.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9ea00640 1 -- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da015bd60 msgr2=0x7f6da0155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9ea00640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da015bd60 0x7f6da0155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9ea00640 1 -- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da0073950 msgr2=0x7f6da01f08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9ea00640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da0073950 0x7f6da01f08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9ea00640 1 -- 172.21.15.102:0/4032510853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da0158600 con 0x7f6da0155610 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9f400640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da0073950 0x7f6da01f08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.539+0000 7f6d9ea00640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da0155610 0x7f6da015cee0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0098c0 tx=0x7f6d8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.540+0000 7f6d97400640 1 -- 172.21.15.102:0/4032510853 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c034040 con 0x7f6da0155610 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.540+0000 7f6d97400640 1 -- 172.21.15.102:0/4032510853 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8c03ada0 con 0x7f6da0155610 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.540+0000 7f6d97400640 1 -- 172.21.15.102:0/4032510853 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c006e10 con 0x7f6da0155610 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.540+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da0158e80 con 0x7f6da0155610 2024-04-20T21:46:07.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.540+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da0158ba0 con 0x7f6da0155610 2024-04-20T21:46:07.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.541+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da0074340 con 0x7f6da0155610 2024-04-20T21:46:07.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.542+0000 7f6d97400640 1 -- 172.21.15.102:0/4032510853 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d8c04b060 con 0x7f6da0155610 2024-04-20T21:46:07.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.543+0000 7f6d97400640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6d8006c920 0x7f6d8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:07.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.543+0000 7f6d97400640 1 -- 172.21.15.102:0/4032510853 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f6d8c0ae310 con 0x7f6da0155610 2024-04-20T21:46:07.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.544+0000 7f6d9f400640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6d8006c920 0x7f6d8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:07.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.544+0000 7f6d9f400640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6d8006c920 0x7f6d8006ede0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009f30 tx=0x7f6d90009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:07.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.548+0000 7f6d97400640 1 -- 172.21.15.102:0/4032510853 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6d8c03f070 con 0x7f6da0155610 2024-04-20T21:46:07.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.779+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6da00c9740 con 0x7f6da0155610 2024-04-20T21:46:07.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.780+0000 7f6d97400640 1 -- 172.21.15.102:0/4032510853 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7f6d8c07b650 con 0x7f6da0155610 2024-04-20T21:46:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:07.780 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:46:07.780 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:46:07.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6d8006c920 msgr2=0x7f6d8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:07.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6d8006c920 0x7f6d8006ede0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009f30 tx=0x7f6d90009190 comp rx=0 tx=0).stop 2024-04-20T21:46:07.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da0155610 msgr2=0x7f6da015cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:07.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da0155610 0x7f6da015cee0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0098c0 tx=0x7f6d8c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 shutdown_connections 2024-04-20T21:46:07.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6da015bd60 0x7f6da0155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da0155610 0x7f6da015cee0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6d8006c920 0x7f6d8006ede0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 --2- 172.21.15.102:0/4032510853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6da0073950 0x7f6da01f08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:07.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 >> 172.21.15.102:0/4032510853 conn(0x7f6da006ebf0 msgr2=0x7f6da0071320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:07.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 shutdown_connections 2024-04-20T21:46:07.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:07.783+0000 7f6da6600640 1 -- 172.21.15.102:0/4032510853 wait complete. 2024-04-20T21:46:07.799 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T21:46:09.801 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:09.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~248s 2024-04-20T21:46:09.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~160s 2024-04-20T21:46:10.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.009+0000 7f5c65600640 1 Processor -- start 2024-04-20T21:46:10.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.010+0000 7f5c65600640 1 -- start start 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.011+0000 7f5c65600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c600caab0 0x7f5c600c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.011+0000 7f5c65600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c60075380 con 0x7f5c60153220 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.011+0000 7f5c65600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c60075620 con 0x7f5c60073b70 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.011+0000 7f5c65600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f5c60158f50 con 0x7f5c600caab0 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.011+0000 7f5c5e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c600caab0 0x7f5c600c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.012+0000 7f5c5e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5c600caab0 0x7f5c600c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37234/0 (socket says 172.21.15.102:37234) 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.012+0000 7f5c5e000640 1 -- 172.21.15.102:0/868108308 learned_addr learned my addr 172.21.15.102:0/868108308 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.012+0000 7f5c5ea00640 1 --1- 172.21.15.102:0/868108308 >> v1:172.21.15.102:6789/0 conn(0x7f5c60153220 0x7f5c6015c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:51484/0 (socket says 172.21.15.102:51484) 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.012+0000 7f5c5e000640 1 -- 172.21.15.102:0/868108308 >> v1:172.21.15.193:6789/0 conn(0x7f5c60073b70 legacy=0x7f5c601638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:10.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.012+0000 7f5c5d600640 1 -- 172.21.15.102:0/868108308 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1414039959 0 0) 0x7f5c60075620 con 0x7f5c60073b70 2024-04-20T21:46:10.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.012+0000 7f5c5e000640 1 -- 172.21.15.102:0/868108308 >> v1:172.21.15.102:6789/0 conn(0x7f5c60153220 legacy=0x7f5c6015c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:10.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.012+0000 7f5c5e000640 1 -- 172.21.15.102:0/868108308 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c601591a0 con 0x7f5c600caab0 2024-04-20T21:46:10.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.013+0000 7f5c5e000640 1 --2- 172.21.15.102:0/868108308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c600caab0 0x7f5c600c7050 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009e50 tx=0x7f5c4802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ba86ed66f94eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:10.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c5d600640 1 -- 172.21.15.102:0/868108308 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4802cbb0 con 0x7f5c600caab0 2024-04-20T21:46:10.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c5d600640 1 -- 172.21.15.102:0/868108308 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4802cd50 con 0x7f5c600caab0 2024-04-20T21:46:10.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c5d600640 1 -- 172.21.15.102:0/868108308 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48034610 con 0x7f5c600caab0 2024-04-20T21:46:10.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c65600640 1 -- 172.21.15.102:0/868108308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c600caab0 msgr2=0x7f5c600c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:10.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c65600640 1 --2- 172.21.15.102:0/868108308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c600caab0 0x7f5c600c7050 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009e50 tx=0x7f5c4802c010 comp rx=0 tx=0).stop 2024-04-20T21:46:10.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c65600640 1 -- 172.21.15.102:0/868108308 shutdown_connections 2024-04-20T21:46:10.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c65600640 1 --2- 172.21.15.102:0/868108308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c600caab0 0x7f5c600c7050 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:10.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c65600640 1 -- 172.21.15.102:0/868108308 >> 172.21.15.102:0/868108308 conn(0x7f5c6006b940 msgr2=0x7f5c6006bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:10.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c65600640 1 -- 172.21.15.102:0/868108308 shutdown_connections 2024-04-20T21:46:10.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.014+0000 7f5c65600640 1 -- 172.21.15.102:0/868108308 wait complete. 2024-04-20T21:46:10.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.017+0000 7f5c65600640 1 Processor -- start 2024-04-20T21:46:10.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.017+0000 7f5c65600640 1 -- start start 2024-04-20T21:46:10.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c65600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c60073b70 0x7f5c60151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:10.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c65600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c600caab0 0x7f5c60149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:10.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c65600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c60153220 0x7f5c6014a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c65600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c60153600 con 0x7f5c60073b70 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c65600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c601597f0 con 0x7f5c60153220 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c65600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5c60159c80 con 0x7f5c600caab0 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c60073b70 0x7f5c60151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c60073b70 0x7f5c60151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37240/0 (socket says 172.21.15.102:37240) 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c5ea00640 1 -- 172.21.15.102:0/3172451590 learned_addr learned my addr 172.21.15.102:0/3172451590 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c5e000640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c600caab0 0x7f5c60149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.018+0000 7f5c5f400640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c60153220 0x7f5c6014a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c5ea00640 1 -- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c600caab0 msgr2=0x7f5c60149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c5ea00640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c600caab0 0x7f5c60149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c5ea00640 1 -- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c60153220 msgr2=0x7f5c6014a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c5ea00640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c60153220 0x7f5c6014a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c5ea00640 1 -- 172.21.15.102:0/3172451590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c60150120 con 0x7f5c60073b70 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c5ea00640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c60073b70 0x7f5c60151950 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000c3f0 tx=0x7f5c5000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:10.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c56a00640 1 -- 172.21.15.102:0/3172451590 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c500074e0 con 0x7f5c60073b70 2024-04-20T21:46:10.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c56a00640 1 -- 172.21.15.102:0/3172451590 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c500047e0 con 0x7f5c60073b70 2024-04-20T21:46:10.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c60150400 con 0x7f5c60073b70 2024-04-20T21:46:10.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c56a00640 1 -- 172.21.15.102:0/3172451590 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50004bd0 con 0x7f5c60073b70 2024-04-20T21:46:10.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.019+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c600cb740 con 0x7f5c60073b70 2024-04-20T21:46:10.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.021+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c60068c30 con 0x7f5c60073b70 2024-04-20T21:46:10.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.022+0000 7f5c56a00640 1 -- 172.21.15.102:0/3172451590 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c5001a020 con 0x7f5c60073b70 2024-04-20T21:46:10.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.022+0000 7f5c56a00640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3c06c8f0 0x7f5c3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:10.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.023+0000 7f5c56a00640 1 -- 172.21.15.102:0/3172451590 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 8197+0+0 (secure 0 0 0) 0x7f5c5008bbd0 con 0x7f5c60073b70 2024-04-20T21:46:10.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.023+0000 7f5c5e000640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3c06c8f0 0x7f5c3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:10.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.024+0000 7f5c5e000640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3c06c8f0 0x7f5c3c06edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5c48035000 tx=0x7f5c48030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:10.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.027+0000 7f5c56a00640 1 -- 172.21.15.102:0/3172451590 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5c50058fa0 con 0x7f5c60073b70 2024-04-20T21:46:10.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.265+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c601568d0 con 0x7f5c60073b70 2024-04-20T21:46:10.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.266+0000 7f5c56a00640 1 -- 172.21.15.102:0/3172451590 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5212 (secure 0 0 0) 0x7f5c5005d2a0 con 0x7f5c60073b70 2024-04-20T21:46:10.265 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":7026,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:45:40.472914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"f","rank":0,"incarnation":98,"state":"up:active","state_seq":436,"addr":"172.21.15.193:6838/2467124512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2467124512},{"type":"v1","addr":"172.21.15.193:6838","nonce":2467124512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-20T21:46:10.265 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 103 2024-04-20T21:46:10.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3c06c8f0 msgr2=0x7f5c3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:10.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3c06c8f0 0x7f5c3c06edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5c48035000 tx=0x7f5c48030000 comp rx=0 tx=0).stop 2024-04-20T21:46:10.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c60073b70 msgr2=0x7f5c60151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c60073b70 0x7f5c60151950 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000c3f0 tx=0x7f5c5000c8b0 comp rx=0 tx=0).stop 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 shutdown_connections 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f5c3c06c8f0 0x7f5c3c06edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c60153220 0x7f5c6014a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c600caab0 0x7f5c60149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 --2- 172.21.15.102:0/3172451590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c60073b70 0x7f5c60151950 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.268+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 >> 172.21.15.102:0/3172451590 conn(0x7f5c6006b940 msgr2=0x7f5c601581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.269+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 shutdown_connections 2024-04-20T21:46:10.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:10.269+0000 7f5c65600640 1 -- 172.21.15.102:0/3172451590 wait complete. 2024-04-20T21:46:10.287 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T21:46:12.287 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:12.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.496+0000 7f9affc00640 1 Processor -- start 2024-04-20T21:46:12.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.496+0000 7f9affc00640 1 -- start start 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.498+0000 7f9affc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9af8155710 0x7f9af815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.498+0000 7f9affc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9af806dd30 0x7f9af80ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.498+0000 7f9affc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af815e930 con 0x7f9af8073da0 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.498+0000 7f9affc00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9af815c250 con 0x7f9af806dd30 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.498+0000 7f9affc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9af815ebd0 con 0x7f9af8155710 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.498+0000 7f9afdc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9af806dd30 0x7f9af80ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9afdc00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9af806dd30 0x7f9af80ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37250/0 (socket says 172.21.15.102:37250) 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9afdc00640 1 -- 172.21.15.102:0/3967047358 learned_addr learned my addr 172.21.15.102:0/3967047358 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9aefe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f9af8073da0 0x7f9af8071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:52964/0 (socket says 172.21.15.102:52964) 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9afd200640 1 --2- 172.21.15.102:0/3967047358 >> v2:172.21.15.193:3300/0 conn(0x7f9af8155710 0x7f9af815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9afdc00640 1 -- 172.21.15.102:0/3967047358 >> v2:172.21.15.193:3300/0 conn(0x7f9af8155710 msgr2=0x7f9af815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:12.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9afdc00640 1 --2- 172.21.15.102:0/3967047358 >> v2:172.21.15.193:3300/0 conn(0x7f9af8155710 0x7f9af815bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9afdc00640 1 -- 172.21.15.102:0/3967047358 >> v1:172.21.15.193:6789/0 conn(0x7f9af8073da0 legacy=0x7f9af8071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9af7e00640 1 -- 172.21.15.102:0/3967047358 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4190446649 0 0) 0x7f9af815e930 con 0x7f9af8073da0 2024-04-20T21:46:12.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.499+0000 7f9afdc00640 1 -- 172.21.15.102:0/3967047358 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af815eeb0 con 0x7f9af806dd30 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.500+0000 7f9afdc00640 1 --2- 172.21.15.102:0/3967047358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af806dd30 0x7f9af80ca140 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9af80cec90 tx=0x7f9af002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdb6bf8d5b219a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.500+0000 7f9af7e00640 1 -- 172.21.15.102:0/3967047358 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af002ca80 con 0x7f9af806dd30 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.500+0000 7f9af7e00640 1 -- 172.21.15.102:0/3967047358 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af002cc20 con 0x7f9af806dd30 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.500+0000 7f9af7e00640 1 -- 172.21.15.102:0/3967047358 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00343f0 con 0x7f9af806dd30 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.501+0000 7f9affc00640 1 -- 172.21.15.102:0/3967047358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af806dd30 msgr2=0x7f9af80ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.501+0000 7f9affc00640 1 --2- 172.21.15.102:0/3967047358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af806dd30 0x7f9af80ca140 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9af80cec90 tx=0x7f9af002bee0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.501+0000 7f9affc00640 1 -- 172.21.15.102:0/3967047358 shutdown_connections 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.501+0000 7f9affc00640 1 --2- 172.21.15.102:0/3967047358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af806dd30 0x7f9af80ca140 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.501+0000 7f9affc00640 1 --2- 172.21.15.102:0/3967047358 >> v2:172.21.15.193:3300/0 conn(0x7f9af8155710 0x7f9af815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.501+0000 7f9affc00640 1 -- 172.21.15.102:0/3967047358 >> 172.21.15.102:0/3967047358 conn(0x7f9af806f9f0 msgr2=0x7f9af806fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:12.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.501+0000 7f9affc00640 1 -- 172.21.15.102:0/3967047358 shutdown_connections 2024-04-20T21:46:12.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.501+0000 7f9affc00640 1 -- 172.21.15.102:0/3967047358 wait complete. 2024-04-20T21:46:12.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.503+0000 7f9affc00640 1 Processor -- start 2024-04-20T21:46:12.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.503+0000 7f9affc00640 1 -- start start 2024-04-20T21:46:12.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9affc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9af806dd30 0x7f9af81f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9affc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8155710 0x7f9af80cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9affc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af81587d0 0x7f9af81f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9affc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9af80725f0 con 0x7f9af8155710 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9affc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9af815f470 con 0x7f9af81587d0 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9affc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9af8072770 con 0x7f9af806dd30 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9aefe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8155710 0x7f9af80cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9aefe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8155710 0x7f9af80cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37258/0 (socket says 172.21.15.102:37258) 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.504+0000 7f9aefe00640 1 -- 172.21.15.102:0/3898344400 learned_addr learned my addr 172.21.15.102:0/3898344400 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9afdc00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af81587d0 0x7f9af81f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9afd200640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9af806dd30 0x7f9af81f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9aefe00640 1 -- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9af806dd30 msgr2=0x7f9af81f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9aefe00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9af806dd30 0x7f9af81f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9aefe00640 1 -- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af81587d0 msgr2=0x7f9af81f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9aefe00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af81587d0 0x7f9af81f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9aefe00640 1 -- 172.21.15.102:0/3898344400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af815a060 con 0x7f9af8155710 2024-04-20T21:46:12.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9afd200640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9af806dd30 0x7f9af81f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:46:12.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9aefe00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8155710 0x7f9af80cde60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000c4e0 tx=0x7f9ae000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:12.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9af5600640 1 -- 172.21.15.102:0/3898344400 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0018070 con 0x7f9af8155710 2024-04-20T21:46:12.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9af5600640 1 -- 172.21.15.102:0/3898344400 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae0007500 con 0x7f9af8155710 2024-04-20T21:46:12.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9af5600640 1 -- 172.21.15.102:0/3898344400 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0004a20 con 0x7f9af8155710 2024-04-20T21:46:12.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.505+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af815a340 con 0x7f9af8155710 2024-04-20T21:46:12.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.506+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af815a900 con 0x7f9af8155710 2024-04-20T21:46:12.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.507+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af8068c30 con 0x7f9af8155710 2024-04-20T21:46:12.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.508+0000 7f9af5600640 1 -- 172.21.15.102:0/3898344400 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ae0004bc0 con 0x7f9af8155710 2024-04-20T21:46:12.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.509+0000 7f9af5600640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ad406c8f0 0x7f9ad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.509+0000 7f9af5600640 1 -- 172.21.15.102:0/3898344400 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f9ae008b950 con 0x7f9af8155710 2024-04-20T21:46:12.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.509+0000 7f9afd200640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ad406c8f0 0x7f9ad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.510+0000 7f9afd200640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ad406c8f0 0x7f9ad406edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009af0 tx=0x7f9ae8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:12.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.513+0000 7f9af5600640 1 -- 172.21.15.102:0/3898344400 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ae0058ca0 con 0x7f9af8155710 2024-04-20T21:46:12.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.751+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9af806e690 con 0x7f9af8155710 2024-04-20T21:46:12.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.752+0000 7f9af5600640 1 -- 172.21.15.102:0/3898344400 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+4375 (secure 0 0 0) 0x7f9ae005cfa0 con 0x7f9af8155710 2024-04-20T21:46:12.751 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:10.945178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:12.751 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 104 2024-04-20T21:46:12.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ad406c8f0 msgr2=0x7f9ad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ad406c8f0 0x7f9ad406edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009af0 tx=0x7f9ae8008040 comp rx=0 tx=0).stop 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8155710 msgr2=0x7f9af80cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8155710 0x7f9af80cde60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000c4e0 tx=0x7f9ae000c9a0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 shutdown_connections 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9ad406c8f0 0x7f9ad406edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af81587d0 0x7f9af81f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8155710 0x7f9af80cde60 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 --2- 172.21.15.102:0/3898344400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9af806dd30 0x7f9af81f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.754+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 >> 172.21.15.102:0/3898344400 conn(0x7f9af806f9f0 msgr2=0x7f9af80c8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.755+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 shutdown_connections 2024-04-20T21:46:12.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.755+0000 7f9affc00640 1 -- 172.21.15.102:0/3898344400 wait complete. 2024-04-20T21:46:12.774 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-04-20T21:46:12.774 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T21:46:12.774 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:12.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.979+0000 7f3524600640 1 Processor -- start 2024-04-20T21:46:12.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.979+0000 7f3524600640 1 -- start start 2024-04-20T21:46:12.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.980+0000 7f3524600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c152170 0x7f351c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.980+0000 7f3524600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f351c158770 0x7f351c070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.980+0000 7f3524600640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f351c158b50 0x7f351c071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.980+0000 7f3524600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f351c0c6dc0 con 0x7f351c158770 2024-04-20T21:46:12.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3524600640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f351c071f70 con 0x7f351c158b50 2024-04-20T21:46:12.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3524600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f351c072210 con 0x7f351c152170 2024-04-20T21:46:12.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f351c158770 0x7f351c070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c152170 0x7f351c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c152170 0x7f351c0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:38596/0 (socket says 172.21.15.102:38596) 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521c00640 1 -- 172.21.15.102:0/2960906157 learned_addr learned my addr 172.21.15.102:0/2960906157 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f351c158770 0x7f351c070df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37264/0 (socket says 172.21.15.102:37264) 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3522600640 1 --2- 172.21.15.102:0/2960906157 >> v2:172.21.15.193:3300/0 conn(0x7f351c158b50 0x7f351c071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521c00640 1 -- 172.21.15.102:0/2960906157 >> v2:172.21.15.193:3300/0 conn(0x7f351c158b50 msgr2=0x7f351c071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521c00640 1 --2- 172.21.15.102:0/2960906157 >> v2:172.21.15.193:3300/0 conn(0x7f351c158b50 0x7f351c071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521c00640 1 -- 172.21.15.102:0/2960906157 >> v2:172.21.15.102:3300/0 conn(0x7f351c158770 msgr2=0x7f351c070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521c00640 1 --2- 172.21.15.102:0/2960906157 >> v2:172.21.15.102:3300/0 conn(0x7f351c158770 0x7f351c070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.981+0000 7f3521c00640 1 -- 172.21.15.102:0/2960906157 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c0724b0 con 0x7f351c152170 2024-04-20T21:46:12.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.982+0000 7f3521c00640 1 --2- 172.21.15.102:0/2960906157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c152170 0x7f351c0c8db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f351c06f980 tx=0x7f351002e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef24dca67d94c311 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:12.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.983+0000 7f351be00640 1 -- 172.21.15.102:0/2960906157 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351003b070 con 0x7f351c152170 2024-04-20T21:46:12.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.983+0000 7f351be00640 1 -- 172.21.15.102:0/2960906157 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3510036440 con 0x7f351c152170 2024-04-20T21:46:12.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.983+0000 7f351be00640 1 -- 172.21.15.102:0/2960906157 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510036810 con 0x7f351c152170 2024-04-20T21:46:12.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.983+0000 7f3524600640 1 -- 172.21.15.102:0/2960906157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c152170 msgr2=0x7f351c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.983+0000 7f3524600640 1 --2- 172.21.15.102:0/2960906157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c152170 0x7f351c0c8db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f351c06f980 tx=0x7f351002e160 comp rx=0 tx=0).stop 2024-04-20T21:46:12.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.983+0000 7f3524600640 1 -- 172.21.15.102:0/2960906157 shutdown_connections 2024-04-20T21:46:12.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.983+0000 7f3524600640 1 --2- 172.21.15.102:0/2960906157 >> v2:172.21.15.193:3300/0 conn(0x7f351c158b50 0x7f351c071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.984+0000 7f3524600640 1 --2- 172.21.15.102:0/2960906157 >> v2:172.21.15.102:3300/0 conn(0x7f351c158770 0x7f351c070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.984+0000 7f3524600640 1 --2- 172.21.15.102:0/2960906157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c152170 0x7f351c0c8db0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.984+0000 7f3524600640 1 -- 172.21.15.102:0/2960906157 >> 172.21.15.102:0/2960906157 conn(0x7f351c0c9390 msgr2=0x7f351c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:12.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.984+0000 7f3524600640 1 -- 172.21.15.102:0/2960906157 shutdown_connections 2024-04-20T21:46:12.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.984+0000 7f3524600640 1 -- 172.21.15.102:0/2960906157 wait complete. 2024-04-20T21:46:12.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.986+0000 7f3524600640 1 Processor -- start 2024-04-20T21:46:12.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.986+0000 7f3524600640 1 -- start start 2024-04-20T21:46:12.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.987+0000 7f3524600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f351c152170 0x7f351c14a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.987+0000 7f3524600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f351c158770 0x7f351c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.987+0000 7f3524600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c158b50 0x7f351c15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.987+0000 7f3524600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f351c0c9f20 con 0x7f351c152170 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.987+0000 7f3524600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f351c158250 con 0x7f351c158770 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.987+0000 7f3524600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f351c157f20 con 0x7f351c158b50 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.987+0000 7f3521c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f351c152170 0x7f351c14a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f351c152170 0x7f351c14a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37266/0 (socket says 172.21.15.102:37266) 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521c00640 1 -- 172.21.15.102:0/1416388920 learned_addr learned my addr 172.21.15.102:0/1416388920 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3522600640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c158b50 0x7f351c15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521200640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f351c158770 0x7f351c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521c00640 1 -- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c158b50 msgr2=0x7f351c15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521c00640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c158b50 0x7f351c15abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521c00640 1 -- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f351c158770 msgr2=0x7f351c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521c00640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f351c158770 0x7f351c0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521c00640 1 -- 172.21.15.102:0/1416388920 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c0c7bf0 con 0x7f351c152170 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521200640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f351c158770 0x7f351c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3521c00640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f351c152170 0x7f351c14a860 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3510000c00 tx=0x7f35100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.988+0000 7f3519600640 1 -- 172.21.15.102:0/1416388920 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510046070 con 0x7f351c152170 2024-04-20T21:46:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.989+0000 7f3519600640 1 -- 172.21.15.102:0/1416388920 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351003d050 con 0x7f351c152170 2024-04-20T21:46:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.989+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c0c7e60 con 0x7f351c152170 2024-04-20T21:46:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.989+0000 7f3519600640 1 -- 172.21.15.102:0/1416388920 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351003b030 con 0x7f351c152170 2024-04-20T21:46:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.989+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c0c8980 con 0x7f351c152170 2024-04-20T21:46:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.990+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c074340 con 0x7f351c152170 2024-04-20T21:46:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.991+0000 7f3519600640 1 -- 172.21.15.102:0/1416388920 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3510049070 con 0x7f351c152170 2024-04-20T21:46:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.992+0000 7f3519600640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f34fc06c920 0x7f34fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.992+0000 7f3519600640 1 -- 172.21.15.102:0/1416388920 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f35100adfb0 con 0x7f351c152170 2024-04-20T21:46:12.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.992+0000 7f3521200640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f34fc06c920 0x7f34fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:12.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.993+0000 7f3521200640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f34fc06c920 0x7f34fc06ede0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f351c14eec0 tx=0x7f3508008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:12.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:12.996+0000 7f3519600640 1 -- 172.21.15.102:0/1416388920 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f351007b2d0 con 0x7f351c152170 2024-04-20T21:46:13.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.238+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f351c0c7490 con 0x7f351c152170 2024-04-20T21:46:13.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.239+0000 7f3519600640 1 -- 172.21.15.102:0/1416388920 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+4375 (secure 0 0 0) 0x7f351007f5d0 con 0x7f351c152170 2024-04-20T21:46:13.238 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:10.945178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:13.238 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 104 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.241+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f34fc06c920 msgr2=0x7f34fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.241+0000 7f3524600640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f34fc06c920 0x7f34fc06ede0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f351c14eec0 tx=0x7f3508008040 comp rx=0 tx=0).stop 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.241+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f351c152170 msgr2=0x7f351c14a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.241+0000 7f3524600640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f351c152170 0x7f351c14a860 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3510000c00 tx=0x7f35100046c0 comp rx=0 tx=0).stop 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.242+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 shutdown_connections 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.242+0000 7f3524600640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f351c158b50 0x7f351c15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.242+0000 7f3524600640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f34fc06c920 0x7f34fc06ede0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.242+0000 7f3524600640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f351c158770 0x7f351c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.242+0000 7f3524600640 1 --2- 172.21.15.102:0/1416388920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f351c152170 0x7f351c14a860 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.242+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 >> 172.21.15.102:0/1416388920 conn(0x7f351c0c9390 msgr2=0x7f351c0c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.242+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 shutdown_connections 2024-04-20T21:46:13.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:13.242+0000 7f3524600640 1 -- 172.21.15.102:0/1416388920 wait complete. 2024-04-20T21:46:13.259 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 104, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 104, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7026": { "addr": "172.21.15.102:6833/1356782221", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 1356782221, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 1356782221, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7026, "incarnation": 104, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:46:10.945178+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7026 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:46:15.260 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:15.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.470+0000 7f6bd2e00640 1 Processor -- start 2024-04-20T21:46:15.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.470+0000 7f6bd2e00640 1 -- start start 2024-04-20T21:46:15.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd2e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6bcc0c75a0 0x7f6bcc0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:15.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd2e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc073c70 0x7f6bcc0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:15.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd2e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bcc153560 con 0x7f6bcc153180 2024-04-20T21:46:15.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd2e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc0c7a70 con 0x7f6bcc0c75a0 2024-04-20T21:46:15.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd2e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc0c7d30 con 0x7f6bcc073c70 2024-04-20T21:46:15.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd2400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc073c70 0x7f6bcc0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:15.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd1000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6bcc0c75a0 0x7f6bcc0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:15.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd2400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc073c70 0x7f6bcc0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:38604/0 (socket says 172.21.15.102:38604) 2024-04-20T21:46:15.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.472+0000 7f6bd2400640 1 -- 172.21.15.102:0/3002690827 learned_addr learned my addr 172.21.15.102:0/3002690827 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.473+0000 7f6bd1000640 1 -- 172.21.15.102:0/3002690827 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc073c70 msgr2=0x7f6bcc0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.473+0000 7f6bd1000640 1 --2- 172.21.15.102:0/3002690827 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc073c70 0x7f6bcc0c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.473+0000 7f6bd1000640 1 -- 172.21.15.102:0/3002690827 >> v1:172.21.15.193:6789/0 conn(0x7f6bcc153180 legacy=0x7f6bcc158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.473+0000 7f6bd1000640 1 -- 172.21.15.102:0/3002690827 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc15c4d0 con 0x7f6bcc0c75a0 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.473+0000 7f6bd2400640 1 --2- 172.21.15.102:0/3002690827 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc073c70 0x7f6bcc0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.474+0000 7f6bd1000640 1 --2- 172.21.15.102:0/3002690827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0c75a0 0x7f6bcc0c5f30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80097a0 tx=0x7f6bb802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fdcb8f388746fff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.474+0000 7f6bcbe00640 1 -- 172.21.15.102:0/3002690827 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8035020 con 0x7f6bcc0c75a0 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.474+0000 7f6bcbe00640 1 -- 172.21.15.102:0/3002690827 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb8031810 con 0x7f6bcc0c75a0 2024-04-20T21:46:15.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.474+0000 7f6bcbe00640 1 -- 172.21.15.102:0/3002690827 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8031be0 con 0x7f6bcc0c75a0 2024-04-20T21:46:15.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.475+0000 7f6bd2e00640 1 -- 172.21.15.102:0/3002690827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0c75a0 msgr2=0x7f6bcc0c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:15.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.475+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/3002690827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0c75a0 0x7f6bcc0c5f30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80097a0 tx=0x7f6bb802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.475+0000 7f6bd2e00640 1 -- 172.21.15.102:0/3002690827 shutdown_connections 2024-04-20T21:46:15.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.475+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/3002690827 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc073c70 0x7f6bcc0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.475+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/3002690827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc0c75a0 0x7f6bcc0c5f30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.475+0000 7f6bd2e00640 1 -- 172.21.15.102:0/3002690827 >> 172.21.15.102:0/3002690827 conn(0x7f6bcc0cc820 msgr2=0x7f6bcc0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:15.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.475+0000 7f6bd2e00640 1 -- 172.21.15.102:0/3002690827 shutdown_connections 2024-04-20T21:46:15.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.475+0000 7f6bd2e00640 1 -- 172.21.15.102:0/3002690827 wait complete. 2024-04-20T21:46:15.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.477+0000 7f6bd2e00640 1 Processor -- start 2024-04-20T21:46:15.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.477+0000 7f6bd2e00640 1 -- start start 2024-04-20T21:46:15.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.478+0000 7f6bd2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bcc073c70 0x7f6bcc15b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:15.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc0c75a0 0x7f6bcc14bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:15.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc153180 0x7f6bcc070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:15.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc159550 con 0x7f6bcc073c70 2024-04-20T21:46:15.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc15ca90 con 0x7f6bcc153180 2024-04-20T21:46:15.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc15d290 con 0x7f6bcc0c75a0 2024-04-20T21:46:15.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc153180 0x7f6bcc070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:15.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd1000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc0c75a0 0x7f6bcc14bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc153180 0x7f6bcc070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:58064/0 (socket says 172.21.15.102:58064) 2024-04-20T21:46:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2400640 1 -- 172.21.15.102:0/1021659406 learned_addr learned my addr 172.21.15.102:0/1021659406 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bcc073c70 0x7f6bcc15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2400640 1 -- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc0c75a0 msgr2=0x7f6bcc14bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2400640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc0c75a0 0x7f6bcc14bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2400640 1 -- 172.21.15.102:0/1021659406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bcc073c70 msgr2=0x7f6bcc15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2400640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bcc073c70 0x7f6bcc15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.479+0000 7f6bd2400640 1 -- 172.21.15.102:0/1021659406 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc14b510 con 0x7f6bcc153180 2024-04-20T21:46:15.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.480+0000 7f6bd2400640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc153180 0x7f6bcc070c40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009e70 tx=0x7f6bc400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:15.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.480+0000 7f6bc9600640 1 -- 172.21.15.102:0/1021659406 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400cbe0 con 0x7f6bcc153180 2024-04-20T21:46:15.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.480+0000 7f6bc9600640 1 -- 172.21.15.102:0/1021659406 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc400cd80 con 0x7f6bcc153180 2024-04-20T21:46:15.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.480+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc14a250 con 0x7f6bcc153180 2024-04-20T21:46:15.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.480+0000 7f6bc9600640 1 -- 172.21.15.102:0/1021659406 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4003bb0 con 0x7f6bcc153180 2024-04-20T21:46:15.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.480+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc1e3440 con 0x7f6bcc153180 2024-04-20T21:46:15.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.481+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc04fd60 con 0x7f6bcc153180 2024-04-20T21:46:15.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.482+0000 7f6bc9600640 1 -- 172.21.15.102:0/1021659406 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bc401b030 con 0x7f6bcc153180 2024-04-20T21:46:15.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.483+0000 7f6bc9600640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6bac06c8f0 0x7f6bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:15.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.483+0000 7f6bc9600640 1 -- 172.21.15.102:0/1021659406 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f6bc408c810 con 0x7f6bcc153180 2024-04-20T21:46:15.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.483+0000 7f6bd1a00640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6bac06c8f0 0x7f6bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:15.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.484+0000 7f6bd1a00640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6bac06c8f0 0x7f6bac06edb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0001780 tx=0x7f6bc0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:15.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.488+0000 7f6bc9600640 1 -- 172.21.15.102:0/1021659406 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6bc4022020 con 0x7f6bcc153180 2024-04-20T21:46:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.731+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bcc0c3690 con 0x7f6bcc153180 2024-04-20T21:46:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.733+0000 7f6bc9600640 1 -- 172.21.15.102:0/1021659406 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+4375 (secure 0 0 0) 0x7f6bcc14b510 con 0x7f6bcc153180 2024-04-20T21:46:15.731 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:15.732 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:10.945178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 104 2024-04-20T21:46:15.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6bac06c8f0 msgr2=0x7f6bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:15.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6bac06c8f0 0x7f6bac06edb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0001780 tx=0x7f6bc0009040 comp rx=0 tx=0).stop 2024-04-20T21:46:15.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc153180 msgr2=0x7f6bcc070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc153180 0x7f6bcc070c40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009e70 tx=0x7f6bc400c490 comp rx=0 tx=0).stop 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 shutdown_connections 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6bac06c8f0 0x7f6bac06edb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bcc153180 0x7f6bcc070c40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bcc0c75a0 0x7f6bcc14bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 --2- 172.21.15.102:0/1021659406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bcc073c70 0x7f6bcc15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 >> 172.21.15.102:0/1021659406 conn(0x7f6bcc0cc820 msgr2=0x7f6bcc150d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 shutdown_connections 2024-04-20T21:46:15.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:15.735+0000 7f6bd2e00640 1 -- 172.21.15.102:0/1021659406 wait complete. 2024-04-20T21:46:15.753 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:16.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~255s 2024-04-20T21:46:16.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~167s 2024-04-20T21:46:17.755 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:17.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.937+0000 7ff970e00640 1 Processor -- start 2024-04-20T21:46:17.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.937+0000 7ff970e00640 1 -- start start 2024-04-20T21:46:17.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.938+0000 7ff970e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff96c155560 0x7ff96c15be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:17.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.938+0000 7ff970e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c0c5cc0 0x7ff96c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.938+0000 7ff970e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff96c0c60c0 0x7ff96c073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.938+0000 7ff970e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c15f0c0 con 0x7ff96c155560 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.938+0000 7ff970e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c15e7b0 con 0x7ff96c0c60c0 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.938+0000 7ff970e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c155940 con 0x7ff96c0c5cc0 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff96c155560 0x7ff96c15be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff96a800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff96c0c60c0 0x7ff96c073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c0c5cc0 0x7ff96c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff96a800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff96c0c60c0 0x7ff96c073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:52196/0 (socket says 172.21.15.102:52196) 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c0c5cc0 0x7ff96c0ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:46580/0 (socket says 172.21.15.102:46580) 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff96a800640 1 -- 172.21.15.102:0/2673755853 learned_addr learned my addr 172.21.15.102:0/2673755853 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:17.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff96c155560 0x7ff96c15be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43338/0 (socket says 172.21.15.102:43338) 2024-04-20T21:46:17.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969e00640 1 -- 172.21.15.102:0/2673755853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c0c5cc0 msgr2=0x7ff96c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:17.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969e00640 1 --2- 172.21.15.102:0/2673755853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c0c5cc0 0x7ff96c0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:17.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969e00640 1 -- 172.21.15.102:0/2673755853 >> v2:172.21.15.193:3300/0 conn(0x7ff96c0c60c0 msgr2=0x7ff96c073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:17.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969e00640 1 --2- 172.21.15.102:0/2673755853 >> v2:172.21.15.193:3300/0 conn(0x7ff96c0c60c0 0x7ff96c073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:17.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.939+0000 7ff969e00640 1 -- 172.21.15.102:0/2673755853 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c0cbea0 con 0x7ff96c155560 2024-04-20T21:46:17.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.940+0000 7ff969e00640 1 --2- 172.21.15.102:0/2673755853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c155560 0x7ff96c15be30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff96c08e7e0 tx=0x7ff95c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5da01c6600441b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:17.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff968a00640 1 -- 172.21.15.102:0/2673755853 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c035020 con 0x7ff96c155560 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff968a00640 1 -- 172.21.15.102:0/2673755853 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c031830 con 0x7ff96c155560 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff968a00640 1 -- 172.21.15.102:0/2673755853 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c031c00 con 0x7ff96c155560 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff970e00640 1 -- 172.21.15.102:0/2673755853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c155560 msgr2=0x7ff96c15be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff970e00640 1 --2- 172.21.15.102:0/2673755853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c155560 0x7ff96c15be30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff96c08e7e0 tx=0x7ff95c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff970e00640 1 -- 172.21.15.102:0/2673755853 shutdown_connections 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff970e00640 1 --2- 172.21.15.102:0/2673755853 >> v2:172.21.15.193:3300/0 conn(0x7ff96c0c60c0 0x7ff96c073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff970e00640 1 --2- 172.21.15.102:0/2673755853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c0c5cc0 0x7ff96c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff970e00640 1 --2- 172.21.15.102:0/2673755853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c155560 0x7ff96c15be30 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.941+0000 7ff970e00640 1 -- 172.21.15.102:0/2673755853 >> 172.21.15.102:0/2673755853 conn(0x7ff96c06cd40 msgr2=0x7ff96c06d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.942+0000 7ff970e00640 1 -- 172.21.15.102:0/2673755853 shutdown_connections 2024-04-20T21:46:17.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.942+0000 7ff970e00640 1 -- 172.21.15.102:0/2673755853 wait complete. 2024-04-20T21:46:17.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.943+0000 7ff970e00640 1 Processor -- start 2024-04-20T21:46:17.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.943+0000 7ff970e00640 1 -- start start 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.944+0000 7ff970e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c0c5cc0 0x7ff96c1f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.944+0000 7ff970e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c155560 0x7ff96c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.944+0000 7ff970e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff96c159070 0x7ff96c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.944+0000 7ff970e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c15ed50 con 0x7ff96c0c5cc0 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff970e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c15ea70 con 0x7ff96c159070 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff970e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c0cbb60 con 0x7ff96c155560 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff969e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c0c5cc0 0x7ff96c1f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff969400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c155560 0x7ff96c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff96a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff96c159070 0x7ff96c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff969400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c155560 0x7ff96c15b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:46590/0 (socket says 172.21.15.102:46590) 2024-04-20T21:46:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff969e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c0c5cc0 0x7ff96c1f1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43340/0 (socket says 172.21.15.102:43340) 2024-04-20T21:46:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff96a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff96c159070 0x7ff96c15bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:52210/0 (socket says 172.21.15.102:52210) 2024-04-20T21:46:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff969400640 1 -- 172.21.15.102:0/3242925651 learned_addr learned my addr 172.21.15.102:0/3242925651 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff96a800640 1 -- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c155560 msgr2=0x7ff96c15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff96a800640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c155560 0x7ff96c15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff96a800640 1 -- 172.21.15.102:0/3242925651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c0c5cc0 msgr2=0x7ff96c1f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff96a800640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c0c5cc0 0x7ff96c1f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff96a800640 1 -- 172.21.15.102:0/3242925651 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c156bc0 con 0x7ff96c159070 2024-04-20T21:46:17.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.945+0000 7ff969e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c0c5cc0 0x7ff96c1f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.946+0000 7ff96a800640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff96c159070 0x7ff96c15bdb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff964009d90 tx=0x7ff96400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.946+0000 7ff962000640 1 -- 172.21.15.102:0/3242925651 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96400cd20 con 0x7ff96c159070 2024-04-20T21:46:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.946+0000 7ff962000640 1 -- 172.21.15.102:0/3242925651 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff964003990 con 0x7ff96c159070 2024-04-20T21:46:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.946+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c159640 con 0x7ff96c159070 2024-04-20T21:46:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.946+0000 7ff962000640 1 -- 172.21.15.102:0/3242925651 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964003d60 con 0x7ff96c159070 2024-04-20T21:46:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.946+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c159c00 con 0x7ff96c159070 2024-04-20T21:46:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.947+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c074340 con 0x7ff96c159070 2024-04-20T21:46:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.948+0000 7ff962000640 1 -- 172.21.15.102:0/3242925651 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff964012050 con 0x7ff96c159070 2024-04-20T21:46:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.949+0000 7ff962000640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff94c06c920 0x7ff94c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.949+0000 7ff962000640 1 -- 172.21.15.102:0/3242925651 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7ff96408c9d0 con 0x7ff96c159070 2024-04-20T21:46:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.949+0000 7ff969e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff94c06c920 0x7ff94c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:17.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.950+0000 7ff969e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff94c06c920 0x7ff94c06ede0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff95c036000 tx=0x7ff95c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:17.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:17.954+0000 7ff962000640 1 -- 172.21.15.102:0/3242925651 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff964010040 con 0x7ff96c159070 2024-04-20T21:46:18.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.199+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff96c0cc480 con 0x7ff96c159070 2024-04-20T21:46:18.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.200+0000 7ff962000640 1 -- 172.21.15.102:0/3242925651 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+4375 (secure 0 0 0) 0x7ff964059c60 con 0x7ff96c159070 2024-04-20T21:46:18.199 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:18.199 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:10.945178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:18.199 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 104 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff94c06c920 msgr2=0x7ff94c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff94c06c920 0x7ff94c06ede0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff95c036000 tx=0x7ff95c03c040 comp rx=0 tx=0).stop 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff96c159070 msgr2=0x7ff96c15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff96c159070 0x7ff96c15bdb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff964009d90 tx=0x7ff96400c490 comp rx=0 tx=0).stop 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 shutdown_connections 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff94c06c920 0x7ff94c06ede0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff96c159070 0x7ff96c15bdb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff96c155560 0x7ff96c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 --2- 172.21.15.102:0/3242925651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff96c0c5cc0 0x7ff96c1f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 >> 172.21.15.102:0/3242925651 conn(0x7ff96c06cd40 msgr2=0x7ff96c0cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:18.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 shutdown_connections 2024-04-20T21:46:18.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:18.203+0000 7ff970e00640 1 -- 172.21.15.102:0/3242925651 wait complete. 2024-04-20T21:46:18.225 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:20.226 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:20.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.415+0000 7f9a4ac00640 1 Processor -- start 2024-04-20T21:46:20.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.415+0000 7f9a4ac00640 1 -- start start 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a4ac00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9a440cd940 0x7f9a4415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a4ac00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9a440ce310 0x7f9a44072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a4ac00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4415f940 con 0x7f9a440c7900 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a4ac00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f9a4415eb60 con 0x7f9a440ce310 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a4ac00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9a4415ee00 con 0x7f9a440cd940 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a48c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9a440ce310 0x7f9a44072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a48c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9a440ce310 0x7f9a44072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43344/0 (socket says 172.21.15.102:43344) 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a48c00640 1 -- 172.21.15.102:0/1204050675 learned_addr learned my addr 172.21.15.102:0/1204050675 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.417+0000 7f9a43400640 1 --2- 172.21.15.102:0/1204050675 >> v2:172.21.15.193:3300/0 conn(0x7f9a440cd940 0x7f9a4415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:20.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.418+0000 7f9a48c00640 1 -- 172.21.15.102:0/1204050675 >> v2:172.21.15.193:3300/0 conn(0x7f9a440cd940 msgr2=0x7f9a4415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.418+0000 7f9a48c00640 1 --2- 172.21.15.102:0/1204050675 >> v2:172.21.15.193:3300/0 conn(0x7f9a440cd940 0x7f9a4415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.418+0000 7f9a48c00640 1 -- 172.21.15.102:0/1204050675 >> v1:172.21.15.193:6789/0 conn(0x7f9a440c7900 legacy=0x7f9a44074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.418+0000 7f9a42a00640 1 -- 172.21.15.102:0/1204050675 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 829259202 0 0) 0x7f9a4415f940 con 0x7f9a440c7900 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.418+0000 7f9a48c00640 1 -- 172.21.15.102:0/1204050675 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4415f0a0 con 0x7f9a440ce310 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.419+0000 7f9a48c00640 1 --2- 172.21.15.102:0/1204050675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440ce310 0x7f9a44072230 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9a440cea70 tx=0x7f9a3c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f498fec2ff715b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.419+0000 7f9a42a00640 1 -- 172.21.15.102:0/1204050675 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c02cb20 con 0x7f9a440ce310 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.419+0000 7f9a42a00640 1 -- 172.21.15.102:0/1204050675 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3c02ccc0 con 0x7f9a440ce310 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.419+0000 7f9a42a00640 1 -- 172.21.15.102:0/1204050675 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c034610 con 0x7f9a440ce310 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.420+0000 7f9a4ac00640 1 -- 172.21.15.102:0/1204050675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440ce310 msgr2=0x7f9a44072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.420+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/1204050675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440ce310 0x7f9a44072230 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9a440cea70 tx=0x7f9a3c02bf80 comp rx=0 tx=0).stop 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.420+0000 7f9a4ac00640 1 -- 172.21.15.102:0/1204050675 shutdown_connections 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.420+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/1204050675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440ce310 0x7f9a44072230 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.420+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/1204050675 >> v2:172.21.15.193:3300/0 conn(0x7f9a440cd940 0x7f9a4415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.420+0000 7f9a4ac00640 1 -- 172.21.15.102:0/1204050675 >> 172.21.15.102:0/1204050675 conn(0x7f9a4406dad0 msgr2=0x7f9a4406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:20.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.420+0000 7f9a4ac00640 1 -- 172.21.15.102:0/1204050675 shutdown_connections 2024-04-20T21:46:20.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.420+0000 7f9a4ac00640 1 -- 172.21.15.102:0/1204050675 wait complete. 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.422+0000 7f9a4ac00640 1 Processor -- start 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.422+0000 7f9a4ac00640 1 -- start start 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a4ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a440c7900 0x7f9a4414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a4ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440cd940 0x7f9a44152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a4ac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a440ce310 0x7f9a44159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a4ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9a440755b0 con 0x7f9a440cd940 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a4ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4415f660 con 0x7f9a440c7900 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a4ac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9a44075730 con 0x7f9a440ce310 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440cd940 0x7f9a44152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:20.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440cd940 0x7f9a44152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43360/0 (socket says 172.21.15.102:43360) 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43400640 1 -- 172.21.15.102:0/2052363600 learned_addr learned my addr 172.21.15.102:0/2052363600 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43e00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a440c7900 0x7f9a4414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a48c00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a440ce310 0x7f9a44159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43400640 1 -- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a440ce310 msgr2=0x7f9a44159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43400640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a440ce310 0x7f9a44159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43400640 1 -- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a440c7900 msgr2=0x7f9a4414e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43400640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a440c7900 0x7f9a4414e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.423+0000 7f9a43400640 1 -- 172.21.15.102:0/2052363600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a44152f60 con 0x7f9a440cd940 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.424+0000 7f9a43400640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440cd940 0x7f9a44152620 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000e9b0 tx=0x7f9a3000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.424+0000 7f9a3be00640 1 -- 172.21.15.102:0/2052363600 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30017070 con 0x7f9a440cd940 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.424+0000 7f9a3be00640 1 -- 172.21.15.102:0/2052363600 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3000c9a0 con 0x7f9a440cd940 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.424+0000 7f9a3be00640 1 -- 172.21.15.102:0/2052363600 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3000cd90 con 0x7f9a440cd940 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.424+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a44153200 con 0x7f9a440cd940 2024-04-20T21:46:20.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.424+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441f18a0 con 0x7f9a440cd940 2024-04-20T21:46:20.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.425+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a44074340 con 0x7f9a440cd940 2024-04-20T21:46:20.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.426+0000 7f9a3be00640 1 -- 172.21.15.102:0/2052363600 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a3001a070 con 0x7f9a440cd940 2024-04-20T21:46:20.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.428+0000 7f9a3be00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9a2406c8f0 0x7f9a2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:20.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.428+0000 7f9a3be00640 1 -- 172.21.15.102:0/2052363600 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f9a3008c470 con 0x7f9a440cd940 2024-04-20T21:46:20.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.428+0000 7f9a43e00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9a2406c8f0 0x7f9a2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:20.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.429+0000 7f9a43e00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9a2406c8f0 0x7f9a2406edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9a34000f60 tx=0x7f9a34009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:20.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.432+0000 7f9a3be00640 1 -- 172.21.15.102:0/2052363600 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9a30059880 con 0x7f9a440cd940 2024-04-20T21:46:20.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.680+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a440c6e40 con 0x7f9a440cd940 2024-04-20T21:46:20.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.682+0000 7f9a3be00640 1 -- 172.21.15.102:0/2052363600 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+4375 (secure 0 0 0) 0x7f9a3005db80 con 0x7f9a440cd940 2024-04-20T21:46:20.681 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:20.681 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:10.945178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:20.681 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 104 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.684+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9a2406c8f0 msgr2=0x7f9a2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.684+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9a2406c8f0 0x7f9a2406edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9a34000f60 tx=0x7f9a34009040 comp rx=0 tx=0).stop 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.684+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440cd940 msgr2=0x7f9a44152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.684+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440cd940 0x7f9a44152620 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000e9b0 tx=0x7f9a3000ee70 comp rx=0 tx=0).stop 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.685+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 shutdown_connections 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.685+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a440ce310 0x7f9a44159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.685+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a440cd940 0x7f9a44152620 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.685+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f9a2406c8f0 0x7f9a2406edb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.685+0000 7f9a4ac00640 1 --2- 172.21.15.102:0/2052363600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a440c7900 0x7f9a4414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.685+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 >> 172.21.15.102:0/2052363600 conn(0x7f9a4406dad0 msgr2=0x7f9a4406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.685+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 shutdown_connections 2024-04-20T21:46:20.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:20.685+0000 7f9a4ac00640 1 -- 172.21.15.102:0/2052363600 wait complete. 2024-04-20T21:46:20.703 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:22.704 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:22.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.895+0000 7f76f5a00640 1 Processor -- start 2024-04-20T21:46:22.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.895+0000 7f76f5a00640 1 -- start start 2024-04-20T21:46:22.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.896+0000 7f76f5a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76f0153180 0x7f76f0158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:22.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.896+0000 7f76f5a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f76f00c75a0 0x7f76f01637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:22.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.896+0000 7f76f5a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f0153560 con 0x7f76f0159380 2024-04-20T21:46:22.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.896+0000 7f76f5a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f76f00c7a70 con 0x7f76f0153180 2024-04-20T21:46:22.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.896+0000 7f76f5a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f76f00c7d30 con 0x7f76f00c75a0 2024-04-20T21:46:22.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76efe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76f0153180 0x7f76f0158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:22.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76efe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f76f0153180 0x7f76f0158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43362/0 (socket says 172.21.15.102:43362) 2024-04-20T21:46:22.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76efe00640 1 -- 172.21.15.102:0/1379386779 learned_addr learned my addr 172.21.15.102:0/1379386779 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:22.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76e7400640 1 --1- 172.21.15.102:0/1379386779 >> v1:172.21.15.193:6789/0 conn(0x7f76f0159380 0x7f76f00c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:46300/0 (socket says 172.21.15.102:46300) 2024-04-20T21:46:22.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76f5000640 1 --2- 172.21.15.102:0/1379386779 >> v2:172.21.15.193:3300/0 conn(0x7f76f00c75a0 0x7f76f01637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:22.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76ef400640 1 -- 172.21.15.102:0/1379386779 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054722741 0 0) 0x7f76f0153560 con 0x7f76f0159380 2024-04-20T21:46:22.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76ef400640 1 -- 172.21.15.102:0/1379386779 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d4003520 con 0x7f76f0159380 2024-04-20T21:46:22.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76efe00640 1 -- 172.21.15.102:0/1379386779 >> v2:172.21.15.193:3300/0 conn(0x7f76f00c75a0 msgr2=0x7f76f01637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:22.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76efe00640 1 --2- 172.21.15.102:0/1379386779 >> v2:172.21.15.193:3300/0 conn(0x7f76f00c75a0 0x7f76f01637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:22.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76efe00640 1 -- 172.21.15.102:0/1379386779 >> v1:172.21.15.193:6789/0 conn(0x7f76f0159380 legacy=0x7f76f00c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:22.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.897+0000 7f76efe00640 1 -- 172.21.15.102:0/1379386779 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f015c4d0 con 0x7f76f0153180 2024-04-20T21:46:22.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.898+0000 7f76efe00640 1 --2- 172.21.15.102:0/1379386779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0153180 0x7f76f0158e40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f76f0159760 tx=0x7f76e002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d402936e9a3c09d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:22.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.898+0000 7f76ef400640 1 -- 172.21.15.102:0/1379386779 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e002cb20 con 0x7f76f0153180 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.898+0000 7f76ef400640 1 -- 172.21.15.102:0/1379386779 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e002ccc0 con 0x7f76f0153180 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.898+0000 7f76ef400640 1 -- 172.21.15.102:0/1379386779 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0034610 con 0x7f76f0153180 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.899+0000 7f76f5a00640 1 -- 172.21.15.102:0/1379386779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0153180 msgr2=0x7f76f0158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.899+0000 7f76f5a00640 1 --2- 172.21.15.102:0/1379386779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0153180 0x7f76f0158e40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f76f0159760 tx=0x7f76e002bf80 comp rx=0 tx=0).stop 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.899+0000 7f76f5a00640 1 -- 172.21.15.102:0/1379386779 shutdown_connections 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.899+0000 7f76f5a00640 1 --2- 172.21.15.102:0/1379386779 >> v2:172.21.15.193:3300/0 conn(0x7f76f00c75a0 0x7f76f01637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.899+0000 7f76f5a00640 1 --2- 172.21.15.102:0/1379386779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0153180 0x7f76f0158e40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.899+0000 7f76f5a00640 1 -- 172.21.15.102:0/1379386779 >> 172.21.15.102:0/1379386779 conn(0x7f76f00cc820 msgr2=0x7f76f00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.899+0000 7f76f5a00640 1 -- 172.21.15.102:0/1379386779 shutdown_connections 2024-04-20T21:46:22.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.899+0000 7f76f5a00640 1 -- 172.21.15.102:0/1379386779 wait complete. 2024-04-20T21:46:22.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.901+0000 7f76f5a00640 1 Processor -- start 2024-04-20T21:46:22.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.901+0000 7f76f5a00640 1 -- start start 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76f00c75a0 0x7f76f01f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76f0153180 0x7f76f0150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0159380 0x7f76f01f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f76f015d290 con 0x7f76f0159380 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f76f015ca90 con 0x7f76f00c75a0 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f76f00c6be0 con 0x7f76f0153180 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0159380 0x7f76f01f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0159380 0x7f76f01f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43368/0 (socket says 172.21.15.102:43368) 2024-04-20T21:46:22.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76f5000640 1 -- 172.21.15.102:0/4235693923 learned_addr learned my addr 172.21.15.102:0/4235693923 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76efe00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76f00c75a0 0x7f76f01f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.902+0000 7f76e7400640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76f0153180 0x7f76f0150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76f5000640 1 -- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76f0153180 msgr2=0x7f76f0150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76f5000640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76f0153180 0x7f76f0150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76f5000640 1 -- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76f00c75a0 msgr2=0x7f76f01f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76f5000640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76f00c75a0 0x7f76f01f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76f5000640 1 -- 172.21.15.102:0/4235693923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f00c5690 con 0x7f76f0159380 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76f5000640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0159380 0x7f76f01f4ff0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f76e800e820 tx=0x7f76e800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76ecc00640 1 -- 172.21.15.102:0/4235693923 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8017070 con 0x7f76f0159380 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76ecc00640 1 -- 172.21.15.102:0/4235693923 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e800ee70 con 0x7f76f0159380 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76ecc00640 1 -- 172.21.15.102:0/4235693923 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e800cb60 con 0x7f76f0159380 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f00c5930 con 0x7f76f0159380 2024-04-20T21:46:22.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.903+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f00cbc00 con 0x7f76f0159380 2024-04-20T21:46:22.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.904+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f004fd60 con 0x7f76f0159380 2024-04-20T21:46:22.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.905+0000 7f76ecc00640 1 -- 172.21.15.102:0/4235693923 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76e801a070 con 0x7f76f0159380 2024-04-20T21:46:22.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.906+0000 7f76ecc00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76cc06c8f0 0x7f76cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:22.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.906+0000 7f76ecc00640 1 -- 172.21.15.102:0/4235693923 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f76e808c860 con 0x7f76f0159380 2024-04-20T21:46:22.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.907+0000 7f76efe00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76cc06c8f0 0x7f76cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:22.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.907+0000 7f76efe00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76cc06c8f0 0x7f76cc06edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f76e0035000 tx=0x7f76e0030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:22.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:22.911+0000 7f76ecc00640 1 -- 172.21.15.102:0/4235693923 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f76e8059af0 con 0x7f76f0159380 2024-04-20T21:46:23.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~261s 2024-04-20T21:46:23.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~173s 2024-04-20T21:46:23.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.190+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76f00c8fc0 con 0x7f76f0159380 2024-04-20T21:46:23.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.192+0000 7f76ecc00640 1 -- 172.21.15.102:0/4235693923 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+4375 (secure 0 0 0) 0x7f76e805ddf0 con 0x7f76f0159380 2024-04-20T21:46:23.191 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:23.191 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:10.945178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:23.191 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 104 2024-04-20T21:46:23.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76cc06c8f0 msgr2=0x7f76cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:23.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76cc06c8f0 0x7f76cc06edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f76e0035000 tx=0x7f76e0030000 comp rx=0 tx=0).stop 2024-04-20T21:46:23.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0159380 msgr2=0x7f76f01f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:23.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0159380 0x7f76f01f4ff0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f76e800e820 tx=0x7f76e800ec20 comp rx=0 tx=0).stop 2024-04-20T21:46:23.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 shutdown_connections 2024-04-20T21:46:23.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76f0159380 0x7f76f01f4ff0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:23.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76f0153180 0x7f76f0150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:23.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f76cc06c8f0 0x7f76cc06edb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:23.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 --2- 172.21.15.102:0/4235693923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76f00c75a0 0x7f76f01f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:23.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 >> 172.21.15.102:0/4235693923 conn(0x7f76f00cc820 msgr2=0x7f76f006f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:23.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 shutdown_connections 2024-04-20T21:46:23.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:23.195+0000 7f76f5a00640 1 -- 172.21.15.102:0/4235693923 wait complete. 2024-04-20T21:46:23.211 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:25.212 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:25.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.423+0000 7f6f4be00640 1 Processor -- start 2024-04-20T21:46:25.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.423+0000 7f6f4be00640 1 -- start start 2024-04-20T21:46:25.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c153180 0x7f6f4c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:25.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4be00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6f4c0c75a0 0x7f6f4c1637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:25.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4be00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f4c153560 con 0x7f6f4c159380 2024-04-20T21:46:25.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4be00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4c0c7a70 con 0x7f6f4c0c75a0 2024-04-20T21:46:25.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c0c7d30 con 0x7f6f4c153180 2024-04-20T21:46:25.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6f4c0c75a0 0x7f6f4c1637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:25.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6f4c0c75a0 0x7f6f4c1637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43370/0 (socket says 172.21.15.102:43370) 2024-04-20T21:46:25.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4b400640 1 -- 172.21.15.102:0/3884958907 learned_addr learned my addr 172.21.15.102:0/3884958907 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:25.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.425+0000 7f6f4aa00640 1 --2- 172.21.15.102:0/3884958907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c153180 0x7f6f4c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:25.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.426+0000 7f6f4b400640 1 -- 172.21.15.102:0/3884958907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c153180 msgr2=0x7f6f4c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:25.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.426+0000 7f6f4b400640 1 --2- 172.21.15.102:0/3884958907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c153180 0x7f6f4c158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.426+0000 7f6f4b400640 1 -- 172.21.15.102:0/3884958907 >> v1:172.21.15.193:6789/0 conn(0x7f6f4c159380 legacy=0x7f6f4c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:25.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.426+0000 7f6f4b400640 1 -- 172.21.15.102:0/3884958907 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c15c4d0 con 0x7f6f4c0c75a0 2024-04-20T21:46:25.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.426+0000 7f6f49600640 1 -- 172.21.15.102:0/3884958907 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2264415526 0 0) 0x7f6f4c153560 con 0x7f6f4c159380 2024-04-20T21:46:25.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.427+0000 7f6f4b400640 1 --2- 172.21.15.102:0/3884958907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c1637c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009e50 tx=0x7f6f4402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571380dda6963a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:25.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.427+0000 7f6f49600640 1 -- 172.21.15.102:0/3884958907 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4402cb00 con 0x7f6f4c0c75a0 2024-04-20T21:46:25.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.427+0000 7f6f49600640 1 -- 172.21.15.102:0/3884958907 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f4402cca0 con 0x7f6f4c0c75a0 2024-04-20T21:46:25.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.427+0000 7f6f49600640 1 -- 172.21.15.102:0/3884958907 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44034610 con 0x7f6f4c0c75a0 2024-04-20T21:46:25.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.428+0000 7f6f4be00640 1 -- 172.21.15.102:0/3884958907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 msgr2=0x7f6f4c1637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:25.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.428+0000 7f6f4be00640 1 --2- 172.21.15.102:0/3884958907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c1637c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009e50 tx=0x7f6f4402bf60 comp rx=0 tx=0).stop 2024-04-20T21:46:25.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.428+0000 7f6f4be00640 1 -- 172.21.15.102:0/3884958907 shutdown_connections 2024-04-20T21:46:25.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.428+0000 7f6f4be00640 1 --2- 172.21.15.102:0/3884958907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c1637c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.428+0000 7f6f4be00640 1 --2- 172.21.15.102:0/3884958907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c153180 0x7f6f4c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.428+0000 7f6f4be00640 1 -- 172.21.15.102:0/3884958907 >> 172.21.15.102:0/3884958907 conn(0x7f6f4c0cc820 msgr2=0x7f6f4c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:25.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.428+0000 7f6f4be00640 1 -- 172.21.15.102:0/3884958907 shutdown_connections 2024-04-20T21:46:25.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.428+0000 7f6f4be00640 1 -- 172.21.15.102:0/3884958907 wait complete. 2024-04-20T21:46:25.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.429+0000 7f6f4be00640 1 Processor -- start 2024-04-20T21:46:25.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.430+0000 7f6f4be00640 1 -- start start 2024-04-20T21:46:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.430+0000 7f6f4be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.430+0000 7f6f4be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c153180 0x7f6f4c1494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.430+0000 7f6f4be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c159380 0x7f6f4c0c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.430+0000 7f6f4be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c15d290 con 0x7f6f4c0c75a0 2024-04-20T21:46:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.430+0000 7f6f4be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c15ca90 con 0x7f6f4c153180 2024-04-20T21:46:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.430+0000 7f6f4be00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c0c6be0 con 0x7f6f4c159380 2024-04-20T21:46:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c153180 0x7f6f4c1494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c153180 0x7f6f4c1494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:52256/0 (socket says 172.21.15.102:52256) 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 -- 172.21.15.102:0/176406989 learned_addr learned my addr 172.21.15.102:0/176406989 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43372/0 (socket says 172.21.15.102:43372) 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4b400640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c159380 0x7f6f4c0c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 -- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c159380 msgr2=0x7f6f4c0c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c159380 0x7f6f4c0c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 -- 172.21.15.102:0/176406989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 msgr2=0x7f6f4c149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 -- 172.21.15.102:0/176406989 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c14a6e0 con 0x7f6f4c153180 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4aa00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.431+0000 7f6f4a000640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c153180 0x7f6f4c1494a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c550 tx=0x7f6f3800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.432+0000 7f6f42a00640 1 -- 172.21.15.102:0/176406989 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38017070 con 0x7f6f4c153180 2024-04-20T21:46:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.432+0000 7f6f42a00640 1 -- 172.21.15.102:0/176406989 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f380035f0 con 0x7f6f4c153180 2024-04-20T21:46:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.432+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c1587b0 con 0x7f6f4c153180 2024-04-20T21:46:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.432+0000 7f6f42a00640 1 -- 172.21.15.102:0/176406989 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f380039e0 con 0x7f6f4c153180 2024-04-20T21:46:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.432+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c158d70 con 0x7f6f4c153180 2024-04-20T21:46:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.433+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c04fd60 con 0x7f6f4c153180 2024-04-20T21:46:25.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.434+0000 7f6f42a00640 1 -- 172.21.15.102:0/176406989 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f3801a070 con 0x7f6f4c153180 2024-04-20T21:46:25.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.435+0000 7f6f42a00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6f2c06c8f0 0x7f6f2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:25.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.435+0000 7f6f42a00640 1 -- 172.21.15.102:0/176406989 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f6f3808b960 con 0x7f6f4c153180 2024-04-20T21:46:25.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.435+0000 7f6f4aa00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6f2c06c8f0 0x7f6f2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:25.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.436+0000 7f6f4aa00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6f2c06c8f0 0x7f6f2c06edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009e90 tx=0x7f6f3c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:25.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.440+0000 7f6f42a00640 1 -- 172.21.15.102:0/176406989 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6f38058d10 con 0x7f6f4c153180 2024-04-20T21:46:25.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.678+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f4c0c2ae0 con 0x7f6f4c153180 2024-04-20T21:46:25.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.680+0000 7f6f42a00640 1 -- 172.21.15.102:0/176406989 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+4379 (secure 0 0 0) 0x7f6f3805d010 con 0x7f6f4c153180 2024-04-20T21:46:25.679 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:25.679 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:23.253791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:25.679 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 105 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6f2c06c8f0 msgr2=0x7f6f2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6f2c06c8f0 0x7f6f2c06edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009e90 tx=0x7f6f3c009190 comp rx=0 tx=0).stop 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c153180 msgr2=0x7f6f4c1494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c153180 0x7f6f4c1494a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c550 tx=0x7f6f3800ca10 comp rx=0 tx=0).stop 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 shutdown_connections 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c159380 0x7f6f4c0c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f6f2c06c8f0 0x7f6f2c06edb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c153180 0x7f6f4c1494a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 --2- 172.21.15.102:0/176406989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f4c0c75a0 0x7f6f4c149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:25.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 >> 172.21.15.102:0/176406989 conn(0x7f6f4c0cc820 msgr2=0x7f6f4c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:25.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 shutdown_connections 2024-04-20T21:46:25.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:25.683+0000 7f6f4be00640 1 -- 172.21.15.102:0/176406989 wait complete. 2024-04-20T21:46:25.700 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 105, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 105, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7026": { "addr": "172.21.15.102:6833/1356782221", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 1356782221, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 1356782221, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7026, "incarnation": 104, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:46:23.253791+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7026 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:46:27.701 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:27.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.912+0000 7f29b4c00640 1 Processor -- start 2024-04-20T21:46:27.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.912+0000 7f29b4c00640 1 -- start start 2024-04-20T21:46:27.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.913+0000 7f29b4c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:27.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.913+0000 7f29b4c00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f29b00c3700 0x7f29b006e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:27.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.913+0000 7f29b4c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b015c220 con 0x7f29b006eb70 2024-04-20T21:46:27.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.913+0000 7f29b4c00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f29b00c9180 con 0x7f29b00c3700 2024-04-20T21:46:27.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.913+0000 7f29b4c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f29b00c4840 con 0x7f29b0152290 2024-04-20T21:46:27.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29aea00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f29b00c3700 0x7f29b006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:27.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29afe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f29b006eb70 0x7f29b00c8330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:44716/0 (socket says 172.21.15.102:44716) 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29af400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29afe00640 1 -- 172.21.15.102:0/434012401 learned_addr learned my addr 172.21.15.102:0/434012401 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29ae000640 1 -- 172.21.15.102:0/434012401 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 188712217 0 0) 0x7f29b015c220 con 0x7f29b006eb70 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29ae000640 1 -- 172.21.15.102:0/434012401 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2994003520 con 0x7f29b006eb70 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29af400640 1 -- 172.21.15.102:0/434012401 >> v2:172.21.15.193:3300/0 conn(0x7f29b00c3700 msgr2=0x7f29b006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29af400640 1 --2- 172.21.15.102:0/434012401 >> v2:172.21.15.193:3300/0 conn(0x7f29b00c3700 0x7f29b006e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29af400640 1 -- 172.21.15.102:0/434012401 >> v1:172.21.15.193:6789/0 conn(0x7f29b006eb70 legacy=0x7f29b00c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29af400640 1 -- 172.21.15.102:0/434012401 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b00c4560 con 0x7f29b0152290 2024-04-20T21:46:27.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.914+0000 7f29aea00640 1 --2- 172.21.15.102:0/434012401 >> v2:172.21.15.193:3300/0 conn(0x7f29b00c3700 0x7f29b006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29af400640 1 --2- 172.21.15.102:0/434012401 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b00cd310 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f29a00097a0 tx=0x7f29a002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647f1a81974f9d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29ae000640 1 -- 172.21.15.102:0/434012401 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0034070 con 0x7f29b0152290 2024-04-20T21:46:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29ae000640 1 -- 172.21.15.102:0/434012401 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a0030800 con 0x7f29b0152290 2024-04-20T21:46:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29ae000640 1 -- 172.21.15.102:0/434012401 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0030bb0 con 0x7f29b0152290 2024-04-20T21:46:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29b4c00640 1 -- 172.21.15.102:0/434012401 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 msgr2=0x7f29b00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29b4c00640 1 --2- 172.21.15.102:0/434012401 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b00cd310 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f29a00097a0 tx=0x7f29a002b940 comp rx=0 tx=0).stop 2024-04-20T21:46:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29b4c00640 1 -- 172.21.15.102:0/434012401 shutdown_connections 2024-04-20T21:46:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29b4c00640 1 --2- 172.21.15.102:0/434012401 >> v2:172.21.15.193:3300/0 conn(0x7f29b00c3700 0x7f29b006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29b4c00640 1 --2- 172.21.15.102:0/434012401 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b00cd310 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.916+0000 7f29b4c00640 1 -- 172.21.15.102:0/434012401 >> 172.21.15.102:0/434012401 conn(0x7f29b00cc820 msgr2=0x7f29b00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.917+0000 7f29b4c00640 1 -- 172.21.15.102:0/434012401 shutdown_connections 2024-04-20T21:46:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.917+0000 7f29b4c00640 1 -- 172.21.15.102:0/434012401 wait complete. 2024-04-20T21:46:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.918+0000 7f29b4c00640 1 Processor -- start 2024-04-20T21:46:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.918+0000 7f29b4c00640 1 -- start start 2024-04-20T21:46:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.919+0000 7f29b4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29b006eb70 0x7f29b0144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.919+0000 7f29b4c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29b00c3700 0x7f29b0150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.919+0000 7f29b4c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b0151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.919+0000 7f29b4c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f29b015ba20 con 0x7f29b006eb70 2024-04-20T21:46:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.919+0000 7f29b4c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29b015b740 con 0x7f29b00c3700 2024-04-20T21:46:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.919+0000 7f29b4c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f29b015bd00 con 0x7f29b0152290 2024-04-20T21:46:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.919+0000 7f29af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29b006eb70 0x7f29b0144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29b006eb70 0x7f29b0144e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44428/0 (socket says 172.21.15.102:44428) 2024-04-20T21:46:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29af400640 1 -- 172.21.15.102:0/3351390973 learned_addr learned my addr 172.21.15.102:0/3351390973 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29afe00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b0151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29aea00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29b00c3700 0x7f29b0150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29af400640 1 -- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 msgr2=0x7f29b0151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29af400640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b0151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29af400640 1 -- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29b00c3700 msgr2=0x7f29b0150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29af400640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29b00c3700 0x7f29b0150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29af400640 1 -- 172.21.15.102:0/3351390973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b00cb670 con 0x7f29b006eb70 2024-04-20T21:46:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29aea00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29b00c3700 0x7f29b0150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.920+0000 7f29af400640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29b006eb70 0x7f29b0144e50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f29a00098c0 tx=0x7f29a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.921+0000 7f29a7400640 1 -- 172.21.15.102:0/3351390973 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0034050 con 0x7f29b006eb70 2024-04-20T21:46:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.921+0000 7f29a7400640 1 -- 172.21.15.102:0/3351390973 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a0003820 con 0x7f29b006eb70 2024-04-20T21:46:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.921+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b015acb0 con 0x7f29b006eb70 2024-04-20T21:46:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.921+0000 7f29a7400640 1 -- 172.21.15.102:0/3351390973 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0003bd0 con 0x7f29b006eb70 2024-04-20T21:46:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.921+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b00cbbf0 con 0x7f29b006eb70 2024-04-20T21:46:27.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.922+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b004fd60 con 0x7f29b006eb70 2024-04-20T21:46:27.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.923+0000 7f29a7400640 1 -- 172.21.15.102:0/3351390973 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29a0037070 con 0x7f29b006eb70 2024-04-20T21:46:27.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.924+0000 7f29a7400640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f299006c920 0x7f299006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:27.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.924+0000 7f29a7400640 1 -- 172.21.15.102:0/3351390973 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f29a00adfe0 con 0x7f29b006eb70 2024-04-20T21:46:27.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.924+0000 7f29aea00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f299006c920 0x7f299006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:27.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.925+0000 7f29aea00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f299006c920 0x7f299006ede0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f29b00cae20 tx=0x7f299c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:27.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:27.929+0000 7f29a7400640 1 -- 172.21.15.102:0/3351390973 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f29a0049060 con 0x7f29b006eb70 2024-04-20T21:46:28.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.170+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29b006dcb0 con 0x7f29b006eb70 2024-04-20T21:46:28.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.171+0000 7f29a7400640 1 -- 172.21.15.102:0/3351390973 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+4379 (secure 0 0 0) 0x7f29a007b320 con 0x7f29b006eb70 2024-04-20T21:46:28.170 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:28.170 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:23.253791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:28.170 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 105 2024-04-20T21:46:28.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f299006c920 msgr2=0x7f299006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:28.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f299006c920 0x7f299006ede0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f29b00cae20 tx=0x7f299c009190 comp rx=0 tx=0).stop 2024-04-20T21:46:28.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29b006eb70 msgr2=0x7f29b0144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:28.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29b006eb70 0x7f29b0144e50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f29a00098c0 tx=0x7f29a00046c0 comp rx=0 tx=0).stop 2024-04-20T21:46:28.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 shutdown_connections 2024-04-20T21:46:28.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29b0152290 0x7f29b0151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:28.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f299006c920 0x7f299006ede0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:28.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29b00c3700 0x7f29b0150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:28.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 --2- 172.21.15.102:0/3351390973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29b006eb70 0x7f29b0144e50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:28.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 >> 172.21.15.102:0/3351390973 conn(0x7f29b00cc820 msgr2=0x7f29b00c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:28.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 shutdown_connections 2024-04-20T21:46:28.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:28.174+0000 7f29b4c00640 1 -- 172.21.15.102:0/3351390973 wait complete. 2024-04-20T21:46:28.192 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:29.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~268s 2024-04-20T21:46:29.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~180s 2024-04-20T21:46:30.193 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:30.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.403+0000 7f135fc00640 1 Processor -- start 2024-04-20T21:46:30.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.403+0000 7f135fc00640 1 -- start start 2024-04-20T21:46:30.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135fc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13580cdb40 0x7f13580728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:30.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135fc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1358154fd0 0x7f1358155440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:30.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135fc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13580c8570 con 0x7f13580cdf20 2024-04-20T21:46:30.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135fc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f1358073e60 con 0x7f1358154fd0 2024-04-20T21:46:30.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135fc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f1358075b40 con 0x7f13580cdb40 2024-04-20T21:46:30.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135d200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13580cdb40 0x7f13580728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:30.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135d200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13580cdb40 0x7f13580728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:46708/0 (socket says 172.21.15.102:46708) 2024-04-20T21:46:30.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135d200640 1 -- 172.21.15.102:0/1321585959 learned_addr learned my addr 172.21.15.102:0/1321585959 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:30.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.405+0000 7f135dc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1358154fd0 0x7f1358155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:30.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.406+0000 7f135d200640 1 -- 172.21.15.102:0/1321585959 >> v2:172.21.15.193:3300/0 conn(0x7f1358154fd0 msgr2=0x7f1358155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:30.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.406+0000 7f135d200640 1 --2- 172.21.15.102:0/1321585959 >> v2:172.21.15.193:3300/0 conn(0x7f1358154fd0 0x7f1358155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.406+0000 7f135d200640 1 -- 172.21.15.102:0/1321585959 >> v1:172.21.15.102:6789/0 conn(0x7f13580cdf20 legacy=0x7f1358075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:30.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.406+0000 7f135d200640 1 -- 172.21.15.102:0/1321585959 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135815b3b0 con 0x7f13580cdb40 2024-04-20T21:46:30.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.407+0000 7f135d200640 1 --2- 172.21.15.102:0/1321585959 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13580cdb40 0x7f13580728a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1348009870 tx=0x7f134802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=101b634a2bb9e390 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:30.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.407+0000 7f1357e00640 1 -- 172.21.15.102:0/1321585959 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134802f830 con 0x7f13580cdb40 2024-04-20T21:46:30.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.407+0000 7f1357e00640 1 -- 172.21.15.102:0/1321585959 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134802f9d0 con 0x7f13580cdb40 2024-04-20T21:46:30.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.407+0000 7f1357e00640 1 -- 172.21.15.102:0/1321585959 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134802fda0 con 0x7f13580cdb40 2024-04-20T21:46:30.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.408+0000 7f135fc00640 1 -- 172.21.15.102:0/1321585959 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13580cdb40 msgr2=0x7f13580728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:30.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.408+0000 7f135fc00640 1 --2- 172.21.15.102:0/1321585959 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13580cdb40 0x7f13580728a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1348009870 tx=0x7f134802ba10 comp rx=0 tx=0).stop 2024-04-20T21:46:30.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.408+0000 7f135fc00640 1 -- 172.21.15.102:0/1321585959 shutdown_connections 2024-04-20T21:46:30.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.408+0000 7f135fc00640 1 --2- 172.21.15.102:0/1321585959 >> v2:172.21.15.193:3300/0 conn(0x7f1358154fd0 0x7f1358155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.408+0000 7f135fc00640 1 --2- 172.21.15.102:0/1321585959 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13580cdb40 0x7f13580728a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.408+0000 7f135fc00640 1 -- 172.21.15.102:0/1321585959 >> 172.21.15.102:0/1321585959 conn(0x7f135806b940 msgr2=0x7f135806bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:30.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.408+0000 7f135fc00640 1 -- 172.21.15.102:0/1321585959 shutdown_connections 2024-04-20T21:46:30.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.408+0000 7f135fc00640 1 -- 172.21.15.102:0/1321585959 wait complete. 2024-04-20T21:46:30.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.410+0000 7f135fc00640 1 Processor -- start 2024-04-20T21:46:30.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.410+0000 7f135fc00640 1 -- start start 2024-04-20T21:46:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.411+0000 7f135fc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13580cdb40 0x7f135815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.411+0000 7f135fc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13580cdf20 0x7f135815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.411+0000 7f135fc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1358154fd0 0x7f13581e8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.411+0000 7f135fc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f135815bbf0 con 0x7f13580cdf20 2024-04-20T21:46:30.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.411+0000 7f135fc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f135815b910 con 0x7f13580cdb40 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.411+0000 7f135fc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f135815bed0 con 0x7f1358154fd0 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13580cdb40 0x7f135815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13580cdb40 0x7f135815ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:57000/0 (socket says 172.21.15.102:57000) 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135d200640 1 -- 172.21.15.102:0/2200198230 learned_addr learned my addr 172.21.15.102:0/2200198230 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f134fe00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13580cdf20 0x7f135815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135dc00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1358154fd0 0x7f13581e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135d200640 1 -- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1358154fd0 msgr2=0x7f13581e8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135d200640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1358154fd0 0x7f13581e8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135d200640 1 -- 172.21.15.102:0/2200198230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13580cdf20 msgr2=0x7f135815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135d200640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13580cdf20 0x7f135815c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.412+0000 7f135d200640 1 -- 172.21.15.102:0/2200198230 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135815d8d0 con 0x7f13580cdb40 2024-04-20T21:46:30.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.413+0000 7f135d200640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13580cdb40 0x7f135815ee80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f134802ba50 tx=0x7f1348004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.413+0000 7f1355600640 1 -- 172.21.15.102:0/2200198230 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348047070 con 0x7f13580cdb40 2024-04-20T21:46:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.413+0000 7f1355600640 1 -- 172.21.15.102:0/2200198230 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1348037c40 con 0x7f13580cdb40 2024-04-20T21:46:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.413+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13580c5aa0 con 0x7f13580cdb40 2024-04-20T21:46:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.413+0000 7f1355600640 1 -- 172.21.15.102:0/2200198230 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348042480 con 0x7f13580cdb40 2024-04-20T21:46:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.413+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135815db70 con 0x7f13580cdb40 2024-04-20T21:46:30.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.414+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1358068c30 con 0x7f13580cdb40 2024-04-20T21:46:30.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.415+0000 7f1355600640 1 -- 172.21.15.102:0/2200198230 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f134804a070 con 0x7f13580cdb40 2024-04-20T21:46:30.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.416+0000 7f1355600640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133406c8f0 0x7f133406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:30.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.417+0000 7f1355600640 1 -- 172.21.15.102:0/2200198230 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f13480ae7c0 con 0x7f13580cdb40 2024-04-20T21:46:30.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.417+0000 7f134fe00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133406c8f0 0x7f133406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:30.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.417+0000 7f134fe00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133406c8f0 0x7f133406edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1358152cc0 tx=0x7f1340009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:30.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.421+0000 7f1355600640 1 -- 172.21.15.102:0/2200198230 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1348033030 con 0x7f13580cdb40 2024-04-20T21:46:30.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.663+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13581540a0 con 0x7f13580cdb40 2024-04-20T21:46:30.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.664+0000 7f1355600640 1 -- 172.21.15.102:0/2200198230 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+4376 (secure 0 0 0) 0x7f134807bb70 con 0x7f13580cdb40 2024-04-20T21:46:30.663 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:30.663 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:28.706261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:30.663 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 106 2024-04-20T21:46:30.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.666+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133406c8f0 msgr2=0x7f133406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.666+0000 7f135fc00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133406c8f0 0x7f133406edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1358152cc0 tx=0x7f1340009040 comp rx=0 tx=0).stop 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.666+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13580cdb40 msgr2=0x7f135815ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.666+0000 7f135fc00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13580cdb40 0x7f135815ee80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f134802ba50 tx=0x7f1348004730 comp rx=0 tx=0).stop 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.666+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 shutdown_connections 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.666+0000 7f135fc00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1358154fd0 0x7f13581e8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.667+0000 7f135fc00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13580cdf20 0x7f135815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.667+0000 7f135fc00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f133406c8f0 0x7f133406edb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.667+0000 7f135fc00640 1 --2- 172.21.15.102:0/2200198230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13580cdb40 0x7f135815ee80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.667+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 >> 172.21.15.102:0/2200198230 conn(0x7f135806b940 msgr2=0x7f13580ca660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.667+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 shutdown_connections 2024-04-20T21:46:30.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:30.667+0000 7f135fc00640 1 -- 172.21.15.102:0/2200198230 wait complete. 2024-04-20T21:46:30.684 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:32.333 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:46:32.341 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:46:32.685 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.893+0000 7f66aa200640 1 Processor -- start 2024-04-20T21:46:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.893+0000 7f66aa200640 1 -- start start 2024-04-20T21:46:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.894+0000 7f66aa200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f66a4151ab0 0x7f66a40cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66aa200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f66a4157ff0 0x7f66a40c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66aa200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a4158420 con 0x7f66a41516d0 2024-04-20T21:46:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66aa200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f66a40c4dd0 con 0x7f66a4157ff0 2024-04-20T21:46:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66aa200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f66a406ef60 con 0x7f66a4151ab0 2024-04-20T21:46:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66a9800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f66a4157ff0 0x7f66a40c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f669be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f66a4151ab0 0x7f66a40cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66a9800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f66a4157ff0 0x7f66a40c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44442/0 (socket says 172.21.15.102:44442) 2024-04-20T21:46:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f669be00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f66a4151ab0 0x7f66a40cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:57002/0 (socket says 172.21.15.102:57002) 2024-04-20T21:46:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66a9800640 1 -- 172.21.15.102:0/345584973 learned_addr learned my addr 172.21.15.102:0/345584973 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66a9800640 1 -- 172.21.15.102:0/345584973 >> v2:172.21.15.193:3300/0 conn(0x7f66a4151ab0 msgr2=0x7f66a40cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66a9800640 1 --2- 172.21.15.102:0/345584973 >> v2:172.21.15.193:3300/0 conn(0x7f66a4151ab0 0x7f66a40cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66a9800640 1 -- 172.21.15.102:0/345584973 >> v1:172.21.15.193:6789/0 conn(0x7f66a41516d0 legacy=0x7f66a40c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.895+0000 7f66a9800640 1 -- 172.21.15.102:0/345584973 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a406f240 con 0x7f66a4157ff0 2024-04-20T21:46:32.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.896+0000 7f66a9800640 1 --2- 172.21.15.102:0/345584973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a40c5820 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f66a40c9250 tx=0x7f669c02e220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0878103be53dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:32.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66a3e00640 1 -- 172.21.15.102:0/345584973 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c00a0d0 con 0x7f66a4157ff0 2024-04-20T21:46:32.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66a3e00640 1 -- 172.21.15.102:0/345584973 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f669c00a270 con 0x7f66a4157ff0 2024-04-20T21:46:32.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66a3e00640 1 -- 172.21.15.102:0/345584973 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c008720 con 0x7f66a4157ff0 2024-04-20T21:46:32.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66aa200640 1 -- 172.21.15.102:0/345584973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 msgr2=0x7f66a40c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:32.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66aa200640 1 --2- 172.21.15.102:0/345584973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a40c5820 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f66a40c9250 tx=0x7f669c02e220 comp rx=0 tx=0).stop 2024-04-20T21:46:32.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66aa200640 1 -- 172.21.15.102:0/345584973 shutdown_connections 2024-04-20T21:46:32.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66aa200640 1 --2- 172.21.15.102:0/345584973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a40c5820 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:32.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66aa200640 1 --2- 172.21.15.102:0/345584973 >> v2:172.21.15.193:3300/0 conn(0x7f66a4151ab0 0x7f66a40cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:32.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66aa200640 1 -- 172.21.15.102:0/345584973 >> 172.21.15.102:0/345584973 conn(0x7f66a40cc820 msgr2=0x7f66a40ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:32.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.897+0000 7f66aa200640 1 -- 172.21.15.102:0/345584973 shutdown_connections 2024-04-20T21:46:32.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.898+0000 7f66aa200640 1 -- 172.21.15.102:0/345584973 wait complete. 2024-04-20T21:46:32.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.899+0000 7f66aa200640 1 Processor -- start 2024-04-20T21:46:32.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.899+0000 7f66aa200640 1 -- start start 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.900+0000 7f66aa200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a41516d0 0x7f66a4155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.900+0000 7f66aa200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f66a4151ab0 0x7f66a414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.900+0000 7f66aa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.900+0000 7f66aa200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f66a40c5300 con 0x7f66a4157ff0 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.900+0000 7f66aa200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66a40c5020 con 0x7f66a41516d0 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.900+0000 7f66aa200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f66a40c4a40 con 0x7f66a4151ab0 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a414c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44450/0 (socket says 172.21.15.102:44450) 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 -- 172.21.15.102:0/1543201745 learned_addr learned my addr 172.21.15.102:0/1543201745 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f669be00640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f66a4151ab0 0x7f66a414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:32.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a8e00640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a41516d0 0x7f66a4155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:32.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 -- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f66a4151ab0 msgr2=0x7f66a414c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:32.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f66a4151ab0 0x7f66a414c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:32.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 -- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a41516d0 msgr2=0x7f66a4155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:32.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a41516d0 0x7f66a4155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:32.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 -- 172.21.15.102:0/1543201745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a414cb40 con 0x7f66a4157ff0 2024-04-20T21:46:32.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.901+0000 7f66a9800640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a414c600 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f669c038040 tx=0x7f669c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:32.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.902+0000 7f66a1600640 1 -- 172.21.15.102:0/1543201745 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c042020 con 0x7f66a4157ff0 2024-04-20T21:46:32.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.902+0000 7f66a1600640 1 -- 172.21.15.102:0/1543201745 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f669c03e610 con 0x7f66a4157ff0 2024-04-20T21:46:32.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.902+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a4157850 con 0x7f66a4157ff0 2024-04-20T21:46:32.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.902+0000 7f66a1600640 1 -- 172.21.15.102:0/1543201745 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c00a620 con 0x7f66a4157ff0 2024-04-20T21:46:32.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.902+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a4152360 con 0x7f66a4157ff0 2024-04-20T21:46:32.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.903+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a414eed0 con 0x7f66a4157ff0 2024-04-20T21:46:32.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.904+0000 7f66a1600640 1 -- 172.21.15.102:0/1543201745 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f669c005d10 con 0x7f66a4157ff0 2024-04-20T21:46:32.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.905+0000 7f66a1600640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f668006c8f0 0x7f668006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:32.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.905+0000 7f66a1600640 1 -- 172.21.15.102:0/1543201745 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f669c0ae510 con 0x7f66a4157ff0 2024-04-20T21:46:32.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.905+0000 7f66a8e00640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f668006c8f0 0x7f668006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:32.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.906+0000 7f66a8e00640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f668006c8f0 0x7f668006edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6694000f60 tx=0x7f6694009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:32.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:32.909+0000 7f66a1600640 1 -- 172.21.15.102:0/1543201745 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f66a414eed0 con 0x7f66a4157ff0 2024-04-20T21:46:33.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.148+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66a404fd60 con 0x7f66a4157ff0 2024-04-20T21:46:33.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.149+0000 7f66a1600640 1 -- 172.21.15.102:0/1543201745 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+4376 (secure 0 0 0) 0x7f669c07b7a0 con 0x7f66a4157ff0 2024-04-20T21:46:33.148 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:33.148 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:28.706261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:33.148 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 106 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.151+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f668006c8f0 msgr2=0x7f668006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.151+0000 7f66aa200640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f668006c8f0 0x7f668006edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6694000f60 tx=0x7f6694009040 comp rx=0 tx=0).stop 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.151+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 msgr2=0x7f66a414c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.151+0000 7f66aa200640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a414c600 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f669c038040 tx=0x7f669c004a40 comp rx=0 tx=0).stop 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.152+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 shutdown_connections 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.152+0000 7f66aa200640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f66a4157ff0 0x7f66a414c600 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.152+0000 7f66aa200640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f66a4151ab0 0x7f66a414c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.152+0000 7f66aa200640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f668006c8f0 0x7f668006edb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.152+0000 7f66aa200640 1 --2- 172.21.15.102:0/1543201745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a41516d0 0x7f66a4155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:33.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.152+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 >> 172.21.15.102:0/1543201745 conn(0x7f66a40cc820 msgr2=0x7f66a41e3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:33.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.152+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 shutdown_connections 2024-04-20T21:46:33.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:33.152+0000 7f66aa200640 1 -- 172.21.15.102:0/1543201745 wait complete. 2024-04-20T21:46:33.168 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:35.169 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:35.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.374+0000 7fe36a400640 1 Processor -- start 2024-04-20T21:46:35.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.374+0000 7fe36a400640 1 -- start start 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.375+0000 7fe36a400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe36414fa50 0x7fe3640c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.375+0000 7fe36a400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe364073a10 0x7fe364158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.375+0000 7fe36a400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3641536d0 con 0x7fe3640c9eb0 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.375+0000 7fe36a400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe364159500 con 0x7fe36414fa50 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.375+0000 7fe36a400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fe3640c76c0 con 0x7fe364073a10 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.376+0000 7fe363400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe36414fa50 0x7fe3640c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.376+0000 7fe363400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe36414fa50 0x7fe3640c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44452/0 (socket says 172.21.15.102:44452) 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.376+0000 7fe363400640 1 -- 172.21.15.102:0/891254178 learned_addr learned my addr 172.21.15.102:0/891254178 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:35.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.376+0000 7fe363e00640 1 --2- 172.21.15.102:0/891254178 >> v2:172.21.15.193:3300/0 conn(0x7fe364073a10 0x7fe364158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:35.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.376+0000 7fe363e00640 1 -- 172.21.15.102:0/891254178 >> v2:172.21.15.102:3300/0 conn(0x7fe36414fa50 msgr2=0x7fe3640c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:35.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.376+0000 7fe363e00640 1 --2- 172.21.15.102:0/891254178 >> v2:172.21.15.102:3300/0 conn(0x7fe36414fa50 0x7fe3640c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.376+0000 7fe363e00640 1 -- 172.21.15.102:0/891254178 >> v1:172.21.15.193:6789/0 conn(0x7fe3640c9eb0 legacy=0x7fe3640c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:35.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.376+0000 7fe363e00640 1 -- 172.21.15.102:0/891254178 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3640c7960 con 0x7fe364073a10 2024-04-20T21:46:35.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.377+0000 7fe363e00640 1 --2- 172.21.15.102:0/891254178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe364073a10 0x7fe364158fc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe364156b00 tx=0x7fe35c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd39e6a593c71166 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:35.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.377+0000 7fe362000640 1 -- 172.21.15.102:0/891254178 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c034070 con 0x7fe364073a10 2024-04-20T21:46:35.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.377+0000 7fe362000640 1 -- 172.21.15.102:0/891254178 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35c030610 con 0x7fe364073a10 2024-04-20T21:46:35.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.377+0000 7fe362000640 1 -- 172.21.15.102:0/891254178 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0309e0 con 0x7fe364073a10 2024-04-20T21:46:35.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.378+0000 7fe36a400640 1 -- 172.21.15.102:0/891254178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe364073a10 msgr2=0x7fe364158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:35.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.378+0000 7fe36a400640 1 --2- 172.21.15.102:0/891254178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe364073a10 0x7fe364158fc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe364156b00 tx=0x7fe35c02b960 comp rx=0 tx=0).stop 2024-04-20T21:46:35.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.378+0000 7fe36a400640 1 -- 172.21.15.102:0/891254178 shutdown_connections 2024-04-20T21:46:35.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.378+0000 7fe36a400640 1 --2- 172.21.15.102:0/891254178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe364073a10 0x7fe364158fc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.378+0000 7fe36a400640 1 --2- 172.21.15.102:0/891254178 >> v2:172.21.15.102:3300/0 conn(0x7fe36414fa50 0x7fe3640c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.378+0000 7fe36a400640 1 -- 172.21.15.102:0/891254178 >> 172.21.15.102:0/891254178 conn(0x7fe3640c9390 msgr2=0x7fe3640c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:35.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.378+0000 7fe36a400640 1 -- 172.21.15.102:0/891254178 shutdown_connections 2024-04-20T21:46:35.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.378+0000 7fe36a400640 1 -- 172.21.15.102:0/891254178 wait complete. 2024-04-20T21:46:35.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.380+0000 7fe36a400640 1 Processor -- start 2024-04-20T21:46:35.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.380+0000 7fe36a400640 1 -- start start 2024-04-20T21:46:35.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.381+0000 7fe36a400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe364073a10 0x7fe3640cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:35.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.381+0000 7fe36a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3640c9eb0 0x7fe3641f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:35.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.381+0000 7fe36a400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe36414fa50 0x7fe3641f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:35.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.381+0000 7fe36a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640c7c00 con 0x7fe3640c9eb0 2024-04-20T21:46:35.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.381+0000 7fe36a400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640c3d90 con 0x7fe36414fa50 2024-04-20T21:46:35.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.381+0000 7fe36a400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe3640c4030 con 0x7fe364073a10 2024-04-20T21:46:35.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.381+0000 7fe362a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3640c9eb0 0x7fe3641f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:35.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe362a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3640c9eb0 0x7fe3641f4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44462/0 (socket says 172.21.15.102:44462) 2024-04-20T21:46:35.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe363e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe36414fa50 0x7fe3641f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:35.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe362a00640 1 -- 172.21.15.102:0/1223537777 learned_addr learned my addr 172.21.15.102:0/1223537777 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:35.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe363400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe364073a10 0x7fe3640cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:35.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe362a00640 1 -- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe364073a10 msgr2=0x7fe3640cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:35.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe362a00640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe364073a10 0x7fe3640cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe362a00640 1 -- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe36414fa50 msgr2=0x7fe3641f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:35.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe362a00640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe36414fa50 0x7fe3641f4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe362a00640 1 -- 172.21.15.102:0/1223537777 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe364072400 con 0x7fe3640c9eb0 2024-04-20T21:46:35.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe363e00640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe36414fa50 0x7fe3641f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:35.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.382+0000 7fe362a00640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3640c9eb0 0x7fe3641f4710 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe35000c430 tx=0x7fe35000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:35.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.383+0000 7fe35b400640 1 -- 172.21.15.102:0/1223537777 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3500052d0 con 0x7fe3640c9eb0 2024-04-20T21:46:35.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.383+0000 7fe35b400640 1 -- 172.21.15.102:0/1223537777 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe350005490 con 0x7fe3640c9eb0 2024-04-20T21:46:35.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.383+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3640726c0 con 0x7fe3640c9eb0 2024-04-20T21:46:35.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.383+0000 7fe35b400640 1 -- 172.21.15.102:0/1223537777 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350007260 con 0x7fe3640c9eb0 2024-04-20T21:46:35.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.383+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe364157d00 con 0x7fe3640c9eb0 2024-04-20T21:46:35.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.384+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36414d5a0 con 0x7fe3640c9eb0 2024-04-20T21:46:35.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.385+0000 7fe35b400640 1 -- 172.21.15.102:0/1223537777 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe350007400 con 0x7fe3640c9eb0 2024-04-20T21:46:35.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.386+0000 7fe35b400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe34406c8f0 0x7fe34406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:35.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.386+0000 7fe35b400640 1 -- 172.21.15.102:0/1223537777 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7fe35008ba30 con 0x7fe3640c9eb0 2024-04-20T21:46:35.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.386+0000 7fe363400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe34406c8f0 0x7fe34406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:35.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.387+0000 7fe363400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe34406c8f0 0x7fe34406edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe354011340 tx=0x7fe35400b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:35.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.391+0000 7fe35b400640 1 -- 172.21.15.102:0/1223537777 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe36414d5a0 con 0x7fe3640c9eb0 2024-04-20T21:46:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.627+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe364074340 con 0x7fe3640c9eb0 2024-04-20T21:46:35.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.628+0000 7fe35b400640 1 -- 172.21.15.102:0/1223537777 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+4376 (secure 0 0 0) 0x7fe3641536d0 con 0x7fe3640c9eb0 2024-04-20T21:46:35.627 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:35.627 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:28.706261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:35.628 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 106 2024-04-20T21:46:35.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe34406c8f0 msgr2=0x7fe34406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:35.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe34406c8f0 0x7fe34406edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe354011340 tx=0x7fe35400b040 comp rx=0 tx=0).stop 2024-04-20T21:46:35.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3640c9eb0 msgr2=0x7fe3641f4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:35.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3640c9eb0 0x7fe3641f4710 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe35000c430 tx=0x7fe35000c8f0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 shutdown_connections 2024-04-20T21:46:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe34406c8f0 0x7fe34406edb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe36414fa50 0x7fe3641f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3640c9eb0 0x7fe3641f4710 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 --2- 172.21.15.102:0/1223537777 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe364073a10 0x7fe3640cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 >> 172.21.15.102:0/1223537777 conn(0x7fe3640c9390 msgr2=0x7fe36414d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 shutdown_connections 2024-04-20T21:46:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:35.631+0000 7fe36a400640 1 -- 172.21.15.102:0/1223537777 wait complete. 2024-04-20T21:46:35.649 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:36.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~274s 2024-04-20T21:46:36.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~187s 2024-04-20T21:46:37.649 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:37.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.858+0000 7faaae200640 1 Processor -- start 2024-04-20T21:46:37.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.858+0000 7faaae200640 1 -- start start 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.859+0000 7faaae200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faaa80c8db0 0x7faaa8070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.859+0000 7faaae200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faaa8071330 0x7faaa81628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaae200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaa8158e80 con 0x7faaa8152170 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaae200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7faaa80c6dc0 con 0x7faaa80c8db0 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaae200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7faaa8071f70 con 0x7faaa8071330 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaa7e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faaa8071330 0x7faaa81628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faa9ea00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faaa80c8db0 0x7faaa8070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaa7400640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7faaa8152170 0x7faaa8158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:37560/0 (socket says 172.21.15.102:37560) 2024-04-20T21:46:37.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaa7400640 1 -- 172.21.15.102:0/3556487903 learned_addr learned my addr 172.21.15.102:0/3556487903 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:37.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaa7e00640 1 -- 172.21.15.102:0/3556487903 >> v2:172.21.15.102:3300/0 conn(0x7faaa80c8db0 msgr2=0x7faaa8070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:37.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaa7e00640 1 --2- 172.21.15.102:0/3556487903 >> v2:172.21.15.102:3300/0 conn(0x7faaa80c8db0 0x7faaa8070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:37.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaa7e00640 1 -- 172.21.15.102:0/3556487903 >> v1:172.21.15.193:6789/0 conn(0x7faaa8152170 legacy=0x7faaa8158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:37.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaa7e00640 1 -- 172.21.15.102:0/3556487903 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa80c91c0 con 0x7faaa8071330 2024-04-20T21:46:37.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.860+0000 7faaa6a00640 1 -- 172.21.15.102:0/3556487903 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049449474 0 0) 0x7faaa8158e80 con 0x7faaa8152170 2024-04-20T21:46:37.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.861+0000 7faaa7e00640 1 --2- 172.21.15.102:0/3556487903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 0x7faaa81628a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faaa806c190 tx=0x7faaa002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9209017052595d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.862+0000 7faaa6a00640 1 -- 172.21.15.102:0/3556487903 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0039070 con 0x7faaa8071330 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.862+0000 7faaa6a00640 1 -- 172.21.15.102:0/3556487903 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa002cc80 con 0x7faaa8071330 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.862+0000 7faaa6a00640 1 -- 172.21.15.102:0/3556487903 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0034660 con 0x7faaa8071330 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.862+0000 7faaae200640 1 -- 172.21.15.102:0/3556487903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 msgr2=0x7faaa81628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.862+0000 7faaae200640 1 --2- 172.21.15.102:0/3556487903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 0x7faaa81628a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faaa806c190 tx=0x7faaa002bed0 comp rx=0 tx=0).stop 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.862+0000 7faaae200640 1 -- 172.21.15.102:0/3556487903 shutdown_connections 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.862+0000 7faaae200640 1 --2- 172.21.15.102:0/3556487903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 0x7faaa81628a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.862+0000 7faaae200640 1 --2- 172.21.15.102:0/3556487903 >> v2:172.21.15.102:3300/0 conn(0x7faaa80c8db0 0x7faaa8070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.863+0000 7faaae200640 1 -- 172.21.15.102:0/3556487903 >> 172.21.15.102:0/3556487903 conn(0x7faaa80c9390 msgr2=0x7faaa80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.863+0000 7faaae200640 1 -- 172.21.15.102:0/3556487903 shutdown_connections 2024-04-20T21:46:37.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.863+0000 7faaae200640 1 -- 172.21.15.102:0/3556487903 wait complete. 2024-04-20T21:46:37.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.864+0000 7faaae200640 1 Processor -- start 2024-04-20T21:46:37.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.864+0000 7faaae200640 1 -- start start 2024-04-20T21:46:37.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.865+0000 7faaae200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 0x7faaa815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.865+0000 7faaae200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaa80c8db0 0x7faaa8145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.865+0000 7faaae200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaa8152170 0x7faaa8146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.865+0000 7faaae200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faaa80724b0 con 0x7faaa8152170 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.865+0000 7faaae200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8072210 con 0x7faaa8071330 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.865+0000 7faaae200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7faaa8158250 con 0x7faaa80c8db0 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 0x7faaa815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaa8152170 0x7faaa8146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faa9ea00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaa80c8db0 0x7faaa8145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 0x7faaa815a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:38244/0 (socket says 172.21.15.102:38244) 2024-04-20T21:46:37.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7400640 1 -- 172.21.15.102:0/1062946954 learned_addr learned my addr 172.21.15.102:0/1062946954 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7e00640 1 -- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaa80c8db0 msgr2=0x7faaa8145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7e00640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaa80c8db0 0x7faaa8145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7e00640 1 -- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 msgr2=0x7faaa815a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7e00640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 0x7faaa815a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7e00640 1 -- 172.21.15.102:0/1062946954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa81f4da0 con 0x7faaa8152170 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.866+0000 7faaa7e00640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaa8152170 0x7faaa8146150 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faaa0033000 tx=0x7faaa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.867+0000 7faa9fe00640 1 -- 172.21.15.102:0/1062946954 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0044020 con 0x7faaa8152170 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.867+0000 7faa9fe00640 1 -- 172.21.15.102:0/1062946954 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa003a040 con 0x7faaa8152170 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.867+0000 7faa9fe00640 1 -- 172.21.15.102:0/1062946954 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0039050 con 0x7faaa8152170 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.867+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa8159b30 con 0x7faaa8152170 2024-04-20T21:46:37.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.867+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa8156c90 con 0x7faaa8152170 2024-04-20T21:46:37.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.868+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa8074340 con 0x7faaa8152170 2024-04-20T21:46:37.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.869+0000 7faa9fe00640 1 -- 172.21.15.102:0/1062946954 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaa0003b90 con 0x7faaa8152170 2024-04-20T21:46:37.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.870+0000 7faa9fe00640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa8406c8f0 0x7faa8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:37.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.870+0000 7faa9fe00640 1 -- 172.21.15.102:0/1062946954 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7faaa00aeb50 con 0x7faaa8152170 2024-04-20T21:46:37.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.870+0000 7faaa7400640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa8406c8f0 0x7faa8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:37.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.871+0000 7faaa7400640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa8406c8f0 0x7faa8406edb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faa98007b10 tx=0x7faa98007a60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:37.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:37.875+0000 7faa9fe00640 1 -- 172.21.15.102:0/1062946954 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faaa004e020 con 0x7faaa8152170 2024-04-20T21:46:38.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.112+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaa80c3460 con 0x7faaa8152170 2024-04-20T21:46:38.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.113+0000 7faa9fe00640 1 -- 172.21.15.102:0/1062946954 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4382 (secure 0 0 0) 0x7faaa007bf00 con 0x7faaa8152170 2024-04-20T21:46:38.112 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:38.112 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:35.827725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:38.112 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 107 2024-04-20T21:46:38.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa8406c8f0 msgr2=0x7faa8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:38.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa8406c8f0 0x7faa8406edb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faa98007b10 tx=0x7faa98007a60 comp rx=0 tx=0).stop 2024-04-20T21:46:38.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaa8152170 msgr2=0x7faaa8146150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:38.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaa8152170 0x7faaa8146150 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faaa0033000 tx=0x7faaa0004a40 comp rx=0 tx=0).stop 2024-04-20T21:46:38.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 shutdown_connections 2024-04-20T21:46:38.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaa8152170 0x7faaa8146150 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:38.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faaa80c8db0 0x7faaa8145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:38.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7faa8406c8f0 0x7faa8406edb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:38.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 --2- 172.21.15.102:0/1062946954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8071330 0x7faaa815a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:38.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 >> 172.21.15.102:0/1062946954 conn(0x7faaa80c9390 msgr2=0x7faaa81585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:38.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 shutdown_connections 2024-04-20T21:46:38.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:38.115+0000 7faaae200640 1 -- 172.21.15.102:0/1062946954 wait complete. 2024-04-20T21:46:38.133 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 107, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 107, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7026": { "addr": "172.21.15.102:6833/1356782221", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 1356782221, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 1356782221, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7026, "incarnation": 104, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:clientreplay", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:46:35.827725+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7026 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:46:40.133 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:40.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.342+0000 7f01b4a00640 1 Processor -- start 2024-04-20T21:46:40.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.342+0000 7f01b4a00640 1 -- start start 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.343+0000 7f01b4a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01b01516d0 0x7f01b00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.343+0000 7f01b4a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 0x7f01b00c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.343+0000 7f01b4a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b0158460 con 0x7f01b0151ab0 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.343+0000 7f01b4a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f01b00c4dd0 con 0x7f01b01516d0 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.343+0000 7f01b4a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f01b006ef60 con 0x7f01b0157ff0 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.344+0000 7f01af400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01b01516d0 0x7f01b00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.344+0000 7f01af400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01b01516d0 0x7f01b00cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56792/0 (socket says 172.21.15.102:56792) 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.344+0000 7f01af400640 1 -- 172.21.15.102:0/586174686 learned_addr learned my addr 172.21.15.102:0/586174686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.344+0000 7f01afe00640 1 --2- 172.21.15.102:0/586174686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 0x7f01b00c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:40.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.344+0000 7f01af400640 1 -- 172.21.15.102:0/586174686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 msgr2=0x7f01b00c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:40.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.344+0000 7f01af400640 1 --2- 172.21.15.102:0/586174686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 0x7f01b00c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.344+0000 7f01af400640 1 -- 172.21.15.102:0/586174686 >> v1:172.21.15.193:6789/0 conn(0x7f01b0151ab0 legacy=0x7f01b00c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:40.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.344+0000 7f01af400640 1 -- 172.21.15.102:0/586174686 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b006f240 con 0x7f01b01516d0 2024-04-20T21:46:40.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.345+0000 7f01af400640 1 --2- 172.21.15.102:0/586174686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b01516d0 0x7f01b00cd310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f01a000bb00 tx=0x7f01a002dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d988301f922415d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:40.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.345+0000 7f01aea00640 1 -- 172.21.15.102:0/586174686 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0034070 con 0x7f01b01516d0 2024-04-20T21:46:40.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.345+0000 7f01aea00640 1 -- 172.21.15.102:0/586174686 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a002ed80 con 0x7f01b01516d0 2024-04-20T21:46:40.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.345+0000 7f01aea00640 1 -- 172.21.15.102:0/586174686 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0038620 con 0x7f01b01516d0 2024-04-20T21:46:40.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.346+0000 7f01b4a00640 1 -- 172.21.15.102:0/586174686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b01516d0 msgr2=0x7f01b00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:40.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.346+0000 7f01b4a00640 1 --2- 172.21.15.102:0/586174686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b01516d0 0x7f01b00cd310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f01a000bb00 tx=0x7f01a002dfd0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.346+0000 7f01b4a00640 1 -- 172.21.15.102:0/586174686 shutdown_connections 2024-04-20T21:46:40.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.346+0000 7f01b4a00640 1 --2- 172.21.15.102:0/586174686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 0x7f01b00c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.346+0000 7f01b4a00640 1 --2- 172.21.15.102:0/586174686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b01516d0 0x7f01b00cd310 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.346+0000 7f01b4a00640 1 -- 172.21.15.102:0/586174686 >> 172.21.15.102:0/586174686 conn(0x7f01b00cc820 msgr2=0x7f01b00ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:40.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.346+0000 7f01b4a00640 1 -- 172.21.15.102:0/586174686 shutdown_connections 2024-04-20T21:46:40.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.346+0000 7f01b4a00640 1 -- 172.21.15.102:0/586174686 wait complete. 2024-04-20T21:46:40.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.348+0000 7f01b4a00640 1 Processor -- start 2024-04-20T21:46:40.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.348+0000 7f01b4a00640 1 -- start start 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01b4a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01b01516d0 0x7f01b014b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b0151ab0 0x7f01b014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01b4a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 0x7f01b014d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01b4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f01b00c5300 con 0x7f01b0151ab0 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01b4a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f01b00c5020 con 0x7f01b01516d0 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01b4a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f01b00c4a40 con 0x7f01b0157ff0 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01af400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01b01516d0 0x7f01b014b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01afe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 0x7f01b014d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01af400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01b01516d0 0x7f01b014b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:38252/0 (socket says 172.21.15.102:38252) 2024-04-20T21:46:40.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01af400640 1 -- 172.21.15.102:0/3649811292 learned_addr learned my addr 172.21.15.102:0/3649811292 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:40.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.349+0000 7f01a6a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b0151ab0 0x7f01b014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:40.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.350+0000 7f01af400640 1 -- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 msgr2=0x7f01b014d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:40.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.350+0000 7f01af400640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 0x7f01b014d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.350+0000 7f01af400640 1 -- 172.21.15.102:0/3649811292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b0151ab0 msgr2=0x7f01b014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:40.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.350+0000 7f01af400640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b0151ab0 0x7f01b014d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.350+0000 7f01af400640 1 -- 172.21.15.102:0/3649811292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b01530f0 con 0x7f01b01516d0 2024-04-20T21:46:40.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.350+0000 7f01af400640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01b01516d0 0x7f01b014b090 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f01a000bc20 tx=0x7f01a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:40.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.350+0000 7f01a7e00640 1 -- 172.21.15.102:0/3649811292 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a002eb80 con 0x7f01b01516d0 2024-04-20T21:46:40.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.351+0000 7f01a7e00640 1 -- 172.21.15.102:0/3649811292 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a00415a0 con 0x7f01b01516d0 2024-04-20T21:46:40.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.351+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b01e24b0 con 0x7f01b01516d0 2024-04-20T21:46:40.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.351+0000 7f01a7e00640 1 -- 172.21.15.102:0/3649811292 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0041990 con 0x7f01b01516d0 2024-04-20T21:46:40.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.351+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b01e2a70 con 0x7f01b01516d0 2024-04-20T21:46:40.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.352+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b004fd60 con 0x7f01b01516d0 2024-04-20T21:46:40.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.353+0000 7f01a7e00640 1 -- 172.21.15.102:0/3649811292 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01a0049020 con 0x7f01b01516d0 2024-04-20T21:46:40.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.354+0000 7f01a7e00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f018c06c8f0 0x7f018c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:40.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.354+0000 7f01a7e00640 1 -- 172.21.15.102:0/3649811292 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f01a00ae910 con 0x7f01b01516d0 2024-04-20T21:46:40.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.354+0000 7f01a6a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f018c06c8f0 0x7f018c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:40.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.355+0000 7f01a6a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f018c06c8f0 0x7f018c06edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f01b01e47f0 tx=0x7f0198009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:40.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.357+0000 7f01a7e00640 1 -- 172.21.15.102:0/3649811292 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f01a0050020 con 0x7f01b01516d0 2024-04-20T21:46:40.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.599+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01b006dcb0 con 0x7f01b01516d0 2024-04-20T21:46:40.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.600+0000 7f01a7e00640 1 -- 172.21.15.102:0/3649811292 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4382 (secure 0 0 0) 0x7f01a007bd70 con 0x7f01b01516d0 2024-04-20T21:46:40.599 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:40.599 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:35.827725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:40.600 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 107 2024-04-20T21:46:40.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f018c06c8f0 msgr2=0x7f018c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:40.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f018c06c8f0 0x7f018c06edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f01b01e47f0 tx=0x7f0198009040 comp rx=0 tx=0).stop 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01b01516d0 msgr2=0x7f01b014b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01b01516d0 0x7f01b014b090 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f01a000bc20 tx=0x7f01a00046c0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 shutdown_connections 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01b0157ff0 0x7f01b014d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01b0151ab0 0x7f01b014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f018c06c8f0 0x7f018c06edb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 --2- 172.21.15.102:0/3649811292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01b01516d0 0x7f01b014b090 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 >> 172.21.15.102:0/3649811292 conn(0x7f01b00cc820 msgr2=0x7f01b014eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.603+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 shutdown_connections 2024-04-20T21:46:40.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:40.604+0000 7f01b4a00640 1 -- 172.21.15.102:0/3649811292 wait complete. 2024-04-20T21:46:40.620 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:42.057 INFO:tasks.ceph.mds.c.smithi102.stderr:terminate called after throwing an instance of 'std::out_of_range' 2024-04-20T21:46:42.057 INFO:tasks.ceph.mds.c.smithi102.stderr: what(): map::at 2024-04-20T21:46:42.057 INFO:tasks.ceph.mds.c.smithi102.stderr:*** Caught signal (Aborted) ** 2024-04-20T21:46:42.057 INFO:tasks.ceph.mds.c.smithi102.stderr: in thread 7febe1800640 thread_name:MR_Finisher 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7febf003e6f0] 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7febf008b94c] 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: 3: raise() 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: 4: abort() 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7febf04a1b21] 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7febf04ad52c] 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7febf04ad597] 2024-04-20T21:46:42.058 INFO:tasks.ceph.mds.c.smithi102.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7febf04ad7f9] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7febf04a4533] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x5599afda9d86] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x5599afeb0cf7] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x5599afef198f] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x5599afe63fef] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x706) [0x5599afe45a76] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x5599afe3eac6] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5599afe3f411] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 17: (Locker::scatter_writebehind_finish(ScatterLock*, boost::intrusive_ptr&)+0xbf) [0x5599afe6342f] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 18: (MDSContext::complete(int)+0x5c) [0x5599aff452ac] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x5599aff456fc] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 20: (MDSLogContextBase::complete(int)+0x53) [0x5599aff45853] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 21: (Finisher::finisher_thread_entry()+0x175) [0x7febf0a30455] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 22: /lib64/libc.so.6(+0x89c02) [0x7febf0089c02] 2024-04-20T21:46:42.059 INFO:tasks.ceph.mds.c.smithi102.stderr: 23: /lib64/libc.so.6(+0x10ec40) [0x7febf010ec40] 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr:2024-04-20T21:46:42.060+0000 7febe1800640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: in thread 7febe1800640 thread_name:MR_Finisher 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7febf003e6f0] 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7febf008b94c] 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 3: raise() 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 4: abort() 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7febf04a1b21] 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7febf04ad52c] 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7febf04ad597] 2024-04-20T21:46:42.060 INFO:tasks.ceph.mds.c.smithi102.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7febf04ad7f9] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7febf04a4533] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x5599afda9d86] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x5599afeb0cf7] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x5599afef198f] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x5599afe63fef] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x706) [0x5599afe45a76] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x5599afe3eac6] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5599afe3f411] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 17: (Locker::scatter_writebehind_finish(ScatterLock*, boost::intrusive_ptr&)+0xbf) [0x5599afe6342f] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 18: (MDSContext::complete(int)+0x5c) [0x5599aff452ac] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x5599aff456fc] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 20: (MDSLogContextBase::complete(int)+0x53) [0x5599aff45853] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 21: (Finisher::finisher_thread_entry()+0x175) [0x7febf0a30455] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 22: /lib64/libc.so.6(+0x89c02) [0x7febf0089c02] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: 23: /lib64/libc.so.6(+0x10ec40) [0x7febf010ec40] 2024-04-20T21:46:42.061 INFO:tasks.ceph.mds.c.smithi102.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:46:42.062 INFO:tasks.ceph.mds.c.smithi102.stderr: 2024-04-20T21:46:42.082 INFO:tasks.ceph.mds.c.smithi102.stderr: 0> 2024-04-20T21:46:42.060+0000 7febe1800640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:46:42.082 INFO:tasks.ceph.mds.c.smithi102.stderr: in thread 7febe1800640 thread_name:MR_Finisher 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7febf003e6f0] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7febf008b94c] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 3: raise() 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 4: abort() 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7febf04a1b21] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7febf04ad52c] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7febf04ad597] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7febf04ad7f9] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7febf04a4533] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x5599afda9d86] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x5599afeb0cf7] 2024-04-20T21:46:42.083 INFO:tasks.ceph.mds.c.smithi102.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x5599afef198f] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x5599afe63fef] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x706) [0x5599afe45a76] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x5599afe3eac6] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5599afe3f411] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 17: (Locker::scatter_writebehind_finish(ScatterLock*, boost::intrusive_ptr&)+0xbf) [0x5599afe6342f] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 18: (MDSContext::complete(int)+0x5c) [0x5599aff452ac] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x5599aff456fc] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 20: (MDSLogContextBase::complete(int)+0x53) [0x5599aff45853] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 21: (Finisher::finisher_thread_entry()+0x175) [0x7febf0a30455] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 22: /lib64/libc.so.6(+0x89c02) [0x7febf0089c02] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 23: /lib64/libc.so.6(+0x10ec40) [0x7febf010ec40] 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:46:42.084 INFO:tasks.ceph.mds.c.smithi102.stderr: 2024-04-20T21:46:42.093 INFO:tasks.ceph.mds.c.smithi102.stderr: -5> 2024-04-20T21:46:42.060+0000 7febe1800640 -1 *** Caught signal (Aborted) ** 2024-04-20T21:46:42.093 INFO:tasks.ceph.mds.c.smithi102.stderr: in thread 7febe1800640 thread_name:MR_Finisher 2024-04-20T21:46:42.093 INFO:tasks.ceph.mds.c.smithi102.stderr: 2024-04-20T21:46:42.093 INFO:tasks.ceph.mds.c.smithi102.stderr: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:46:42.093 INFO:tasks.ceph.mds.c.smithi102.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7febf003e6f0] 2024-04-20T21:46:42.093 INFO:tasks.ceph.mds.c.smithi102.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7febf008b94c] 2024-04-20T21:46:42.093 INFO:tasks.ceph.mds.c.smithi102.stderr: 3: raise() 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 4: abort() 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7febf04a1b21] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7febf04ad52c] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 7: /lib64/libstdc++.so.6(+0xad597) [0x7febf04ad597] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7febf04ad7f9] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 9: (std::__throw_out_of_range(char const*)+0x41) [0x7febf04a4533] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 10: (MDCache::project_rstat_inode_to_frag(boost::intrusive_ptr const&, CInode*, CDir*, snapid_t, int, SnapRealm*)+0x9e6) [0x5599afda9d86] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 11: (CDir::assimilate_dirty_rstat_inodes(boost::intrusive_ptr&)+0x147) [0x5599afeb0cf7] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 12: (CInode::finish_scatter_gather_update(int, boost::intrusive_ptr&)+0x164f) [0x5599afef198f] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 13: (Locker::scatter_writebehind(ScatterLock*)+0x17f) [0x5599afe63fef] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 14: (Locker::eval_gather(SimpleLock*, bool, bool*, std::vector >*)+0x706) [0x5599afe45a76] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 15: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x236) [0x5599afe3eac6] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 16: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5599afe3f411] 2024-04-20T21:46:42.094 INFO:tasks.ceph.mds.c.smithi102.stderr: 17: (Locker::scatter_writebehind_finish(ScatterLock*, boost::intrusive_ptr&)+0xbf) [0x5599afe6342f] 2024-04-20T21:46:42.095 INFO:tasks.ceph.mds.c.smithi102.stderr: 18: (MDSContext::complete(int)+0x5c) [0x5599aff452ac] 2024-04-20T21:46:42.095 INFO:tasks.ceph.mds.c.smithi102.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x5599aff456fc] 2024-04-20T21:46:42.095 INFO:tasks.ceph.mds.c.smithi102.stderr: 20: (MDSLogContextBase::complete(int)+0x53) [0x5599aff45853] 2024-04-20T21:46:42.095 INFO:tasks.ceph.mds.c.smithi102.stderr: 21: (Finisher::finisher_thread_entry()+0x175) [0x7febf0a30455] 2024-04-20T21:46:42.095 INFO:tasks.ceph.mds.c.smithi102.stderr: 22: /lib64/libc.so.6(+0x89c02) [0x7febf0089c02] 2024-04-20T21:46:42.095 INFO:tasks.ceph.mds.c.smithi102.stderr: 23: /lib64/libc.so.6(+0x10ec40) [0x7febf010ec40] 2024-04-20T21:46:42.095 INFO:tasks.ceph.mds.c.smithi102.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-20T21:46:42.095 INFO:tasks.ceph.mds.c.smithi102.stderr: 2024-04-20T21:46:42.621 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:42.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.826+0000 7efc89c00640 1 Processor -- start 2024-04-20T21:46:42.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.826+0000 7efc89c00640 1 -- start start 2024-04-20T21:46:42.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc89c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc840c7290 0x7efc840c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:42.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc89c00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efc84071a70 0x7efc8406d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:42.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc89c00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc84071040 con 0x7efc84151c90 2024-04-20T21:46:42.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc89c00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7efc840c9200 con 0x7efc84071a70 2024-04-20T21:46:42.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc89c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efc84070df0 con 0x7efc840c7290 2024-04-20T21:46:42.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc83400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efc84071a70 0x7efc8406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:42.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc83400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efc84071a70 0x7efc8406d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56804/0 (socket says 172.21.15.102:56804) 2024-04-20T21:46:42.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc83400640 1 -- 172.21.15.102:0/3445726605 learned_addr learned my addr 172.21.15.102:0/3445726605 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:42.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc82a00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7efc84151c90 0x7efc84158240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:37590/0 (socket says 172.21.15.102:37590) 2024-04-20T21:46:42.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.828+0000 7efc82000640 1 --2- 172.21.15.102:0/3445726605 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc840c7290 0x7efc840c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:42.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.829+0000 7efc81600640 1 -- 172.21.15.102:0/3445726605 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254221233 0 0) 0x7efc84071040 con 0x7efc84151c90 2024-04-20T21:46:42.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.829+0000 7efc81600640 1 -- 172.21.15.102:0/3445726605 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc68003520 con 0x7efc84151c90 2024-04-20T21:46:42.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.829+0000 7efc83400640 1 -- 172.21.15.102:0/3445726605 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc840c7290 msgr2=0x7efc840c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:42.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.829+0000 7efc83400640 1 --2- 172.21.15.102:0/3445726605 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc840c7290 0x7efc840c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:42.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.829+0000 7efc83400640 1 -- 172.21.15.102:0/3445726605 >> v1:172.21.15.193:6789/0 conn(0x7efc84151c90 legacy=0x7efc84158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:42.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.829+0000 7efc83400640 1 -- 172.21.15.102:0/3445726605 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc84071320 con 0x7efc84071a70 2024-04-20T21:46:42.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.829+0000 7efc82000640 1 --2- 172.21.15.102:0/3445726605 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc840c7290 0x7efc840c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:42.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.830+0000 7efc83400640 1 --2- 172.21.15.102:0/3445726605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc84071a70 0x7efc8406d270 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00d150 tx=0x7efc7c02e1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de13ac6f42f05b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:42.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.830+0000 7efc81600640 1 -- 172.21.15.102:0/3445726605 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00b0d0 con 0x7efc84071a70 2024-04-20T21:46:42.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.830+0000 7efc81600640 1 -- 172.21.15.102:0/3445726605 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c00b270 con 0x7efc84071a70 2024-04-20T21:46:42.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.830+0000 7efc81600640 1 -- 172.21.15.102:0/3445726605 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c009600 con 0x7efc84071a70 2024-04-20T21:46:42.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.831+0000 7efc89c00640 1 -- 172.21.15.102:0/3445726605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc84071a70 msgr2=0x7efc8406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:42.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.831+0000 7efc89c00640 1 --2- 172.21.15.102:0/3445726605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc84071a70 0x7efc8406d270 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00d150 tx=0x7efc7c02e1f0 comp rx=0 tx=0).stop 2024-04-20T21:46:42.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.831+0000 7efc89c00640 1 -- 172.21.15.102:0/3445726605 shutdown_connections 2024-04-20T21:46:42.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.831+0000 7efc89c00640 1 --2- 172.21.15.102:0/3445726605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc84071a70 0x7efc8406d270 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:42.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.831+0000 7efc89c00640 1 --2- 172.21.15.102:0/3445726605 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc840c7290 0x7efc840c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:42.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.831+0000 7efc89c00640 1 -- 172.21.15.102:0/3445726605 >> 172.21.15.102:0/3445726605 conn(0x7efc840c9390 msgr2=0x7efc840c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:42.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.831+0000 7efc89c00640 1 -- 172.21.15.102:0/3445726605 shutdown_connections 2024-04-20T21:46:42.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.831+0000 7efc89c00640 1 -- 172.21.15.102:0/3445726605 wait complete. 2024-04-20T21:46:42.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.833+0000 7efc89c00640 1 Processor -- start 2024-04-20T21:46:42.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.833+0000 7efc89c00640 1 -- start start 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc89c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc84071a70 0x7efc8406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc89c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc840c7290 0x7efc840cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc89c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc84151c90 0x7efc841f1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc89c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc84157a40 con 0x7efc840c7290 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc89c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc84157db0 con 0x7efc84071a70 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc89c00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efc840ca1f0 con 0x7efc84151c90 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc840c7290 0x7efc840cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc84071a70 0x7efc8406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc840c7290 0x7efc840cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56816/0 (socket says 172.21.15.102:56816) 2024-04-20T21:46:42.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82000640 1 -- 172.21.15.102:0/2486632223 learned_addr learned my addr 172.21.15.102:0/2486632223 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc83400640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc84151c90 0x7efc841f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82a00640 1 -- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc84151c90 msgr2=0x7efc841f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82a00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc84151c90 0x7efc841f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82a00640 1 -- 172.21.15.102:0/2486632223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc840c7290 msgr2=0x7efc840cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82a00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc840c7290 0x7efc840cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82a00640 1 -- 172.21.15.102:0/2486632223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc841f4570 con 0x7efc84071a70 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc83400640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc84151c90 0x7efc841f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.834+0000 7efc82000640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc840c7290 0x7efc840cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.835+0000 7efc82a00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc84071a70 0x7efc8406eff0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c3c0 tx=0x7efc74001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.835+0000 7efc7aa00640 1 -- 172.21.15.102:0/2486632223 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc740074e0 con 0x7efc84071a70 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.835+0000 7efc7aa00640 1 -- 172.21.15.102:0/2486632223 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc740076a0 con 0x7efc84071a70 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.835+0000 7efc7aa00640 1 -- 172.21.15.102:0/2486632223 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74004a20 con 0x7efc84071a70 2024-04-20T21:46:42.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.835+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc841f4810 con 0x7efc84071a70 2024-04-20T21:46:42.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.835+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc841f4dd0 con 0x7efc84071a70 2024-04-20T21:46:42.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.836+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc84074340 con 0x7efc84071a70 2024-04-20T21:46:42.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.837+0000 7efc7aa00640 1 -- 172.21.15.102:0/2486632223 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc7401a020 con 0x7efc84071a70 2024-04-20T21:46:42.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.838+0000 7efc7aa00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efc6406c8f0 0x7efc6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:42.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.838+0000 7efc7aa00640 1 -- 172.21.15.102:0/2486632223 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7efc7408bc80 con 0x7efc84071a70 2024-04-20T21:46:42.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.838+0000 7efc82000640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efc6406c8f0 0x7efc6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:42.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.839+0000 7efc82000640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efc6406c8f0 0x7efc6406edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efc70009560 tx=0x7efc70009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:42.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:42.842+0000 7efc7aa00640 1 -- 172.21.15.102:0/2486632223 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efc74058fa0 con 0x7efc84071a70 2024-04-20T21:46:42.890 INFO:tasks.ceph.mds.c.smithi102.stderr:daemon-helper: command crashed with signal 6 2024-04-20T21:46:42.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~281s 2024-04-20T21:46:42.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~193s 2024-04-20T21:46:43.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.092+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc841e20a0 con 0x7efc84071a70 2024-04-20T21:46:43.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.093+0000 7efc7aa00640 1 -- 172.21.15.102:0/2486632223 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4382 (secure 0 0 0) 0x7efc7405d2a0 con 0x7efc84071a70 2024-04-20T21:46:43.093 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:43.093 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:35.827725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:43.093 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 107 2024-04-20T21:46:43.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efc6406c8f0 msgr2=0x7efc6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:43.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efc6406c8f0 0x7efc6406edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efc70009560 tx=0x7efc70009190 comp rx=0 tx=0).stop 2024-04-20T21:46:43.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc84071a70 msgr2=0x7efc8406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:43.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc84071a70 0x7efc8406eff0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c3c0 tx=0x7efc74001e70 comp rx=0 tx=0).stop 2024-04-20T21:46:43.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 shutdown_connections 2024-04-20T21:46:43.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc84151c90 0x7efc841f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:43.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc840c7290 0x7efc840cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:43.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efc6406c8f0 0x7efc6406edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:43.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 --2- 172.21.15.102:0/2486632223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc84071a70 0x7efc8406eff0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:43.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 >> 172.21.15.102:0/2486632223 conn(0x7efc840c9390 msgr2=0x7efc84071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:43.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 shutdown_connections 2024-04-20T21:46:43.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:43.096+0000 7efc89c00640 1 -- 172.21.15.102:0/2486632223 wait complete. 2024-04-20T21:46:43.115 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:45.116 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:45.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.321+0000 7f0297800640 1 Processor -- start 2024-04-20T21:46:45.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.321+0000 7f0297800640 1 -- start start 2024-04-20T21:46:45.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.322+0000 7f0297800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0290151c90 0x7f02900c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:45.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.322+0000 7f0297800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0290071040 con 0x7f0290071a70 2024-04-20T21:46:45.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.322+0000 7f0297800640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0290071320 con 0x7f02900c7290 2024-04-20T21:46:45.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.322+0000 7f0297800640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f02900c9200 con 0x7f0290151c90 2024-04-20T21:46:45.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f0295800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0290071a70 0x7f029006d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:44154/0 (socket says 172.21.15.102:44154) 2024-04-20T21:46:45.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f0295800640 1 -- 172.21.15.102:0/3095940423 learned_addr learned my addr 172.21.15.102:0/3095940423 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:45.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f0294e00640 1 --2- 172.21.15.102:0/3095940423 >> v2:172.21.15.193:3300/0 conn(0x7f0290151c90 0x7f02900c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:45.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f028f400640 1 -- 172.21.15.102:0/3095940423 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334073857 0 0) 0x7f0290071040 con 0x7f0290071a70 2024-04-20T21:46:45.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f028f400640 1 -- 172.21.15.102:0/3095940423 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0278003520 con 0x7f0290071a70 2024-04-20T21:46:45.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f028f400640 1 -- 172.21.15.102:0/3095940423 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 263534939 0 0) 0x7f0290071320 con 0x7f02900c7290 2024-04-20T21:46:45.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f028f400640 1 -- 172.21.15.102:0/3095940423 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0290071040 con 0x7f02900c7290 2024-04-20T21:46:45.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f0294e00640 1 -- 172.21.15.102:0/3095940423 >> v1:172.21.15.193:6789/0 conn(0x7f02900c7290 legacy=0x7f0290158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:45.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f0294e00640 1 -- 172.21.15.102:0/3095940423 >> v1:172.21.15.102:6789/0 conn(0x7f0290071a70 legacy=0x7f029006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:45.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.323+0000 7f0294e00640 1 -- 172.21.15.102:0/3095940423 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0290071600 con 0x7f0290151c90 2024-04-20T21:46:45.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.324+0000 7f0294e00640 1 --2- 172.21.15.102:0/3095940423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290151c90 0x7f02900c8cc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f02840097a0 tx=0x7f028402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0a244729807b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:45.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f028f400640 1 -- 172.21.15.102:0/3095940423 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028402f810 con 0x7f0290151c90 2024-04-20T21:46:45.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f028f400640 1 -- 172.21.15.102:0/3095940423 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028402f9b0 con 0x7f0290151c90 2024-04-20T21:46:45.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f028f400640 1 -- 172.21.15.102:0/3095940423 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028402fd80 con 0x7f0290151c90 2024-04-20T21:46:45.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f0297800640 1 -- 172.21.15.102:0/3095940423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290151c90 msgr2=0x7f02900c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:45.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f0297800640 1 --2- 172.21.15.102:0/3095940423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290151c90 0x7f02900c8cc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f02840097a0 tx=0x7f028402b9f0 comp rx=0 tx=0).stop 2024-04-20T21:46:45.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f0297800640 1 -- 172.21.15.102:0/3095940423 shutdown_connections 2024-04-20T21:46:45.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f0297800640 1 --2- 172.21.15.102:0/3095940423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290151c90 0x7f02900c8cc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:45.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f0297800640 1 -- 172.21.15.102:0/3095940423 >> 172.21.15.102:0/3095940423 conn(0x7f02900c9390 msgr2=0x7f02900c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:45.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.325+0000 7f0297800640 1 -- 172.21.15.102:0/3095940423 shutdown_connections 2024-04-20T21:46:45.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.326+0000 7f0297800640 1 -- 172.21.15.102:0/3095940423 wait complete. 2024-04-20T21:46:45.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.327+0000 7f0297800640 1 Processor -- start 2024-04-20T21:46:45.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.327+0000 7f0297800640 1 -- start start 2024-04-20T21:46:45.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0297800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290071a70 0x7f0290147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:45.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0297800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02900c7290 0x7f029015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:45.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0297800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0290151c90 0x7f0290143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0297800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0290157db0 con 0x7f0290151c90 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0297800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0290070df0 con 0x7f0290071a70 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0297800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f02900ca1f0 con 0x7f02900c7290 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0295800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0290151c90 0x7f0290143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0295800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0290151c90 0x7f0290143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56824/0 (socket says 172.21.15.102:56824) 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0295800640 1 -- 172.21.15.102:0/1086377438 learned_addr learned my addr 172.21.15.102:0/1086377438 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.328+0000 7f0294e00640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290071a70 0x7f0290147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f028fe00640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02900c7290 0x7f029015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f0295800640 1 -- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02900c7290 msgr2=0x7f029015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f0295800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02900c7290 0x7f029015b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f0295800640 1 -- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290071a70 msgr2=0x7f0290147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f0295800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290071a70 0x7f0290147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f0295800640 1 -- 172.21.15.102:0/1086377438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0290151480 con 0x7f0290151c90 2024-04-20T21:46:45.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f0295800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0290151c90 0x7f0290143180 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f028800caf0 tx=0x7f028800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:45.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f028cc00640 1 -- 172.21.15.102:0/1086377438 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288017070 con 0x7f0290151c90 2024-04-20T21:46:45.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f028cc00640 1 -- 172.21.15.102:0/1086377438 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028800ac40 con 0x7f0290151c90 2024-04-20T21:46:45.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.329+0000 7f028cc00640 1 -- 172.21.15.102:0/1086377438 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288012400 con 0x7f0290151c90 2024-04-20T21:46:45.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.330+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02901595d0 con 0x7f0290151c90 2024-04-20T21:46:45.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.330+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0290159b70 con 0x7f0290151c90 2024-04-20T21:46:45.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.331+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0290074340 con 0x7f0290151c90 2024-04-20T21:46:45.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.332+0000 7f028cc00640 1 -- 172.21.15.102:0/1086377438 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f028801a070 con 0x7f0290151c90 2024-04-20T21:46:45.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.332+0000 7f028cc00640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f027006c8f0 0x7f027006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:45.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.333+0000 7f028cc00640 1 -- 172.21.15.102:0/1086377438 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f028808c260 con 0x7f0290151c90 2024-04-20T21:46:45.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.333+0000 7f0294e00640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f027006c8f0 0x7f027006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:45.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.333+0000 7f0294e00640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f027006c8f0 0x7f027006edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0284036000 tx=0x7f0284008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:45.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.338+0000 7f028cc00640 1 -- 172.21.15.102:0/1086377438 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0288059610 con 0x7f0290151c90 2024-04-20T21:46:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.577+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02900c8aa0 con 0x7f0290151c90 2024-04-20T21:46:45.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.578+0000 7f028cc00640 1 -- 172.21.15.102:0/1086377438 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4382 (secure 0 0 0) 0x7f028805d910 con 0x7f0290151c90 2024-04-20T21:46:45.577 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:45.577 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:35.827725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:45.577 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 107 2024-04-20T21:46:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f027006c8f0 msgr2=0x7f027006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f027006c8f0 0x7f027006edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0284036000 tx=0x7f0284008030 comp rx=0 tx=0).stop 2024-04-20T21:46:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0290151c90 msgr2=0x7f0290143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0290151c90 0x7f0290143180 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f028800caf0 tx=0x7f028800a060 comp rx=0 tx=0).stop 2024-04-20T21:46:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 shutdown_connections 2024-04-20T21:46:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0290151c90 0x7f0290143180 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02900c7290 0x7f029015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f027006c8f0 0x7f027006edb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 --2- 172.21.15.102:0/1086377438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0290071a70 0x7f0290147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 >> 172.21.15.102:0/1086377438 conn(0x7f02900c9390 msgr2=0x7f02901544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 shutdown_connections 2024-04-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:45.581+0000 7f0297800640 1 -- 172.21.15.102:0/1086377438 wait complete. 2024-04-20T21:46:45.600 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:47.601 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:47.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.809+0000 7fa577200640 1 Processor -- start 2024-04-20T21:46:47.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.809+0000 7fa577200640 1 -- start start 2024-04-20T21:46:47.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa577200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa5701618a0 0x7fa570163c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa577200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa570158e80 con 0x7fa5700c8db0 2024-04-20T21:46:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa577200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5700c91c0 con 0x7fa570152170 2024-04-20T21:46:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa577200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa5700c6dc0 con 0x7fa5701618a0 2024-04-20T21:46:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa56fe00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fa570152170 0x7fa570158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:40698/0 (socket says 172.21.15.102:40698) 2024-04-20T21:46:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa56fe00640 1 -- 172.21.15.102:0/3879619476 learned_addr learned my addr 172.21.15.102:0/3879619476 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:47.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa575200640 1 --2- 172.21.15.102:0/3879619476 >> v2:172.21.15.102:3300/0 conn(0x7fa5701618a0 0x7fa570163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa56f400640 1 -- 172.21.15.102:0/3879619476 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2778120651 0 0) 0x7fa5700c91c0 con 0x7fa570152170 2024-04-20T21:46:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa56f400640 1 -- 172.21.15.102:0/3879619476 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa554003520 con 0x7fa570152170 2024-04-20T21:46:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa56f400640 1 -- 172.21.15.102:0/3879619476 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373071724 0 0) 0x7fa570158e80 con 0x7fa5700c8db0 2024-04-20T21:46:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.811+0000 7fa56f400640 1 -- 172.21.15.102:0/3879619476 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5700c91c0 con 0x7fa5700c8db0 2024-04-20T21:46:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.812+0000 7fa575200640 1 -- 172.21.15.102:0/3879619476 >> v1:172.21.15.193:6789/0 conn(0x7fa570152170 legacy=0x7fa570158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.812+0000 7fa575200640 1 -- 172.21.15.102:0/3879619476 >> v1:172.21.15.102:6789/0 conn(0x7fa5700c8db0 legacy=0x7fa570070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:47.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.812+0000 7fa575200640 1 -- 172.21.15.102:0/3879619476 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa570071f70 con 0x7fa5701618a0 2024-04-20T21:46:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.813+0000 7fa575200640 1 --2- 172.21.15.102:0/3879619476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5701618a0 0x7fa570163c90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa568009d90 tx=0x7fa56802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab9068367d2902b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.813+0000 7fa56f400640 1 -- 172.21.15.102:0/3879619476 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568039070 con 0x7fa5701618a0 2024-04-20T21:46:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.813+0000 7fa56f400640 1 -- 172.21.15.102:0/3879619476 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56802ccf0 con 0x7fa5701618a0 2024-04-20T21:46:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.813+0000 7fa56f400640 1 -- 172.21.15.102:0/3879619476 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568034660 con 0x7fa5701618a0 2024-04-20T21:46:47.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.814+0000 7fa577200640 1 -- 172.21.15.102:0/3879619476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5701618a0 msgr2=0x7fa570163c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:47.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.814+0000 7fa577200640 1 --2- 172.21.15.102:0/3879619476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5701618a0 0x7fa570163c90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa568009d90 tx=0x7fa56802bf40 comp rx=0 tx=0).stop 2024-04-20T21:46:47.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.814+0000 7fa577200640 1 -- 172.21.15.102:0/3879619476 shutdown_connections 2024-04-20T21:46:47.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.814+0000 7fa577200640 1 --2- 172.21.15.102:0/3879619476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5701618a0 0x7fa570163c90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:47.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.814+0000 7fa577200640 1 -- 172.21.15.102:0/3879619476 >> 172.21.15.102:0/3879619476 conn(0x7fa5700c9390 msgr2=0x7fa5700c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:47.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.814+0000 7fa577200640 1 -- 172.21.15.102:0/3879619476 shutdown_connections 2024-04-20T21:46:47.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.814+0000 7fa577200640 1 -- 172.21.15.102:0/3879619476 wait complete. 2024-04-20T21:46:47.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.815+0000 7fa577200640 1 Processor -- start 2024-04-20T21:46:47.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.816+0000 7fa577200640 1 -- start start 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.816+0000 7fa577200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5700c8db0 0x7fa5701f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.816+0000 7fa577200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa570152170 0x7fa5701590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa577200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5701618a0 0x7fa570151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa577200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5700724b0 con 0x7fa570152170 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa577200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa570072210 con 0x7fa5701618a0 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa577200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa570158250 con 0x7fa5700c8db0 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa567400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa570152170 0x7fa5701590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa567400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa570152170 0x7fa5701590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59738/0 (socket says 172.21.15.102:59738) 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa567400640 1 -- 172.21.15.102:0/1863733905 learned_addr learned my addr 172.21.15.102:0/1863733905 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa575200640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5701618a0 0x7fa570151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:47.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa56fe00640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5700c8db0 0x7fa5701f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa567400640 1 -- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5700c8db0 msgr2=0x7fa5701f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa567400640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5700c8db0 0x7fa5701f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa567400640 1 -- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5701618a0 msgr2=0x7fa570151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa567400640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5701618a0 0x7fa570151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.817+0000 7fa567400640 1 -- 172.21.15.102:0/1863733905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5701616d0 con 0x7fa570152170 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.818+0000 7fa567400640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa570152170 0x7fa5701590e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c430 tx=0x7fa55800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.818+0000 7fa56cc00640 1 -- 172.21.15.102:0/1863733905 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558017070 con 0x7fa570152170 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.818+0000 7fa56cc00640 1 -- 172.21.15.102:0/1863733905 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5580052f0 con 0x7fa570152170 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.818+0000 7fa56cc00640 1 -- 172.21.15.102:0/1863733905 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558003830 con 0x7fa570152170 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.818+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5701f4680 con 0x7fa570152170 2024-04-20T21:46:47.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.818+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5701f4920 con 0x7fa570152170 2024-04-20T21:46:47.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.819+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa570074340 con 0x7fa570152170 2024-04-20T21:46:47.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.820+0000 7fa56cc00640 1 -- 172.21.15.102:0/1863733905 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa55801a070 con 0x7fa570152170 2024-04-20T21:46:47.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.821+0000 7fa56cc00640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa54c06c8f0 0x7fa54c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:47.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.821+0000 7fa56cc00640 1 -- 172.21.15.102:0/1863733905 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7fa55808b9a0 con 0x7fa570152170 2024-04-20T21:46:47.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.822+0000 7fa56fe00640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa54c06c8f0 0x7fa54c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:47.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.823+0000 7fa56fe00640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa54c06c8f0 0x7fa54c06edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa5600011e0 tx=0x7fa560007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:47.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:47.835+0000 7fa56cc00640 1 -- 172.21.15.102:0/1863733905 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa558058c30 con 0x7fa570152170 2024-04-20T21:46:48.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.081+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa570149a30 con 0x7fa570152170 2024-04-20T21:46:48.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.083+0000 7fa56cc00640 1 -- 172.21.15.102:0/1863733905 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4382 (secure 0 0 0) 0x7fa55805cf30 con 0x7fa570152170 2024-04-20T21:46:48.082 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:48.082 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:35.827725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:48.082 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 107 2024-04-20T21:46:48.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.085+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa54c06c8f0 msgr2=0x7fa54c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:48.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.085+0000 7fa577200640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa54c06c8f0 0x7fa54c06edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa5600011e0 tx=0x7fa560007d30 comp rx=0 tx=0).stop 2024-04-20T21:46:48.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.085+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa570152170 msgr2=0x7fa5701590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:48.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.085+0000 7fa577200640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa570152170 0x7fa5701590e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c430 tx=0x7fa55800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:46:48.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.086+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 shutdown_connections 2024-04-20T21:46:48.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.086+0000 7fa577200640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa54c06c8f0 0x7fa54c06edb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:48.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.086+0000 7fa577200640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5701618a0 0x7fa570151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:48.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.086+0000 7fa577200640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa570152170 0x7fa5701590e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:48.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.086+0000 7fa577200640 1 --2- 172.21.15.102:0/1863733905 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5700c8db0 0x7fa5701f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:48.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.086+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 >> 172.21.15.102:0/1863733905 conn(0x7fa5700c9390 msgr2=0x7fa57006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:48.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.086+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 shutdown_connections 2024-04-20T21:46:48.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:48.086+0000 7fa577200640 1 -- 172.21.15.102:0/1863733905 wait complete. 2024-04-20T21:46:48.104 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:49.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~288s 2024-04-20T21:46:49.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-20T21:46:49.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~200s 2024-04-20T21:46:50.105 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:50.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.316+0000 7f5187800640 1 Processor -- start 2024-04-20T21:46:50.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.316+0000 7f5187800640 1 -- start start 2024-04-20T21:46:50.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f5187800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f518015c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:50.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f5187800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f518015bfd0 0x7f51800ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:50.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f5187800640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5180073a20 con 0x7f518006c550 2024-04-20T21:46:50.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f5187800640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f518015b510 con 0x7f518015bfd0 2024-04-20T21:46:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f5187800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f5180073cc0 con 0x7f51801554f0 2024-04-20T21:46:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f517fe00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f518006c550 0x7f51800c98f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37816/0 (socket says 172.21.15.102:37816) 2024-04-20T21:46:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f517fe00640 1 -- 172.21.15.102:0/1610188036 learned_addr learned my addr 172.21.15.102:0/1610188036 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f5185800640 1 --2- 172.21.15.102:0/1610188036 >> v2:172.21.15.102:3300/0 conn(0x7f518015bfd0 0x7f51800ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.318+0000 7f5184e00640 1 --2- 172.21.15.102:0/1610188036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f518015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.319+0000 7f5184e00640 1 -- 172.21.15.102:0/1610188036 >> v2:172.21.15.102:3300/0 conn(0x7f518015bfd0 msgr2=0x7f51800ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.319+0000 7f5184e00640 1 --2- 172.21.15.102:0/1610188036 >> v2:172.21.15.102:3300/0 conn(0x7f518015bfd0 0x7f51800ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.319+0000 7f517f400640 1 -- 172.21.15.102:0/1610188036 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980172031 0 0) 0x7f5180073a20 con 0x7f518006c550 2024-04-20T21:46:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.319+0000 7f5184e00640 1 -- 172.21.15.102:0/1610188036 >> v1:172.21.15.102:6789/0 conn(0x7f518006c550 legacy=0x7f51800c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.319+0000 7f5184e00640 1 -- 172.21.15.102:0/1610188036 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518015b7d0 con 0x7f51801554f0 2024-04-20T21:46:50.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.320+0000 7f5184e00640 1 --2- 172.21.15.102:0/1610188036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f518015c810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5180073bf0 tx=0x7f516802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b1b5fa0d7d964ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:50.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.321+0000 7f517f400640 1 -- 172.21.15.102:0/1610188036 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168035020 con 0x7f51801554f0 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.321+0000 7f517f400640 1 -- 172.21.15.102:0/1610188036 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5168031800 con 0x7f51801554f0 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.321+0000 7f517f400640 1 -- 172.21.15.102:0/1610188036 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168031bd0 con 0x7f51801554f0 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.321+0000 7f5187800640 1 -- 172.21.15.102:0/1610188036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 msgr2=0x7f518015c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.321+0000 7f5187800640 1 --2- 172.21.15.102:0/1610188036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f518015c810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5180073bf0 tx=0x7f516802b8a0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.322+0000 7f5187800640 1 -- 172.21.15.102:0/1610188036 shutdown_connections 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.322+0000 7f5187800640 1 --2- 172.21.15.102:0/1610188036 >> v2:172.21.15.102:3300/0 conn(0x7f518015bfd0 0x7f51800ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.322+0000 7f5187800640 1 --2- 172.21.15.102:0/1610188036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f518015c810 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.322+0000 7f5187800640 1 -- 172.21.15.102:0/1610188036 >> 172.21.15.102:0/1610188036 conn(0x7f518006ebf0 msgr2=0x7f518006efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.322+0000 7f5187800640 1 -- 172.21.15.102:0/1610188036 shutdown_connections 2024-04-20T21:46:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.322+0000 7f5187800640 1 -- 172.21.15.102:0/1610188036 wait complete. 2024-04-20T21:46:50.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.323+0000 7f5187800640 1 Processor -- start 2024-04-20T21:46:50.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.323+0000 7f5187800640 1 -- start start 2024-04-20T21:46:50.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.324+0000 7f5187800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f518006c550 0x7f51800cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:50.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.324+0000 7f5187800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f5180156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:50.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.324+0000 7f5187800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f518015bfd0 0x7f5180156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:50.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.324+0000 7f5187800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51800ca000 con 0x7f518006c550 2024-04-20T21:46:50.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.324+0000 7f5187800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51800736e0 con 0x7f518015bfd0 2024-04-20T21:46:50.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.324+0000 7f5187800640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f51801558d0 con 0x7f51801554f0 2024-04-20T21:46:50.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f518006c550 0x7f51800cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f518006c550 0x7f51800cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59760/0 (socket says 172.21.15.102:59760) 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 -- 172.21.15.102:0/435371878 learned_addr learned my addr 172.21.15.102:0/435371878 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f517fe00640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f5180156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5185800640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f518015bfd0 0x7f5180156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 -- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 msgr2=0x7f5180156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f5180156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 -- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f518015bfd0 msgr2=0x7f5180156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f518015bfd0 0x7f5180156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 -- 172.21.15.102:0/435371878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51800c78b0 con 0x7f518006c550 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f517fe00640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f5180156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5185800640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f518015bfd0 0x7f5180156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:46:50.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.325+0000 7f5184e00640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f518006c550 0x7f51800cca00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5168000c00 tx=0x7f51680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:50.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.326+0000 7f517cc00640 1 -- 172.21.15.102:0/435371878 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168035050 con 0x7f518006c550 2024-04-20T21:46:50.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.326+0000 7f517cc00640 1 -- 172.21.15.102:0/435371878 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516803bcf0 con 0x7f518006c550 2024-04-20T21:46:50.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.326+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51800c7b20 con 0x7f518006c550 2024-04-20T21:46:50.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.326+0000 7f517cc00640 1 -- 172.21.15.102:0/435371878 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168006e10 con 0x7f518006c550 2024-04-20T21:46:50.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.326+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51800c6c50 con 0x7f518006c550 2024-04-20T21:46:50.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.327+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5180074340 con 0x7f518006c550 2024-04-20T21:46:50.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.328+0000 7f517cc00640 1 -- 172.21.15.102:0/435371878 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5168038020 con 0x7f518006c550 2024-04-20T21:46:50.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.329+0000 7f517cc00640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f515c06c8f0 0x7f515c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:50.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.329+0000 7f517cc00640 1 -- 172.21.15.102:0/435371878 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f51680ae550 con 0x7f518006c550 2024-04-20T21:46:50.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.329+0000 7f517fe00640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f515c06c8f0 0x7f515c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.330+0000 7f517fe00640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f515c06c8f0 0x7f515c06edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51800cfc70 tx=0x7f5170007f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:50.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.334+0000 7f517cc00640 1 -- 172.21.15.102:0/435371878 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5168040070 con 0x7f518006c550 2024-04-20T21:46:50.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.571+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51800c7e00 con 0x7f518006c550 2024-04-20T21:46:50.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.572+0000 7f517cc00640 1 -- 172.21.15.102:0/435371878 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4382 (secure 0 0 0) 0x7f516807bec0 con 0x7f518006c550 2024-04-20T21:46:50.572 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:50.572 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:35.827725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:50.572 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 107 2024-04-20T21:46:50.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f515c06c8f0 msgr2=0x7f515c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:50.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f515c06c8f0 0x7f515c06edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51800cfc70 tx=0x7f5170007f90 comp rx=0 tx=0).stop 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f518006c550 msgr2=0x7f51800cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f518006c550 0x7f51800cca00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5168000c00 tx=0x7f51680046c0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 shutdown_connections 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f515c06c8f0 0x7f515c06edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f518015bfd0 0x7f5180156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f51801554f0 0x7f5180156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 --2- 172.21.15.102:0/435371878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f518006c550 0x7f51800cca00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 >> 172.21.15.102:0/435371878 conn(0x7f518006ebf0 msgr2=0x7f51800cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 shutdown_connections 2024-04-20T21:46:50.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:50.575+0000 7f5187800640 1 -- 172.21.15.102:0/435371878 wait complete. 2024-04-20T21:46:50.594 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 107, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 107, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7026": { "addr": "172.21.15.102:6833/1356782221", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6832", "nonce": 1356782221, "type": "v2" }, { "addr": "172.21.15.102:6833", "nonce": 1356782221, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7026, "incarnation": 104, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:clientreplay", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:46:35.827725+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7026 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:46:52.596 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:52.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.807+0000 7ff503e00640 1 Processor -- start 2024-04-20T21:46:52.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.808+0000 7ff503e00640 1 -- start start 2024-04-20T21:46:52.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.809+0000 7ff503e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc1532a0 0x7ff4fc0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:52.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.809+0000 7ff503e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4fc159ff0 con 0x7ff4fc159060 2024-04-20T21:46:52.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.809+0000 7ff503e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4fc0c9150 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.809+0000 7ff503e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc15cae0 con 0x7ff4fc1532a0 2024-04-20T21:46:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.810+0000 7ff501400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc1532a0 0x7ff4fc0c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.810+0000 7ff500a00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff4fc0c7e30 0x7ff4fc15cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:40708/0 (socket says 172.21.15.102:40708) 2024-04-20T21:46:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.810+0000 7ff500a00640 1 -- 172.21.15.102:0/1800565010 learned_addr learned my addr 172.21.15.102:0/1800565010 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.810+0000 7ff501400640 1 -- 172.21.15.102:0/1800565010 >> v1:172.21.15.193:6789/0 conn(0x7ff4fc0c7e30 legacy=0x7ff4fc15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.810+0000 7ff501400640 1 -- 172.21.15.102:0/1800565010 >> v1:172.21.15.102:6789/0 conn(0x7ff4fc159060 legacy=0x7ff4fc06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.810+0000 7ff4fbe00640 1 -- 172.21.15.102:0/1800565010 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426060037 0 0) 0x7ff4fc0c9150 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.810+0000 7ff501400640 1 -- 172.21.15.102:0/1800565010 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc0c9430 con 0x7ff4fc1532a0 2024-04-20T21:46:52.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.811+0000 7ff501400640 1 --2- 172.21.15.102:0/1800565010 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc1532a0 0x7ff4fc0c85f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc15cc60 tx=0x7ff4f002b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cfe0fa26373190d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:52.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.812+0000 7ff4fbe00640 1 -- 172.21.15.102:0/1800565010 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0037070 con 0x7ff4fc1532a0 2024-04-20T21:46:52.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.812+0000 7ff4fbe00640 1 -- 172.21.15.102:0/1800565010 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f0030710 con 0x7ff4fc1532a0 2024-04-20T21:46:52.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.812+0000 7ff4fbe00640 1 -- 172.21.15.102:0/1800565010 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0030ae0 con 0x7ff4fc1532a0 2024-04-20T21:46:52.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.812+0000 7ff503e00640 1 -- 172.21.15.102:0/1800565010 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc1532a0 msgr2=0x7ff4fc0c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:52.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.812+0000 7ff503e00640 1 --2- 172.21.15.102:0/1800565010 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc1532a0 0x7ff4fc0c85f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc15cc60 tx=0x7ff4f002b760 comp rx=0 tx=0).stop 2024-04-20T21:46:52.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.813+0000 7ff503e00640 1 -- 172.21.15.102:0/1800565010 shutdown_connections 2024-04-20T21:46:52.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.813+0000 7ff503e00640 1 --2- 172.21.15.102:0/1800565010 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc1532a0 0x7ff4fc0c85f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.813+0000 7ff503e00640 1 -- 172.21.15.102:0/1800565010 >> 172.21.15.102:0/1800565010 conn(0x7ff4fc06f9f0 msgr2=0x7ff4fc06fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.813+0000 7ff503e00640 1 -- 172.21.15.102:0/1800565010 shutdown_connections 2024-04-20T21:46:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.813+0000 7ff503e00640 1 -- 172.21.15.102:0/1800565010 wait complete. 2024-04-20T21:46:52.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.814+0000 7ff503e00640 1 Processor -- start 2024-04-20T21:46:52.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.814+0000 7ff503e00640 1 -- start start 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.815+0000 7ff503e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc0c7e30 0x7ff4fc1e3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.815+0000 7ff503e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4fc1532a0 0x7ff4fc071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.815+0000 7ff503e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4fc159060 0x7ff4fc151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.815+0000 7ff503e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc153680 con 0x7ff4fc159060 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.815+0000 7ff503e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc15c770 con 0x7ff4fc1532a0 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.815+0000 7ff503e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc153800 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff500a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4fc1532a0 0x7ff4fc071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc0c7e30 0x7ff4fc1e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4fc159060 0x7ff4fc151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc0c7e30 0x7ff4fc1e3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:49424/0 (socket says 172.21.15.102:49424) 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 -- 172.21.15.102:0/2928759060 learned_addr learned my addr 172.21.15.102:0/2928759060 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:52.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 -- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4fc1532a0 msgr2=0x7ff4fc071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:52.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4fc1532a0 0x7ff4fc071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:52.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 -- 172.21.15.102:0/2928759060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4fc159060 msgr2=0x7ff4fc151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:52.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4fc159060 0x7ff4fc151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:52.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 -- 172.21.15.102:0/2928759060 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc072640 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff500a00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4fc1532a0 0x7ff4fc071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:52.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501e00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4fc159060 0x7ff4fc151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:52.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.816+0000 7ff501400640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc0c7e30 0x7ff4fc1e3900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009720 tx=0x7ff4f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:52.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.817+0000 7ff4f9600640 1 -- 172.21.15.102:0/2928759060 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0042070 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.817+0000 7ff4f9600640 1 -- 172.21.15.102:0/2928759060 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f0032a40 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.817+0000 7ff4f9600640 1 -- 172.21.15.102:0/2928759060 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0037040 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.817+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc1e45b0 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.817+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc1e4850 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.818+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4fc068c30 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.819+0000 7ff4f9600640 1 -- 172.21.15.102:0/2928759060 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4f0045070 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.820+0000 7ff4f9600640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff4dc06c920 0x7ff4dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:52.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.820+0000 7ff4f9600640 1 -- 172.21.15.102:0/2928759060 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7ff4f00ae2b0 con 0x7ff4fc0c7e30 2024-04-20T21:46:52.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.820+0000 7ff500a00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff4dc06c920 0x7ff4dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:52.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.821+0000 7ff500a00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff4dc06c920 0x7ff4dc06ede0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc158870 tx=0x7ff4e8007e40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:52.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:52.825+0000 7ff4f9600640 1 -- 172.21.15.102:0/2928759060 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff4f007b6c0 con 0x7ff4fc0c7e30 2024-04-20T21:46:53.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.065+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4fc1e4a90 con 0x7ff4fc0c7e30 2024-04-20T21:46:53.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.066+0000 7ff4f9600640 1 -- 172.21.15.102:0/2928759060 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4382 (secure 0 0 0) 0x7ff4f007f9c0 con 0x7ff4fc0c7e30 2024-04-20T21:46:53.065 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:53.066 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:35.827725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:53.066 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 107 2024-04-20T21:46:53.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff4dc06c920 msgr2=0x7ff4dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:53.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff4dc06c920 0x7ff4dc06ede0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc158870 tx=0x7ff4e8007e40 comp rx=0 tx=0).stop 2024-04-20T21:46:53.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc0c7e30 msgr2=0x7ff4fc1e3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:53.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc0c7e30 0x7ff4fc1e3900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009720 tx=0x7ff4f00046c0 comp rx=0 tx=0).stop 2024-04-20T21:46:53.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 shutdown_connections 2024-04-20T21:46:53.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4fc159060 0x7ff4fc151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:53.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff4dc06c920 0x7ff4dc06ede0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:53.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4fc1532a0 0x7ff4fc071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:53.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 --2- 172.21.15.102:0/2928759060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4fc0c7e30 0x7ff4fc1e3900 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:53.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 >> 172.21.15.102:0/2928759060 conn(0x7ff4fc06f9f0 msgr2=0x7ff4fc152760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:53.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 shutdown_connections 2024-04-20T21:46:53.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:53.069+0000 7ff503e00640 1 -- 172.21.15.102:0/2928759060 wait complete. 2024-04-20T21:46:53.088 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:55.088 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:55.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.300+0000 7f9642400640 1 Processor -- start 2024-04-20T21:46:55.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.300+0000 7f9642400640 1 -- start start 2024-04-20T21:46:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f9642400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c0c7290 0x7f963c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f9642400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f963c071a70 0x7f963c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f9642400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f963c071040 con 0x7f963c151c90 2024-04-20T21:46:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f9642400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f963c0c9200 con 0x7f963c071a70 2024-04-20T21:46:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f9642400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f963c070df0 con 0x7f963c0c7290 2024-04-20T21:46:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f963b400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f963c151c90 0x7f963c158240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37830/0 (socket says 172.21.15.102:37830) 2024-04-20T21:46:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f963b400640 1 -- 172.21.15.102:0/1785329678 learned_addr learned my addr 172.21.15.102:0/1785329678 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f9632a00640 1 --2- 172.21.15.102:0/1785329678 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c0c7290 0x7f963c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:55.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.302+0000 7f963be00640 1 --2- 172.21.15.102:0/1785329678 >> v2:172.21.15.193:3300/0 conn(0x7f963c071a70 0x7f963c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:55.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.303+0000 7f9632a00640 1 -- 172.21.15.102:0/1785329678 >> v2:172.21.15.193:3300/0 conn(0x7f963c071a70 msgr2=0x7f963c06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:46:55.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.303+0000 7f9632a00640 1 --2- 172.21.15.102:0/1785329678 >> v2:172.21.15.193:3300/0 conn(0x7f963c071a70 0x7f963c06d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.303+0000 7f9632a00640 1 -- 172.21.15.102:0/1785329678 >> v1:172.21.15.102:6789/0 conn(0x7f963c151c90 legacy=0x7f963c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:55.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.303+0000 7f963aa00640 1 -- 172.21.15.102:0/1785329678 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3558524095 0 0) 0x7f963c071040 con 0x7f963c151c90 2024-04-20T21:46:55.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.303+0000 7f9632a00640 1 -- 172.21.15.102:0/1785329678 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c071320 con 0x7f963c0c7290 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.304+0000 7f9632a00640 1 --2- 172.21.15.102:0/1785329678 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c0c7290 0x7f963c0c8cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f962400a940 tx=0x7f962402ce10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5318e993c4f9284 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.304+0000 7f963aa00640 1 -- 172.21.15.102:0/1785329678 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962403a070 con 0x7f963c0c7290 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.304+0000 7f963aa00640 1 -- 172.21.15.102:0/1785329678 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962402dbc0 con 0x7f963c0c7290 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.304+0000 7f963aa00640 1 -- 172.21.15.102:0/1785329678 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962400b040 con 0x7f963c0c7290 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.305+0000 7f9642400640 1 -- 172.21.15.102:0/1785329678 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c0c7290 msgr2=0x7f963c0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.305+0000 7f9642400640 1 --2- 172.21.15.102:0/1785329678 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c0c7290 0x7f963c0c8cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f962400a940 tx=0x7f962402ce10 comp rx=0 tx=0).stop 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.305+0000 7f9642400640 1 -- 172.21.15.102:0/1785329678 shutdown_connections 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.305+0000 7f9642400640 1 --2- 172.21.15.102:0/1785329678 >> v2:172.21.15.193:3300/0 conn(0x7f963c071a70 0x7f963c06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.305+0000 7f9642400640 1 --2- 172.21.15.102:0/1785329678 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c0c7290 0x7f963c0c8cc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.305+0000 7f9642400640 1 -- 172.21.15.102:0/1785329678 >> 172.21.15.102:0/1785329678 conn(0x7f963c0c9390 msgr2=0x7f963c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.305+0000 7f9642400640 1 -- 172.21.15.102:0/1785329678 shutdown_connections 2024-04-20T21:46:55.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.305+0000 7f9642400640 1 -- 172.21.15.102:0/1785329678 wait complete. 2024-04-20T21:46:55.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.307+0000 7f9642400640 1 Processor -- start 2024-04-20T21:46:55.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.307+0000 7f9642400640 1 -- start start 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.307+0000 7f9642400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c071a70 0x7f963c1e0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9642400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f963c0c7290 0x7f963c1efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9642400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963c151c90 0x7f963c146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9642400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f963c157a40 con 0x7f963c0c7290 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9642400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f963c157db0 con 0x7f963c151c90 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9642400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f963c0ca1f0 con 0x7f963c071a70 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9632a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f963c0c7290 0x7f963c1efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9632a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f963c0c7290 0x7f963c1efce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59786/0 (socket says 172.21.15.102:59786) 2024-04-20T21:46:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9632a00640 1 -- 172.21.15.102:0/2797589137 learned_addr learned my addr 172.21.15.102:0/2797589137 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:55.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f963be00640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963c151c90 0x7f963c146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:55.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f963b400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c071a70 0x7f963c1e0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9632a00640 1 -- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c071a70 msgr2=0x7f963c1e0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9632a00640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c071a70 0x7f963c1e0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9632a00640 1 -- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963c151c90 msgr2=0x7f963c146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9632a00640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963c151c90 0x7f963c146160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.308+0000 7f9632a00640 1 -- 172.21.15.102:0/2797589137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c1ef550 con 0x7f963c0c7290 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.309+0000 7f963be00640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963c151c90 0x7f963c146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.309+0000 7f9632a00640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f963c0c7290 0x7f963c1efce0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9624038040 tx=0x7f96240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.309+0000 7f9633e00640 1 -- 172.21.15.102:0/2797589137 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624045070 con 0x7f963c0c7290 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.309+0000 7f9633e00640 1 -- 172.21.15.102:0/2797589137 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9624033550 con 0x7f963c0c7290 2024-04-20T21:46:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.309+0000 7f9633e00640 1 -- 172.21.15.102:0/2797589137 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962403a070 con 0x7f963c0c7290 2024-04-20T21:46:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.309+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c1f38b0 con 0x7f963c0c7290 2024-04-20T21:46:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.310+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c14c240 con 0x7f963c0c7290 2024-04-20T21:46:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.311+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c151840 con 0x7f963c0c7290 2024-04-20T21:46:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.312+0000 7f9633e00640 1 -- 172.21.15.102:0/2797589137 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9624048070 con 0x7f963c0c7290 2024-04-20T21:46:55.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.313+0000 7f9633e00640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f961806c8f0 0x7f961806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:55.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.313+0000 7f9633e00640 1 -- 172.21.15.102:0/2797589137 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 8283+0+0 (secure 0 0 0) 0x7f96240adf30 con 0x7f963c0c7290 2024-04-20T21:46:55.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.313+0000 7f963b400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f961806c8f0 0x7f961806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:55.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.314+0000 7f963b400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f961806c8f0 0x7f961806edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f962c007fd0 tx=0x7f962c007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.317+0000 7f9633e00640 1 -- 172.21.15.102:0/2797589137 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f963c151840 con 0x7f963c0c7290 2024-04-20T21:46:55.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.556+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f963c074340 con 0x7f963c0c7290 2024-04-20T21:46:55.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.557+0000 7f9633e00640 1 -- 172.21.15.102:0/2797589137 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4382 (secure 0 0 0) 0x7f962407b250 con 0x7f963c0c7290 2024-04-20T21:46:55.556 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:55.556 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:35.827725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7026":{"gid":7026,"name":"c","rank":0,"incarnation":104,"state":"up:clientreplay","state_seq":14,"addr":"172.21.15.102:6833/1356782221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1356782221},{"type":"v1","addr":"172.21.15.102:6833","nonce":1356782221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:55.556 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 107 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f961806c8f0 msgr2=0x7f961806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f961806c8f0 0x7f961806edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f962c007fd0 tx=0x7f962c007de0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f963c0c7290 msgr2=0x7f963c1efce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f963c0c7290 0x7f963c1efce0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9624038040 tx=0x7f96240046c0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 shutdown_connections 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f961806c8f0 0x7f961806edb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963c151c90 0x7f963c146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f963c0c7290 0x7f963c1efce0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 --2- 172.21.15.102:0/2797589137 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f963c071a70 0x7f963c1e0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 >> 172.21.15.102:0/2797589137 conn(0x7f963c0c9390 msgr2=0x7f963c1493a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 shutdown_connections 2024-04-20T21:46:55.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:55.560+0000 7f9642400640 1 -- 172.21.15.102:0/2797589137 wait complete. 2024-04-20T21:46:55.578 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:46:55.943 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~294s 2024-04-20T21:46:55.943 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-04-20T21:46:55.943 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~206s 2024-04-20T21:46:57.579 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:46:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.790+0000 7ff89ac00640 1 Processor -- start 2024-04-20T21:46:57.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.790+0000 7ff89ac00640 1 -- start start 2024-04-20T21:46:57.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.792+0000 7ff89ac00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff894155530 0x7ff89415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.792+0000 7ff89ac00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff89415ef60 con 0x7ff8940cc470 2024-04-20T21:46:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.792+0000 7ff89ac00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff89415f240 con 0x7ff8940cb6a0 2024-04-20T21:46:57.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.792+0000 7ff89ac00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7ff8940c5ca0 con 0x7ff894155530 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.792+0000 7ff898e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff8940cc470 0x7ff8940cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37838/0 (socket says 172.21.15.102:37838) 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.792+0000 7ff898e00640 1 -- 172.21.15.102:0/2187863506 learned_addr learned my addr 172.21.15.102:0/2187863506 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.792+0000 7ff899800640 1 --2- 172.21.15.102:0/2187863506 >> v2:172.21.15.193:3300/0 conn(0x7ff894155530 0x7ff89415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.793+0000 7ff893e00640 1 -- 172.21.15.102:0/2187863506 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992519115 0 0) 0x7ff89415ef60 con 0x7ff8940cc470 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.793+0000 7ff893e00640 1 -- 172.21.15.102:0/2187863506 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff878003520 con 0x7ff8940cc470 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.793+0000 7ff893e00640 1 -- 172.21.15.102:0/2187863506 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260725255 0 0) 0x7ff89415f240 con 0x7ff8940cb6a0 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.793+0000 7ff893e00640 1 -- 172.21.15.102:0/2187863506 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff89415ef60 con 0x7ff8940cb6a0 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.793+0000 7ff899800640 1 -- 172.21.15.102:0/2187863506 >> v1:172.21.15.193:6789/0 conn(0x7ff8940cb6a0 legacy=0x7ff8941659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.793+0000 7ff899800640 1 -- 172.21.15.102:0/2187863506 >> v1:172.21.15.102:6789/0 conn(0x7ff8940cc470 legacy=0x7ff8940cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:57.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.793+0000 7ff899800640 1 -- 172.21.15.102:0/2187863506 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89415f520 con 0x7ff894155530 2024-04-20T21:46:57.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.794+0000 7ff899800640 1 --2- 172.21.15.102:0/2187863506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 0x7ff89415e640 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0097a0 tx=0x7ff88c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=165cc177edd0ad47 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:57.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.794+0000 7ff893e00640 1 -- 172.21.15.102:0/2187863506 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c034070 con 0x7ff894155530 2024-04-20T21:46:57.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.794+0000 7ff893e00640 1 -- 172.21.15.102:0/2187863506 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88c030610 con 0x7ff894155530 2024-04-20T21:46:57.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.794+0000 7ff893e00640 1 -- 172.21.15.102:0/2187863506 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0309c0 con 0x7ff894155530 2024-04-20T21:46:57.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.795+0000 7ff89ac00640 1 -- 172.21.15.102:0/2187863506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 msgr2=0x7ff89415e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:57.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.795+0000 7ff89ac00640 1 --2- 172.21.15.102:0/2187863506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 0x7ff89415e640 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0097a0 tx=0x7ff88c02b960 comp rx=0 tx=0).stop 2024-04-20T21:46:57.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.795+0000 7ff89ac00640 1 -- 172.21.15.102:0/2187863506 shutdown_connections 2024-04-20T21:46:57.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.795+0000 7ff89ac00640 1 --2- 172.21.15.102:0/2187863506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 0x7ff89415e640 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:57.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.795+0000 7ff89ac00640 1 -- 172.21.15.102:0/2187863506 >> 172.21.15.102:0/2187863506 conn(0x7ff894075080 msgr2=0x7ff894075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:57.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.795+0000 7ff89ac00640 1 -- 172.21.15.102:0/2187863506 shutdown_connections 2024-04-20T21:46:57.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.795+0000 7ff89ac00640 1 -- 172.21.15.102:0/2187863506 wait complete. 2024-04-20T21:46:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.797+0000 7ff89ac00640 1 Processor -- start 2024-04-20T21:46:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.797+0000 7ff89ac00640 1 -- start start 2024-04-20T21:46:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff89ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8940cb6a0 0x7ff89415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff89ac00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8940cc470 0x7ff8940ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff89ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 0x7ff8941464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff89ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940cd280 con 0x7ff8940cb6a0 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff89ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940cd5f0 con 0x7ff894155530 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff89ac00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7ff8940ccbe0 con 0x7ff8940cc470 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff899800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8940cb6a0 0x7ff89415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff899800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8940cb6a0 0x7ff89415e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59790/0 (socket says 172.21.15.102:59790) 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff899800640 1 -- 172.21.15.102:0/221339091 learned_addr learned my addr 172.21.15.102:0/221339091 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff89a200640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 0x7ff8941464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.798+0000 7ff898e00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8940cc470 0x7ff8940ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff899800640 1 -- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8940cc470 msgr2=0x7ff8940ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff899800640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8940cc470 0x7ff8940ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff899800640 1 -- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 msgr2=0x7ff8941464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff899800640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 0x7ff8941464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff899800640 1 -- 172.21.15.102:0/221339091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89415c230 con 0x7ff8940cb6a0 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff89a200640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 0x7ff8941464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:46:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff899800640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8940cb6a0 0x7ff89415e0c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff88c000c00 tx=0x7ff88c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:57.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff891600640 1 -- 172.21.15.102:0/221339091 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c034050 con 0x7ff8940cb6a0 2024-04-20T21:46:57.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff891600640 1 -- 172.21.15.102:0/221339091 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88c0044c0 con 0x7ff8940cb6a0 2024-04-20T21:46:57.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff891600640 1 -- 172.21.15.102:0/221339091 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c006e10 con 0x7ff8940cb6a0 2024-04-20T21:46:57.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89415d290 con 0x7ff8940cb6a0 2024-04-20T21:46:57.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.799+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89415c4d0 con 0x7ff8940cb6a0 2024-04-20T21:46:57.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.801+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89404fd60 con 0x7ff8940cb6a0 2024-04-20T21:46:57.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.801+0000 7ff891600640 1 -- 172.21.15.102:0/221339091 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff88c037070 con 0x7ff8940cb6a0 2024-04-20T21:46:57.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.802+0000 7ff891600640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:46:57.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.803+0000 7ff891600640 1 -- 172.21.15.102:0/221339091 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7ff88c0ae2d0 con 0x7ff8940cb6a0 2024-04-20T21:46:57.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.803+0000 7ff898e00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:46:57.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.803+0000 7ff898e00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff87406c8f0 0x7ff87406edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff894153d00 tx=0x7ff880001ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:46:57.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:57.807+0000 7ff891600640 1 -- 172.21.15.102:0/221339091 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff88c07b620 con 0x7ff8940cb6a0 2024-04-20T21:46:58.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.051+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8940cc9c0 con 0x7ff8940cb6a0 2024-04-20T21:46:58.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.052+0000 7ff891600640 1 -- 172.21.15.102:0/221339091 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+3548 (secure 0 0 0) 0x7ff88c07f920 con 0x7ff8940cb6a0 2024-04-20T21:46:58.051 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:46:58.051 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:55.958174+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:46:58.051 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 108 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff87406c8f0 msgr2=0x7ff87406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff87406c8f0 0x7ff87406edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff894153d00 tx=0x7ff880001ab0 comp rx=0 tx=0).stop 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8940cb6a0 msgr2=0x7ff89415e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8940cb6a0 0x7ff89415e0c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff88c000c00 tx=0x7ff88c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 shutdown_connections 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7ff87406c8f0 0x7ff87406edb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff894155530 0x7ff8941464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8940cc470 0x7ff8940ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 --2- 172.21.15.102:0/221339091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8940cb6a0 0x7ff89415e0c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:46:58.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 >> 172.21.15.102:0/221339091 conn(0x7ff894075080 msgr2=0x7ff894153090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:46:58.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 shutdown_connections 2024-04-20T21:46:58.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:46:58.055+0000 7ff89ac00640 1 -- 172.21.15.102:0/221339091 wait complete. 2024-04-20T21:46:58.072 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:47:00.073 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:47:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.282+0000 7f8d6de00640 1 Processor -- start 2024-04-20T21:47:00.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.282+0000 7f8d6de00640 1 -- start start 2024-04-20T21:47:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.284+0000 7f8d6de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d6815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.284+0000 7f8d6de00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6815ef60 con 0x7f8d680cb6a0 2024-04-20T21:47:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.284+0000 7f8d6de00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6815f240 con 0x7f8d680cc470 2024-04-20T21:47:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.284+0000 7f8d6de00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8d680c5ca0 con 0x7f8d68155530 2024-04-20T21:47:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.284+0000 7f8d6d400640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8d680cb6a0 0x7f8d681659d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43612/0 (socket says 172.21.15.102:43612) 2024-04-20T21:47:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.284+0000 7f8d6d400640 1 -- 172.21.15.102:0/2181378595 learned_addr learned my addr 172.21.15.102:0/2181378595 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:00.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.284+0000 7f8d6ca00640 1 --2- 172.21.15.102:0/2181378595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d6815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:00.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.285+0000 7f8d67400640 1 -- 172.21.15.102:0/2181378595 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3552487536 0 0) 0x7f8d6815ef60 con 0x7f8d680cb6a0 2024-04-20T21:47:00.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.285+0000 7f8d67400640 1 -- 172.21.15.102:0/2181378595 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d4c003520 con 0x7f8d680cb6a0 2024-04-20T21:47:00.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.285+0000 7f8d67400640 1 -- 172.21.15.102:0/2181378595 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559294056 0 0) 0x7f8d6815f240 con 0x7f8d680cc470 2024-04-20T21:47:00.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.285+0000 7f8d67400640 1 -- 172.21.15.102:0/2181378595 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d6815ef60 con 0x7f8d680cc470 2024-04-20T21:47:00.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.285+0000 7f8d6ca00640 1 -- 172.21.15.102:0/2181378595 >> v1:172.21.15.193:6789/0 conn(0x7f8d680cc470 legacy=0x7f8d680cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:00.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.285+0000 7f8d6ca00640 1 -- 172.21.15.102:0/2181378595 >> v1:172.21.15.102:6789/0 conn(0x7f8d680cb6a0 legacy=0x7f8d681659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:00.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.285+0000 7f8d6ca00640 1 -- 172.21.15.102:0/2181378595 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6815f520 con 0x7f8d68155530 2024-04-20T21:47:00.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.286+0000 7f8d6ca00640 1 --2- 172.21.15.102:0/2181378595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d6815e640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8d580097d0 tx=0x7f8d5802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b571a1df98b445e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:00.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.286+0000 7f8d67400640 1 -- 172.21.15.102:0/2181378595 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58034070 con 0x7f8d68155530 2024-04-20T21:47:00.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.286+0000 7f8d67400640 1 -- 172.21.15.102:0/2181378595 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d58030880 con 0x7f8d68155530 2024-04-20T21:47:00.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.286+0000 7f8d67400640 1 -- 172.21.15.102:0/2181378595 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58030c50 con 0x7f8d68155530 2024-04-20T21:47:00.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.287+0000 7f8d6de00640 1 -- 172.21.15.102:0/2181378595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 msgr2=0x7f8d6815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:00.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.287+0000 7f8d6de00640 1 --2- 172.21.15.102:0/2181378595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d6815e640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8d580097d0 tx=0x7f8d5802b9c0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.287+0000 7f8d6de00640 1 -- 172.21.15.102:0/2181378595 shutdown_connections 2024-04-20T21:47:00.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.287+0000 7f8d6de00640 1 --2- 172.21.15.102:0/2181378595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d6815e640 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.287+0000 7f8d6de00640 1 -- 172.21.15.102:0/2181378595 >> 172.21.15.102:0/2181378595 conn(0x7f8d68075080 msgr2=0x7f8d68075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:00.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.287+0000 7f8d6de00640 1 -- 172.21.15.102:0/2181378595 shutdown_connections 2024-04-20T21:47:00.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.287+0000 7f8d6de00640 1 -- 172.21.15.102:0/2181378595 wait complete. 2024-04-20T21:47:00.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.289+0000 7f8d6de00640 1 Processor -- start 2024-04-20T21:47:00.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.290+0000 7f8d6de00640 1 -- start start 2024-04-20T21:47:00.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d680cb6a0 0x7f8d680b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:00.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d680cc470 0x7f8d6814ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:00.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6de00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d681e6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:00.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680cd280 con 0x7f8d680cc470 2024-04-20T21:47:00.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6de00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680cd5f0 con 0x7f8d680cb6a0 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6de00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8d680ccbe0 con 0x7f8d68155530 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d680cc470 0x7f8d6814ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d680cc470 0x7f8d6814ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41464/0 (socket says 172.21.15.102:41464) 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d67e00640 1 -- 172.21.15.102:0/2316745029 learned_addr learned my addr 172.21.15.102:0/2316745029 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6ca00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d680cb6a0 0x7f8d680b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6d400640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d681e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6ca00640 1 -- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 msgr2=0x7f8d681e6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6ca00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d681e6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6ca00640 1 -- 172.21.15.102:0/2316745029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d680cc470 msgr2=0x7f8d6814ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6ca00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d680cc470 0x7f8d6814ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.291+0000 7f8d6ca00640 1 -- 172.21.15.102:0/2316745029 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6814f490 con 0x7f8d680cb6a0 2024-04-20T21:47:00.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.292+0000 7f8d67e00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d680cc470 0x7f8d6814ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:47:00.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.292+0000 7f8d6ca00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d680cb6a0 0x7f8d680b52c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8d58033000 tx=0x7f8d580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:00.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.292+0000 7f8d64c00640 1 -- 172.21.15.102:0/2316745029 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58034040 con 0x7f8d680cb6a0 2024-04-20T21:47:00.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.292+0000 7f8d64c00640 1 -- 172.21.15.102:0/2316745029 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5803ada0 con 0x7f8d680cb6a0 2024-04-20T21:47:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.292+0000 7f8d64c00640 1 -- 172.21.15.102:0/2316745029 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58006e10 con 0x7f8d680cb6a0 2024-04-20T21:47:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.292+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d680b61e0 con 0x7f8d680cb6a0 2024-04-20T21:47:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.292+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d680b67a0 con 0x7f8d680cb6a0 2024-04-20T21:47:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.294+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d68074340 con 0x7f8d680cb6a0 2024-04-20T21:47:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.294+0000 7f8d64c00640 1 -- 172.21.15.102:0/2316745029 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d5804b060 con 0x7f8d680cb6a0 2024-04-20T21:47:00.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.295+0000 7f8d64c00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d4406c920 0x7f8d4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:00.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.295+0000 7f8d64c00640 1 -- 172.21.15.102:0/2316745029 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7f8d580ae5c0 con 0x7f8d680cb6a0 2024-04-20T21:47:00.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.296+0000 7f8d67e00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d4406c920 0x7f8d4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:00.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.296+0000 7f8d67e00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d4406c920 0x7f8d4406ede0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8d68151fb0 tx=0x7f8d50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:00.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.300+0000 7f8d64c00640 1 -- 172.21.15.102:0/2316745029 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8d68074340 con 0x7f8d680cb6a0 2024-04-20T21:47:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.540+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d68154370 con 0x7f8d680cb6a0 2024-04-20T21:47:00.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.542+0000 7f8d64c00640 1 -- 172.21.15.102:0/2316745029 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+3548 (secure 0 0 0) 0x7f8d68154370 con 0x7f8d680cb6a0 2024-04-20T21:47:00.541 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:47:00.541 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:55.958174+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:47:00.541 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 108 2024-04-20T21:47:00.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d4406c920 msgr2=0x7f8d4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:00.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d4406c920 0x7f8d4406ede0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8d68151fb0 tx=0x7f8d50009040 comp rx=0 tx=0).stop 2024-04-20T21:47:00.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d680cb6a0 msgr2=0x7f8d680b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:00.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d680cb6a0 0x7f8d680b52c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8d58033000 tx=0x7f8d580046c0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 shutdown_connections 2024-04-20T21:47:00.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d68155530 0x7f8d681e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d680cc470 0x7f8d6814ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8d4406c920 0x7f8d4406ede0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 --2- 172.21.15.102:0/2316745029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d680cb6a0 0x7f8d680b52c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:00.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 >> 172.21.15.102:0/2316745029 conn(0x7f8d68075080 msgr2=0x7f8d68150e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:00.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.545+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 shutdown_connections 2024-04-20T21:47:00.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:00.546+0000 7f8d6de00640 1 -- 172.21.15.102:0/2316745029 wait complete. 2024-04-20T21:47:00.563 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:47:02.453 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:47:02.456 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~301s 2024-04-20T21:47:02.456 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-04-20T21:47:02.456 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~213s 2024-04-20T21:47:02.456 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-04-20T21:47:02.456 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:47:02.456 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:47:02.461 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:47:02.551 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:47:02.551 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:47:02.564 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:47:02.655 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:47:02.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.793+0000 7f2f71e00640 1 Processor -- start 2024-04-20T21:47:02.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.793+0000 7f2f71e00640 1 -- start start 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.794+0000 7f2f71e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c14fa50 0x7f2f6c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.794+0000 7f2f71e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2f6c073a10 0x7f2f6c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.794+0000 7f2f71e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c1536d0 con 0x7f2f6c0c9eb0 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.794+0000 7f2f71e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c159500 con 0x7f2f6c073a10 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.794+0000 7f2f71e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c0c76c0 con 0x7f2f6c14fa50 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c14fa50 0x7f2f6c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f62000640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f2f6c0c9eb0 0x7f2f6c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:59852/0 (socket says 172.21.15.102:59852) 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f62000640 1 -- 172.21.15.102:0/3449159874 learned_addr learned my addr 172.21.15.102:0/3449159874 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f6b400640 1 --2- 172.21.15.102:0/3449159874 >> v2:172.21.15.102:3300/0 conn(0x7f2f6c073a10 0x7f2f6c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f6aa00640 1 -- 172.21.15.102:0/3449159874 >> v2:172.21.15.102:3300/0 conn(0x7f2f6c073a10 msgr2=0x7f2f6c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f6aa00640 1 --2- 172.21.15.102:0/3449159874 >> v2:172.21.15.102:3300/0 conn(0x7f2f6c073a10 0x7f2f6c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f6aa00640 1 -- 172.21.15.102:0/3449159874 >> v1:172.21.15.193:6789/0 conn(0x7f2f6c0c9eb0 legacy=0x7f2f6c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f6a000640 1 -- 172.21.15.102:0/3449159874 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422206754 0 0) 0x7f2f54001110 con 0x7f2f6c0c9eb0 2024-04-20T21:47:02.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.795+0000 7f2f6aa00640 1 -- 172.21.15.102:0/3449159874 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c0c7960 con 0x7f2f6c14fa50 2024-04-20T21:47:02.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.796+0000 7f2f6aa00640 1 --2- 172.21.15.102:0/3449159874 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c14fa50 0x7f2f6c0c4830 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0097a0 tx=0x7f2f5c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4837c2176c799d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:02.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.796+0000 7f2f6a000640 1 -- 172.21.15.102:0/3449159874 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c034070 con 0x7f2f6c14fa50 2024-04-20T21:47:02.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.796+0000 7f2f6a000640 1 -- 172.21.15.102:0/3449159874 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c030610 con 0x7f2f6c14fa50 2024-04-20T21:47:02.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.796+0000 7f2f6a000640 1 -- 172.21.15.102:0/3449159874 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0309e0 con 0x7f2f6c14fa50 2024-04-20T21:47:02.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.797+0000 7f2f71e00640 1 -- 172.21.15.102:0/3449159874 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c14fa50 msgr2=0x7f2f6c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.797+0000 7f2f71e00640 1 --2- 172.21.15.102:0/3449159874 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c14fa50 0x7f2f6c0c4830 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0097a0 tx=0x7f2f5c02b960 comp rx=0 tx=0).stop 2024-04-20T21:47:02.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.797+0000 7f2f71e00640 1 -- 172.21.15.102:0/3449159874 shutdown_connections 2024-04-20T21:47:02.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.797+0000 7f2f71e00640 1 --2- 172.21.15.102:0/3449159874 >> v2:172.21.15.102:3300/0 conn(0x7f2f6c073a10 0x7f2f6c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.797+0000 7f2f71e00640 1 --2- 172.21.15.102:0/3449159874 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c14fa50 0x7f2f6c0c4830 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.797+0000 7f2f71e00640 1 -- 172.21.15.102:0/3449159874 >> 172.21.15.102:0/3449159874 conn(0x7f2f6c0c9390 msgr2=0x7f2f6c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:02.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.797+0000 7f2f71e00640 1 -- 172.21.15.102:0/3449159874 shutdown_connections 2024-04-20T21:47:02.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.797+0000 7f2f71e00640 1 -- 172.21.15.102:0/3449159874 wait complete. 2024-04-20T21:47:02.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.799+0000 7f2f71e00640 1 Processor -- start 2024-04-20T21:47:02.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.799+0000 7f2f71e00640 1 -- start start 2024-04-20T21:47:02.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f71e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c073a10 0x7f2f6c14a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f71e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f6c0c9eb0 0x7f2f6c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f71e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f6c14fa50 0x7f2f6c14e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f71e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0c7c00 con 0x7f2f6c14fa50 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f71e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0c3d90 con 0x7f2f6c0c9eb0 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f71e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c0c4030 con 0x7f2f6c073a10 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c073a10 0x7f2f6c14a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c073a10 0x7f2f6c14a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50270/0 (socket says 172.21.15.102:50270) 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f6aa00640 1 -- 172.21.15.102:0/1254756811 learned_addr learned my addr 172.21.15.102:0/1254756811 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f62000640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f6c0c9eb0 0x7f2f6c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.800+0000 7f2f6b400640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f6c14fa50 0x7f2f6c14e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f6aa00640 1 -- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f6c0c9eb0 msgr2=0x7f2f6c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f6aa00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f6c0c9eb0 0x7f2f6c156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f6aa00640 1 -- 172.21.15.102:0/1254756811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f6c14fa50 msgr2=0x7f2f6c14e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f6aa00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f6c14fa50 0x7f2f6c14e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f6aa00640 1 -- 172.21.15.102:0/1254756811 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c1541b0 con 0x7f2f6c073a10 2024-04-20T21:47:02.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f6b400640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f6c14fa50 0x7f2f6c14e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:47:02.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f6aa00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c073a10 0x7f2f6c14a9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0098c0 tx=0x7f2f5c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:02.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f63e00640 1 -- 172.21.15.102:0/1254756811 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c034040 con 0x7f2f6c073a10 2024-04-20T21:47:02.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f63e00640 1 -- 172.21.15.102:0/1254756811 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c03aaa0 con 0x7f2f6c073a10 2024-04-20T21:47:02.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f63e00640 1 -- 172.21.15.102:0/1254756811 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0443f0 con 0x7f2f6c073a10 2024-04-20T21:47:02.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c154490 con 0x7f2f6c073a10 2024-04-20T21:47:02.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.801+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c154a50 con 0x7f2f6c073a10 2024-04-20T21:47:02.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.802+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c074340 con 0x7f2f6c073a10 2024-04-20T21:47:02.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.803+0000 7f2f63e00640 1 -- 172.21.15.102:0/1254756811 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f5c04b060 con 0x7f2f6c073a10 2024-04-20T21:47:02.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.804+0000 7f2f63e00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.804+0000 7f2f63e00640 1 -- 172.21.15.102:0/1254756811 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7f2f5c0ae3e0 con 0x7f2f6c073a10 2024-04-20T21:47:02.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.804+0000 7f2f62000640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.805+0000 7f2f62000640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f4806c8f0 0x7f2f4806edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c14af30 tx=0x7f2f54007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:02.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.808+0000 7f2f63e00640 1 -- 172.21.15.102:0/1254756811 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2f5c03f070 con 0x7f2f6c073a10 2024-04-20T21:47:02.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.878+0000 7f0347e00640 1 Processor -- start 2024-04-20T21:47:02.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.878+0000 7f0347e00640 1 -- start start 2024-04-20T21:47:02.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.879+0000 7f0347e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0340162e40 0x7f034006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.879+0000 7f0347e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f034006d2e0 con 0x7f0340163660 2024-04-20T21:47:02.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.879+0000 7f0347e00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f034006d5c0 con 0x7f0340163220 2024-04-20T21:47:02.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.879+0000 7f0347e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f03400c7290 con 0x7f0340162e40 2024-04-20T21:47:02.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.879+0000 7f0345400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0340162e40 0x7f034006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.879+0000 7f0345e00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0340163660 0x7f03400c7a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43622/0 (socket says 172.21.15.102:43622) 2024-04-20T21:47:02.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.879+0000 7f0345e00640 1 -- 172.21.15.102:0/1379219350 learned_addr learned my addr 172.21.15.102:0/1379219350 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:02.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.880+0000 7f0345400640 1 -- 172.21.15.102:0/1379219350 >> v1:172.21.15.193:6789/0 conn(0x7f0340163220 legacy=0x7f0340159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.880+0000 7f033fe00640 1 -- 172.21.15.102:0/1379219350 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025589623 0 0) 0x7f034006d2e0 con 0x7f0340163660 2024-04-20T21:47:02.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.880+0000 7f0345400640 1 -- 172.21.15.102:0/1379219350 >> v1:172.21.15.102:6789/0 conn(0x7f0340163660 legacy=0x7f03400c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.880+0000 7f0345400640 1 -- 172.21.15.102:0/1379219350 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034006d8a0 con 0x7f0340162e40 2024-04-20T21:47:02.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.881+0000 7f0345400640 1 --2- 172.21.15.102:0/1379219350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f034006c9e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0330009720 tx=0x7f033002b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3adb8467586b116 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:02.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.881+0000 7f033fe00640 1 -- 172.21.15.102:0/1379219350 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330038070 con 0x7f0340162e40 2024-04-20T21:47:02.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.881+0000 7f033fe00640 1 -- 172.21.15.102:0/1379219350 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0330031520 con 0x7f0340162e40 2024-04-20T21:47:02.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.881+0000 7f033fe00640 1 -- 172.21.15.102:0/1379219350 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03300318f0 con 0x7f0340162e40 2024-04-20T21:47:02.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.882+0000 7f0347e00640 1 -- 172.21.15.102:0/1379219350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 msgr2=0x7f034006c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.882+0000 7f0347e00640 1 --2- 172.21.15.102:0/1379219350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f034006c9e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0330009720 tx=0x7f033002b7c0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.882+0000 7f0347e00640 1 -- 172.21.15.102:0/1379219350 shutdown_connections 2024-04-20T21:47:02.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.882+0000 7f0347e00640 1 --2- 172.21.15.102:0/1379219350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f034006c9e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.882+0000 7f0347e00640 1 -- 172.21.15.102:0/1379219350 >> 172.21.15.102:0/1379219350 conn(0x7f03400c9390 msgr2=0x7f03400c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:02.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.882+0000 7f0347e00640 1 -- 172.21.15.102:0/1379219350 shutdown_connections 2024-04-20T21:47:02.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.882+0000 7f0347e00640 1 -- 172.21.15.102:0/1379219350 wait complete. 2024-04-20T21:47:02.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.884+0000 7f0347e00640 1 Processor -- start 2024-04-20T21:47:02.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.884+0000 7f0347e00640 1 -- start start 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.884+0000 7f0347e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f03401515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0347e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0340163220 0x7f03400c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0347e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0340163660 0x7f0340162220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0347e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f03400c9080 con 0x7f0340163660 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0347e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03400c7570 con 0x7f0340162e40 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0347e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f034006cf20 con 0x7f0340163220 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f03401515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f03401515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51196/0 (socket says 172.21.15.102:51196) 2024-04-20T21:47:02.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345400640 1 -- 172.21.15.102:0/1719581146 learned_addr learned my addr 172.21.15.102:0/1719581146 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345e00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0340163660 0x7f0340162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0344a00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0340163220 0x7f03400c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345400640 1 -- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0340163220 msgr2=0x7f03400c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345400640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0340163220 0x7f03400c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345400640 1 -- 172.21.15.102:0/1719581146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0340163660 msgr2=0x7f0340162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345400640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0340163660 0x7f0340162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.885+0000 7f0345400640 1 -- 172.21.15.102:0/1719581146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0340150920 con 0x7f0340162e40 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.886+0000 7f0345400640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f03401515b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0330000c00 tx=0x7f03300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.886+0000 7f033d600640 1 -- 172.21.15.102:0/1719581146 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330043070 con 0x7f0340162e40 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.886+0000 7f033d600640 1 -- 172.21.15.102:0/1719581146 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033003f930 con 0x7f0340162e40 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.886+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03400c4860 con 0x7f0340162e40 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.886+0000 7f033d600640 1 -- 172.21.15.102:0/1719581146 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330038040 con 0x7f0340162e40 2024-04-20T21:47:02.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.886+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0340150640 con 0x7f0340162e40 2024-04-20T21:47:02.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.888+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0340072a50 con 0x7f0340162e40 2024-04-20T21:47:02.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.888+0000 7f033d600640 1 -- 172.21.15.102:0/1719581146 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0330046070 con 0x7f0340162e40 2024-04-20T21:47:02.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.889+0000 7f033d600640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f032406c920 0x7f032406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:02.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.890+0000 7f033d600640 1 -- 172.21.15.102:0/1719581146 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7f03300ae250 con 0x7f0340162e40 2024-04-20T21:47:02.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.890+0000 7f0344a00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f032406c920 0x7f032406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:02.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.891+0000 7f0344a00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f032406c920 0x7f032406ede0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f03400cc570 tx=0x7f0334009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:02.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:02.897+0000 7f033d600640 1 -- 172.21.15.102:0/1719581146 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0340072a50 con 0x7f0340162e40 2024-04-20T21:47:03.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.058+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f6c06d600 con 0x7f2f6c073a10 2024-04-20T21:47:03.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.059+0000 7f2f63e00640 1 -- 172.21.15.102:0/1254756811 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+3548 (secure 0 0 0) 0x7f2f5c07bc60 con 0x7f2f6c073a10 2024-04-20T21:47:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:47:03.058 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:55.958174+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:47:03.058 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 108 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.062+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f4806c8f0 msgr2=0x7f2f4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.062+0000 7f2f71e00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f4806c8f0 0x7f2f4806edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c14af30 tx=0x7f2f54007b50 comp rx=0 tx=0).stop 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.062+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c073a10 msgr2=0x7f2f6c14a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.062+0000 7f2f71e00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c073a10 0x7f2f6c14a9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0098c0 tx=0x7f2f5c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.063+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 shutdown_connections 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.063+0000 7f2f71e00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f6c14fa50 0x7f2f6c14e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.063+0000 7f2f71e00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.063+0000 7f2f71e00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f6c0c9eb0 0x7f2f6c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.063+0000 7f2f71e00640 1 --2- 172.21.15.102:0/1254756811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f6c073a10 0x7f2f6c14a9f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.063+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 >> 172.21.15.102:0/1254756811 conn(0x7f2f6c0c9390 msgr2=0x7f2f6c06f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:03.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.063+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 shutdown_connections 2024-04-20T21:47:03.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.063+0000 7f2f71e00640 1 -- 172.21.15.102:0/1254756811 wait complete. 2024-04-20T21:47:03.082 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 108, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T21:16:54.017441+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 108, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4279": { "addr": "172.21.15.193:6837/72219413", "addrs": { "addrvec": [ { "addr": "172.21.15.193:6834", "nonce": 72219413, "type": "v2" }, { "addr": "172.21.15.193:6837", "nonce": 72219413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4279, "incarnation": 108, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 66, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T21:46:55.958174+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4279 } } } ], "standbys": [ { "addr": "172.21.15.102:6837/4172033112", "addrs": { "addrvec": [ { "addr": "172.21.15.102:6836", "nonce": 4172033112, "type": "v2" }, { "addr": "172.21.15.102:6837", "nonce": 4172033112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4262, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-20T21:47:03.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.108+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0340074340 con 0x7f0340162e40 2024-04-20T21:47:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.110+0000 7f033d600640 1 -- 172.21.15.102:0/1719581146 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+21439 (secure 0 0 0) 0x7f0340150920 con 0x7f0340162e40 2024-04-20T21:47:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:47:03.109 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":66,"fsid":"1c72cacb-b320-4a89-8b72-ff890245905f","created":"2024-04-20T21:16:30.439499+0000","modified":"2024-04-20T21:46:55.958242+0000","last_up_change":"2024-04-20T21:16:41.779245+0000","last_in_change":"2024-04-20T21:16:32.814840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:16:42.688017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:16:51.795761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:16:52.983434+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T21:16:56.582297+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc15372f-7389-4989-b1f9-32dce20f8e38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6809","nonce":1191557030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6811","nonce":1191557030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6815","nonce":1191557030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1191557030},{"type":"v1","addr":"172.21.15.102:6813","nonce":1191557030}]},"public_addr":"172.21.15.102:6809/1191557030","cluster_addr":"172.21.15.102:6811/1191557030","heartbeat_back_addr":"172.21.15.102:6815/1191557030","heartbeat_front_addr":"172.21.15.102:6813/1191557030","state":["exists","up"]},{"osd":1,"uuid":"9ad26a2f-4aff-44d2-a3cb-8392ff8a66a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6801","nonce":100719841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6803","nonce":100719841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6807","nonce":100719841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":100719841},{"type":"v1","addr":"172.21.15.102:6805","nonce":100719841}]},"public_addr":"172.21.15.102:6801/100719841","cluster_addr":"172.21.15.102:6803/100719841","heartbeat_back_addr":"172.21.15.102:6807/100719841","heartbeat_front_addr":"172.21.15.102:6805/100719841","state":["exists","up"]},{"osd":2,"uuid":"4582fd67-4f82-4bee-9055-5905a095036b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6817","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6819","nonce":1144653737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6821","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6822","nonce":1144653737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6830","nonce":1144653737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":1144653737},{"type":"v1","addr":"172.21.15.102:6826","nonce":1144653737}]},"public_addr":"172.21.15.102:6819/1144653737","cluster_addr":"172.21.15.102:6822/1144653737","heartbeat_back_addr":"172.21.15.102:6830/1144653737","heartbeat_front_addr":"172.21.15.102:6826/1144653737","state":["exists","up"]},{"osd":3,"uuid":"32da37a5-4aee-4279-a79a-2b689b1ebd48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6818","nonce":2815998409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6823","nonce":2815998409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6829","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6831","nonce":2815998409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6825","nonce":2815998409},{"type":"v1","addr":"172.21.15.102:6827","nonce":2815998409}]},"public_addr":"172.21.15.102:6818/2815998409","cluster_addr":"172.21.15.102:6823/2815998409","heartbeat_back_addr":"172.21.15.102:6831/2815998409","heartbeat_front_addr":"172.21.15.102:6827/2815998409","state":["exists","up"]},{"osd":4,"uuid":"32489a66-07e4-4062-a82b-4183f3b727e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6801","nonce":2469990595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6803","nonce":2469990595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6809","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6811","nonce":2469990595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6805","nonce":2469990595},{"type":"v1","addr":"172.21.15.193:6807","nonce":2469990595}]},"public_addr":"172.21.15.193:6801/2469990595","cluster_addr":"172.21.15.193:6803/2469990595","heartbeat_back_addr":"172.21.15.193:6811/2469990595","heartbeat_front_addr":"172.21.15.193:6807/2469990595","state":["exists","up"]},{"osd":5,"uuid":"bc20150c-0f8e-498b-996c-83f43f44964a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6806","nonce":1109454637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6810","nonce":1109454637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6815","nonce":1109454637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1109454637},{"type":"v1","addr":"172.21.15.193:6813","nonce":1109454637}]},"public_addr":"172.21.15.193:6806/1109454637","cluster_addr":"172.21.15.193:6810/1109454637","heartbeat_back_addr":"172.21.15.193:6815/1109454637","heartbeat_front_addr":"172.21.15.193:6813/1109454637","state":["exists","up"]},{"osd":6,"uuid":"cbdceaf5-95a2-481b-831a-3edbfe220712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6817","nonce":1341503027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6819","nonce":1341503027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6825","nonce":1341503027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1341503027},{"type":"v1","addr":"172.21.15.193:6822","nonce":1341503027}]},"public_addr":"172.21.15.193:6817/1341503027","cluster_addr":"172.21.15.193:6819/1341503027","heartbeat_back_addr":"172.21.15.193:6825/1341503027","heartbeat_front_addr":"172.21.15.193:6822/1341503027","state":["exists","up"]},{"osd":7,"uuid":"4f7cd742-6ea8-43ff-a38b-f68c893a5c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6823","nonce":2963386825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6827","nonce":2963386825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6831","nonce":2963386825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":2963386825},{"type":"v1","addr":"172.21.15.193:6829","nonce":2963386825}]},"public_addr":"172.21.15.193:6823/2963386825","cluster_addr":"172.21.15.193:6827/2963386825","heartbeat_back_addr":"172.21.15.193:6831/2963386825","heartbeat_front_addr":"172.21.15.193:6829/2963386825","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.151009+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.952475+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.001724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:38.212388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.962961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.932794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.970607+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:16:37.975081+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6835/2467124512":"2024-04-21T21:46:10.944621+0000","172.21.15.102:6833/2976263880":"2024-04-21T21:45:10.928031+0000","172.21.15.193:6836/2076039362":"2024-04-21T21:43:40.902616+0000","172.21.15.102:6834/36459307":"2024-04-21T21:31:25.670488+0000","172.21.15.102:6833/713712414":"2024-04-21T21:23:30.536944+0000","172.21.15.102:6835/2873333699":"2024-04-21T21:19:40.461600+0000","172.21.15.102:6833/2239658043":"2024-04-21T21:28:35.625696+0000","172.21.15.102:6834/1017522496":"2024-04-21T21:22:20.501478+0000","172.21.15.102:6832/2239658043":"2024-04-21T21:28:35.625696+0000","172.21.15.102:6832/1043030921":"2024-04-21T21:33:05.702536+0000","172.21.15.102:6834/1097388930":"2024-04-21T21:26:10.588429+0000","172.21.15.102:6832/2976263880":"2024-04-21T21:45:10.928031+0000","172.21.15.102:6832/327932375":"2024-04-21T21:39:30.821390+0000","172.21.15.102:6834/945751373":"2024-04-21T21:38:20.801433+0000","172.21.15.193:6838/2467124512":"2024-04-21T21:46:10.944621+0000","172.21.15.102:6833/2795763369":"2024-04-21T21:21:20.485686+0000","172.21.15.102:6835/1097388930":"2024-04-21T21:26:10.588429+0000","172.21.15.102:6835/1017522496":"2024-04-21T21:22:20.501478+0000","172.21.15.102:6835/2459661167":"2024-04-21T21:42:55.884532+0000","172.21.15.102:6832/2795763369":"2024-04-21T21:21:20.485686+0000","172.21.15.102:6833/1043030921":"2024-04-21T21:33:05.702536+0000","172.21.15.102:6834/2655303061":"2024-04-21T21:34:50.733957+0000","172.21.15.102:6832/713712414":"2024-04-21T21:23:30.536944+0000","172.21.15.102:6833/327932375":"2024-04-21T21:39:30.821390+0000","172.21.15.102:6835/2655303061":"2024-04-21T21:34:50.733957+0000","172.21.15.102:6832/3985712430":"2024-04-21T21:35:50.750525+0000","172.21.15.102:6835/36459307":"2024-04-21T21:31:25.670488+0000","172.21.15.102:6832/2637555410":"2024-04-21T21:42:10.872593+0000","172.21.15.102:6833/1356782221":"2024-04-21T21:46:55.958136+0000","172.21.15.102:6834/2873333699":"2024-04-21T21:19:40.461600+0000","172.21.15.102:6833/3985712430":"2024-04-21T21:35:50.750525+0000","172.21.15.102:6835/945751373":"2024-04-21T21:38:20.801433+0000","172.21.15.102:6834/2459661167":"2024-04-21T21:42:55.884532+0000","172.21.15.102:6832/1356782221":"2024-04-21T21:46:55.958136+0000","172.21.15.193:6839/2076039362":"2024-04-21T21:43:40.902616+0000","172.21.15.102:6834/3586768191":"2024-04-21T21:41:25.854747+0000","172.21.15.102:6835/3586768191":"2024-04-21T21:41:25.854747+0000","172.21.15.102:6833/2637555410":"2024-04-21T21:42:10.872593+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:47:03.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f032406c920 msgr2=0x7f032406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f032406c920 0x7f032406ede0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f03400cc570 tx=0x7f0334009040 comp rx=0 tx=0).stop 2024-04-20T21:47:03.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 msgr2=0x7f03401515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f03401515b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0330000c00 tx=0x7f03300046c0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 shutdown_connections 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0340163660 0x7f0340162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0340163220 0x7f03400c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f032406c920 0x7f032406ede0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 --2- 172.21.15.102:0/1719581146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0340162e40 0x7f03401515b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 >> 172.21.15.102:0/1719581146 conn(0x7f03400c9390 msgr2=0x7f0340159210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 shutdown_connections 2024-04-20T21:47:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.113+0000 7f0347e00640 1 -- 172.21.15.102:0/1719581146 wait complete. 2024-04-20T21:47:03.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:16:42.688017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:16:51.795761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:16:52.983434+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 505, 'snap_epoch': 50, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T21:16:56.582297+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 505, 'snap_epoch': 50, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T21:47:03.131 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:47:03.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.379+0000 7fbc11600640 1 Processor -- start 2024-04-20T21:47:03.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.379+0000 7fbc11600640 1 -- start start 2024-04-20T21:47:03.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.380+0000 7fbc11600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.380+0000 7fbc11600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbc0c0cd060 0x7fbc0c0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.380+0000 7fbc11600640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc0c158f80 con 0x7fbc0c0c5c50 2024-04-20T21:47:03.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.380+0000 7fbc11600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fbc0c159de0 con 0x7fbc0c0cd060 2024-04-20T21:47:03.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.380+0000 7fbc11600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0c159260 con 0x7fbc0c152d80 2024-04-20T21:47:03.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc03e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbc0c0cd060 0x7fbc0c0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc0be00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc10c00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbc0c0c5c50 0x7fbc0c15c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43628/0 (socket says 172.21.15.102:43628) 2024-04-20T21:47:03.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc10c00640 1 -- 172.21.15.102:0/309611046 learned_addr learned my addr 172.21.15.102:0/309611046 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:03.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc0be00640 1 -- 172.21.15.102:0/309611046 >> v2:172.21.15.102:3300/0 conn(0x7fbc0c0cd060 msgr2=0x7fbc0c0c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:47:03.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc0be00640 1 --2- 172.21.15.102:0/309611046 >> v2:172.21.15.102:3300/0 conn(0x7fbc0c0cd060 0x7fbc0c0c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc0be00640 1 -- 172.21.15.102:0/309611046 >> v1:172.21.15.102:6789/0 conn(0x7fbc0c0c5c50 legacy=0x7fbc0c15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc0be00640 1 -- 172.21.15.102:0/309611046 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c159540 con 0x7fbc0c152d80 2024-04-20T21:47:03.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.381+0000 7fbc03e00640 1 --2- 172.21.15.102:0/309611046 >> v2:172.21.15.102:3300/0 conn(0x7fbc0c0cd060 0x7fbc0c0c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:47:03.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.382+0000 7fbc0be00640 1 --2- 172.21.15.102:0/309611046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c073ba0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0097a0 tx=0x7fbbfc02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8b8c49d5223156 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:03.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc0b400640 1 -- 172.21.15.102:0/309611046 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc034070 con 0x7fbc0c152d80 2024-04-20T21:47:03.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc0b400640 1 -- 172.21.15.102:0/309611046 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbfc030800 con 0x7fbc0c152d80 2024-04-20T21:47:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc0b400640 1 -- 172.21.15.102:0/309611046 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc030bd0 con 0x7fbc0c152d80 2024-04-20T21:47:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc11600640 1 -- 172.21.15.102:0/309611046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 msgr2=0x7fbc0c073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc11600640 1 --2- 172.21.15.102:0/309611046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c073ba0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0097a0 tx=0x7fbbfc02b940 comp rx=0 tx=0).stop 2024-04-20T21:47:03.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc11600640 1 -- 172.21.15.102:0/309611046 shutdown_connections 2024-04-20T21:47:03.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc11600640 1 --2- 172.21.15.102:0/309611046 >> v2:172.21.15.102:3300/0 conn(0x7fbc0c0cd060 0x7fbc0c0c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc11600640 1 --2- 172.21.15.102:0/309611046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c073ba0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc11600640 1 -- 172.21.15.102:0/309611046 >> 172.21.15.102:0/309611046 conn(0x7fbc0c0cc540 msgr2=0x7fbc0c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:03.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc11600640 1 -- 172.21.15.102:0/309611046 shutdown_connections 2024-04-20T21:47:03.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.383+0000 7fbc11600640 1 -- 172.21.15.102:0/309611046 wait complete. 2024-04-20T21:47:03.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.385+0000 7fbc11600640 1 Processor -- start 2024-04-20T21:47:03.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.385+0000 7fbc11600640 1 -- start start 2024-04-20T21:47:03.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc11600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0c0c5c50 0x7fbc0c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc11600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0c0cd060 0x7fbc0c0c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc11600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc11600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c153160 con 0x7fbc0c0cd060 2024-04-20T21:47:03.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc11600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c159b00 con 0x7fbc0c0c5c50 2024-04-20T21:47:03.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc11600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0c1532e0 con 0x7fbc0c152d80 2024-04-20T21:47:03.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0c0cd060 0x7fbc0c0c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc10c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0c0c5c50 0x7fbc0c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc10c00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50298/0 (socket says 172.21.15.102:50298) 2024-04-20T21:47:03.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.386+0000 7fbc10c00640 1 -- 172.21.15.102:0/1265934630 learned_addr learned my addr 172.21.15.102:0/1265934630 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:03.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc10c00640 1 -- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0c0c5c50 msgr2=0x7fbc0c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc10c00640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0c0c5c50 0x7fbc0c15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc10c00640 1 -- 172.21.15.102:0/1265934630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0c0cd060 msgr2=0x7fbc0c0c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc10c00640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0c0cd060 0x7fbc0c0c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc10c00640 1 -- 172.21.15.102:0/1265934630 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c1f03c0 con 0x7fbc0c152d80 2024-04-20T21:47:03.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc03e00640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0c0cd060 0x7fbc0c0c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:47:03.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc10c00640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c155aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400cab0 tx=0x7fbc0400cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:03.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc08c00640 1 -- 172.21.15.102:0/1265934630 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04017070 con 0x7fbc0c152d80 2024-04-20T21:47:03.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc08c00640 1 -- 172.21.15.102:0/1265934630 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0400ac30 con 0x7fbc0c152d80 2024-04-20T21:47:03.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c15ac00 con 0x7fbc0c152d80 2024-04-20T21:47:03.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.387+0000 7fbc08c00640 1 -- 172.21.15.102:0/1265934630 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04012400 con 0x7fbc0c152d80 2024-04-20T21:47:03.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.388+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c1f0960 con 0x7fbc0c152d80 2024-04-20T21:47:03.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.389+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0c04fcf0 con 0x7fbc0c152d80 2024-04-20T21:47:03.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.390+0000 7fbc08c00640 1 -- 172.21.15.102:0/1265934630 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc0401a070 con 0x7fbc0c152d80 2024-04-20T21:47:03.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.391+0000 7fbc08c00640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbbe806c8f0 0x7fbbe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.391+0000 7fbc08c00640 1 -- 172.21.15.102:0/1265934630 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7fbc0408c210 con 0x7fbc0c152d80 2024-04-20T21:47:03.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.391+0000 7fbc0be00640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbbe806c8f0 0x7fbbe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.392+0000 7fbc0be00640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbbe806c8f0 0x7fbbe806edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc036000 tx=0x7fbbfc03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:03.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.399+0000 7fbc08c00640 1 -- 172.21.15.102:0/1265934630 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbc040594d0 con 0x7fbc0c152d80 2024-04-20T21:47:03.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.597+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbc0c151430 con 0x7fbc0c152d80 2024-04-20T21:47:03.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.598+0000 7fbc08c00640 1 -- 172.21.15.102:0/1265934630 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7fbc0405d7d0 con 0x7fbc0c152d80 2024-04-20T21:47:03.597 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.600+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbbe806c8f0 msgr2=0x7fbbe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.600+0000 7fbc11600640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbbe806c8f0 0x7fbbe806edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc036000 tx=0x7fbbfc03b040 comp rx=0 tx=0).stop 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.600+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 msgr2=0x7fbc0c155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.600+0000 7fbc11600640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c155aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400cab0 tx=0x7fbc0400cf70 comp rx=0 tx=0).stop 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.601+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 shutdown_connections 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.601+0000 7fbc11600640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc0c152d80 0x7fbc0c155aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.601+0000 7fbc11600640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc0c0cd060 0x7fbc0c0c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.601+0000 7fbc11600640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fbbe806c8f0 0x7fbbe806edb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.601+0000 7fbc11600640 1 --2- 172.21.15.102:0/1265934630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0c0c5c50 0x7fbc0c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.601+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 >> 172.21.15.102:0/1265934630 conn(0x7fbc0c0cc540 msgr2=0x7fbc0c151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.601+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 shutdown_connections 2024-04-20T21:47:03.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.601+0000 7fbc11600640 1 -- 172.21.15.102:0/1265934630 wait complete. 2024-04-20T21:47:03.620 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:47:03.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.828+0000 7f87b0e00640 1 Processor -- start 2024-04-20T21:47:03.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.828+0000 7f87b0e00640 1 -- start start 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87b0e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f87ac152f80 0x7f87ac0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87b0e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f87ac153360 0x7f87ac072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87b0e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ac0c47c0 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87b0e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f87ac0c9630 con 0x7f87ac152f80 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87b0e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f87ac0c98f0 con 0x7f87ac153360 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87aa800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f87ac0c9dc0 0x7f87ac0c6130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43634/0 (socket says 172.21.15.102:43634) 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87a9400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f87ac153360 0x7f87ac072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87aa800640 1 -- 172.21.15.102:0/479632809 learned_addr learned my addr 172.21.15.102:0/479632809 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:03.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.831+0000 7f87a9e00640 1 --2- 172.21.15.102:0/479632809 >> v2:172.21.15.102:3300/0 conn(0x7f87ac152f80 0x7f87ac0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.832+0000 7f87a9e00640 1 -- 172.21.15.102:0/479632809 >> v2:172.21.15.193:3300/0 conn(0x7f87ac153360 msgr2=0x7f87ac072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.832+0000 7f87a9e00640 1 --2- 172.21.15.102:0/479632809 >> v2:172.21.15.193:3300/0 conn(0x7f87ac153360 0x7f87ac072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.832+0000 7f87a9e00640 1 -- 172.21.15.102:0/479632809 >> v1:172.21.15.102:6789/0 conn(0x7f87ac0c9dc0 legacy=0x7f87ac0c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.832+0000 7f87a9e00640 1 -- 172.21.15.102:0/479632809 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac0c8960 con 0x7f87ac152f80 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.832+0000 7f87a8a00640 1 -- 172.21.15.102:0/479632809 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248223962 0 0) 0x7f87ac0c47c0 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.833+0000 7f87a9e00640 1 --2- 172.21.15.102:0/479632809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac152f80 0x7f87ac0c7c40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0c7340 tx=0x7f879c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e9bd44f96e648c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.833+0000 7f87a8a00640 1 -- 172.21.15.102:0/479632809 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c035020 con 0x7f87ac152f80 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.833+0000 7f87a8a00640 1 -- 172.21.15.102:0/479632809 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c031830 con 0x7f87ac152f80 2024-04-20T21:47:03.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.833+0000 7f87a8a00640 1 -- 172.21.15.102:0/479632809 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c031c00 con 0x7f87ac152f80 2024-04-20T21:47:03.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.834+0000 7f87b0e00640 1 -- 172.21.15.102:0/479632809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac152f80 msgr2=0x7f87ac0c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.834+0000 7f87b0e00640 1 --2- 172.21.15.102:0/479632809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac152f80 0x7f87ac0c7c40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0c7340 tx=0x7f879c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.834+0000 7f87b0e00640 1 -- 172.21.15.102:0/479632809 shutdown_connections 2024-04-20T21:47:03.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.834+0000 7f87b0e00640 1 --2- 172.21.15.102:0/479632809 >> v2:172.21.15.193:3300/0 conn(0x7f87ac153360 0x7f87ac072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.834+0000 7f87b0e00640 1 --2- 172.21.15.102:0/479632809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac152f80 0x7f87ac0c7c40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.834+0000 7f87b0e00640 1 -- 172.21.15.102:0/479632809 >> 172.21.15.102:0/479632809 conn(0x7f87ac06cf20 msgr2=0x7f87ac06e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:03.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.834+0000 7f87b0e00640 1 -- 172.21.15.102:0/479632809 shutdown_connections 2024-04-20T21:47:03.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.834+0000 7f87b0e00640 1 -- 172.21.15.102:0/479632809 wait complete. 2024-04-20T21:47:03.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.836+0000 7f87b0e00640 1 Processor -- start 2024-04-20T21:47:03.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.836+0000 7f87b0e00640 1 -- start start 2024-04-20T21:47:03.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87b0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac0c9dc0 0x7f87ac0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87b0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87ac152f80 0x7f87ac151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87b0e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87ac153360 0x7f87ac1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87b0e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac0c69c0 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87b0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac0c6840 con 0x7f87ac152f80 2024-04-20T21:47:03.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87b0e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac14bf00 con 0x7f87ac153360 2024-04-20T21:47:03.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac0c9dc0 0x7f87ac0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87ac152f80 0x7f87ac151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac0c9dc0 0x7f87ac0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41512/0 (socket says 172.21.15.102:41512) 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9e00640 1 -- 172.21.15.102:0/766403938 learned_addr learned my addr 172.21.15.102:0/766403938 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87aa800640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87ac153360 0x7f87ac1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9e00640 1 -- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87ac153360 msgr2=0x7f87ac1401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87ac153360 0x7f87ac1401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9e00640 1 -- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87ac152f80 msgr2=0x7f87ac151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87ac152f80 0x7f87ac151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.837+0000 7f87a9e00640 1 -- 172.21.15.102:0/766403938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac149f00 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.838+0000 7f87a9e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac0c9dc0 0x7f87ac0cbc00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f879c034040 tx=0x7f879c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.838+0000 7f87a2000640 1 -- 172.21.15.102:0/766403938 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c035040 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.838+0000 7f87a2000640 1 -- 172.21.15.102:0/766403938 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c004230 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.838+0000 7f87a2000640 1 -- 172.21.15.102:0/766403938 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c03a640 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.838+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ac14a1e0 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.838+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ac14c430 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.839+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ac04fcf0 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.840+0000 7f87a2000640 1 -- 172.21.15.102:0/766403938 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f879c038020 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.841+0000 7f87a2000640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f878c06c920 0x7f878c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:03.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.841+0000 7f87a2000640 1 -- 172.21.15.102:0/766403938 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7f879c0ae1d0 con 0x7f87ac0c9dc0 2024-04-20T21:47:03.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.841+0000 7f87a9400640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f878c06c920 0x7f878c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:03.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.842+0000 7f87a9400640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f878c06c920 0x7f878c06ede0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f87ac069520 tx=0x7f8798009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:03.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:03.846+0000 7f87a2000640 1 -- 172.21.15.102:0/766403938 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f879c07b4f0 con 0x7f87ac0c9dc0 2024-04-20T21:47:04.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.053+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f87ac1568c0 con 0x7f87ac0c9dc0 2024-04-20T21:47:04.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.053+0000 7f87a2000640 1 -- 172.21.15.102:0/766403938 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7f879c07f7f0 con 0x7f87ac0c9dc0 2024-04-20T21:47:04.053 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.056+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f878c06c920 msgr2=0x7f878c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.056+0000 7f87b0e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f878c06c920 0x7f878c06ede0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f87ac069520 tx=0x7f8798009190 comp rx=0 tx=0).stop 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.056+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac0c9dc0 msgr2=0x7f87ac0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.056+0000 7f87b0e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac0c9dc0 0x7f87ac0cbc00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f879c034040 tx=0x7f879c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.056+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 shutdown_connections 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.056+0000 7f87b0e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87ac153360 0x7f87ac1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.057+0000 7f87b0e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f878c06c920 0x7f878c06ede0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.057+0000 7f87b0e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87ac152f80 0x7f87ac151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.057+0000 7f87b0e00640 1 --2- 172.21.15.102:0/766403938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87ac0c9dc0 0x7f87ac0cbc00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.057+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 >> 172.21.15.102:0/766403938 conn(0x7f87ac06cf20 msgr2=0x7f87ac067730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:04.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.057+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 shutdown_connections 2024-04-20T21:47:04.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.057+0000 7f87b0e00640 1 -- 172.21.15.102:0/766403938 wait complete. 2024-04-20T21:47:04.073 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:47:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.261+0000 7efd65200640 1 Processor -- start 2024-04-20T21:47:04.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.261+0000 7efd65200640 1 -- start start 2024-04-20T21:47:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd65200640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efd60153210 0x7efd6015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd65200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd600c6ba0 0x7efd60159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd65200640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd600c44d0 con 0x7efd600c9eb0 2024-04-20T21:47:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd65200640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7efd601596f0 con 0x7efd60153210 2024-04-20T21:47:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd65200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efd600c47b0 con 0x7efd600c6ba0 2024-04-20T21:47:04.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd5de00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efd60153210 0x7efd6015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd5e800640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efd600c9eb0 0x7efd60072fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43640/0 (socket says 172.21.15.102:43640) 2024-04-20T21:47:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd5e800640 1 -- 172.21.15.102:0/1663159982 learned_addr learned my addr 172.21.15.102:0/1663159982 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.263+0000 7efd5d400640 1 --2- 172.21.15.102:0/1663159982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd600c6ba0 0x7efd60159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.264+0000 7efd5d400640 1 -- 172.21.15.102:0/1663159982 >> v2:172.21.15.102:3300/0 conn(0x7efd60153210 msgr2=0x7efd6015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:47:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.264+0000 7efd5d400640 1 --2- 172.21.15.102:0/1663159982 >> v2:172.21.15.102:3300/0 conn(0x7efd60153210 0x7efd6015c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.264+0000 7efd5d400640 1 -- 172.21.15.102:0/1663159982 >> v1:172.21.15.102:6789/0 conn(0x7efd600c9eb0 legacy=0x7efd60072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.264+0000 7efd5d400640 1 -- 172.21.15.102:0/1663159982 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd600c4a90 con 0x7efd600c6ba0 2024-04-20T21:47:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.265+0000 7efd5d400640 1 --2- 172.21.15.102:0/1663159982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd600c6ba0 0x7efd60159b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0097a0 tx=0x7efd4c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebfe226cc3173d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.265+0000 7efd5ca00640 1 -- 172.21.15.102:0/1663159982 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c034070 con 0x7efd600c6ba0 2024-04-20T21:47:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.265+0000 7efd5ca00640 1 -- 172.21.15.102:0/1663159982 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4c030800 con 0x7efd600c6ba0 2024-04-20T21:47:04.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.265+0000 7efd5ca00640 1 -- 172.21.15.102:0/1663159982 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c030bd0 con 0x7efd600c6ba0 2024-04-20T21:47:04.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.266+0000 7efd65200640 1 -- 172.21.15.102:0/1663159982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd600c6ba0 msgr2=0x7efd60159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.266+0000 7efd65200640 1 --2- 172.21.15.102:0/1663159982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd600c6ba0 0x7efd60159b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0097a0 tx=0x7efd4c02b940 comp rx=0 tx=0).stop 2024-04-20T21:47:04.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.266+0000 7efd65200640 1 -- 172.21.15.102:0/1663159982 shutdown_connections 2024-04-20T21:47:04.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.266+0000 7efd65200640 1 --2- 172.21.15.102:0/1663159982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd600c6ba0 0x7efd60159b80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.266+0000 7efd65200640 1 --2- 172.21.15.102:0/1663159982 >> v2:172.21.15.102:3300/0 conn(0x7efd60153210 0x7efd6015c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.266+0000 7efd65200640 1 -- 172.21.15.102:0/1663159982 >> 172.21.15.102:0/1663159982 conn(0x7efd600c9390 msgr2=0x7efd600c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:04.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.266+0000 7efd65200640 1 -- 172.21.15.102:0/1663159982 shutdown_connections 2024-04-20T21:47:04.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.266+0000 7efd65200640 1 -- 172.21.15.102:0/1663159982 wait complete. 2024-04-20T21:47:04.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.268+0000 7efd65200640 1 Processor -- start 2024-04-20T21:47:04.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.268+0000 7efd65200640 1 -- start start 2024-04-20T21:47:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd65200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd600c6ba0 0x7efd6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd65200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd600c9eb0 0x7efd60148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd65200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd60153210 0x7efd60148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd65200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd600736d0 con 0x7efd600c6ba0 2024-04-20T21:47:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd65200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efd60159030 con 0x7efd600c9eb0 2024-04-20T21:47:04.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd65200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7efd60073850 con 0x7efd60153210 2024-04-20T21:47:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd5d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd600c9eb0 0x7efd60148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd5e800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd60153210 0x7efd60148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd5d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd600c9eb0 0x7efd60148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51242/0 (socket says 172.21.15.102:51242) 2024-04-20T21:47:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd5d400640 1 -- 172.21.15.102:0/1084355640 learned_addr learned my addr 172.21.15.102:0/1084355640 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:04.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.269+0000 7efd5de00640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd600c6ba0 0x7efd6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd5d400640 1 -- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd60153210 msgr2=0x7efd60148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd5d400640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd60153210 0x7efd60148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd5d400640 1 -- 172.21.15.102:0/1084355640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd600c6ba0 msgr2=0x7efd6006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd5d400640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd600c6ba0 0x7efd6006e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd5d400640 1 -- 172.21.15.102:0/1084355640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd601e2540 con 0x7efd600c9eb0 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd5d400640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd600c9eb0 0x7efd60148110 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efd4c02bf50 tx=0x7efd4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd56000640 1 -- 172.21.15.102:0/1084355640 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c048070 con 0x7efd600c9eb0 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd56000640 1 -- 172.21.15.102:0/1084355640 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4c030600 con 0x7efd600c9eb0 2024-04-20T21:47:04.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd56000640 1 -- 172.21.15.102:0/1084355640 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c034040 con 0x7efd600c9eb0 2024-04-20T21:47:04.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.270+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd601e27e0 con 0x7efd600c9eb0 2024-04-20T21:47:04.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.271+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd601e2da0 con 0x7efd600c9eb0 2024-04-20T21:47:04.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.272+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd60157db0 con 0x7efd600c9eb0 2024-04-20T21:47:04.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.273+0000 7efd56000640 1 -- 172.21.15.102:0/1084355640 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd4c04b070 con 0x7efd600c9eb0 2024-04-20T21:47:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.274+0000 7efd56000640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.274+0000 7efd56000640 1 -- 172.21.15.102:0/1084355640 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7efd4c0ae0e0 con 0x7efd600c9eb0 2024-04-20T21:47:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.274+0000 7efd5de00640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.274+0000 7efd5de00640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efd50009aa0 tx=0x7efd50009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:04.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.282+0000 7efd56000640 1 -- 172.21.15.102:0/1084355640 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efd60157db0 con 0x7efd600c9eb0 2024-04-20T21:47:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.479+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd60158370 con 0x7efd600c9eb0 2024-04-20T21:47:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.479+0000 7efd56000640 1 -- 172.21.15.102:0/1084355640 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7efd60158370 con 0x7efd600c9eb0 2024-04-20T21:47:04.479 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-20T21:47:04.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 msgr2=0x7efd4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efd50009aa0 tx=0x7efd50009190 comp rx=0 tx=0).stop 2024-04-20T21:47:04.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd600c9eb0 msgr2=0x7efd60148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd600c9eb0 0x7efd60148110 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efd4c02bf50 tx=0x7efd4c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 shutdown_connections 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efd60153210 0x7efd60148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7efd4006c8f0 0x7efd4006edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd600c9eb0 0x7efd60148110 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 --2- 172.21.15.102:0/1084355640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd600c6ba0 0x7efd6006e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 >> 172.21.15.102:0/1084355640 conn(0x7efd600c9390 msgr2=0x7efd6014b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 shutdown_connections 2024-04-20T21:47:04.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.482+0000 7efd65200640 1 -- 172.21.15.102:0/1084355640 wait complete. 2024-04-20T21:47:04.499 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T21:47:04.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.692+0000 7f54ccc00640 1 Processor -- start 2024-04-20T21:47:04.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.692+0000 7f54ccc00640 1 -- start start 2024-04-20T21:47:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54ccc00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f54c80c3b20 0x7f54c815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54ccc00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c806e370 con 0x7f54c806f0f0 2024-04-20T21:47:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54ccc00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c80c4a50 con 0x7f54c8151ee0 2024-04-20T21:47:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54ccc00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f54c806ec60 con 0x7f54c80c3b20 2024-04-20T21:47:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54bea00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f54c806f0f0 0x7f54c80c8130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43644/0 (socket says 172.21.15.102:43644) 2024-04-20T21:47:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54bea00640 1 -- 172.21.15.102:0/2927259962 learned_addr learned my addr 172.21.15.102:0/2927259962 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54c7400640 1 --1- 172.21.15.102:0/2927259962 >> v1:172.21.15.193:6789/0 conn(0x7f54c8151ee0 0x7f54c80c5010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:59876/0 (socket says 172.21.15.102:59876) 2024-04-20T21:47:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54c7e00640 1 --2- 172.21.15.102:0/2927259962 >> v2:172.21.15.193:3300/0 conn(0x7f54c80c3b20 0x7f54c815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54c6a00640 1 -- 172.21.15.102:0/2927259962 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633689431 0 0) 0x7f54c806e370 con 0x7f54c806f0f0 2024-04-20T21:47:04.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54c7e00640 1 -- 172.21.15.102:0/2927259962 >> v1:172.21.15.193:6789/0 conn(0x7f54c8151ee0 legacy=0x7f54c80c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54c7e00640 1 -- 172.21.15.102:0/2927259962 >> v1:172.21.15.102:6789/0 conn(0x7f54c806f0f0 legacy=0x7f54c80c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.694+0000 7f54c7e00640 1 -- 172.21.15.102:0/2927259962 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c80c4d30 con 0x7f54c80c3b20 2024-04-20T21:47:04.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.695+0000 7f54c7e00640 1 --2- 172.21.15.102:0/2927259962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c80c3b20 0x7f54c815bc90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009d90 tx=0x7f54c002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c96e2bf59ba9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54c6a00640 1 -- 172.21.15.102:0/2927259962 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0039070 con 0x7f54c80c3b20 2024-04-20T21:47:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54c6a00640 1 -- 172.21.15.102:0/2927259962 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c002cd00 con 0x7f54c80c3b20 2024-04-20T21:47:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54c6a00640 1 -- 172.21.15.102:0/2927259962 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0034660 con 0x7f54c80c3b20 2024-04-20T21:47:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54ccc00640 1 -- 172.21.15.102:0/2927259962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c80c3b20 msgr2=0x7f54c815bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54ccc00640 1 --2- 172.21.15.102:0/2927259962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c80c3b20 0x7f54c815bc90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009d90 tx=0x7f54c002bf50 comp rx=0 tx=0).stop 2024-04-20T21:47:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54ccc00640 1 -- 172.21.15.102:0/2927259962 shutdown_connections 2024-04-20T21:47:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54ccc00640 1 --2- 172.21.15.102:0/2927259962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c80c3b20 0x7f54c815bc90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54ccc00640 1 -- 172.21.15.102:0/2927259962 >> 172.21.15.102:0/2927259962 conn(0x7f54c80cc620 msgr2=0x7f54c80cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.696+0000 7f54ccc00640 1 -- 172.21.15.102:0/2927259962 shutdown_connections 2024-04-20T21:47:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.697+0000 7f54ccc00640 1 -- 172.21.15.102:0/2927259962 wait complete. 2024-04-20T21:47:04.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.698+0000 7f54ccc00640 1 Processor -- start 2024-04-20T21:47:04.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.698+0000 7f54ccc00640 1 -- start start 2024-04-20T21:47:04.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54ccc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c806f0f0 0x7f54c81f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c80c3b20 0x7f54c81594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54ccc00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54c8151ee0 0x7f54c81e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54ccc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f54c80c8840 con 0x7f54c80c3b20 2024-04-20T21:47:04.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54ccc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54c806e8f0 con 0x7f54c806f0f0 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54ccc00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f54c80c89c0 con 0x7f54c8151ee0 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54c7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c806f0f0 0x7f54c81f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54bea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c80c3b20 0x7f54c81594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54c7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c806f0f0 0x7f54c81f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:51254/0 (socket says 172.21.15.102:51254) 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54c7400640 1 -- 172.21.15.102:0/3009807619 learned_addr learned my addr 172.21.15.102:0/3009807619 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.699+0000 7f54c7e00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54c8151ee0 0x7f54c81e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54c7400640 1 -- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54c8151ee0 msgr2=0x7f54c81e32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54c7400640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54c8151ee0 0x7f54c81e32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54c7400640 1 -- 172.21.15.102:0/3009807619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c80c3b20 msgr2=0x7f54c81594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54c7400640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c80c3b20 0x7f54c81594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54c7400640 1 -- 172.21.15.102:0/3009807619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c815bae0 con 0x7f54c806f0f0 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54c7400640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c806f0f0 0x7f54c81f1620 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c3e0 tx=0x7f54b800c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54bfe00640 1 -- 172.21.15.102:0/3009807619 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80074e0 con 0x7f54c806f0f0 2024-04-20T21:47:04.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54bfe00640 1 -- 172.21.15.102:0/3009807619 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b8007680 con 0x7f54c806f0f0 2024-04-20T21:47:04.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c815bda0 con 0x7f54c806f0f0 2024-04-20T21:47:04.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.700+0000 7f54bfe00640 1 -- 172.21.15.102:0/3009807619 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80049e0 con 0x7f54c806f0f0 2024-04-20T21:47:04.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.701+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c81e13a0 con 0x7f54c806f0f0 2024-04-20T21:47:04.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.702+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c814bf60 con 0x7f54c806f0f0 2024-04-20T21:47:04.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.702+0000 7f54bfe00640 1 -- 172.21.15.102:0/3009807619 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54b801a020 con 0x7f54c806f0f0 2024-04-20T21:47:04.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.703+0000 7f54bfe00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54a406c8f0 0x7f54a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:04.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.703+0000 7f54bfe00640 1 -- 172.21.15.102:0/3009807619 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7f54b808bc90 con 0x7f54c806f0f0 2024-04-20T21:47:04.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.704+0000 7f54bea00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54a406c8f0 0x7f54a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:04.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.704+0000 7f54bea00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54a406c8f0 0x7f54a406edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f54b0007f70 tx=0x7f54b0007f00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:04.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.708+0000 7f54bfe00640 1 -- 172.21.15.102:0/3009807619 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f54c814bf60 con 0x7f54c806f0f0 2024-04-20T21:47:04.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.909+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f54c8073350 con 0x7f54c806f0f0 2024-04-20T21:47:04.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.909+0000 7f54bfe00640 1 -- 172.21.15.102:0/3009807619 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v66) ==== 103+0+11 (secure 0 0 0) 0x7f54c8073350 con 0x7f54c806f0f0 2024-04-20T21:47:04.908 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-20T21:47:04.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54a406c8f0 msgr2=0x7f54a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54a406c8f0 0x7f54a406edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f54b0007f70 tx=0x7f54b0007f00 comp rx=0 tx=0).stop 2024-04-20T21:47:04.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c806f0f0 msgr2=0x7f54c81f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c806f0f0 0x7f54c81f1620 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c3e0 tx=0x7f54b800c8a0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 shutdown_connections 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f54c8151ee0 0x7f54c81e32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c80c3b20 0x7f54c81594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f54a406c8f0 0x7f54a406edb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 --2- 172.21.15.102:0/3009807619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c806f0f0 0x7f54c81f1620 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 >> 172.21.15.102:0/3009807619 conn(0x7f54c80cc620 msgr2=0x7f54c814ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 shutdown_connections 2024-04-20T21:47:04.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:04.912+0000 7f54ccc00640 1 -- 172.21.15.102:0/3009807619 wait complete. 2024-04-20T21:47:04.931 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:47:05.083 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:47:05.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.139+0000 7f2248a00640 1 Processor -- start 2024-04-20T21:47:05.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.139+0000 7f2248a00640 1 -- start start 2024-04-20T21:47:05.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.140+0000 7f2248a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f224415e890 0x7f22440cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.140+0000 7f2248a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22440cdb30 0x7f22440cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.140+0000 7f2248a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22440ccdd0 con 0x7f22440cb980 2024-04-20T21:47:05.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.140+0000 7f2248a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f224415ed00 con 0x7f22440cdb30 2024-04-20T21:47:05.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.140+0000 7f2248a00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f224415f7e0 con 0x7f224415e890 2024-04-20T21:47:05.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2242800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22440cdb30 0x7f22440cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2242800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f22440cdb30 0x7f22440cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41546/0 (socket says 172.21.15.102:41546) 2024-04-20T21:47:05.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2242800640 1 -- 172.21.15.102:0/1294348491 learned_addr learned my addr 172.21.15.102:0/1294348491 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:05.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2241400640 1 --2- 172.21.15.102:0/1294348491 >> v2:172.21.15.193:3300/0 conn(0x7f224415e890 0x7f22440cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2241e00640 1 --1- 172.21.15.102:0/1294348491 >> v1:172.21.15.102:6789/0 conn(0x7f22440cb980 0x7f22440cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43660/0 (socket says 172.21.15.102:43660) 2024-04-20T21:47:05.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2242800640 1 -- 172.21.15.102:0/1294348491 >> v2:172.21.15.193:3300/0 conn(0x7f224415e890 msgr2=0x7f22440cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:47:05.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2242800640 1 --2- 172.21.15.102:0/1294348491 >> v2:172.21.15.193:3300/0 conn(0x7f224415e890 0x7f22440cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2242800640 1 -- 172.21.15.102:0/1294348491 >> v1:172.21.15.102:6789/0 conn(0x7f22440cb980 legacy=0x7f22440cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.141+0000 7f2242800640 1 -- 172.21.15.102:0/1294348491 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224415ef40 con 0x7f22440cdb30 2024-04-20T21:47:05.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.142+0000 7f2242800640 1 --2- 172.21.15.102:0/1294348491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cdb30 0x7f22440cdfa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f22440ce6a0 tx=0x7f223c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aed036ed146916c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.142+0000 7f2240a00640 1 -- 172.21.15.102:0/1294348491 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c02cb10 con 0x7f22440cdb30 2024-04-20T21:47:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2240a00640 1 -- 172.21.15.102:0/1294348491 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f223c02ccb0 con 0x7f22440cdb30 2024-04-20T21:47:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2240a00640 1 -- 172.21.15.102:0/1294348491 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c02ce60 con 0x7f22440cdb30 2024-04-20T21:47:05.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2248a00640 1 -- 172.21.15.102:0/1294348491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cdb30 msgr2=0x7f22440cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2248a00640 1 --2- 172.21.15.102:0/1294348491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cdb30 0x7f22440cdfa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f22440ce6a0 tx=0x7f223c02bf70 comp rx=0 tx=0).stop 2024-04-20T21:47:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2248a00640 1 -- 172.21.15.102:0/1294348491 shutdown_connections 2024-04-20T21:47:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2248a00640 1 --2- 172.21.15.102:0/1294348491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cdb30 0x7f22440cdfa0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2248a00640 1 --2- 172.21.15.102:0/1294348491 >> v2:172.21.15.193:3300/0 conn(0x7f224415e890 0x7f22440cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2248a00640 1 -- 172.21.15.102:0/1294348491 >> 172.21.15.102:0/1294348491 conn(0x7f2244075080 msgr2=0x7f2244075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2248a00640 1 -- 172.21.15.102:0/1294348491 shutdown_connections 2024-04-20T21:47:05.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.143+0000 7f2248a00640 1 -- 172.21.15.102:0/1294348491 wait complete. 2024-04-20T21:47:05.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.145+0000 7f2248a00640 1 Processor -- start 2024-04-20T21:47:05.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.145+0000 7f2248a00640 1 -- start start 2024-04-20T21:47:05.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.146+0000 7f2248a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cb980 0x7f22441f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.146+0000 7f2248a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22440cdb30 0x7f22441e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.146+0000 7f2248a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f224415e890 0x7f224415d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.146+0000 7f2248a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22440cd330 con 0x7f22440cb980 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.146+0000 7f2248a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f224415f500 con 0x7f22440cdb30 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.146+0000 7f2248a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f22440cd610 con 0x7f224415e890 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.146+0000 7f2241e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cb980 0x7f22441f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2242800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f224415e890 0x7f224415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22440cdb30 0x7f22441e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cb980 0x7f22441f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41558/0 (socket says 172.21.15.102:41558) 2024-04-20T21:47:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241e00640 1 -- 172.21.15.102:0/136363974 learned_addr learned my addr 172.21.15.102:0/136363974 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:05.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241e00640 1 -- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f224415e890 msgr2=0x7f224415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241e00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f224415e890 0x7f224415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241e00640 1 -- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22440cdb30 msgr2=0x7f22441e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241e00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22440cdb30 0x7f22441e5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241e00640 1 -- 172.21.15.102:0/136363974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22441e48b0 con 0x7f22440cb980 2024-04-20T21:47:05.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.147+0000 7f2241e00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cb980 0x7f22441f2730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f223400c3c0 tx=0x7f223400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:05.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.148+0000 7f223a000640 1 -- 172.21.15.102:0/136363974 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340074e0 con 0x7f22440cb980 2024-04-20T21:47:05.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.148+0000 7f223a000640 1 -- 172.21.15.102:0/136363974 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2234007680 con 0x7f22440cb980 2024-04-20T21:47:05.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.148+0000 7f223a000640 1 -- 172.21.15.102:0/136363974 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234004a00 con 0x7f22440cb980 2024-04-20T21:47:05.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.148+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22441e4b70 con 0x7f22440cb980 2024-04-20T21:47:05.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.148+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22441e5130 con 0x7f22440cb980 2024-04-20T21:47:05.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.149+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2244074340 con 0x7f22440cb980 2024-04-20T21:47:05.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.150+0000 7f223a000640 1 -- 172.21.15.102:0/136363974 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f223401a020 con 0x7f22440cb980 2024-04-20T21:47:05.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.151+0000 7f223a000640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f222406c920 0x7f222406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.152+0000 7f223a000640 1 -- 172.21.15.102:0/136363974 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7f223408bce0 con 0x7f22440cb980 2024-04-20T21:47:05.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.152+0000 7f2241400640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f222406c920 0x7f222406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.152+0000 7f2241400640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f222406c920 0x7f222406ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f22440cca00 tx=0x7f2230009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:05.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.156+0000 7f223a000640 1 -- 172.21.15.102:0/136363974 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2234058fa0 con 0x7f22440cb980 2024-04-20T21:47:05.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.276+0000 7fa24f200640 1 Processor -- start 2024-04-20T21:47:05.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.276+0000 7fa24f200640 1 -- start start 2024-04-20T21:47:05.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24f200640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa250155530 0x7fa25015e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cc470 0x7fa2500cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24f200640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa25015ef60 con 0x7fa2500cdfc0 2024-04-20T21:47:05.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24f200640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa2500c5ca0 con 0x7fa250155530 2024-04-20T21:47:05.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24f200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa2500cd5f0 con 0x7fa2500cc470 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24e800640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fa2500cdfc0 0x7fa2500cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:59878/0 (socket says 172.21.15.102:59878) 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cc470 0x7fa2500cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24e800640 1 -- 172.21.15.102:0/386071817 learned_addr learned my addr 172.21.15.102:0/386071817 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24de00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa250155530 0x7fa25015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.278+0000 7fa24ca00640 1 -- 172.21.15.102:0/386071817 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369953126 0 0) 0x7fa25015ef60 con 0x7fa2500cdfc0 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.279+0000 7fa24ca00640 1 -- 172.21.15.102:0/386071817 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa234003520 con 0x7fa2500cdfc0 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.279+0000 7fa24d400640 1 -- 172.21.15.102:0/386071817 >> v2:172.21.15.193:3300/0 conn(0x7fa250155530 msgr2=0x7fa25015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.279+0000 7fa24d400640 1 --2- 172.21.15.102:0/386071817 >> v2:172.21.15.193:3300/0 conn(0x7fa250155530 0x7fa25015e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.279+0000 7fa24d400640 1 -- 172.21.15.102:0/386071817 >> v1:172.21.15.193:6789/0 conn(0x7fa2500cdfc0 legacy=0x7fa2500cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.279+0000 7fa24d400640 1 -- 172.21.15.102:0/386071817 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25015f240 con 0x7fa2500cc470 2024-04-20T21:47:05.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.279+0000 7fa24de00640 1 --2- 172.21.15.102:0/386071817 >> v2:172.21.15.193:3300/0 conn(0x7fa250155530 0x7fa25015e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.280+0000 7fa24d400640 1 --2- 172.21.15.102:0/386071817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cc470 0x7fa2500cda80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0097a0 tx=0x7fa23c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108c657420800228 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.280+0000 7fa24ca00640 1 -- 172.21.15.102:0/386071817 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c035020 con 0x7fa2500cc470 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.280+0000 7fa24ca00640 1 -- 172.21.15.102:0/386071817 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23c031810 con 0x7fa2500cc470 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.280+0000 7fa24ca00640 1 -- 172.21.15.102:0/386071817 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c031be0 con 0x7fa2500cc470 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.281+0000 7fa24f200640 1 -- 172.21.15.102:0/386071817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cc470 msgr2=0x7fa2500cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.281+0000 7fa24f200640 1 --2- 172.21.15.102:0/386071817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cc470 0x7fa2500cda80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0097a0 tx=0x7fa23c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.281+0000 7fa24f200640 1 -- 172.21.15.102:0/386071817 shutdown_connections 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.281+0000 7fa24f200640 1 --2- 172.21.15.102:0/386071817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cc470 0x7fa2500cda80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.281+0000 7fa24f200640 1 --2- 172.21.15.102:0/386071817 >> v2:172.21.15.193:3300/0 conn(0x7fa250155530 0x7fa25015e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.281+0000 7fa24f200640 1 -- 172.21.15.102:0/386071817 >> 172.21.15.102:0/386071817 conn(0x7fa250075080 msgr2=0x7fa250075490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.281+0000 7fa24f200640 1 -- 172.21.15.102:0/386071817 shutdown_connections 2024-04-20T21:47:05.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.281+0000 7fa24f200640 1 -- 172.21.15.102:0/386071817 wait complete. 2024-04-20T21:47:05.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.282+0000 7fa24f200640 1 Processor -- start 2024-04-20T21:47:05.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.283+0000 7fa24f200640 1 -- start start 2024-04-20T21:47:05.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.283+0000 7fa24f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2500cc470 0x7fa2501f1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.283+0000 7fa24f200640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cdfc0 0x7fa2500cff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.283+0000 7fa24f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa250155530 0x7fa2500d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.283+0000 7fa24f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa2500cd280 con 0x7fa2500cc470 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.283+0000 7fa24f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2500ccfa0 con 0x7fa250155530 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.283+0000 7fa24f200640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa2500ccbe0 con 0x7fa2500cdfc0 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cdfc0 0x7fa2500cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cdfc0 0x7fa2500cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.102:50374/0 (socket says 172.21.15.102:50374) 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 -- 172.21.15.102:0/2402290509 learned_addr learned my addr 172.21.15.102:0/2402290509 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24e800640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa250155530 0x7fa2500d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24de00640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2500cc470 0x7fa2501f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 -- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa250155530 msgr2=0x7fa2500d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa250155530 0x7fa2500d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 -- 172.21.15.102:0/2402290509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2500cc470 msgr2=0x7fa2501f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2500cc470 0x7fa2501f1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 -- 172.21.15.102:0/2402290509 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2501f1e40 con 0x7fa2500cdfc0 2024-04-20T21:47:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.284+0000 7fa24d400640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cdfc0 0x7fa2500cff10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009890 tx=0x7fa23c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.285+0000 7fa246000640 1 -- 172.21.15.102:0/2402290509 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c035040 con 0x7fa2500cdfc0 2024-04-20T21:47:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.285+0000 7fa246000640 1 -- 172.21.15.102:0/2402290509 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23c03ad30 con 0x7fa2500cdfc0 2024-04-20T21:47:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.285+0000 7fa246000640 1 -- 172.21.15.102:0/2402290509 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0466a0 con 0x7fa2500cdfc0 2024-04-20T21:47:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.285+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2500cf7b0 con 0x7fa2500cdfc0 2024-04-20T21:47:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.285+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa250148ae0 con 0x7fa2500cdfc0 2024-04-20T21:47:05.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.286+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25004fd60 con 0x7fa2500cdfc0 2024-04-20T21:47:05.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.287+0000 7fa246000640 1 -- 172.21.15.102:0/2402290509 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa23c038020 con 0x7fa2500cdfc0 2024-04-20T21:47:05.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.288+0000 7fa246000640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa23006c920 0x7fa23006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:05.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.288+0000 7fa246000640 1 -- 172.21.15.102:0/2402290509 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7fa23c0ae6e0 con 0x7fa2500cdfc0 2024-04-20T21:47:05.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.288+0000 7fa24de00640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa23006c920 0x7fa23006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:05.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.289+0000 7fa24de00640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa23006c920 0x7fa23006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa240009560 tx=0x7fa240009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.293+0000 7fa246000640 1 -- 172.21.15.102:0/2402290509 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa23c07b910 con 0x7fa2500cdfc0 2024-04-20T21:47:05.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.375+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f22440c7830 con 0x7f22440cb980 2024-04-20T21:47:05.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.376+0000 7f223a000640 1 -- 172.21.15.102:0/136363974 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 38 entries v66) ==== 81+0+2346 (secure 0 0 0) 0x7f223405d2a0 con 0x7f22440cb980 2024-04-20T21:47:05.375 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.193:6835/2467124512 2024-04-21T21:46:10.944621+0000 2024-04-20T21:47:05.375 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/2976263880 2024-04-21T21:45:10.928031+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.193:6836/2076039362 2024-04-21T21:43:40.902616+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6834/36459307 2024-04-21T21:31:25.670488+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/713712414 2024-04-21T21:23:30.536944+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6835/2873333699 2024-04-21T21:19:40.461600+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/2239658043 2024-04-21T21:28:35.625696+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6834/1017522496 2024-04-21T21:22:20.501478+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/2239658043 2024-04-21T21:28:35.625696+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/1043030921 2024-04-21T21:33:05.702536+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6834/1097388930 2024-04-21T21:26:10.588429+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/2976263880 2024-04-21T21:45:10.928031+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/327932375 2024-04-21T21:39:30.821390+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6834/945751373 2024-04-21T21:38:20.801433+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.193:6838/2467124512 2024-04-21T21:46:10.944621+0000 2024-04-20T21:47:05.376 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/2795763369 2024-04-21T21:21:20.485686+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6835/1097388930 2024-04-21T21:26:10.588429+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6835/1017522496 2024-04-21T21:22:20.501478+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6835/2459661167 2024-04-21T21:42:55.884532+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/2795763369 2024-04-21T21:21:20.485686+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/1043030921 2024-04-21T21:33:05.702536+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6834/2655303061 2024-04-21T21:34:50.733957+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/713712414 2024-04-21T21:23:30.536944+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/327932375 2024-04-21T21:39:30.821390+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6835/2655303061 2024-04-21T21:34:50.733957+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/3985712430 2024-04-21T21:35:50.750525+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6835/36459307 2024-04-21T21:31:25.670488+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/2637555410 2024-04-21T21:42:10.872593+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/1356782221 2024-04-21T21:46:55.958136+0000 2024-04-20T21:47:05.377 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6834/2873333699 2024-04-21T21:19:40.461600+0000 2024-04-20T21:47:05.378 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/3985712430 2024-04-21T21:35:50.750525+0000 2024-04-20T21:47:05.378 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6835/945751373 2024-04-21T21:38:20.801433+0000 2024-04-20T21:47:05.378 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6834/2459661167 2024-04-21T21:42:55.884532+0000 2024-04-20T21:47:05.378 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6832/1356782221 2024-04-21T21:46:55.958136+0000 2024-04-20T21:47:05.378 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.193:6839/2076039362 2024-04-21T21:43:40.902616+0000 2024-04-20T21:47:05.378 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6834/3586768191 2024-04-21T21:41:25.854747+0000 2024-04-20T21:47:05.378 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6835/3586768191 2024-04-21T21:41:25.854747+0000 2024-04-20T21:47:05.378 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6833/2637555410 2024-04-21T21:42:10.872593+0000 2024-04-20T21:47:05.379 INFO:teuthology.orchestra.run.smithi102.stderr:listed 38 entries 2024-04-20T21:47:05.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f222406c920 msgr2=0x7f222406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f222406c920 0x7f222406ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f22440cca00 tx=0x7f2230009190 comp rx=0 tx=0).stop 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cb980 msgr2=0x7f22441f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cb980 0x7f22441f2730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f223400c3c0 tx=0x7f223400c880 comp rx=0 tx=0).stop 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 shutdown_connections 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f224415e890 0x7f224415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f222406c920 0x7f222406ede0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22440cdb30 0x7f22441e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 --2- 172.21.15.102:0/136363974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22440cb980 0x7f22441f2730 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 >> 172.21.15.102:0/136363974 conn(0x7f2244075080 msgr2=0x7f22440cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 shutdown_connections 2024-04-20T21:47:05.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.379+0000 7f2248a00640 1 -- 172.21.15.102:0/136363974 wait complete. 2024-04-20T21:47:05.397 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-20T21:47:05.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:47:05.397 DEBUG:teuthology.orchestra.run.smithi102:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-20T21:47:05.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.540+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2500cc9c0 con 0x7fa2500cdfc0 2024-04-20T21:47:05.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.541+0000 7fa246000640 1 -- 172.21.15.102:0/2402290509 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+3548 (secure 0 0 0) 0x7fa23c07fc10 con 0x7fa2500cdfc0 2024-04-20T21:47:05.540 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:47:05.540 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:46:55.958174+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:47:05.540 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 108 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa23006c920 msgr2=0x7fa23006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa23006c920 0x7fa23006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa240009560 tx=0x7fa240009190 comp rx=0 tx=0).stop 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cdfc0 msgr2=0x7fa2500cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cdfc0 0x7fa2500cff10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009890 tx=0x7fa23c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 shutdown_connections 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa23006c920 0x7fa23006ede0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa250155530 0x7fa2500d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa2500cdfc0 0x7fa2500cff10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 --2- 172.21.15.102:0/2402290509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2500cc470 0x7fa2501f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 >> 172.21.15.102:0/2402290509 conn(0x7fa250075080 msgr2=0x7fa250154190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.543+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 shutdown_connections 2024-04-20T21:47:05.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:05.544+0000 7fa24f200640 1 -- 172.21.15.102:0/2402290509 wait complete. 2024-04-20T21:47:05.567 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:47:05.790 INFO:teuthology.orchestra.run.smithi102.stderr:umount: /home/ubuntu/cephtest/mnt.0: target is busy. 2024-04-20T21:47:05.790 INFO:tasks.workunit.client.0.smithi102.stderr:warning: unable to access 'src/client/hadoop/.gitattributes': Input/output error 2024-04-20T21:47:05.792 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-20T21:47:05.793 DEBUG:tasks.cephfs.kernel_mount:Killing processes on client.0... 2024-04-20T21:47:05.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:47:05.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo PATH=/usr/sbin:$PATH lsof ; ps auxf 2024-04-20T21:47:05.824 INFO:tasks.workunit.client.0.smithi102.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh: line 24: 51649 Bus error (core dumped) git reset --hard $ver 2024-04-20T21:47:05.824 DEBUG:teuthology.orchestra.run:got remote process result: 135 2024-04-20T21:47:05.824 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2024-04-20T21:47:05.824 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-20T21:47:06.002 INFO:teuthology.orchestra.run.smithi102.stderr:lsof: WARNING: can't stat() ceph file system /home/ubuntu/cephtest/mnt.0 2024-04-20T21:47:06.002 INFO:teuthology.orchestra.run.smithi102.stderr: Output information may be incomplete. 2024-04-20T21:47:06.900 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi102 with status 135: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2024-04-20T21:47:07.125 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d4ff550fa1e6496383b27826043ebdfc Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi102 with status 135: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2024-04-20T21:47:07.127 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2024-04-20T21:47:07.136 INFO:tasks.mds_thrash:joining mds_thrasher 2024-04-20T21:47:07.568 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:47:07.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.762+0000 7fa413e00640 1 Processor -- start 2024-04-20T21:47:07.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.762+0000 7fa413e00640 1 -- start start 2024-04-20T21:47:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa413e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa4140b1f40 0x7fa41415c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa413e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa4140c7a40 0x7fa414165a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa413e00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4140caa30 con 0x7fa4140b2320 2024-04-20T21:47:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa413e00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fa4140c2190 con 0x7fa4140b1f40 2024-04-20T21:47:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa413e00640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7fa4140c98f0 con 0x7fa4140c7a40 2024-04-20T21:47:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa40be00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa4140b2320 0x7fa41415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43664/0 (socket says 172.21.15.102:43664) 2024-04-20T21:47:07.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa40be00640 1 -- 172.21.15.102:0/1107212796 learned_addr learned my addr 172.21.15.102:0/1107212796 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:07.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa413400640 1 --2- 172.21.15.102:0/1107212796 >> v2:172.21.15.193:3300/0 conn(0x7fa4140c7a40 0x7fa414165a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:07.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.764+0000 7fa412a00640 1 --2- 172.21.15.102:0/1107212796 >> v2:172.21.15.102:3300/0 conn(0x7fa4140b1f40 0x7fa41415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:07.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.765+0000 7fa412000640 1 -- 172.21.15.102:0/1107212796 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4109119618 0 0) 0x7fa4140caa30 con 0x7fa4140b2320 2024-04-20T21:47:07.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.765+0000 7fa413400640 1 -- 172.21.15.102:0/1107212796 >> v2:172.21.15.102:3300/0 conn(0x7fa4140b1f40 msgr2=0x7fa41415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:47:07.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.765+0000 7fa413400640 1 --2- 172.21.15.102:0/1107212796 >> v2:172.21.15.102:3300/0 conn(0x7fa4140b1f40 0x7fa41415c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:07.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.765+0000 7fa413400640 1 -- 172.21.15.102:0/1107212796 >> v1:172.21.15.102:6789/0 conn(0x7fa4140b2320 legacy=0x7fa41415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:07.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.765+0000 7fa413400640 1 -- 172.21.15.102:0/1107212796 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41415f250 con 0x7fa4140c7a40 2024-04-20T21:47:07.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.766+0000 7fa413400640 1 --2- 172.21.15.102:0/1107212796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140c7a40 0x7fa414165a70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa4140c2310 tx=0x7fa40c02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4624eabeaacca51d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:07.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.766+0000 7fa412000640 1 -- 172.21.15.102:0/1107212796 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c039070 con 0x7fa4140c7a40 2024-04-20T21:47:07.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.766+0000 7fa412000640 1 -- 172.21.15.102:0/1107212796 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c02cd00 con 0x7fa4140c7a40 2024-04-20T21:47:07.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.766+0000 7fa412000640 1 -- 172.21.15.102:0/1107212796 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c034660 con 0x7fa4140c7a40 2024-04-20T21:47:07.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.767+0000 7fa413e00640 1 -- 172.21.15.102:0/1107212796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140c7a40 msgr2=0x7fa414165a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:07.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.767+0000 7fa413e00640 1 --2- 172.21.15.102:0/1107212796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140c7a40 0x7fa414165a70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa4140c2310 tx=0x7fa40c02bf50 comp rx=0 tx=0).stop 2024-04-20T21:47:07.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.767+0000 7fa413e00640 1 -- 172.21.15.102:0/1107212796 shutdown_connections 2024-04-20T21:47:07.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.767+0000 7fa413e00640 1 --2- 172.21.15.102:0/1107212796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140c7a40 0x7fa414165a70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:07.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.767+0000 7fa413e00640 1 --2- 172.21.15.102:0/1107212796 >> v2:172.21.15.102:3300/0 conn(0x7fa4140b1f40 0x7fa41415c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:07.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.767+0000 7fa413e00640 1 -- 172.21.15.102:0/1107212796 >> 172.21.15.102:0/1107212796 conn(0x7fa4140c9b40 msgr2=0x7fa4140c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:07.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.767+0000 7fa413e00640 1 -- 172.21.15.102:0/1107212796 shutdown_connections 2024-04-20T21:47:07.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.767+0000 7fa413e00640 1 -- 172.21.15.102:0/1107212796 wait complete. 2024-04-20T21:47:07.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.768+0000 7fa413e00640 1 Processor -- start 2024-04-20T21:47:07.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.769+0000 7fa413e00640 1 -- start start 2024-04-20T21:47:07.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.769+0000 7fa413e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4140b1f40 0x7fa4141f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:07.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.769+0000 7fa413e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140b2320 0x7fa41406d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:07.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.769+0000 7fa413e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4140c7a40 0x7fa41406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:07.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.769+0000 7fa413e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa4140c9320 con 0x7fa4140b1f40 2024-04-20T21:47:07.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa413e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4140c9610 con 0x7fa4140b2320 2024-04-20T21:47:07.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa413e00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fa41415cb60 con 0x7fa4140c7a40 2024-04-20T21:47:07.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4140b1f40 0x7fa4141f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4140b1f40 0x7fa4141f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41582/0 (socket says 172.21.15.102:41582) 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 -- 172.21.15.102:0/4185086904 learned_addr learned my addr 172.21.15.102:0/4185086904 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa40be00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140b2320 0x7fa41406d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa413400640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4140c7a40 0x7fa41406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 -- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4140c7a40 msgr2=0x7fa41406d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4140c7a40 0x7fa41406d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 -- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140b2320 msgr2=0x7fa41406d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140b2320 0x7fa41406d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:07.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 -- 172.21.15.102:0/4185086904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4141f1000 con 0x7fa4140b1f40 2024-04-20T21:47:07.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa40be00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140b2320 0x7fa41406d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:47:07.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.770+0000 7fa412a00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4140b1f40 0x7fa4141f1aa0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa40400e8a0 tx=0x7fa40400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:07.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.771+0000 7fa40b400640 1 -- 172.21.15.102:0/4185086904 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404017070 con 0x7fa4140b1f40 2024-04-20T21:47:07.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.771+0000 7fa40b400640 1 -- 172.21.15.102:0/4185086904 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40400c940 con 0x7fa4140b1f40 2024-04-20T21:47:07.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.771+0000 7fa40b400640 1 -- 172.21.15.102:0/4185086904 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40400cd30 con 0x7fa4140b1f40 2024-04-20T21:47:07.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.771+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa414071ad0 con 0x7fa4140b1f40 2024-04-20T21:47:07.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.771+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4141f12a0 con 0x7fa4140b1f40 2024-04-20T21:47:07.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.772+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4140b1210 con 0x7fa4140b1f40 2024-04-20T21:47:07.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.773+0000 7fa40b400640 1 -- 172.21.15.102:0/4185086904 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa40401a070 con 0x7fa4140b1f40 2024-04-20T21:47:07.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.774+0000 7fa40b400640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3f006c8f0 0x7fa3f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:07.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.774+0000 7fa40b400640 1 -- 172.21.15.102:0/4185086904 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7fa40408c890 con 0x7fa4140b1f40 2024-04-20T21:47:07.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.774+0000 7fa40be00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3f006c8f0 0x7fa3f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:07.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.775+0000 7fa40be00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3f006c8f0 0x7fa3f006edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa41406ec70 tx=0x7fa3fc007de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:07.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:07.779+0000 7fa40b400640 1 -- 172.21.15.102:0/4185086904 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa4140b1210 con 0x7fa4140b1f40 2024-04-20T21:47:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.021+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa41404fd60 con 0x7fa4140b1f40 2024-04-20T21:47:08.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.022+0000 7fa40b400640 1 -- 172.21.15.102:0/4185086904 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+3553 (secure 0 0 0) 0x7fa404059ac0 con 0x7fa4140b1f40 2024-04-20T21:47:08.021 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:47:08.021 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":109,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:47:07.748327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":108,"state":"up:reconnect","state_seq":456,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:47:08.021 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 109 2024-04-20T21:47:08.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.024+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3f006c8f0 msgr2=0x7fa3f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:08.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.024+0000 7fa413e00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3f006c8f0 0x7fa3f006edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa41406ec70 tx=0x7fa3fc007de0 comp rx=0 tx=0).stop 2024-04-20T21:47:08.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4140b1f40 msgr2=0x7fa4141f1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:08.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4140b1f40 0x7fa4141f1aa0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa40400e8a0 tx=0x7fa40400ed60 comp rx=0 tx=0).stop 2024-04-20T21:47:08.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 shutdown_connections 2024-04-20T21:47:08.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4140c7a40 0x7fa41406d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:08.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fa3f006c8f0 0x7fa3f006edb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:08.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4140b2320 0x7fa41406d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:08.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 --2- 172.21.15.102:0/4185086904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4140b1f40 0x7fa4141f1aa0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:08.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 >> 172.21.15.102:0/4185086904 conn(0x7fa4140c9b40 msgr2=0x7fa4140c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:08.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 shutdown_connections 2024-04-20T21:47:08.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:08.025+0000 7fa413e00640 1 -- 172.21.15.102:0/4185086904 wait complete. 2024-04-20T21:47:08.041 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:47:08.608 INFO:teuthology.orchestra.run.smithi102.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-04-20T21:47:08.609 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.609 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.609 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-04-20T21:47:08.609 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 598426 62136 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-20T21:47:08.609 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.609 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.609 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.610 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.610 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.610 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.610 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.610 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.610 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 21219 62233 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-20T21:47:08.610 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-20T21:47:08.610 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.611 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-20T21:47:08.612 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 3w CHR 1,11 0t0 10 /dev/kmsg 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 4u a_inode 0,15 0 50 [eventpoll:5,6,8,9,11,14,15,16,17,18,19,44,45,52,60,75,80,86,94,105,106,107,108,109,110,111,113,115,120,122,125,126...] 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 5u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.613 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 6r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 7r DIR 0,26 0 1 /sys/fs/cgroup 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 8u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 9u a_inode 0,15 0 50 [eventpoll:10,13] 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 10r REG 0,21 0 6470 /proc/1/mountinfo 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 11r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 12u netlink 0t0 8471 AUDIT 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 13r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 14r REG 0,21 0 4026532078 /proc/swaps 2024-04-20T21:47:08.614 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 15u netlink 0t0 8463 KOBJECT_UEVENT 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 16u unix 0xffff88810abfbc00 0t0 8465 /run/systemd/private type=STREAM (LISTEN) 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 17u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 18r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 19r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 20u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 22u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 23u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 25u unix 0xffff88810abfe900 0t0 8516 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 26u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.616 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 27r CHR 10,235 0t0 110 /dev/autofs 2024-04-20T21:47:08.616 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 28u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.616 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 29u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.616 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 30u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.616 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 31u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.616 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 32u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.616 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 33u a_inode 0,15 0 50 bpf-prog 2024-04-20T21:47:08.616 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 44u unix 0xffff88811056a580 0t0 6478 /run/systemd/io.system.ManagedOOM type=STREAM (LISTEN) 2024-04-20T21:47:08.617 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 45u unix 0xffff888110569e00 0t0 6477 /run/systemd/userdb/io.systemd.DynamicUser type=STREAM (LISTEN) 2024-04-20T21:47:08.617 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 52u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.617 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 60u unix 0xffff888108507800 0t0 5895 type=STREAM (CONNECTED) 2024-04-20T21:47:08.617 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 75u unix 0xffff888110568000 0t0 6472 /run/systemd/notify type=DGRAM (CONNECTED) 2024-04-20T21:47:08.617 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 80u unix 0xffff888110568780 0t0 6473 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.617 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 81u unix 0xffff888110568f00 0t0 6474 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.617 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 84u unix 0xffff88810abfc380 0t0 8663 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2024-04-20T21:47:08.617 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 85u netlink 0t0 8491 KOBJECT_UEVENT 2024-04-20T21:47:08.618 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 86u unix 0xffff88810abfad00 0t0 8665 /run/.heim_org.h5l.kcm-socket type=STREAM (LISTEN) 2024-04-20T21:47:08.618 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 87u unix 0xffff88810abfe180 0t0 8489 /run/udev/control type=SEQPACKET (LISTEN) 2024-04-20T21:47:08.618 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 88u unix 0xffff888110428f00 0t0 106 /run/systemd/journal/socket type=DGRAM (CONNECTED) 2024-04-20T21:47:08.618 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 89u unix 0xffff888110429680 0t0 108 /run/systemd/journal/stdout type=STREAM (LISTEN) 2024-04-20T21:47:08.618 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 93u unix 0xffff888110428780 0t0 104 /run/systemd/journal/dev-log type=DGRAM (CONNECTED) 2024-04-20T21:47:08.618 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 94u unix 0xffff88810abfcb00 0t0 8475 /run/lvm/lvmpolld.socket type=STREAM (LISTEN) 2024-04-20T21:47:08.618 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 95u unix 0xffff88810abfd280 0t0 8477 /run/rpcbind.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.618 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 96u IPv4 7420 0t0 TCP *:sunrpc (LISTEN) 2024-04-20T21:47:08.619 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 97u IPv4 7426 0t0 UDP *:sunrpc 2024-04-20T21:47:08.619 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 98u IPv6 3297 0t0 TCP *:sunrpc (LISTEN) 2024-04-20T21:47:08.619 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 99u IPv6 3303 0t0 UDP *:sunrpc 2024-04-20T21:47:08.619 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 100r FIFO 0,14 0t0 6570 pipe 2024-04-20T21:47:08.619 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 103u unix 0xffff888105633c00 0t0 36981 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.619 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 105u unix 0xffff888107874380 0t0 3576 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.619 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 106u unix 0xffff88810ac2c380 0t0 4465 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.620 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 107u unix 0xffff88810ac29680 0t0 6762 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.620 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 108u unix 0xffff88810ac29e00 0t0 6760 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.620 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 109u unix 0xffff88810ad1a580 0t0 4351 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.620 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 110u unix 0xffff88811056d280 0t0 4348 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.620 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 111u unix 0xffff88811042d280 0t0 4347 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.620 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 112u unix 0xffff88811042bc00 0t0 4344 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.620 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 113u unix 0xffff88810ac2e900 0t0 4337 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.620 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 114u unix 0xffff88810ad1bc00 0t0 4315 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.621 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 115u unix 0xffff88810ad1e900 0t0 368 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.621 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 120u FIFO 0,25 0t0 429 /run/initctl 2024-04-20T21:47:08.621 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 122u unix 0xffff88810abfda00 0t0 8487 /run/systemd/coredump type=SEQPACKET (LISTEN) 2024-04-20T21:47:08.621 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 125u FIFO 0,25 0t0 423 /run/dmeventd-server 2024-04-20T21:47:08.621 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 1 root 126u FIFO 0,25 0t0 424 /run/dmeventd-client 2024-04-20T21:47:08.621 INFO:teuthology.orchestra.run.smithi102.stdout:kthreadd 2 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.621 INFO:teuthology.orchestra.run.smithi102.stdout:kthreadd 2 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.621 INFO:teuthology.orchestra.run.smithi102.stdout:kthreadd 2 root txt unknown /proc/2/exe 2024-04-20T21:47:08.622 INFO:teuthology.orchestra.run.smithi102.stdout:pool_work 3 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.622 INFO:teuthology.orchestra.run.smithi102.stdout:pool_work 3 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.622 INFO:teuthology.orchestra.run.smithi102.stdout:pool_work 3 root txt unknown /proc/3/exe 2024-04-20T21:47:08.622 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 4 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.622 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 4 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.622 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 4 root txt unknown /proc/4/exe 2024-04-20T21:47:08.622 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 5 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.622 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 5 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 5 root txt unknown /proc/5/exe 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 6 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 6 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 6 root txt unknown /proc/6/exe 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 9 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 9 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 9 root txt unknown /proc/9/exe 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 11 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.623 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 11 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.624 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 11 root txt unknown /proc/11/exe 2024-04-20T21:47:08.624 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 12 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.624 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 12 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.624 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 12 root txt unknown /proc/12/exe 2024-04-20T21:47:08.624 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 13 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.624 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 13 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.624 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 13 root txt unknown /proc/13/exe 2024-04-20T21:47:08.624 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 14 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.625 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 14 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.625 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_tasks 14 root txt unknown /proc/14/exe 2024-04-20T21:47:08.625 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 15 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.625 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 15 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.625 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 15 root txt unknown /proc/15/exe 2024-04-20T21:47:08.625 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_preem 16 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.625 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_preem 16 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.625 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_preem 16 root txt unknown /proc/16/exe 2024-04-20T21:47:08.626 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_exp_p 17 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.626 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_exp_p 17 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.626 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_exp_p 17 root txt unknown /proc/17/exe 2024-04-20T21:47:08.626 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_exp_g 18 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.626 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_exp_g 18 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.626 INFO:teuthology.orchestra.run.smithi102.stdout:rcu_exp_g 18 root txt unknown /proc/18/exe 2024-04-20T21:47:08.626 INFO:teuthology.orchestra.run.smithi102.stdout:migration 19 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.626 INFO:teuthology.orchestra.run.smithi102.stdout:migration 19 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.627 INFO:teuthology.orchestra.run.smithi102.stdout:migration 19 root txt unknown /proc/19/exe 2024-04-20T21:47:08.627 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/0 20 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.627 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/0 20 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.627 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/0 20 root txt unknown /proc/20/exe 2024-04-20T21:47:08.627 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/1 21 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.627 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/1 21 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.627 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/1 21 root txt unknown /proc/21/exe 2024-04-20T21:47:08.627 INFO:teuthology.orchestra.run.smithi102.stdout:migration 22 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.628 INFO:teuthology.orchestra.run.smithi102.stdout:migration 22 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.628 INFO:teuthology.orchestra.run.smithi102.stdout:migration 22 root txt unknown /proc/22/exe 2024-04-20T21:47:08.628 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 23 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.628 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 23 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.628 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 23 root txt unknown /proc/23/exe 2024-04-20T21:47:08.628 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 25 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.628 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 25 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.628 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 25 root txt unknown /proc/25/exe 2024-04-20T21:47:08.629 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/2 26 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.629 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/2 26 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.629 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/2 26 root txt unknown /proc/26/exe 2024-04-20T21:47:08.629 INFO:teuthology.orchestra.run.smithi102.stdout:migration 27 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.629 INFO:teuthology.orchestra.run.smithi102.stdout:migration 27 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.629 INFO:teuthology.orchestra.run.smithi102.stdout:migration 27 root txt unknown /proc/27/exe 2024-04-20T21:47:08.629 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 28 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.629 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 28 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 28 root txt unknown /proc/28/exe 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 30 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 30 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 30 root txt unknown /proc/30/exe 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/3 31 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/3 31 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/3 31 root txt unknown /proc/31/exe 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:migration 32 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:migration 32 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.631 INFO:teuthology.orchestra.run.smithi102.stdout:migration 32 root txt unknown /proc/32/exe 2024-04-20T21:47:08.631 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 33 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.631 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 33 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.631 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 33 root txt unknown /proc/33/exe 2024-04-20T21:47:08.631 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 35 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.631 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 35 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.631 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 35 root txt unknown /proc/35/exe 2024-04-20T21:47:08.631 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/4 36 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.632 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/4 36 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.632 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/4 36 root txt unknown /proc/36/exe 2024-04-20T21:47:08.632 INFO:teuthology.orchestra.run.smithi102.stdout:migration 37 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.632 INFO:teuthology.orchestra.run.smithi102.stdout:migration 37 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.632 INFO:teuthology.orchestra.run.smithi102.stdout:migration 37 root txt unknown /proc/37/exe 2024-04-20T21:47:08.632 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 38 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.632 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 38 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.632 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 38 root txt unknown /proc/38/exe 2024-04-20T21:47:08.633 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 40 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.633 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 40 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.633 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 40 root txt unknown /proc/40/exe 2024-04-20T21:47:08.633 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/5 41 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.633 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/5 41 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.633 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/5 41 root txt unknown /proc/41/exe 2024-04-20T21:47:08.633 INFO:teuthology.orchestra.run.smithi102.stdout:migration 42 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.633 INFO:teuthology.orchestra.run.smithi102.stdout:migration 42 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.634 INFO:teuthology.orchestra.run.smithi102.stdout:migration 42 root txt unknown /proc/42/exe 2024-04-20T21:47:08.634 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 43 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.634 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 43 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.634 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 43 root txt unknown /proc/43/exe 2024-04-20T21:47:08.634 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 45 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.634 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 45 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.634 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 45 root txt unknown /proc/45/exe 2024-04-20T21:47:08.634 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/6 46 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.635 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/6 46 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.635 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/6 46 root txt unknown /proc/46/exe 2024-04-20T21:47:08.635 INFO:teuthology.orchestra.run.smithi102.stdout:migration 47 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.635 INFO:teuthology.orchestra.run.smithi102.stdout:migration 47 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.635 INFO:teuthology.orchestra.run.smithi102.stdout:migration 47 root txt unknown /proc/47/exe 2024-04-20T21:47:08.635 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 48 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.635 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 48 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.635 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 48 root txt unknown /proc/48/exe 2024-04-20T21:47:08.636 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.636 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.636 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50 root txt unknown /proc/50/exe 2024-04-20T21:47:08.636 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/7 51 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.636 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/7 51 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.636 INFO:teuthology.orchestra.run.smithi102.stdout:cpuhp/7 51 root txt unknown /proc/51/exe 2024-04-20T21:47:08.636 INFO:teuthology.orchestra.run.smithi102.stdout:migration 52 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.636 INFO:teuthology.orchestra.run.smithi102.stdout:migration 52 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:migration 52 root txt unknown /proc/52/exe 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 53 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 53 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:ksoftirqd 53 root txt unknown /proc/53/exe 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 55 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 55 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 55 root txt unknown /proc/55/exe 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:kdevtmpfs 56 root cwd DIR 0,6 3320 1 / 2024-04-20T21:47:08.637 INFO:teuthology.orchestra.run.smithi102.stdout:kdevtmpfs 56 root rtd DIR 0,6 3320 1 / 2024-04-20T21:47:08.638 INFO:teuthology.orchestra.run.smithi102.stdout:kdevtmpfs 56 root txt unknown /proc/56/exe 2024-04-20T21:47:08.638 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 57 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.638 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 57 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.638 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 57 root txt unknown /proc/57/exe 2024-04-20T21:47:08.638 INFO:teuthology.orchestra.run.smithi102.stdout:kauditd 58 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.638 INFO:teuthology.orchestra.run.smithi102.stdout:kauditd 58 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.638 INFO:teuthology.orchestra.run.smithi102.stdout:kauditd 58 root txt unknown /proc/58/exe 2024-04-20T21:47:08.638 INFO:teuthology.orchestra.run.smithi102.stdout:khungtask 59 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.639 INFO:teuthology.orchestra.run.smithi102.stdout:khungtask 59 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.639 INFO:teuthology.orchestra.run.smithi102.stdout:khungtask 59 root txt unknown /proc/59/exe 2024-04-20T21:47:08.639 INFO:teuthology.orchestra.run.smithi102.stdout:oom_reape 61 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.639 INFO:teuthology.orchestra.run.smithi102.stdout:oom_reape 61 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.639 INFO:teuthology.orchestra.run.smithi102.stdout:oom_reape 61 root txt unknown /proc/61/exe 2024-04-20T21:47:08.639 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 63 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.640 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 63 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.640 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 63 root txt unknown /proc/63/exe 2024-04-20T21:47:08.640 INFO:teuthology.orchestra.run.smithi102.stdout:kcompactd 64 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.640 INFO:teuthology.orchestra.run.smithi102.stdout:kcompactd 64 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.640 INFO:teuthology.orchestra.run.smithi102.stdout:kcompactd 64 root txt unknown /proc/64/exe 2024-04-20T21:47:08.640 INFO:teuthology.orchestra.run.smithi102.stdout:ksmd 65 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.640 INFO:teuthology.orchestra.run.smithi102.stdout:ksmd 65 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.640 INFO:teuthology.orchestra.run.smithi102.stdout:ksmd 65 root txt unknown /proc/65/exe 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:khugepage 66 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:khugepage 66 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:khugepage 66 root txt unknown /proc/66/exe 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 67 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 67 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 67 root txt unknown /proc/67/exe 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 68 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 68 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.641 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 68 root txt unknown /proc/68/exe 2024-04-20T21:47:08.642 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 69 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.642 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 69 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.642 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 69 root txt unknown /proc/69/exe 2024-04-20T21:47:08.642 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 70 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.642 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 70 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.642 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 70 root txt unknown /proc/70/exe 2024-04-20T21:47:08.642 INFO:teuthology.orchestra.run.smithi102.stdout:irq/9-acp 78 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.643 INFO:teuthology.orchestra.run.smithi102.stdout:irq/9-acp 78 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.643 INFO:teuthology.orchestra.run.smithi102.stdout:irq/9-acp 78 root txt unknown /proc/78/exe 2024-04-20T21:47:08.643 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 79 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.643 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 79 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.643 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 79 root txt unknown /proc/79/exe 2024-04-20T21:47:08.643 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 80 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.643 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 80 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.643 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 80 root txt unknown /proc/80/exe 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 81 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 81 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 81 root txt unknown /proc/81/exe 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 82 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 82 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 82 root txt unknown /proc/82/exe 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 83 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 83 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.644 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 83 root txt unknown /proc/83/exe 2024-04-20T21:47:08.645 INFO:teuthology.orchestra.run.smithi102.stdout:watchdogd 84 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.645 INFO:teuthology.orchestra.run.smithi102.stdout:watchdogd 84 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.645 INFO:teuthology.orchestra.run.smithi102.stdout:watchdogd 84 root txt unknown /proc/84/exe 2024-04-20T21:47:08.645 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 85 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.645 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 85 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.645 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 85 root txt unknown /proc/85/exe 2024-04-20T21:47:08.645 INFO:teuthology.orchestra.run.smithi102.stdout:kswapd0 86 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.645 INFO:teuthology.orchestra.run.smithi102.stdout:kswapd0 86 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.646 INFO:teuthology.orchestra.run.smithi102.stdout:kswapd0 86 root txt unknown /proc/86/exe 2024-04-20T21:47:08.646 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 88 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.646 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 88 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.646 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 88 root txt unknown /proc/88/exe 2024-04-20T21:47:08.646 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 92 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.646 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 92 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.646 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 92 root txt unknown /proc/92/exe 2024-04-20T21:47:08.646 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_0 93 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_0 93 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_0 93 root txt unknown /proc/93/exe 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 94 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 94 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 94 root txt unknown /proc/94/exe 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_1 95 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_1 95 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_1 95 root txt unknown /proc/95/exe 2024-04-20T21:47:08.647 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 96 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 96 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 96 root txt unknown /proc/96/exe 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_2 97 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_2 97 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_2 97 root txt unknown /proc/97/exe 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 98 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 98 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 98 root txt unknown /proc/98/exe 2024-04-20T21:47:08.648 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_3 99 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.649 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_3 99 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.649 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_3 99 root txt unknown /proc/99/exe 2024-04-20T21:47:08.649 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 100 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.649 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 100 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.649 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 100 root txt unknown /proc/100/exe 2024-04-20T21:47:08.649 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_4 104 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.649 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_4 104 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.649 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_4 104 root txt unknown /proc/104/exe 2024-04-20T21:47:08.650 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 105 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.650 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 105 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.650 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 105 root txt unknown /proc/105/exe 2024-04-20T21:47:08.650 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_5 106 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.650 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_5 106 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.650 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_5 106 root txt unknown /proc/106/exe 2024-04-20T21:47:08.650 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 107 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.650 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 107 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.651 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 107 root txt unknown /proc/107/exe 2024-04-20T21:47:08.651 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_6 108 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.651 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_6 108 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.651 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_6 108 root txt unknown /proc/108/exe 2024-04-20T21:47:08.651 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 109 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.651 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 109 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.651 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 109 root txt unknown /proc/109/exe 2024-04-20T21:47:08.652 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_7 110 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.652 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_7 110 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.652 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_7 110 root txt unknown /proc/110/exe 2024-04-20T21:47:08.652 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 111 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.652 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 111 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.652 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 111 root txt unknown /proc/111/exe 2024-04-20T21:47:08.652 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_8 112 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.652 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_8 112 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_8 112 root txt unknown /proc/112/exe 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 113 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 113 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 113 root txt unknown /proc/113/exe 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_9 114 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_9 114 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:scsi_eh_9 114 root txt unknown /proc/114/exe 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 115 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.653 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 115 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.654 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 115 root txt unknown /proc/115/exe 2024-04-20T21:47:08.654 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 116 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.654 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 116 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.654 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 116 root txt unknown /proc/116/exe 2024-04-20T21:47:08.654 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 117 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.657 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 117 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.657 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 117 root txt unknown /proc/117/exe 2024-04-20T21:47:08.657 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 119 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.657 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 119 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.658 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 119 root txt unknown /proc/119/exe 2024-04-20T21:47:08.658 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 128 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.658 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 128 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.658 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 128 root txt unknown /proc/128/exe 2024-04-20T21:47:08.658 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 129 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.658 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 129 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.658 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 129 root txt unknown /proc/129/exe 2024-04-20T21:47:08.658 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 130 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 130 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 130 root txt unknown /proc/130/exe 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 131 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 131 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 131 root txt unknown /proc/131/exe 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 132 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 132 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 132 root txt unknown /proc/132/exe 2024-04-20T21:47:08.659 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 141 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.660 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 141 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.660 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 141 root txt unknown /proc/141/exe 2024-04-20T21:47:08.660 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 250 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.660 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 250 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.660 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 250 root txt unknown /proc/250/exe 2024-04-20T21:47:08.660 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 252 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.660 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 252 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.660 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 252 root txt unknown /proc/252/exe 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 255 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 255 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 255 root txt unknown /proc/255/exe 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 261 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 261 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 261 root txt unknown /proc/261/exe 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 365 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 365 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.661 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 365 root txt unknown /proc/365/exe 2024-04-20T21:47:08.662 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 411 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.662 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 411 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.662 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 411 root txt unknown /proc/411/exe 2024-04-20T21:47:08.662 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 412 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.662 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 412 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.662 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 412 root txt unknown /proc/412/exe 2024-04-20T21:47:08.662 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 413 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.662 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 413 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 413 root txt unknown /proc/413/exe 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 435 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 435 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 435 root txt unknown /proc/435/exe 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:jbd2/sda1 448 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:jbd2/sda1 448 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:jbd2/sda1 448 root txt unknown /proc/448/exe 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 449 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.663 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 449 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 449 root txt unknown /proc/449/exe 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root txt REG 8,1 185640 18590 /usr/lib/systemd/systemd-journald 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 0,25 8388608 59 /run/log/journal/7fc8828e06d0482694bcf4a447d13cb6/system.journal 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.664 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.665 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.665 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.665 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.665 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.665 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.665 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.665 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.665 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.666 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-20T21:47:08.666 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.666 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.666 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-20T21:47:08.666 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.666 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-20T21:47:08.666 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.666 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.667 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root mem REG 0,25 8 56 /run/systemd/journal/kernel-seqnum 2024-04-20T21:47:08.668 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.668 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 1w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.668 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 2w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.668 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 3u unix 0xffff888110428f00 0t0 106 /run/systemd/journal/socket type=DGRAM (CONNECTED) 2024-04-20T21:47:08.668 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 4u unix 0xffff888110429680 0t0 108 /run/systemd/journal/stdout type=STREAM (LISTEN) 2024-04-20T21:47:08.668 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 5u unix 0xffff888110428780 0t0 104 /run/systemd/journal/dev-log type=DGRAM (CONNECTED) 2024-04-20T21:47:08.668 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 6w CHR 1,11 0t0 10 /dev/kmsg 2024-04-20T21:47:08.668 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 7u a_inode 0,15 0 50 [eventpoll:3,4,5,8,9,10,11,12,13,15,17,18,19,20,22,23,24,26,27,28,29,30] 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 8u CHR 1,11 0t0 10 /dev/kmsg 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 9u unix 0xffff88810ac2bc00 0t0 7451 /run/systemd/journal/io.systemd.journal type=STREAM (LISTEN) 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 10r REG 0,21 0 2524 /proc/sys/kernel/hostname 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 11u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 12u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 13u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 14u unix 0xffff88810ac2b480 0t0 7452 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 15u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.669 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 16u REG 0,25 8388608 59 /run/log/journal/7fc8828e06d0482694bcf4a447d13cb6/system.journal 2024-04-20T21:47:08.670 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 17u unix 0xffff88810ad1e900 0t0 368 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.670 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 18u unix 0xffff88810ac2e900 0t0 4337 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.670 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 19u unix 0xffff88811042bc00 0t0 4344 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.670 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 20u unix 0xffff88810ad1bc00 0t0 4315 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.670 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 22u unix 0xffff888105633c00 0t0 36981 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.670 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 23u unix 0xffff88811042d280 0t0 4347 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.670 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 24u unix 0xffff88811056d280 0t0 4348 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.670 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 26u unix 0xffff88810ad1a580 0t0 4351 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.671 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 27u unix 0xffff88810ac29e00 0t0 6760 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.671 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 28u unix 0xffff88810ac29680 0t0 6762 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.671 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 29u unix 0xffff888107874380 0t0 3576 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.671 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-j 526 root 30u unix 0xffff88810ac2c380 0t0 4465 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2024-04-20T21:47:08.671 INFO:teuthology.orchestra.run.smithi102.stdout:ext4lazyi 532 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.671 INFO:teuthology.orchestra.run.smithi102.stdout:ext4lazyi 532 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.671 INFO:teuthology.orchestra.run.smithi102.stdout:ext4lazyi 532 root txt unknown /proc/532/exe 2024-04-20T21:47:08.671 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 537 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.672 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 537 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.672 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 537 root txt unknown /proc/537/exe 2024-04-20T21:47:08.672 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.672 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.672 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root txt REG 8,1 598008 9078 /usr/bin/udevadm 2024-04-20T21:47:08.672 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 11730661 64009 /etc/udev/hwdb.bin 2024-04-20T21:47:08.672 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 598426 62136 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-20T21:47:08.672 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 10610 264155 /usr/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 285055 264154 /usr/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 553079 264151 /usr/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 176703 264149 /usr/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.673 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.674 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.674 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.674 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.674 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.674 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.674 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.674 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.674 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-20T21:47:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-20T21:47:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-20T21:47:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 21219 62233 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-20T21:47:08.676 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.677 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 1u unix 0xffff88810ad1e180 0t0 2678 type=STREAM (CONNECTED) 2024-04-20T21:47:08.677 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 2u unix 0xffff88810ad1e180 0t0 2678 type=STREAM (CONNECTED) 2024-04-20T21:47:08.677 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 3u netlink 0t0 8491 KOBJECT_UEVENT 2024-04-20T21:47:08.677 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 4u unix 0xffff88810abfe180 0t0 8489 /run/udev/control type=SEQPACKET (LISTEN) 2024-04-20T21:47:08.677 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 5u unix 0xffff88810ad1f080 0t0 2690 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.677 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 6u unix 0xffff88810ad1f800 0t0 2694 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.677 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 7u unix 0xffff88810ad18000 0t0 2695 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.677 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 9u a_inode 0,15 0 50 [eventpoll:3,4,6,8,10,11,14] 2024-04-20T21:47:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 10u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 11u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 12r REG 8,1 11730661 64009 /etc/udev/hwdb.bin 2024-04-20T21:47:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-u 546 root 14u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 598 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 598 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 598 root txt unknown /proc/598/exe 2024-04-20T21:47:08.679 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.679 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.679 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc txt REG 8,1 61400 13830 /usr/bin/rpcbind 2024-04-20T21:47:08.679 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.679 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.679 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.679 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.679 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-20T21:47:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.682 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.682 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 1u unix 0xffff88810ad1c380 0t0 2763 type=STREAM (CONNECTED) 2024-04-20T21:47:08.682 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 2u unix 0xffff88810ad1c380 0t0 2763 type=STREAM (CONNECTED) 2024-04-20T21:47:08.682 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 3u unix 0xffff88810abfd280 0t0 8477 /run/rpcbind.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.682 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 4u IPv4 7420 0t0 TCP *:sunrpc (LISTEN) 2024-04-20T21:47:08.682 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 5u IPv4 7426 0t0 UDP *:sunrpc 2024-04-20T21:47:08.682 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 6u IPv6 3297 0t0 TCP *:sunrpc (LISTEN) 2024-04-20T21:47:08.682 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 7u IPv6 3303 0t0 UDP *:sunrpc 2024-04-20T21:47:08.683 INFO:teuthology.orchestra.run.smithi102.stdout:rpcbind 614 rpc 8rW REG 0,25 0 984 /run/rpcbind/rpcbind.lock 2024-04-20T21:47:08.683 INFO:teuthology.orchestra.run.smithi102.stdout:kipmi0 615 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.683 INFO:teuthology.orchestra.run.smithi102.stdout:kipmi0 615 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.683 INFO:teuthology.orchestra.run.smithi102.stdout:kipmi0 615 root txt unknown /proc/615/exe 2024-04-20T21:47:08.683 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.683 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.683 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root txt REG 8,1 140632 9260 /usr/sbin/auditd 2024-04-20T21:47:08.683 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.684 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.685 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.685 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.685 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.685 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 147792 7213 /usr/lib64/libauparse.so.0.0.0 2024-04-20T21:47:08.685 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.685 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.685 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.685 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.686 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.686 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 3u netlink 0t0 5866 AUDIT 2024-04-20T21:47:08.686 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 4w REG 8,1 2495748 64044 /var/log/audit/audit.log 2024-04-20T21:47:08.686 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 6u unix 0xffff888108504b00 0t0 5869 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.686 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 7u unix 0xffff888108504380 0t0 5868 type=STREAM (CONNECTED) 2024-04-20T21:47:08.686 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 8u unix 0xffff888108505280 0t0 5870 type=STREAM (CONNECTED) 2024-04-20T21:47:08.686 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 9u unix 0xffff888108505a00 0t0 5871 type=STREAM (CONNECTED) 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 root 10u a_inode 0,15 0 50 [eventfd:0] 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root txt REG 8,1 140632 9260 /usr/sbin/auditd 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.687 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 147792 7213 /usr/lib64/libauparse.so.0.0.0 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 3u netlink 0t0 5866 AUDIT 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 4w REG 8,1 2495748 64044 /var/log/audit/audit.log 2024-04-20T21:47:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 6u unix 0xffff888108504b00 0t0 5869 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.690 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 7u unix 0xffff888108504380 0t0 5868 type=STREAM (CONNECTED) 2024-04-20T21:47:08.690 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 8u unix 0xffff888108505280 0t0 5870 type=STREAM (CONNECTED) 2024-04-20T21:47:08.690 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 9u unix 0xffff888108505a00 0t0 5871 type=STREAM (CONNECTED) 2024-04-20T21:47:08.690 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 621 auditd root 10u a_inode 0,15 0 50 [eventfd:0] 2024-04-20T21:47:08.690 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.690 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.690 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root txt REG 8,1 140632 9260 /usr/sbin/auditd 2024-04-20T21:47:08.690 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.691 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.691 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.691 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.691 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.691 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.691 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.691 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.691 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 147792 7213 /usr/lib64/libauparse.so.0.0.0 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.692 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.693 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.693 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 3u netlink 0t0 5866 AUDIT 2024-04-20T21:47:08.693 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 4w REG 8,1 2495748 64044 /var/log/audit/audit.log 2024-04-20T21:47:08.693 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 6u unix 0xffff888108504b00 0t0 5869 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.693 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 7u unix 0xffff888108504380 0t0 5868 type=STREAM (CONNECTED) 2024-04-20T21:47:08.693 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 8u unix 0xffff888108505280 0t0 5870 type=STREAM (CONNECTED) 2024-04-20T21:47:08.693 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 9u unix 0xffff888108505a00 0t0 5871 type=STREAM (CONNECTED) 2024-04-20T21:47:08.693 INFO:teuthology.orchestra.run.smithi102.stdout:auditd 620 623 auditd root 10u a_inode 0,15 0 50 [eventfd:0] 2024-04-20T21:47:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root txt REG 8,1 19624 9398 /usr/sbin/sedispatch 2024-04-20T21:47:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.695 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 147792 7213 /usr/lib64/libauparse.so.0.0.0 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 343752 7611 /usr/lib64/libdbus-1.so.3.19.13 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root 0u unix 0xffff888108502d00 0t0 5867 type=STREAM (CONNECTED) 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:sedispatc 622 root 3u unix 0xffff888100eaa580 0t0 8914 type=STREAM (CONNECTED) 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 628 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.696 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 628 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.697 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 628 root txt unknown /proc/628/exe 2024-04-20T21:47:08.697 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 629 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.697 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 629 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.697 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 629 root txt unknown /proc/629/exe 2024-04-20T21:47:08.697 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.697 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.697 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus txt REG 8,1 131896 8696 /usr/bin/dbus-broker-launch 2024-04-20T21:47:08.697 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus DEL REG 0,1 2048 /memfd:dbus-broker-log 2024-04-20T21:47:08.698 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.698 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.698 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.698 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.698 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.698 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.698 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.698 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.699 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.699 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.699 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.699 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.699 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.699 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.699 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.699 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 1u unix 0xffff88810ac2e180 0t0 7575 type=STREAM (CONNECTED) 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 2u unix 0xffff88810ac2e180 0t0 7575 type=STREAM (CONNECTED) 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 3u unix 0xffff88810abfc380 0t0 8663 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 4u unix 0xffff88810ac2cb00 0t0 7585 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 5u a_inode 0,15 0 50 [eventpoll:6,7,8,9,10,11] 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 6u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 7r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.700 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 8u unix 0xffff88810ac28000 0t0 7587 type=STREAM (CONNECTED) 2024-04-20T21:47:08.701 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 9u unix 0xffff88810ac28f00 0t0 7589 type=STREAM (CONNECTED) 2024-04-20T21:47:08.701 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 10u REG 0,5 0 963 pidfd 2024-04-20T21:47:08.701 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 11u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.701 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 662 dbus 12u REG 0,1 2097152 2048 /memfd:dbus-broker-log (deleted) 2024-04-20T21:47:08.701 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.701 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.701 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus txt REG 8,1 236336 8695 /usr/bin/dbus-broker 2024-04-20T21:47:08.701 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.702 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.702 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.702 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.702 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.702 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.702 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.702 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 1u unix 0xffff88810ac2e180 0t0 7575 type=STREAM (CONNECTED) 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 2u unix 0xffff88810ac2e180 0t0 7575 type=STREAM (CONNECTED) 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 3u sock 0,9 0t0 8670 protocol: NETLINK 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 4u unix 0xffff88810ac2cb00 0t0 7585 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 5u a_inode 0,15 0 50 [eventpoll:6,7,8,9,10,11,12,14,16,17,18] 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 6u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 7u unix 0xffff88810abfc380 0t0 8663 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2024-04-20T21:47:08.704 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 8u unix 0xffff888108507080 0t0 8671 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2024-04-20T21:47:08.704 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 9u unix 0xffff88810ac28780 0t0 7588 type=STREAM (CONNECTED) 2024-04-20T21:47:08.704 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 10u unix 0xffff88810ac2ad00 0t0 8673 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2024-04-20T21:47:08.704 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 11u unix 0xffff88810ad1d280 0t0 7590 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2024-04-20T21:47:08.704 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 12u unix 0xffff88811056f800 0t0 8685 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2024-04-20T21:47:08.704 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 14u unix 0xffff88810ad18780 0t0 8689 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2024-04-20T21:47:08.704 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 16u unix 0xffff888100eae900 0t0 3694 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2024-04-20T21:47:08.704 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 17u unix 0xffff888100ea8000 0t0 7707 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:dbus-brok 663 dbus 18u unix 0xffff888108502580 0t0 15038 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root txt REG 8,1 3748632 9116 /usr/sbin/NetworkManager 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 317672 50025 /usr/lib64/NetworkManager/1.47.4-1.el9/libnm-settings-plugin-ifcfg-rh.so 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 343752 7611 /usr/lib64/libdbus-1.so.3.19.13 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 62456 7265 /usr/lib64/libjansson.so.4.14.0 2024-04-20T21:47:08.705 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 31792 8437 /usr/lib64/libteamdctl.so.0.1.5 2024-04-20T21:47:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 57128 48336 /usr/lib64/NetworkManager/1.47.4-1.el9/libnm-device-plugin-team.so 2024-04-20T21:47:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.707 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.708 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 626240 7471 /usr/lib64/libhogweed.so.6.8 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.709 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 94112 7384 /usr/lib64/libtasn1.so.6.6.0 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 2337504 7477 /usr/lib64/libgnutls.so.30.37.1 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.711 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.712 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.712 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.712 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 32408 8817 /usr/lib64/libndp.so.0.2.0 2024-04-20T21:47:08.712 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:08.717 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.718 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.718 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.718 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.718 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.718 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.718 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.718 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.719 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.719 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 1u unix 0xffff88811042cb00 0t0 513 type=STREAM (CONNECTED) 2024-04-20T21:47:08.719 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 2u unix 0xffff88811042cb00 0t0 513 type=STREAM (CONNECTED) 2024-04-20T21:47:08.719 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 3u a_inode 0,15 0 50 [eventfd:3] 2024-04-20T21:47:08.719 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 4u a_inode 0,15 0 50 [eventfd:4] 2024-04-20T21:47:08.719 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 5u unix 0xffff88811056e900 0t0 6685 type=DGRAM (UNCONNECTED) 2024-04-20T21:47:08.719 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 6u unix 0xffff888110569680 0t0 6686 type=STREAM (CONNECTED) 2024-04-20T21:47:08.719 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 7u a_inode 0,15 0 50 [eventfd:5] 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 8r REG 0,4 0 4026531840 net 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 9r REG 0,4 0 4026532500 mnt 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 10u netlink 0t0 6690 KOBJECT_UEVENT 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 11u netlink 0t0 6691 GENERIC 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 12u netlink 0t0 6692 GENERIC 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 13u netlink 0t0 6693 ROUTE 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 14r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 15r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.720 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 16u netlink 0t0 4367 KOBJECT_UEVENT 2024-04-20T21:47:08.721 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 17u unix 0xffff88810ad1da00 0t0 2860 type=STREAM (CONNECTED) 2024-04-20T21:47:08.721 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 18u a_inode 0,15 0 50 [eventpoll] 2024-04-20T21:47:08.721 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 19u a_inode 0,15 0 50 [eventpoll:21,26] 2024-04-20T21:47:08.721 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 20w FIFO 0,25 0t0 1094 /run/systemd/inhibit/1.ref 2024-04-20T21:47:08.721 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 21u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.721 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 23u a_inode 0,15 0 50 [eventpoll:24,25] 2024-04-20T21:47:08.721 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 24u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.721 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 25u pack 598 0t0 ARP type=SOCK_DGRAM 2024-04-20T21:47:08.722 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 root 26u IPv4 599 0t0 UDP smithi102.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2024-04-20T21:47:08.722 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.722 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.722 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root txt REG 8,1 3748632 9116 /usr/sbin/NetworkManager 2024-04-20T21:47:08.722 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.722 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 317672 50025 /usr/lib64/NetworkManager/1.47.4-1.el9/libnm-settings-plugin-ifcfg-rh.so 2024-04-20T21:47:08.722 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 343752 7611 /usr/lib64/libdbus-1.so.3.19.13 2024-04-20T21:47:08.722 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 62456 7265 /usr/lib64/libjansson.so.4.14.0 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 31792 8437 /usr/lib64/libteamdctl.so.0.1.5 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 57128 48336 /usr/lib64/NetworkManager/1.47.4-1.el9/libnm-device-plugin-team.so 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:08.723 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.724 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.725 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.725 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.725 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.725 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.725 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.725 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:08.725 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.725 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.726 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:08.726 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:08.726 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.726 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.726 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.726 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 626240 7471 /usr/lib64/libhogweed.so.6.8 2024-04-20T21:47:08.726 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:08.726 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.727 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.727 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.727 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.727 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.727 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:08.727 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.727 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.727 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 94112 7384 /usr/lib64/libtasn1.so.6.6.0 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.728 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 2337504 7477 /usr/lib64/libgnutls.so.30.37.1 2024-04-20T21:47:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 32408 8817 /usr/lib64/libndp.so.0.2.0 2024-04-20T21:47:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:08.729 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.730 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.730 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.730 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.730 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.730 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.730 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.730 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.730 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 1u unix 0xffff88811042cb00 0t0 513 type=STREAM (CONNECTED) 2024-04-20T21:47:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 2u unix 0xffff88811042cb00 0t0 513 type=STREAM (CONNECTED) 2024-04-20T21:47:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 3u a_inode 0,15 0 50 [eventfd:3] 2024-04-20T21:47:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 4u a_inode 0,15 0 50 [eventfd:4] 2024-04-20T21:47:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 5u unix 0xffff88811056e900 0t0 6685 type=DGRAM (UNCONNECTED) 2024-04-20T21:47:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 6u unix 0xffff888110569680 0t0 6686 type=STREAM (CONNECTED) 2024-04-20T21:47:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 7u a_inode 0,15 0 50 [eventfd:5] 2024-04-20T21:47:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 8r REG 0,4 0 4026531840 net 2024-04-20T21:47:08.732 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 9r REG 0,4 0 4026532500 mnt 2024-04-20T21:47:08.732 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 10u netlink 0t0 6690 KOBJECT_UEVENT 2024-04-20T21:47:08.732 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 11u netlink 0t0 6691 GENERIC 2024-04-20T21:47:08.732 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 12u netlink 0t0 6692 GENERIC 2024-04-20T21:47:08.732 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 13u netlink 0t0 6693 ROUTE 2024-04-20T21:47:08.732 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 14r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.732 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 15r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.732 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 16u netlink 0t0 4367 KOBJECT_UEVENT 2024-04-20T21:47:08.733 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 17u unix 0xffff88810ad1da00 0t0 2860 type=STREAM (CONNECTED) 2024-04-20T21:47:08.733 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 18u a_inode 0,15 0 50 [eventpoll] 2024-04-20T21:47:08.733 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 19u a_inode 0,15 0 50 [eventpoll:21,26] 2024-04-20T21:47:08.733 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 20w FIFO 0,25 0t0 1094 /run/systemd/inhibit/1.ref 2024-04-20T21:47:08.733 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 21u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.733 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 23u a_inode 0,15 0 50 [eventpoll:24,25] 2024-04-20T21:47:08.733 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 24u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.733 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 25u pack 598 0t0 ARP type=SOCK_DGRAM 2024-04-20T21:47:08.734 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 679 gmain root 26u IPv4 599 0t0 UDP smithi102.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2024-04-20T21:47:08.734 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.734 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.734 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root txt REG 8,1 3748632 9116 /usr/sbin/NetworkManager 2024-04-20T21:47:08.734 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.734 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 317672 50025 /usr/lib64/NetworkManager/1.47.4-1.el9/libnm-settings-plugin-ifcfg-rh.so 2024-04-20T21:47:08.734 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 343752 7611 /usr/lib64/libdbus-1.so.3.19.13 2024-04-20T21:47:08.734 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 62456 7265 /usr/lib64/libjansson.so.4.14.0 2024-04-20T21:47:08.735 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 31792 8437 /usr/lib64/libteamdctl.so.0.1.5 2024-04-20T21:47:08.735 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 57128 48336 /usr/lib64/NetworkManager/1.47.4-1.el9/libnm-device-plugin-team.so 2024-04-20T21:47:08.735 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.735 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.735 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.735 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.735 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.735 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:08.737 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.738 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.738 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:08.738 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:08.738 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.738 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.738 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.738 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 626240 7471 /usr/lib64/libhogweed.so.6.8 2024-04-20T21:47:08.738 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:08.739 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.739 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.739 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.739 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.739 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.739 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:08.739 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.739 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.740 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.740 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:08.740 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 94112 7384 /usr/lib64/libtasn1.so.6.6.0 2024-04-20T21:47:08.740 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:08.740 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.740 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.740 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.740 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 2337504 7477 /usr/lib64/libgnutls.so.30.37.1 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 32408 8817 /usr/lib64/libndp.so.0.2.0 2024-04-20T21:47:08.741 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:08.742 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.742 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.742 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.742 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.742 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.742 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.742 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.742 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.743 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.743 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 1u unix 0xffff88811042cb00 0t0 513 type=STREAM (CONNECTED) 2024-04-20T21:47:08.743 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 2u unix 0xffff88811042cb00 0t0 513 type=STREAM (CONNECTED) 2024-04-20T21:47:08.743 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 3u a_inode 0,15 0 50 [eventfd:3] 2024-04-20T21:47:08.743 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 4u a_inode 0,15 0 50 [eventfd:4] 2024-04-20T21:47:08.743 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 5u unix 0xffff88811056e900 0t0 6685 type=DGRAM (UNCONNECTED) 2024-04-20T21:47:08.743 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 6u unix 0xffff888110569680 0t0 6686 type=STREAM (CONNECTED) 2024-04-20T21:47:08.743 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 7u a_inode 0,15 0 50 [eventfd:5] 2024-04-20T21:47:08.744 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 8r REG 0,4 0 4026531840 net 2024-04-20T21:47:08.744 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 9r REG 0,4 0 4026532500 mnt 2024-04-20T21:47:08.744 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 10u netlink 0t0 6690 KOBJECT_UEVENT 2024-04-20T21:47:08.744 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 11u netlink 0t0 6691 GENERIC 2024-04-20T21:47:08.744 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 12u netlink 0t0 6692 GENERIC 2024-04-20T21:47:08.744 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 13u netlink 0t0 6693 ROUTE 2024-04-20T21:47:08.744 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 14r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.744 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 15r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 16u netlink 0t0 4367 KOBJECT_UEVENT 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 17u unix 0xffff88810ad1da00 0t0 2860 type=STREAM (CONNECTED) 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 18u a_inode 0,15 0 50 [eventpoll] 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 19u a_inode 0,15 0 50 [eventpoll:21,26] 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 20w FIFO 0,25 0t0 1094 /run/systemd/inhibit/1.ref 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 21u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 23u a_inode 0,15 0 50 [eventpoll:24,25] 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 24u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.745 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 25u pack 598 0t0 ARP type=SOCK_DGRAM 2024-04-20T21:47:08.746 INFO:teuthology.orchestra.run.smithi102.stdout:NetworkMa 664 680 gdbus root 26u IPv4 599 0t0 UDP smithi102.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2024-04-20T21:47:08.746 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.746 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.746 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root txt REG 8,1 61904 9699 /usr/sbin/irqbalance 2024-04-20T21:47:08.746 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.746 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.746 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.746 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root 1u unix 0xffff88811042e180 0t0 522 type=STREAM (CONNECTED) 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root 2u unix 0xffff88811042e180 0t0 522 type=STREAM (CONNECTED) 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root 3u a_inode 0,15 0 50 [eventfd:1] 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root 4u a_inode 0,15 0 50 [eventfd:2] 2024-04-20T21:47:08.747 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root 5u unix 0xffff88810abfb480 0t0 8678 /run/irqbalance/irqbalance667.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.748 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 root 6u unix 0xffff888100ea9e00 0t0 8818 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.748 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.748 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.748 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root txt REG 8,1 61904 9699 /usr/sbin/irqbalance 2024-04-20T21:47:08.748 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.748 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.748 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.748 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root 1u unix 0xffff88811042e180 0t0 522 type=STREAM (CONNECTED) 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root 2u unix 0xffff88811042e180 0t0 522 type=STREAM (CONNECTED) 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root 3u a_inode 0,15 0 50 [eventfd:1] 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root 4u a_inode 0,15 0 50 [eventfd:2] 2024-04-20T21:47:08.749 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root 5u unix 0xffff88810abfb480 0t0 8678 /run/irqbalance/irqbalance667.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.750 INFO:teuthology.orchestra.run.smithi102.stdout:irqbalanc 667 672 gmain root 6u unix 0xffff888100ea9e00 0t0 8818 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.750 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.750 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.750 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt txt REG 8,1 32280 8775 /usr/bin/lsmd 2024-04-20T21:47:08.750 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.750 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt mem REG 8,1 56912 8294 /usr/lib64/libconfig.so.11.0.2 2024-04-20T21:47:08.750 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.750 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.751 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt 1u unix 0xffff88811056da00 0t0 6655 type=STREAM (CONNECTED) 2024-04-20T21:47:08.751 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt 2u unix 0xffff88811056da00 0t0 6655 type=STREAM (CONNECTED) 2024-04-20T21:47:08.751 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt 3uW REG 0,25 0 1055 /run/lsm/ipc/.lsmd-ipc-lock 2024-04-20T21:47:08.751 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt 5u unix 0xffff88811056cb00 0t0 6659 /var/run/lsm/ipc/sim type=STREAM (LISTEN) 2024-04-20T21:47:08.751 INFO:teuthology.orchestra.run.smithi102.stdout:lsmd 668 libstoragemgmt 6u unix 0xffff88810abf8780 0t0 8682 /var/run/lsm/ipc/simc type=STREAM (LISTEN) 2024-04-20T21:47:08.751 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.751 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.751 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root txt REG 8,1 630408 9978 /usr/sbin/smartd 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.752 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.753 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.753 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.753 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.753 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.753 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.753 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.753 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.753 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.754 INFO:teuthology.orchestra.run.smithi102.stdout:smartd 669 root 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.754 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.754 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.754 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root txt REG 8,1 303000 18592 /usr/lib/systemd/systemd-logind 2024-04-20T21:47:08.754 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root DEL REG 8,1 62234 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-20T21:47:08.754 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root DEL REG 8,1 62167 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-20T21:47:08.754 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:08.754 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.758 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.758 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-20T21:47:08.758 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.758 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 1u unix 0xffff88810ad19e00 0t0 2795 type=STREAM (CONNECTED) 2024-04-20T21:47:08.758 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 2u unix 0xffff88810ad19e00 0t0 2795 type=STREAM (CONNECTED) 2024-04-20T21:47:08.758 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 3u unix 0xffff88810ad18f00 0t0 2820 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.758 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 4u a_inode 0,15 0 50 [eventpoll:5,6,7,8,9,10,11,12,13,14,15,16,17,18,20] 2024-04-20T21:47:08.758 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 5u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 6u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 7r REG 0,22 4096 25236 /sys/devices/virtual/tty/tty0/active 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 8u netlink 0t0 2831 KOBJECT_UEVENT 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 9r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 10r FIFO 0,25 0t0 1094 /run/systemd/inhibit/1.ref 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 11u netlink 0t0 2832 KOBJECT_UEVENT 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 12u netlink 0t0 2833 KOBJECT_UEVENT 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 13u netlink 0t0 2834 KOBJECT_UEVENT 2024-04-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 14u unix 0xffff88810ad1cb00 0t0 2835 type=STREAM (CONNECTED) 2024-04-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 15u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 16u CHR 13,65 0t0 136 /dev/input/event1 2024-04-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 17u CHR 13,64 0t0 135 /dev/input/event0 2024-04-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 18u CHR 13,66 0t0 157 /dev/input/event2 2024-04-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 19u CHR 4,6 0t0 25 /dev/tty6 2024-04-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:systemd-l 671 root 20r FIFO 0,25 0t0 1496 /run/systemd/sessions/4.ref 2024-04-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root txt REG 8,1 968944 12006 /usr/sbin/sshd 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root 1u unix 0xffff88810ac2a580 0t0 7605 type=STREAM (CONNECTED) 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root 2u unix 0xffff88810ac2a580 0t0 7605 type=STREAM (CONNECTED) 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root 3u IPv4 7627 0t0 TCP *:ssh (LISTEN) 2024-04-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 710 root 4u IPv6 7629 0t0 TCP *:ssh (LISTEN) 2024-04-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root txt REG 8,1 127680 13852 /usr/sbin/gssproxy 2024-04-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 70960 13842 /usr/lib64/libev.so.4.0.0 2024-04-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 16168 13849 /usr/lib64/libverto-libev.so.1.0.0 2024-04-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 15912 7247 /usr/lib64/libbasicobjects.so.0.1.0 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 20304 7451 /usr/lib64/libpath_utils.so.1.0.1 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 58392 7249 /usr/lib64/libcollection.so.4.1.1 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.767 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 142168 8878 /usr/lib64/libgssrpc.so.4.2 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 16056 7251 /usr/lib64/libref_array.so.1.2.1 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 120640 7453 /usr/lib64/libini_config.so.5.2.1 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 28472 8875 /usr/lib64/libverto.so.1.0.0 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 58368 7172 /usr/lib64/libpopt.so.0.0.1 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.768 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.769 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.769 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.769 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.769 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.769 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.769 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.769 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.769 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 1u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.770 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 2u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.770 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 3u unix 0xffff88810ab0da00 0t0 3501 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.770 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 4r FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.770 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 5u a_inode 0,15 0 50 [eventpoll:4,6,7,8] 2024-04-20T21:47:08.770 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 6u a_inode 0,15 0 50 [eventfd:12] 2024-04-20T21:47:08.770 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 7u unix 0xffff88810ab0d280 0t0 3502 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.770 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 8u unix 0xffff88810ab0f800 0t0 3503 /run/gssproxy.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.770 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 root 9w FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.771 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.771 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.771 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root txt REG 8,1 127680 13852 /usr/sbin/gssproxy 2024-04-20T21:47:08.771 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.771 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.771 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.771 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.771 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 70960 13842 /usr/lib64/libev.so.4.0.0 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 16168 13849 /usr/lib64/libverto-libev.so.1.0.0 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.772 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 15912 7247 /usr/lib64/libbasicobjects.so.0.1.0 2024-04-20T21:47:08.773 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 20304 7451 /usr/lib64/libpath_utils.so.1.0.1 2024-04-20T21:47:08.773 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 58392 7249 /usr/lib64/libcollection.so.4.1.1 2024-04-20T21:47:08.773 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.773 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.773 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.773 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.773 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.773 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 142168 8878 /usr/lib64/libgssrpc.so.4.2 2024-04-20T21:47:08.774 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.774 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 16056 7251 /usr/lib64/libref_array.so.1.2.1 2024-04-20T21:47:08.774 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 120640 7453 /usr/lib64/libini_config.so.5.2.1 2024-04-20T21:47:08.774 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 28472 8875 /usr/lib64/libverto.so.1.0.0 2024-04-20T21:47:08.774 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.774 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.774 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 58368 7172 /usr/lib64/libpopt.so.0.0.1 2024-04-20T21:47:08.774 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.775 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 1u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.776 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 2u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.776 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 3u unix 0xffff88810ab0da00 0t0 3501 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.776 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 4r FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.776 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 5u a_inode 0,15 0 50 [eventpoll:4,6,7,8] 2024-04-20T21:47:08.776 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 6u a_inode 0,15 0 50 [eventfd:12] 2024-04-20T21:47:08.776 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 7u unix 0xffff88810ab0d280 0t0 3502 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.776 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 8u unix 0xffff88810ab0f800 0t0 3503 /run/gssproxy.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.776 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 719 gssproxy root 9w FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root txt REG 8,1 127680 13852 /usr/sbin/gssproxy 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 70960 13842 /usr/lib64/libev.so.4.0.0 2024-04-20T21:47:08.777 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 16168 13849 /usr/lib64/libverto-libev.so.1.0.0 2024-04-20T21:47:08.778 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.778 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.778 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.778 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.778 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.778 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.778 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 15912 7247 /usr/lib64/libbasicobjects.so.0.1.0 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 20304 7451 /usr/lib64/libpath_utils.so.1.0.1 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 58392 7249 /usr/lib64/libcollection.so.4.1.1 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.779 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 142168 8878 /usr/lib64/libgssrpc.so.4.2 2024-04-20T21:47:08.780 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.780 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 16056 7251 /usr/lib64/libref_array.so.1.2.1 2024-04-20T21:47:08.780 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 120640 7453 /usr/lib64/libini_config.so.5.2.1 2024-04-20T21:47:08.780 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 28472 8875 /usr/lib64/libverto.so.1.0.0 2024-04-20T21:47:08.780 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.780 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.780 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 58368 7172 /usr/lib64/libpopt.so.0.0.1 2024-04-20T21:47:08.780 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.781 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 1u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 2u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 3u unix 0xffff88810ab0da00 0t0 3501 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 4r FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 5u a_inode 0,15 0 50 [eventpoll:4,6,7,8] 2024-04-20T21:47:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 6u a_inode 0,15 0 50 [eventfd:12] 2024-04-20T21:47:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 7u unix 0xffff88810ab0d280 0t0 3502 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 8u unix 0xffff88810ab0f800 0t0 3503 /run/gssproxy.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 720 gssproxy root 9w FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.783 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.783 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.783 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root txt REG 8,1 127680 13852 /usr/sbin/gssproxy 2024-04-20T21:47:08.783 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.783 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.783 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.783 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.783 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 70960 13842 /usr/lib64/libev.so.4.0.0 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 16168 13849 /usr/lib64/libverto-libev.so.1.0.0 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.784 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 15912 7247 /usr/lib64/libbasicobjects.so.0.1.0 2024-04-20T21:47:08.785 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 20304 7451 /usr/lib64/libpath_utils.so.1.0.1 2024-04-20T21:47:08.785 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 58392 7249 /usr/lib64/libcollection.so.4.1.1 2024-04-20T21:47:08.785 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.785 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.785 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.785 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.785 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.785 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 142168 8878 /usr/lib64/libgssrpc.so.4.2 2024-04-20T21:47:08.786 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.786 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 16056 7251 /usr/lib64/libref_array.so.1.2.1 2024-04-20T21:47:08.786 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 120640 7453 /usr/lib64/libini_config.so.5.2.1 2024-04-20T21:47:08.786 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 28472 8875 /usr/lib64/libverto.so.1.0.0 2024-04-20T21:47:08.786 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.786 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.786 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 58368 7172 /usr/lib64/libpopt.so.0.0.1 2024-04-20T21:47:08.786 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.787 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.787 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.787 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.787 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.787 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.787 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.787 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.787 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.788 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 1u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.788 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 2u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.788 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 3u unix 0xffff88810ab0da00 0t0 3501 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.788 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 4r FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.788 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 5u a_inode 0,15 0 50 [eventpoll:4,6,7,8] 2024-04-20T21:47:08.788 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 6u a_inode 0,15 0 50 [eventfd:12] 2024-04-20T21:47:08.788 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 7u unix 0xffff88810ab0d280 0t0 3502 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.788 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 8u unix 0xffff88810ab0f800 0t0 3503 /run/gssproxy.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.789 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 721 gssproxy root 9w FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.789 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.789 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.789 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root txt REG 8,1 127680 13852 /usr/sbin/gssproxy 2024-04-20T21:47:08.789 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.789 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.789 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.789 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 70960 13842 /usr/lib64/libev.so.4.0.0 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 16168 13849 /usr/lib64/libverto-libev.so.1.0.0 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.790 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.791 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 15912 7247 /usr/lib64/libbasicobjects.so.0.1.0 2024-04-20T21:47:08.791 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 20304 7451 /usr/lib64/libpath_utils.so.1.0.1 2024-04-20T21:47:08.791 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 58392 7249 /usr/lib64/libcollection.so.4.1.1 2024-04-20T21:47:08.791 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.791 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.791 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.791 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.791 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.792 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 142168 8878 /usr/lib64/libgssrpc.so.4.2 2024-04-20T21:47:08.792 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.792 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 16056 7251 /usr/lib64/libref_array.so.1.2.1 2024-04-20T21:47:08.792 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 120640 7453 /usr/lib64/libini_config.so.5.2.1 2024-04-20T21:47:08.792 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 28472 8875 /usr/lib64/libverto.so.1.0.0 2024-04-20T21:47:08.792 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.792 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.792 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 58368 7172 /usr/lib64/libpopt.so.0.0.1 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.793 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.794 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 1u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.794 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 2u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.794 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 3u unix 0xffff88810ab0da00 0t0 3501 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.794 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 4r FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.794 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 5u a_inode 0,15 0 50 [eventpoll:4,6,7,8] 2024-04-20T21:47:08.794 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 6u a_inode 0,15 0 50 [eventfd:12] 2024-04-20T21:47:08.794 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 7u unix 0xffff88810ab0d280 0t0 3502 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.794 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 8u unix 0xffff88810ab0f800 0t0 3503 /run/gssproxy.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.795 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 722 gssproxy root 9w FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.795 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.795 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.795 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root txt REG 8,1 127680 13852 /usr/sbin/gssproxy 2024-04-20T21:47:08.795 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.795 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.795 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.795 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.796 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 70960 13842 /usr/lib64/libev.so.4.0.0 2024-04-20T21:47:08.796 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 16168 13849 /usr/lib64/libverto-libev.so.1.0.0 2024-04-20T21:47:08.796 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.796 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.796 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.796 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.796 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.796 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 15912 7247 /usr/lib64/libbasicobjects.so.0.1.0 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 20304 7451 /usr/lib64/libpath_utils.so.1.0.1 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 58392 7249 /usr/lib64/libcollection.so.4.1.1 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.797 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.798 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 142168 8878 /usr/lib64/libgssrpc.so.4.2 2024-04-20T21:47:08.798 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.798 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 16056 7251 /usr/lib64/libref_array.so.1.2.1 2024-04-20T21:47:08.798 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 120640 7453 /usr/lib64/libini_config.so.5.2.1 2024-04-20T21:47:08.798 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 28472 8875 /usr/lib64/libverto.so.1.0.0 2024-04-20T21:47:08.798 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.798 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.798 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 58368 7172 /usr/lib64/libpopt.so.0.0.1 2024-04-20T21:47:08.799 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.799 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.799 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.799 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.799 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.799 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.799 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.799 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 1u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 2u unix 0xffff88810ac2f080 0t0 7601 type=STREAM (CONNECTED) 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 3u unix 0xffff88810ab0da00 0t0 3501 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 4r FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 5u a_inode 0,15 0 50 [eventpoll:4,6,7,8] 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 6u a_inode 0,15 0 50 [eventfd:12] 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 7u unix 0xffff88810ab0d280 0t0 3502 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.800 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 8u unix 0xffff88810ab0f800 0t0 3503 /run/gssproxy.sock type=STREAM (LISTEN) 2024-04-20T21:47:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:gssproxy 718 723 gssproxy root 9w FIFO 0,14 0t0 3506 pipe 2024-04-20T21:47:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root cwd DIR 8,1 4096 62221 /var/spool/at 2024-04-20T21:47:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root txt REG 8,1 32680 9927 /usr/sbin/atd 2024-04-20T21:47:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.801 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.802 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.802 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.802 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.802 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.802 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.802 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.802 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.802 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.803 INFO:teuthology.orchestra.run.smithi102.stdout:atd 725 root 3uW REG 0,25 4 1115 /run/atd.pid 2024-04-20T21:47:08.803 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.803 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.803 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root txt REG 8,1 78168 8715 /usr/sbin/crond 2024-04-20T21:47:08.803 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.803 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.803 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.803 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.804 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.805 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.806 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.806 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root 1u unix 0xffff88810ac2d280 0t0 7639 type=STREAM (CONNECTED) 2024-04-20T21:47:08.806 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root 2u unix 0xffff88810ac2d280 0t0 7639 type=STREAM (CONNECTED) 2024-04-20T21:47:08.806 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root 3uW REG 0,25 4 1114 /run/crond.pid 2024-04-20T21:47:08.806 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root 4u unix 0xffff88810ac2da00 0t0 7642 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.806 INFO:teuthology.orchestra.run.smithi102.stdout:crond 726 root 5r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.806 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.806 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe txt REG 8,1 80984 9093 /usr/sbin/nrpe 2024-04-20T21:47:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe 1w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe 2w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe 3u IPv4 2913 0t0 TCP *:nrpe (LISTEN) 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe 4u unix 0xffff888107873480 0t0 2905 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:nrpe 748 nrpe 5u IPv6 2914 0t0 TCP *:nrpe (LISTEN) 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 598426 62136 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-20T21:47:08.808 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.809 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.809 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.809 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.809 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.809 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.809 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.809 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.809 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-20T21:47:08.810 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.810 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-20T21:47:08.810 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.810 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.810 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.810 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-04-20T21:47:08.810 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.810 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.811 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 21219 62233 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 1u unix 0xffff888107873c00 0t0 2951 type=STREAM (CONNECTED) 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 2u unix 0xffff888107873c00 0t0 2951 type=STREAM (CONNECTED) 2024-04-20T21:47:08.812 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 3u unix 0xffff88810858d280 0t0 6828 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.813 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 4u a_inode 0,15 0 50 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-04-20T21:47:08.813 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 5u a_inode 0,15 0 50 [signalfd] 2024-04-20T21:47:08.813 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 6r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.813 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 7r DIR 0,26 0 3277 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-04-20T21:47:08.813 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 8u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.813 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 9u a_inode 0,15 0 50 [eventpoll:10,13] 2024-04-20T21:47:08.813 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 10r REG 0,21 0 6817 /proc/874/mountinfo 2024-04-20T21:47:08.813 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 11r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.814 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 12u unix 0xffff888100ea8780 0t0 8797 /run/user/1000/bus type=STREAM (LISTEN) 2024-04-20T21:47:08.814 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 13r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.814 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 14r REG 0,21 0 4026532078 /proc/swaps 2024-04-20T21:47:08.814 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 15u netlink 0t0 8779 KOBJECT_UEVENT 2024-04-20T21:47:08.814 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 16u unix 0xffff88810abfa580 0t0 8784 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-04-20T21:47:08.814 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 17u unix 0xffff88810abff080 0t0 8785 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.814 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 18u unix 0xffff88810abf8f00 0t0 8786 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.814 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 19u unix 0xffff88811056b480 0t0 8787 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-04-20T21:47:08.815 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 20u unix 0xffff88810ac2f800 0t0 8789 type=STREAM (CONNECTED) 2024-04-20T21:47:08.815 INFO:teuthology.orchestra.run.smithi102.stdout:systemd 874 ubuntu 21u a_inode 0,15 0 50 [timerfd] 2024-04-20T21:47:08.815 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.815 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.815 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-04-20T21:47:08.815 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:08.815 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:08.816 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:08.816 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:08.816 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:08.816 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:08.816 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 44264 30088 /usr/lib64/security/pam_namespace.so 2024-04-20T21:47:08.816 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 15296 30106 /usr/lib64/security/pam_umask.so 2024-04-20T21:47:08.816 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:08.816 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 27720 30096 /usr/lib64/security/pam_selinux.so 2024-04-20T21:47:08.817 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:08.817 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu DEL REG 8,1 846 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-20T21:47:08.817 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.817 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.817 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.817 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.817 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.817 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 15360 30084 /usr/lib64/security/pam_loginuid.so 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.818 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.819 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.819 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.819 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-20T21:47:08.819 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.819 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.819 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.819 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-20T21:47:08.819 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.820 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-20T21:47:08.820 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-04-20T21:47:08.820 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.820 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.820 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-20T21:47:08.820 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.820 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu DEL REG 8,1 848 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-20T21:47:08.820 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu 1u unix 0xffff888107873c00 0t0 2951 type=STREAM (CONNECTED) 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu 2u unix 0xffff888107873c00 0t0 2951 type=STREAM (CONNECTED) 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu 3u a_inode 0,15 0 50 [eventfd:9] 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu 4u a_inode 0,15 0 50 [eventfd:10] 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu 6w FIFO 0,14 0t0 2955 pipe 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:(sd-pam) 880 ubuntu 7u unix 0xffff88810858cb00 0t0 6820 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.821 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.822 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.822 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root txt REG 8,1 58248 11941 /usr/sbin/agetty 2024-04-20T21:47:08.822 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.822 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.822 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.822 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.822 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root 0u CHR 4,1 0t0 20 /dev/tty1 2024-04-20T21:47:08.822 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root 1u CHR 4,1 0t0 20 /dev/tty1 2024-04-20T21:47:08.823 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root 2u CHR 4,1 0t0 20 /dev/tty1 2024-04-20T21:47:08.823 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1122 root 4r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.823 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.823 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.823 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root txt REG 8,1 58248 11941 /usr/sbin/agetty 2024-04-20T21:47:08.823 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.823 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.823 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.824 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root 0u CHR 4,65 0t0 113 /dev/ttyS1 2024-04-20T21:47:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root 1u CHR 4,65 0t0 113 /dev/ttyS1 2024-04-20T21:47:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root 2u CHR 4,65 0t0 113 /dev/ttyS1 2024-04-20T21:47:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:agetty 1125 root 4r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.825 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.826 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.826 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root txt REG 8,1 800352 9743 /usr/sbin/rsyslogd 2024-04-20T21:47:08.826 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 0,25 8388608 59 /run/log/journal/7fc8828e06d0482694bcf4a447d13cb6/system.journal 2024-04-20T21:47:08.826 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 37784 54075 /usr/lib64/rsyslog/imjournal.so 2024-04-20T21:47:08.826 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 42336 54082 /usr/lib64/rsyslog/imuxsock.so 2024-04-20T21:47:08.826 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 32904 54083 /usr/lib64/rsyslog/lmnet.so 2024-04-20T21:47:08.826 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.826 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:08.827 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.828 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 52856 9685 /usr/lib64/libfastjson.so.4.3.0 2024-04-20T21:47:08.828 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 16032 9684 /usr/lib64/libestr.so.0.0.0 2024-04-20T21:47:08.828 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.828 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.828 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.828 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.828 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 1w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.828 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 2w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 3r CHR 1,9 0t0 9 /dev/urandom 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 4u unix 0xffff888107872d00 0t0 10269 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 5w REG 8,1 1844728 62206 /var/log/messages 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 6r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 7w REG 8,1 709949 62207 /var/log/secure 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 root 9r REG 0,25 8388608 59 /run/log/journal/7fc8828e06d0482694bcf4a447d13cb6/system.journal 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.829 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root txt REG 8,1 800352 9743 /usr/sbin/rsyslogd 2024-04-20T21:47:08.830 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 0,25 8388608 59 /run/log/journal/7fc8828e06d0482694bcf4a447d13cb6/system.journal 2024-04-20T21:47:08.830 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 37784 54075 /usr/lib64/rsyslog/imjournal.so 2024-04-20T21:47:08.830 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 42336 54082 /usr/lib64/rsyslog/imuxsock.so 2024-04-20T21:47:08.830 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 32904 54083 /usr/lib64/rsyslog/lmnet.so 2024-04-20T21:47:08.830 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.830 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.830 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.830 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.831 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.831 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.831 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.831 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.831 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.831 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:08.831 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 52856 9685 /usr/lib64/libfastjson.so.4.3.0 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 16032 9684 /usr/lib64/libestr.so.0.0.0 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 1w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 2w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 3r CHR 1,9 0t0 9 /dev/urandom 2024-04-20T21:47:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 4u unix 0xffff888107872d00 0t0 10269 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 5w REG 8,1 1844728 62206 /var/log/messages 2024-04-20T21:47:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 6r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 7w REG 8,1 709949 62207 /var/log/secure 2024-04-20T21:47:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1653 in:imjour root 9r REG 0,25 8388608 59 /run/log/journal/7fc8828e06d0482694bcf4a447d13cb6/system.journal 2024-04-20T21:47:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root txt REG 8,1 800352 9743 /usr/sbin/rsyslogd 2024-04-20T21:47:08.834 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 0,25 8388608 59 /run/log/journal/7fc8828e06d0482694bcf4a447d13cb6/system.journal 2024-04-20T21:47:08.834 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 37784 54075 /usr/lib64/rsyslog/imjournal.so 2024-04-20T21:47:08.834 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 42336 54082 /usr/lib64/rsyslog/imuxsock.so 2024-04-20T21:47:08.834 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 32904 54083 /usr/lib64/rsyslog/lmnet.so 2024-04-20T21:47:08.834 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.834 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.834 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.834 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 52856 9685 /usr/lib64/libfastjson.so.4.3.0 2024-04-20T21:47:08.835 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 16032 9684 /usr/lib64/libestr.so.0.0.0 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 1w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 2w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 3r CHR 1,9 0t0 9 /dev/urandom 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 4u unix 0xffff888107872d00 0t0 10269 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.836 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 5w REG 8,1 1844728 62206 /var/log/messages 2024-04-20T21:47:08.837 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 6r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.837 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 7w REG 8,1 709949 62207 /var/log/secure 2024-04-20T21:47:08.837 INFO:teuthology.orchestra.run.smithi102.stdout:rsyslogd 1648 1654 rs:main root 9r REG 0,25 8388608 59 /run/log/journal/7fc8828e06d0482694bcf4a447d13cb6/system.journal 2024-04-20T21:47:08.837 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.837 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.837 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.837 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.837 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.838 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.838 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.838 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.838 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.838 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.838 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.838 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.838 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.844 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.846 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.846 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.846 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.846 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.846 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.846 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.846 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7718 gmain polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.847 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.848 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.848 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.848 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.848 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.848 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.848 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.848 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.848 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.849 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.850 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.850 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.850 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.850 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.850 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.850 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.850 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.850 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.851 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.852 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.852 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7719 gdbus polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.852 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.852 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.852 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.852 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.852 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.852 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.853 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.855 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.856 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.856 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.856 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.856 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.856 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.856 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.856 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.856 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.857 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.857 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7720 JS\x20Hel polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.857 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.857 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.857 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.857 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.857 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.857 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.858 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.858 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.858 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.858 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.858 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.858 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.858 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.858 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.859 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.860 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.861 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.861 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.861 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.861 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.861 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.861 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.861 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.861 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.862 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.862 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7721 JS\x20Hel polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.862 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.862 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.862 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.862 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.862 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.862 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.863 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.863 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.863 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.863 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.863 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.863 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.863 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.863 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.864 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.865 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.865 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.865 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.865 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.865 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.865 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.865 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.865 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.866 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.866 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.866 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.866 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.866 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.866 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.866 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.866 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7722 JS\x20Hel polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.867 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.868 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.869 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.869 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.869 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.869 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.869 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.869 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.869 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.869 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.870 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.870 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.870 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.870 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.870 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.870 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.870 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.870 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.871 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.871 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.871 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.871 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.871 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.871 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.871 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.871 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7723 JS\x20Hel polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.872 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.873 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.873 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.873 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.873 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.873 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.873 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.873 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.873 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.874 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7724 JS\x20Hel polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.878 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.878 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.878 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.878 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.878 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.878 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.878 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.878 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7725 JS\x20Hel polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.881 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.882 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.882 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.882 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.882 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.882 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.882 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.882 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.882 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.883 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.884 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.884 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.884 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.884 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.884 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.884 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.884 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.884 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.885 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.885 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.885 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.885 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.885 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.885 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.885 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.885 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.886 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.886 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.886 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.886 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.886 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7726 JS\x20Hel polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.886 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.886 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.886 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.887 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.888 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.888 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.888 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.888 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.888 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.888 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.888 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.888 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.889 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.890 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.890 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.890 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.890 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.890 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.890 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.890 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.890 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.891 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.891 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.891 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.891 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7727 JS\x20Hel polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.891 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.891 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.891 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd txt REG 8,1 145440 46534 /usr/lib/polkit-1/polkitd 2024-04-20T21:47:08.891 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.892 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.892 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.892 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.892 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.892 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.892 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.892 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.892 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 489696 7483 /usr/lib64/libpcre.so.1.2.12 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 29097776 34975 /usr/lib64/polkit-1/libmozjs-78.so 2024-04-20T21:47:08.893 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.894 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 19592 7652 /usr/lib64/libgmodule-2.0.so.0.6800.4 2024-04-20T21:47:08.894 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.894 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:08.894 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.894 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 1303216 7650 /usr/lib64/libglib-2.0.so.0.6800.4 2024-04-20T21:47:08.894 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 375264 7654 /usr/lib64/libgobject-2.0.so.0.6800.4 2024-04-20T21:47:08.894 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 1945176 7648 /usr/lib64/libgio-2.0.so.0.6800.4 2024-04-20T21:47:08.894 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 131296 7661 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2024-04-20T21:47:08.895 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.895 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.895 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.895 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.895 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 3r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.895 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 4u a_inode 0,15 0 50 [eventfd:13] 2024-04-20T21:47:08.895 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 5u a_inode 0,15 0 50 [eventfd:14] 2024-04-20T21:47:08.895 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 6u unix 0xffff888108501e00 0t0 21847 type=STREAM (CONNECTED) 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 7u a_inode 0,15 0 50 [eventfd:15] 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 8r a_inode 0,15 0 50 inotify 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 9u a_inode 0,15 0 50 [eventfd:16] 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:polkitd 7716 7728 polkitd polkitd 10u unix 0xffff888107877800 0t0 22856 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8172 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8172 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8172 root txt unknown /proc/8172/exe 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8202 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.896 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8202 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.897 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8202 root txt unknown /proc/8202/exe 2024-04-20T21:47:08.897 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8231 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.897 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8231 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.897 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8231 root txt unknown /proc/8231/exe 2024-04-20T21:47:08.897 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8260 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.897 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8260 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.897 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8260 root txt unknown /proc/8260/exe 2024-04-20T21:47:08.897 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8289 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8289 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8289 root txt unknown /proc/8289/exe 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8453 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8453 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8453 root txt unknown /proc/8453/exe 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8454 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8454 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8454 root txt unknown /proc/8454/exe 2024-04-20T21:47:08.898 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8455 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8455 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8455 root txt unknown /proc/8455/exe 2024-04-20T21:47:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8456 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8456 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8456 root txt unknown /proc/8456/exe 2024-04-20T21:47:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8457 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8457 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8457 root txt unknown /proc/8457/exe 2024-04-20T21:47:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8458 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8458 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8458 root txt unknown /proc/8458/exe 2024-04-20T21:47:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8459 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8459 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8459 root txt unknown /proc/8459/exe 2024-04-20T21:47:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8460 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8460 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.901 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8460 root txt unknown /proc/8460/exe 2024-04-20T21:47:08.901 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8461 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.901 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8461 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.901 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 8461 root txt unknown /proc/8461/exe 2024-04-20T21:47:08.901 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 8462 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.901 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 8462 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.901 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 8462 root txt unknown /proc/8462/exe 2024-04-20T21:47:08.901 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.902 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.902 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root txt REG 8,1 968944 12006 /usr/sbin/sshd 2024-04-20T21:47:08.902 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:08.902 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 19632 30080 /usr/lib64/security/pam_lastlog.so 2024-04-20T21:47:08.902 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 15296 30106 /usr/lib64/security/pam_umask.so 2024-04-20T21:47:08.902 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.902 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.902 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 15376 30087 /usr/lib64/security/pam_motd.so 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 44264 30088 /usr/lib64/security/pam_namespace.so 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 27720 30096 /usr/lib64/security/pam_selinux.so 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:08.903 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 15360 30084 /usr/lib64/security/pam_loginuid.so 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 15344 30089 /usr/lib64/security/pam_nologin.so 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 19472 30098 /usr/lib64/security/pam_sepermit.so 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.904 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-20T21:47:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root 3u unix 0xffff88810ab0f080 0t0 24539 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root 4u IPv4 27329 0t0 TCP smithi102.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:56482 (ESTABLISHED) 2024-04-20T21:47:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root 6u unix 0xffff88810ab0cb00 0t0 24514 type=STREAM (CONNECTED) 2024-04-20T21:47:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root 7w FIFO 0,25 0t0 1496 /run/systemd/sessions/4.ref 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9392 root 8u unix 0xffff88810ab0e180 0t0 24543 type=STREAM (CONNECTED) 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu txt REG 8,1 968944 12006 /usr/sbin/sshd 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 19632 30080 /usr/lib64/security/pam_lastlog.so 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 15296 30106 /usr/lib64/security/pam_umask.so 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 15376 30087 /usr/lib64/security/pam_motd.so 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 44264 30088 /usr/lib64/security/pam_namespace.so 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 27720 30096 /usr/lib64/security/pam_selinux.so 2024-04-20T21:47:08.910 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 15360 30084 /usr/lib64/security/pam_loginuid.so 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 15344 30089 /usr/lib64/security/pam_nologin.so 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 19472 30098 /usr/lib64/security/pam_sepermit.so 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:08.911 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-20T21:47:08.912 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.912 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.912 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.912 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.912 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.912 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.912 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-20T21:47:08.912 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.913 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.914 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.914 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.914 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.914 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.914 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.914 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 915856 7605 /usr/lib64/libsystemd.so.0.35.0 2024-04-20T21:47:08.914 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.914 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 0u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 1u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 3u unix 0xffff88810ab0f080 0t0 24539 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 4u IPv4 27329 0t0 TCP smithi102.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:56482 (ESTABLISHED) 2024-04-20T21:47:08.915 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 5u unix 0xffff88810ab08f00 0t0 24542 type=STREAM (CONNECTED) 2024-04-20T21:47:08.916 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 6u unix 0xffff88810ab0cb00 0t0 24514 type=STREAM (CONNECTED) 2024-04-20T21:47:08.916 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 7w FIFO 0,25 0t0 1496 /run/systemd/sessions/4.ref 2024-04-20T21:47:08.916 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 11w FIFO 0,14 0t0 44839 pipe 2024-04-20T21:47:08.916 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 12r FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:08.916 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 13w FIFO 0,14 0t0 44843 pipe 2024-04-20T21:47:08.916 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 14r FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:08.916 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 15r FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:08.916 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 16w FIFO 0,14 0t0 53295 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 17r FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 18r FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 19w FIFO 0,14 0t0 51393 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 20r FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 21r FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 22w FIFO 0,14 0t0 51397 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 23r FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 24r FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 25w FIFO 0,14 0t0 53300 pipe 2024-04-20T21:47:08.918 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 26r FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:08.918 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 27r FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:08.918 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 29r FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:08.918 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 30r FIFO 0,14 0t0 115786 pipe 2024-04-20T21:47:08.918 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 31r FIFO 0,14 0t0 115791 pipe 2024-04-20T21:47:08.918 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 32r FIFO 0,14 0t0 115787 pipe 2024-04-20T21:47:08.918 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 34w FIFO 0,14 0t0 49881 pipe 2024-04-20T21:47:08.918 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 35r FIFO 0,14 0t0 115792 pipe 2024-04-20T21:47:08.919 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 36r FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:08.919 INFO:teuthology.orchestra.run.smithi102.stdout:sshd 9395 ubuntu 38r FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:08.919 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.919 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.919 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony txt REG 8,1 370264 9943 /usr/sbin/chronyd 2024-04-20T21:47:08.919 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 626240 7471 /usr/lib64/libhogweed.so.6.8 2024-04-20T21:47:08.919 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:08.919 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 94112 7384 /usr/lib64/libtasn1.so.6.6.0 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 2337504 7477 /usr/lib64/libgnutls.so.30.37.1 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.920 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony 1w CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony 2u CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony 3u unix 0xffff88817bc16180 0t0 31803 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony 5u IPv4 31808 0t0 UDP localhost:323 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony 6u IPv6 31809 0t0 UDP localhost:323 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony 7u unix 0xffff88817bc15a00 0t0 31810 /run/chrony/chronyd.sock type=DGRAM (CONNECTED) 2024-04-20T21:47:08.921 INFO:teuthology.orchestra.run.smithi102.stdout:chronyd 9455 chrony 8u sock 0,9 0t0 84171 protocol: UDP 2024-04-20T21:47:08.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph txt REG 8,1 15624 7981 /usr/bin/python3.9 2024-04-20T21:47:08.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:08.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph 1u unix 0xffff888105633480 0t0 36744 type=STREAM (CONNECTED) 2024-04-20T21:47:08.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-cras 9769 ceph 2u unix 0xffff888105633480 0t0 36744 type=STREAM (CONNECTED) 2024-04-20T21:47:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16409 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16409 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16409 root txt unknown /proc/16409/exe 2024-04-20T21:47:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16410 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16410 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16410 root txt unknown /proc/16410/exe 2024-04-20T21:47:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16411 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16411 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16411 root txt unknown /proc/16411/exe 2024-04-20T21:47:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16412 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16412 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16412 root txt unknown /proc/16412/exe 2024-04-20T21:47:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16413 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16413 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16413 root txt unknown /proc/16413/exe 2024-04-20T21:47:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16414 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16414 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16414 root txt unknown /proc/16414/exe 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16415 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16415 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16415 root txt unknown /proc/16415/exe 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16416 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16416 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16416 root txt unknown /proc/16416/exe 2024-04-20T21:47:08.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16528 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16528 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16528 root txt unknown /proc/16528/exe 2024-04-20T21:47:08.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16529 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16529 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16529 root txt unknown /proc/16529/exe 2024-04-20T21:47:08.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16530 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16530 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16530 root txt unknown /proc/16530/exe 2024-04-20T21:47:08.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16531 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16531 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16531 root txt unknown /proc/16531/exe 2024-04-20T21:47:08.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16532 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16532 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16532 root txt unknown /proc/16532/exe 2024-04-20T21:47:08.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16533 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16533 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16533 root txt unknown /proc/16533/exe 2024-04-20T21:47:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16534 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16534 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16534 root txt unknown /proc/16534/exe 2024-04-20T21:47:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16535 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16535 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16535 root txt unknown /proc/16535/exe 2024-04-20T21:47:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16643 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16643 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16643 root txt unknown /proc/16643/exe 2024-04-20T21:47:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16644 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16644 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16644 root txt unknown /proc/16644/exe 2024-04-20T21:47:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16645 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16645 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16645 root txt unknown /proc/16645/exe 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16646 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16646 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16646 root txt unknown /proc/16646/exe 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16647 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16647 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16647 root txt unknown /proc/16647/exe 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16648 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16648 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16648 root txt unknown /proc/16648/exe 2024-04-20T21:47:08.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16649 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16649 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16649 root txt unknown /proc/16649/exe 2024-04-20T21:47:08.933 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16650 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.933 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16650 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.933 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16650 root txt unknown /proc/16650/exe 2024-04-20T21:47:08.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16727 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16727 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16727 root txt unknown /proc/16727/exe 2024-04-20T21:47:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16728 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16728 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16728 root txt unknown /proc/16728/exe 2024-04-20T21:47:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16729 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16729 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16729 root txt unknown /proc/16729/exe 2024-04-20T21:47:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16730 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16730 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16730 root txt unknown /proc/16730/exe 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16731 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16731 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16731 root txt unknown /proc/16731/exe 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16732 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16732 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16732 root txt unknown /proc/16732/exe 2024-04-20T21:47:08.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16733 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16733 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16733 root txt unknown /proc/16733/exe 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16734 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16734 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:xfsaild/d 16734 root txt unknown /proc/16734/exe 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16780 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16780 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 16780 root txt unknown /proc/16780/exe 2024-04-20T21:47:08.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 17942 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 17942 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 17942 root txt unknown /proc/17942/exe 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 19104 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 19104 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 19104 root txt unknown /proc/19104/exe 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 20268 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 20268 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 20268 root txt unknown /proc/20268/exe 2024-04-20T21:47:08.937 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:08.938 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.938 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root txt REG 8,1 185464 9243 /usr/bin/sudo 2024-04-20T21:47:08.938 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:08.938 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.938 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.938 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.938 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.938 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:08.939 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:08.939 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:08.939 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:08.939 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:08.939 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:08.939 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:08.939 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:08.939 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.940 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.941 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.941 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.941 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.941 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.941 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.941 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.941 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.941 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 472552 47567 /usr/libexec/sudo/sudoers.so 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:08.942 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.943 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.943 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.943 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.943 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.943 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.943 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.943 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.943 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 128024 47563 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.944 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root 0r FIFO 0,14 0t0 44839 pipe 2024-04-20T21:47:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root 3r FIFO 0,14 0t0 44850 pipe 2024-04-20T21:47:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root 4w FIFO 0,14 0t0 44850 pipe 2024-04-20T21:47:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root 5u netlink 0t0 44857 AUDIT 2024-04-20T21:47:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22049 root 6u unix 0xffff88817bc6e900 0t0 44858 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root txt REG 8,1 185464 9243 /usr/bin/sudo 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:08.947 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:08.947 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:08.947 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:08.947 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:08.947 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:08.947 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:08.947 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.947 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 472552 47567 /usr/libexec/sudo/sudoers.so 2024-04-20T21:47:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.950 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:08.950 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.950 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.950 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:08.950 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:08.950 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.950 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:08.950 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:08.951 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 128024 47563 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root 0r FIFO 0,14 0t0 44843 pipe 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:08.952 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:08.953 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root 3r FIFO 0,14 0t0 48504 pipe 2024-04-20T21:47:08.953 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root 4w FIFO 0,14 0t0 48504 pipe 2024-04-20T21:47:08.953 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root 5u netlink 0t0 48509 AUDIT 2024-04-20T21:47:08.953 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22065 root 6u unix 0xffff888162352d00 0t0 48510 type=DGRAM (CONNECTED) 2024-04-20T21:47:08.953 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:08.953 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.953 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root txt REG 8,1 15624 7981 /usr/bin/python3.9 2024-04-20T21:47:08.953 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.954 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:08.954 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.954 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.954 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.954 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.954 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.954 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.954 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.955 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root 0r FIFO 0,14 0t0 44839 pipe 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22087 root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root txt REG 8,1 15624 7981 /usr/bin/python3.9 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:08.956 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root 0r FIFO 0,14 0t0 44843 pipe 2024-04-20T21:47:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:08.959 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22102 root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:08.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:08.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:08.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:08.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:08.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:08.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:08.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:08.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:08.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:08.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:08.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:08.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:08.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:08.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:08.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:08.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:08.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:08.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:08.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:08.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:08.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:08.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:08.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:08.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:08.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:08.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:08.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:08.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:08.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:08.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:08.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:08.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:08.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:08.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:08.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:08.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:08.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:08.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:08.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:08.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:08.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:08.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:08.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:08.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:08.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:08.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:08.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:08.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:08.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:08.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:08.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:08.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:08.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:08.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:08.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:08.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:08.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:08.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:08.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:08.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:08.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:08.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:08.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:08.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:08.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:08.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:08.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:08.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:08.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:08.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:08.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:08.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22107 log root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:08.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:08.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:08.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:08.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:08.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:08.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:08.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:08.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:08.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:08.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:08.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:08.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:08.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:08.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:08.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:08.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:08.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:08.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:08.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:08.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:08.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:08.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:08.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:08.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:08.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:08.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:08.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:08.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:08.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22108 service root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:08.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:08.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:08.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:08.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:08.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:08.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:08.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:08.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:08.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:08.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:08.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:08.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:08.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:08.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:08.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:08.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:08.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:08.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:08.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:08.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:08.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:08.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:08.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:08.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:08.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:08.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:08.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22109 admin_soc root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22110 signal_ha root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22111 rocksdb:l root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22112 rocksdb:h root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22128 ceph-mon root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22129 fn_monsto root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22130 msgr-work root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22131 msgr-work root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22132 msgr-work root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22133 OpHistory root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22134 fin root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22135 safe_time root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22136 cpu_tp root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22137 cpu_tp root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22138 cpu_tp root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22139 cpu_tp root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22140 ms_dispat root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22141 ms_local root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22142 safe_time root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22143 ms_dispat root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 22144 ms_local root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root txt REG 8,1 11309240 63407 /usr/bin/ceph-mon 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 0,15 46043 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 1w FIFO 0,14 0t0 44840 pipe 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 2w FIFO 0,14 0t0 44841 pipe 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 3r FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 4w FIFO 0,14 0t0 46038 pipe 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 5w REG 8,1 100947328 393329 /var/log/ceph/ceph-mon.a.log 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 6uW REG 0,25 6 1603 /run/ceph/ceph-mon.a.pid 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 7r FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 8w FIFO 0,14 0t0 46040 pipe 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 9u unix 0xffff88810a755280 0t0 46041 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 10r FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 11w FIFO 0,14 0t0 46042 pipe 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 12u a_inode 0,15 0 46043 [io_uring] 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 13r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 14uW REG 253,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 15r DIR 253,4 4096 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 16r REG 253,4 4052311 50331788 /var/lib/ceph/mon/ceph-a/store.db/000042.sst 2024-04-20T21:47:09.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 18w REG 253,4 2607 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 19u a_inode 0,15 0 50 [eventpoll:20,28,29,33] 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 20r FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 21w FIFO 0,14 0t0 46049 pipe 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 22u a_inode 0,15 0 50 [eventpoll:23,40] 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 23r FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 24w FIFO 0,14 0t0 46050 pipe 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 25u a_inode 0,15 0 50 [eventpoll:26,30,41] 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 26r FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 27w FIFO 0,14 0t0 46051 pipe 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 28u IPv4 47086 0t0 TCP smithi102.front.sepia.ceph.com:3300 (LISTEN) 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 29u IPv4 47087 0t0 TCP smithi102.front.sepia.ceph.com:smc-https (LISTEN) 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 30u IPv4 49542 0t0 TCP smithi102.front.sepia.ceph.com:39552->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 31r FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 32w FIFO 0,14 0t0 51258 pipe 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 33u IPv4 44865 0t0 TCP smithi102.front.sepia.ceph.com:45348->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 34r FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 35w FIFO 0,14 0t0 51259 pipe 2024-04-20T21:47:09.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 36r FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 37w FIFO 0,14 0t0 51260 pipe 2024-04-20T21:47:09.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 38w REG 8,1 130606 393332 /var/log/ceph/ceph.audit.log 2024-04-20T21:47:09.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 39w REG 8,1 224765 393323 /var/log/ceph/ceph.log 2024-04-20T21:47:09.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 40u IPv4 82933 0t0 TCP smithi102.front.sepia.ceph.com:55010->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 41u IPv4 47819 0t0 TCP smithi102.front.sepia.ceph.com:3300->smithi102.front.sepia.ceph.com:50040 (ESTABLISHED) 2024-04-20T21:47:09.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon 22105 40007 rstore_co root 43w REG 253,4 1483997 50331785 /var/lib/ceph/mon/ceph-a/store.db/000040.log 2024-04-20T21:47:09.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22145 log root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22147 msgr-work root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22148 msgr-work root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22149 msgr-work root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22154 service root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22155 admin_soc root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22156 ceph_time root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22157 fn_anonym root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22158 signal_ha root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22159 mgrsb-fin root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22161 ms_dispat root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22162 ms_local root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22163 io_contex root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22164 io_contex root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22165 safe_time root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22166 safe_time root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22167 safe_time root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22168 fn_anonym root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22169 flusher root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root txt REG 8,1 4766000 63442 /usr/bin/ceph-mgr 2024-04-20T21:47:09.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 783584 401080 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 331936 401058 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 41112936 63052 /usr/lib64/libopenblaso-r0.3.21.so 2024-04-20T21:47:09.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 654400 401091 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 7042240 400449 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2024-04-20T21:47:09.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 2883944 62660 /usr/lib64/libgfortran.so.5.0.0 2024-04-20T21:47:09.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 3870704 63064 /usr/lib64/libflexiblas.so.3.0 2024-04-20T21:47:09.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 3784896 400499 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 294392 7381 /usr/lib64/libgomp.so.1.0.0 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 382824 400450 /usr/lib64/flexiblas/libflexiblas_netlib.so 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 299944 62656 /usr/lib64/libquadmath.so.0.0.0 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 384760 8947 /usr/lib64/libdevmapper.so.1.02 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 554440 8737 /usr/lib64/libcryptsetup.so.12.9.0 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 8231664 63215 /usr/lib64/librbd.so.1.19.0 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 1129584 63218 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 357336 7473 /usr/lib64/libnettle.so.8.8 2024-04-20T21:47:09.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 2154416 524833 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2024-04-20T21:47:09.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 2177136 62324 /usr/lib64/libcephfs.so.2.0.0 2024-04-20T21:47:09.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 721976 524831 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-04-20T21:47:09.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 78952 39964 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 65240 401745 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 60264 401085 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 237384 401060 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 87608 401083 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 96304 401081 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 283680 36207 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 70784 401082 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 177920 401086 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 73976 400866 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 155704 400968 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 132136 400498 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 75928 7267 /usr/lib64/libjson-c.so.5.0.0 2024-04-20T21:47:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 40328 400447 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2024-04-20T21:47:09.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 32104 39986 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 130560 36204 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 20504 400969 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 198064 7258 /usr/lib64/libexpat.so.1.8.10 2024-04-20T21:47:09.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 20952 36211 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 119784 34895 /usr/lib64/ossl-modules/legacy.so 2024-04-20T21:47:09.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 124280 39969 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 71536 36221 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 64328 36190 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 15464 36201 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 16192 393813 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 1116976 39987 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 213488 39975 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 20648 39971 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 136048 9292 /usr/lib64/libyaml-0.so.2.0.9 2024-04-20T21:47:09.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 246392 56894 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 515320 62668 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 38856 36203 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 41496 36217 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 24272 39972 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 198136 63427 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 49328 525574 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2024-04-20T21:47:09.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 50344 39959 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 69168 36209 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 36440 7195 /usr/lib64/libuuid.so.1.3.0 2024-04-20T21:47:09.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 1129040 62326 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 16104 36192 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 15624 39977 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 97064 39974 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 101336 39973 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 68120 36216 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 107864 36206 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 50168 39966 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 15680 36212 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 1954224 62321 /usr/lib64/librados.so.2.0.0 2024-04-20T21:47:09.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:09.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 1294816 7203 /usr/lib64/libsqlite3.so.0.8.6 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 435584 62619 /usr/lib64/libcephsqlite.so 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 1w FIFO 0,14 0t0 44844 pipe 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 2w FIFO 0,14 0t0 44845 pipe 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 3r FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 4w FIFO 0,14 0t0 50401 pipe 2024-04-20T21:47:09.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,25] 2024-04-20T21:47:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 6r FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 7w FIFO 0,14 0t0 44867 pipe 2024-04-20T21:47:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,23] 2024-04-20T21:47:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 9r FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 10w FIFO 0,14 0t0 44868 pipe 2024-04-20T21:47:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12] 2024-04-20T21:47:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 12r FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 13w FIFO 0,14 0t0 44869 pipe 2024-04-20T21:47:09.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 14w REG 8,1 1951364 393330 /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:47:09.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 15uW REG 0,25 6 1605 /run/ceph/ceph-mgr.y.pid 2024-04-20T21:47:09.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 16r FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 17w FIFO 0,14 0t0 49544 pipe 2024-04-20T21:47:09.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 18u unix 0xffff88815c744b00 0t0 49545 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 19r FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 20w FIFO 0,14 0t0 49549 pipe 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 21r FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 22w FIFO 0,14 0t0 49550 pipe 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 23u IPv4 113224 0t0 TCP smithi102.front.sepia.ceph.com:57270->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr 22106 22170 safe_time root 25u IPv4 47089 0t0 TCP smithi102.front.sepia.ceph.com:39572->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root txt REG 8,1 185464 9243 /usr/bin/sudo 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:09.675 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.676 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.676 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:09.676 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.676 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:09.676 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:09.676 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:09.676 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:09.676 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:09.677 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:09.677 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:09.677 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:09.677 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.677 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.677 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.677 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:09.677 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.679 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.679 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.679 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.679 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.679 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:09.679 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.679 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.679 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.680 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:09.680 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.680 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:09.680 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 472552 47567 /usr/libexec/sudo/sudoers.so 2024-04-20T21:47:09.680 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.680 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:09.680 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:09.680 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:09.681 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:09.681 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:09.681 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:09.681 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:09.681 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:09.681 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.681 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:09.681 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 128024 47563 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-04-20T21:47:09.682 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.682 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:09.682 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:09.682 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.682 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.682 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:09.682 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root 0r FIFO 0,14 0t0 53295 pipe 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root 3r FIFO 0,14 0t0 49773 pipe 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root 4w FIFO 0,14 0t0 49773 pipe 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root 5u netlink 0t0 52255 AUDIT 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22274 root 6u unix 0xffff88810a757080 0t0 52256 type=DGRAM (CONNECTED) 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.683 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root txt REG 8,1 185464 9243 /usr/bin/sudo 2024-04-20T21:47:09.684 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:09.684 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.684 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.684 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:09.684 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.684 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:09.684 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:09.684 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:09.685 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:09.685 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:09.685 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.685 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:09.685 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:09.685 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:09.685 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:09.685 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:09.688 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 472552 47567 /usr/libexec/sudo/sudoers.so 2024-04-20T21:47:09.688 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.688 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.688 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:09.688 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:09.688 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:09.688 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:09.688 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:09.689 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:09.689 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:09.689 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:09.689 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:09.689 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:09.689 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:09.689 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.689 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:09.690 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 128024 47563 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-04-20T21:47:09.690 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.690 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:09.690 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:09.690 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.690 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.690 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:09.691 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root 0r FIFO 0,14 0t0 51393 pipe 2024-04-20T21:47:09.691 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:09.691 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:09.691 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root 3r FIFO 0,14 0t0 53321 pipe 2024-04-20T21:47:09.691 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root 4w FIFO 0,14 0t0 53321 pipe 2024-04-20T21:47:09.691 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root 5u netlink 0t0 49780 AUDIT 2024-04-20T21:47:09.691 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22276 root 6u unix 0xffff88815c746180 0t0 49781 type=DGRAM (CONNECTED) 2024-04-20T21:47:09.691 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root txt REG 8,1 185464 9243 /usr/bin/sudo 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:09.692 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:09.693 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:09.693 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:09.693 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:09.693 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:09.693 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:09.693 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:09.693 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:09.694 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:09.694 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:09.694 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.694 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.694 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.694 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.694 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.694 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.695 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.695 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.695 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.695 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.695 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.695 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:09.695 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.695 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 472552 47567 /usr/libexec/sudo/sudoers.so 2024-04-20T21:47:09.696 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.697 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:09.697 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:09.697 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:09.697 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:09.697 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:09.697 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:09.697 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:09.698 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:09.698 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.698 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:09.698 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 128024 47563 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-04-20T21:47:09.698 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.698 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:09.698 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:09.698 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.699 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.699 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:09.699 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root 0r FIFO 0,14 0t0 51397 pipe 2024-04-20T21:47:09.699 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:09.699 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:09.699 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root 3r FIFO 0,14 0t0 47767 pipe 2024-04-20T21:47:09.699 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root 4w FIFO 0,14 0t0 47767 pipe 2024-04-20T21:47:09.699 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root 5u netlink 0t0 48593 AUDIT 2024-04-20T21:47:09.700 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22281 root 6u unix 0xffff888162353c00 0t0 48594 type=DGRAM (CONNECTED) 2024-04-20T21:47:09.700 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.700 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.700 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root txt REG 8,1 185464 9243 /usr/bin/sudo 2024-04-20T21:47:09.700 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:09.700 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:09.700 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.701 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:09.701 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.701 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:09.701 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:09.701 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:09.701 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:09.701 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:09.701 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:09.702 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:09.702 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:09.702 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:09.702 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:09.702 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.702 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.702 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.702 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:09.703 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.703 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.703 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.703 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.703 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.703 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.703 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.703 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:09.704 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.704 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.704 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:09.704 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.704 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.704 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.704 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.704 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.705 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:09.705 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:09.705 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 472552 47567 /usr/libexec/sudo/sudoers.so 2024-04-20T21:47:09.705 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.705 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:09.705 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:09.705 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:09.705 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:09.706 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:09.706 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:09.706 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:09.706 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:09.706 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.706 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:09.706 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 128024 47563 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-04-20T21:47:09.706 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.707 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:09.707 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:09.707 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.707 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.707 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:09.707 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root 0r FIFO 0,14 0t0 53300 pipe 2024-04-20T21:47:09.707 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:09.707 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:09.708 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root 3r FIFO 0,14 0t0 48588 pipe 2024-04-20T21:47:09.708 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root 4w FIFO 0,14 0t0 48588 pipe 2024-04-20T21:47:09.708 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root 5u netlink 0t0 48599 AUDIT 2024-04-20T21:47:09.708 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 22286 root 6u unix 0xffff888162351680 0t0 48600 type=DGRAM (CONNECTED) 2024-04-20T21:47:09.708 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.708 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.708 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root txt REG 8,1 15624 7981 /usr/bin/python3.9 2024-04-20T21:47:09.708 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.709 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.710 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.710 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.710 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.710 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.710 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.710 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.710 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root 0r FIFO 0,14 0t0 53295 pipe 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22356 root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root txt REG 8,1 15624 7981 /usr/bin/python3.9 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.711 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.712 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.712 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.712 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.712 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.712 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.712 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.712 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.712 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.713 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.713 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.713 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.713 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.713 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.713 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.713 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.713 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.714 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.714 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root 0r FIFO 0,14 0t0 51393 pipe 2024-04-20T21:47:09.714 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:09.714 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22358 root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:09.714 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.714 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.714 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root txt REG 8,1 15624 7981 /usr/bin/python3.9 2024-04-20T21:47:09.714 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.715 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.716 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.717 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root 0r FIFO 0,14 0t0 51397 pipe 2024-04-20T21:47:09.717 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:09.717 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22362 root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:09.717 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.717 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.717 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root txt REG 8,1 15624 7981 /usr/bin/python3.9 2024-04-20T21:47:09.717 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:09.717 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:09.718 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.718 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.718 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.718 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.718 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.718 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.718 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.718 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.719 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.719 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.719 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:09.719 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.719 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:09.719 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.719 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:09.719 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root 0r FIFO 0,14 0t0 53300 pipe 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 22363 root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 14w REG 8,1 1509610730 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 14w REG 8,1 1509541031 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22382 log root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 14w REG 8,1 1509541031 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22384 msgr-work root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 14w REG 8,1 1509541031 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22385 msgr-work root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 14w REG 8,1 1509541319 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22386 msgr-work root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 14w REG 8,1 1509541319 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22394 service root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 14w REG 8,1 1509541319 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22395 admin_soc root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 14w REG 8,1 1509541319 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22448 signal_ha root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 14w REG 8,1 1509541319 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22449 io_contex root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 14w REG 8,1 1509541319 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22450 io_contex root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 14w REG 8,1 1509541319 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22462 OpHistory root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 14w REG 8,1 1509541607 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:09.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:09.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:09.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:09.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:09.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:09.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:09.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:09.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:09.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:09.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:09.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:09.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:09.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:09.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:09.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:09.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22463 ceph_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:09.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:09.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:09.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:09.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:09.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:09.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:09.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:09.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:09.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:09.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:09.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:09.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:09.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:09.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:09.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:09.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:09.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:09.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:09.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:09.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:09.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:09.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:09.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:09.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:09.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:09.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:09.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:09.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:09.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:09.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:09.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:09.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:09.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:09.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:09.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:09.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:09.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:09.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:09.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:09.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:09.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:09.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:09.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:09.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:09.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:09.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:09.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:09.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 14w REG 8,1 1509541607 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:09.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:09.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:09.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:09.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:09.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:09.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:09.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:09.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:09.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:09.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:09.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:09.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:09.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:09.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:09.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:09.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:09.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:09.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:09.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:09.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:09.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22491 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 14w REG 8,1 1509541607 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22519 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 14w REG 8,1 1509541607 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22529 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 14w REG 8,1 1509541607 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22537 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 14w REG 8,1 1509541607 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22545 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 14w REG 8,1 1509541895 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22555 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 14w REG 8,1 1509541895 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22563 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 14w REG 8,1 1509541895 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22953 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 14w REG 8,1 1509541895 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22954 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 14w REG 8,1 1509541895 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22955 safe_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 14w REG 8,1 1509541895 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22956 safe_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 14w REG 8,1 1509541895 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22957 safe_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 14w REG 8,1 1509542183 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22958 safe_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 14w REG 8,1 1509542183 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22959 fn_anonym root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 14w REG 8,1 1509542183 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22962 rocksdb:l root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 14w REG 8,1 1509542183 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22963 rocksdb:l root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 14w REG 8,1 1509542183 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22964 rocksdb:l root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 14w REG 8,1 1509542183 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 22965 rocksdb:h root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 14w REG 8,1 1509542183 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 24431 bstore_ai root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 14w REG 8,1 1509542587 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25245 bstore_ai root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 14w REG 8,1 1509542587 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25426 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 14w REG 8,1 1509542587 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25427 cfin root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 14w REG 8,1 1509542587 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25428 bstore_kv root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 14w REG 8,1 1509542587 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25429 bstore_kv root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 14w REG 8,1 1509542587 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25430 bstore_me root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 14w REG 8,1 1509542587 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25431 ms_dispat root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 14w REG 8,1 1509542875 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25432 ms_local root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 14w REG 8,1 1509542875 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25433 safe_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 14w REG 8,1 1509542875 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25434 safe_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 14w REG 8,1 1509542875 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25435 ms_dispat root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 14w REG 8,1 1509542875 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25436 ms_local root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 14w REG 8,1 1509542875 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25437 ms_dispat root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 14w REG 8,1 1509543163 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25438 ms_local root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 14w REG 8,1 1509543163 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25439 ms_dispat root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 14w REG 8,1 1509543163 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25440 ms_local root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 14w REG 8,1 1509543163 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25441 ms_dispat root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 14w REG 8,1 1509543163 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25442 ms_local root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.789 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:47:10.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 14w REG 8,1 1509543163 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25443 ms_dispat root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 14w REG 8,1 1509543163 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25444 ms_local root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 14w REG 8,1 1509543451 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25445 ms_dispat root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 14w REG 8,1 1509557231 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25446 ms_local root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 14w REG 8,1 1509557231 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25447 fn_anonym root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 14w REG 8,1 1509557231 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25448 finisher root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 14w REG 8,1 1509557231 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25449 safe_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:10.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:10.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:10.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 14w REG 8,1 1509571377 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:10.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:10.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:10.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:10.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:10.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:10.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:10.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:10.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:10.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:10.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:10.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:10.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:10.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:10.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:10.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:10.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:10.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:10.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:10.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:10.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25450 safe_time root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:10.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:10.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:10.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:10.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:10.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:10.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:10.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:10.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:10.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:10.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:10.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:10.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:10.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:10.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:10.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:10.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:10.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:10.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:10.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:10.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:10.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:10.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:10.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:10.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:10.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:10.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:10.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:10.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:10.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:10.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:10.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:10.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:10.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:10.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:10.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:10.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:10.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:10.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:10.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 14w REG 8,1 1509596627 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25451 ceph-osd root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 14w REG 8,1 1509596915 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25452 osd_srv_a root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 14w REG 8,1 1509600996 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25453 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 14w REG 8,1 1509605574 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25454 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 14w REG 8,1 1509605574 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25455 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 14w REG 8,1 1509605574 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25456 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 14w REG 8,1 1509605574 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25457 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 14w REG 8,1 1509605862 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25458 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 14w REG 8,1 1509605862 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25459 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 14w REG 8,1 1509605862 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25460 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 14w REG 8,1 1509605862 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25461 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 14w REG 8,1 1509605862 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25462 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 14w REG 8,1 1509605862 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25463 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 14w REG 8,1 1509605862 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25464 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 14w REG 8,1 1509610730 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25465 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 14w REG 8,1 1509610730 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25466 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 14w REG 8,1 1509610730 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25467 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 14w REG 8,1 1509610730 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25468 tp_osd_tp root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root DEL REG 0,18 48639 /[aio] 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root DEL REG 0,18 48636 /[aio] 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 0,15 51478 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 1w FIFO 0,14 0t0 53296 pipe 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 2w FIFO 0,14 0t0 53297 pipe 2024-04-20T21:47:11.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 3r FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 4w FIFO 0,14 0t0 53357 pipe 2024-04-20T21:47:11.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,67,69,73,75,77,83,84,87,90,97,101,103] 2024-04-20T21:47:11.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 6r FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 7w FIFO 0,14 0t0 53359 pipe 2024-04-20T21:47:11.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 8u a_inode 0,15 0 50 [eventpoll:9,59,68,70,72,76,78,79,80,81,92,94,95,96] 2024-04-20T21:47:11.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 9r FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 10w FIFO 0,14 0t0 53360 pipe 2024-04-20T21:47:11.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 11u a_inode 0,15 0 50 [eventpoll:12,62,65,71,74,82,85,86,88,89,91,93,98,99] 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 12r FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 13w FIFO 0,14 0t0 53361 pipe 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 14w REG 8,1 1509610730 393325 /var/log/ceph/ceph-osd.0.log 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 15uW REG 0,25 6 1610 /run/ceph/ceph-osd.0.pid 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 16r FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 17w FIFO 0,14 0t0 51431 pipe 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 18u unix 0xffff888100eac380 0t0 51432 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 19u IPv4 50512 0t0 TCP *:6808 (LISTEN) 2024-04-20T21:47:11.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 20u IPv4 50522 0t0 TCP *:6809 (LISTEN) 2024-04-20T21:47:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 21u IPv4 50533 0t0 TCP *:6810 (LISTEN) 2024-04-20T21:47:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 22u IPv4 50545 0t0 TCP *:6811 (LISTEN) 2024-04-20T21:47:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 23u IPv4 50558 0t0 TCP *:6812 (LISTEN) 2024-04-20T21:47:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 24u IPv4 50572 0t0 TCP *:6813 (LISTEN) 2024-04-20T21:47:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 25u IPv4 50587 0t0 TCP *:6814 (LISTEN) 2024-04-20T21:47:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 26u IPv4 50603 0t0 TCP *:6815 (LISTEN) 2024-04-20T21:47:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 27r FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 28w FIFO 0,14 0t0 44931 pipe 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 29r FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 30w FIFO 0,14 0t0 44932 pipe 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 31u sock 0,9 0t0 51475 protocol: UDPv6 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 58u a_inode 0,15 0 51478 [io_uring] 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 59u IPv4 52506 0t0 TCP smithi102.front.sepia.ceph.com:39676->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 60r FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 61w FIFO 0,14 0t0 51491 pipe 2024-04-20T21:47:11.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 62u IPv4 97616 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40956 (ESTABLISHED) 2024-04-20T21:47:11.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 63r FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 64w FIFO 0,14 0t0 51492 pipe 2024-04-20T21:47:11.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 65u IPv4 97637 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38908 (ESTABLISHED) 2024-04-20T21:47:11.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 66u IPv4 83015 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:44310 (ESTABLISHED) 2024-04-20T21:47:11.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 67u IPv4 110799 0t0 TCP smithi102.front.sepia.ceph.com:40476->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 68u IPv4 88214 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:55950 (ESTABLISHED) 2024-04-20T21:47:11.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 69u IPv4 97640 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:40970 (ESTABLISHED) 2024-04-20T21:47:11.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 70u IPv4 85946 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51496 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 71u IPv4 108494 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi193.front.sepia.ceph.com:47884 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 72u IPv4 85949 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:43676 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 73u IPv4 108497 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:46114 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 74u IPv4 109047 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi102.front.sepia.ceph.com:37160 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 75u IPv4 106778 0t0 TCP smithi102.front.sepia.ceph.com:49544->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 76u IPv4 111680 0t0 TCP smithi102.front.sepia.ceph.com:52708->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 77u IPv4 112744 0t0 TCP smithi102.front.sepia.ceph.com:49116->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 78u IPv4 108717 0t0 TCP smithi102.front.sepia.ceph.com:42326->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 79u IPv4 107376 0t0 TCP smithi102.front.sepia.ceph.com:39854->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 80u IPv4 108716 0t0 TCP smithi102.front.sepia.ceph.com:56154->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 81u IPv4 111681 0t0 TCP smithi102.front.sepia.ceph.com:35448->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 82u IPv4 106757 0t0 TCP smithi102.front.sepia.ceph.com:51756->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 83u IPv4 112745 0t0 TCP smithi102.front.sepia.ceph.com:50438->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 84u IPv4 105525 0t0 TCP smithi102.front.sepia.ceph.com:43070->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 85u IPv4 97619 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:38898 (ESTABLISHED) 2024-04-20T21:47:11.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 86u IPv4 110833 0t0 TCP smithi102.front.sepia.ceph.com:34132->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:11.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 87u IPv4 110333 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi102.front.sepia.ceph.com:34436 (ESTABLISHED) 2024-04-20T21:47:11.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 88u IPv4 99164 0t0 TCP smithi102.front.sepia.ceph.com:53252->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 89u IPv4 109133 0t0 TCP smithi102.front.sepia.ceph.com:37832->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:11.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 90u IPv4 88217 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:51536 (ESTABLISHED) 2024-04-20T21:47:11.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 91u IPv4 109315 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:46138 (ESTABLISHED) 2024-04-20T21:47:11.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 92u IPv4 111379 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:45970 (ESTABLISHED) 2024-04-20T21:47:11.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 93u IPv4 113921 0t0 TCP smithi102.front.sepia.ceph.com:42462->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 94u IPv4 110110 0t0 TCP smithi102.front.sepia.ceph.com:41920->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 95u IPv4 111841 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:40496 (ESTABLISHED) 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 96u IPv4 116766 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:35026 (ESTABLISHED) 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 97u IPv4 110336 0t0 TCP smithi102.front.sepia.ceph.com:6812->smithi102.front.sepia.ceph.com:58452 (ESTABLISHED) 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 98u IPv4 107426 0t0 TCP smithi102.front.sepia.ceph.com:6808->smithi193.front.sepia.ceph.com:44888 (ESTABLISHED) 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 99u IPv4 113157 0t0 TCP smithi102.front.sepia.ceph.com:38030->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 101u IPv4 108006 0t0 TCP smithi102.front.sepia.ceph.com:6810->smithi193.front.sepia.ceph.com:33828 (ESTABLISHED) 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22370 25469 osd_srv_h root 103u IPv4 83018 0t0 TCP smithi102.front.sepia.ceph.com:6814->smithi193.front.sepia.ceph.com:41044 (ESTABLISHED) 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 14w REG 8,1 1797613621 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 14w REG 8,1 1797479276 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22373 log root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 14w REG 8,1 1797493055 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22375 msgr-work root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 14w REG 8,1 1797505372 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22376 msgr-work root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 14w REG 8,1 1797508464 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22377 msgr-work root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 14w REG 8,1 1797508464 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22390 service root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 14w REG 8,1 1797508464 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22391 admin_soc root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 14w REG 8,1 1797508464 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22444 signal_ha root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 14w REG 8,1 1797508464 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22445 io_contex root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 14w REG 8,1 1797508464 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22446 io_contex root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 14w REG 8,1 1797508464 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22460 OpHistory root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 14w REG 8,1 1797513335 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22461 ceph_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 14w REG 8,1 1797513335 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22489 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 14w REG 8,1 1797513335 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22521 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 14w REG 8,1 1797564443 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22523 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 14w REG 8,1 1797605638 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22531 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 14w REG 8,1 1797605638 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22539 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 14w REG 8,1 1797605638 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22547 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 14w REG 8,1 1797605926 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22557 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 14w REG 8,1 1797605926 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22565 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 14w REG 8,1 1797605926 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22566 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 14w REG 8,1 1797605926 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22567 safe_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 14w REG 8,1 1797605926 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22568 safe_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 14w REG 8,1 1797605926 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22569 safe_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 14w REG 8,1 1797605926 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22570 safe_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 14w REG 8,1 1797606214 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22571 fn_anonym root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 14w REG 8,1 1797606214 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:11.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:11.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:11.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:11.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:11.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:11.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:11.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:11.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:11.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:11.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22574 rocksdb:l root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:11.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:11.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:11.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:11.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:11.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:11.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:11.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:11.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:11.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:11.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:11.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:11.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:11.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:11.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:11.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:11.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:11.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:11.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:11.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:11.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:11.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:11.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:11.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 14w REG 8,1 1797606214 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:11.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:11.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:11.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:11.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:11.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:11.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22575 rocksdb:l root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 14w REG 8,1 1797606214 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22576 rocksdb:l root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 14w REG 8,1 1797606214 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 22577 rocksdb:h root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 14w REG 8,1 1797606214 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24067 bstore_ai root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 14w REG 8,1 1797606214 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24613 bstore_ai root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 14w REG 8,1 1797609092 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24976 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 14w REG 8,1 1797609092 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24977 cfin root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 14w REG 8,1 1797609092 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24978 bstore_kv root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 14w REG 8,1 1797609092 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24979 bstore_kv root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 14w REG 8,1 1797609092 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 24980 bstore_me root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 14w REG 8,1 1797609092 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25162 ms_dispat root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 14w REG 8,1 1797609092 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25163 ms_local root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 14w REG 8,1 1797609092 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25164 safe_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 14w REG 8,1 1797609380 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25165 safe_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 14w REG 8,1 1797609380 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25166 ms_dispat root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 14w REG 8,1 1797609380 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25167 ms_local root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 14w REG 8,1 1797609380 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25168 ms_dispat root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 14w REG 8,1 1797609591 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25169 ms_local root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 14w REG 8,1 1797609591 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25170 ms_dispat root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 14w REG 8,1 1797609591 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25171 ms_local root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 14w REG 8,1 1797609879 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25172 ms_dispat root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 14w REG 8,1 1797609879 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25173 ms_local root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 14w REG 8,1 1797609879 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25174 ms_dispat root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 14w REG 8,1 1797609879 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25175 ms_local root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 14w REG 8,1 1797609879 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25176 ms_dispat root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 14w REG 8,1 1797609879 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25177 ms_local root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 14w REG 8,1 1797609879 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25178 fn_anonym root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 14w REG 8,1 1797612757 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25179 finisher root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 14w REG 8,1 1797612757 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25180 safe_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 14w REG 8,1 1797612757 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25181 safe_time root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 14w REG 8,1 1797612757 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25182 ceph-osd root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 14w REG 8,1 1797612757 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25183 osd_srv_a root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 14w REG 8,1 1797612757 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25184 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 14w REG 8,1 1797613045 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25185 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 14w REG 8,1 1797613045 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25186 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 14w REG 8,1 1797613045 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25187 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 14w REG 8,1 1797613045 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25188 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 14w REG 8,1 1797613045 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25189 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 14w REG 8,1 1797613045 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25190 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 14w REG 8,1 1797613045 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25191 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 14w REG 8,1 1797613045 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25192 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 14w REG 8,1 1797613333 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25193 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 14w REG 8,1 1797613333 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25194 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:12.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:12.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:12.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:12.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:12.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:12.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:12.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:12.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:12.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:12.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:12.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:12.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:12.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:12.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:12.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:12.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:12.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:12.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:12.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:12.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:12.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:12.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:12.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:12.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:12.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:12.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:12.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:12.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:12.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:12.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:12.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:12.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:12.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:12.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:12.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:12.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:12.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:12.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:12.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:12.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:12.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 14w REG 8,1 1797613333 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:12.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:12.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:12.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:12.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:12.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:12.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:12.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:12.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:12.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:12.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:12.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:12.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:12.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:12.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:12.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:12.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:12.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:12.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:12.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:12.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:12.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:12.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:12.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:12.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:12.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25195 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:12.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:12.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:12.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:12.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:12.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:12.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:12.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:12.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:12.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:12.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:12.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:12.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:12.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:12.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:12.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:12.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:12.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:12.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:12.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:12.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:12.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:12.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:12.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:12.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:12.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:13.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 14w REG 8,1 1797613333 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:13.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:13.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:13.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:13.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:13.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:13.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:13.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:13.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:13.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:13.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:13.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:13.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:13.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:13.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:13.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:13.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:13.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:13.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:13.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:13.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:13.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:13.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:13.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:13.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:13.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:13.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:13.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:13.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:13.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:13.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:13.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25196 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:13.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:13.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:13.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 14w REG 8,1 1797613333 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:13.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:13.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:13.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:13.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:13.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:13.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:13.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:13.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:13.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:13.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:13.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:13.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:13.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:13.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:13.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:13.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:13.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:13.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:13.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:13.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:13.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:13.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:13.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:13.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25197 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:13.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:13.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:13.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 14w REG 8,1 1797613333 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:13.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:13.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:13.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:13.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:13.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:13.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:13.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:13.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:13.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:13.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:13.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:13.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:13.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:13.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:13.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:13.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:13.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:13.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:13.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25198 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:13.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:13.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:13.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:13.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:13.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:13.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 14w REG 8,1 1797613333 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:13.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:13.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:13.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:13.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:13.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:13.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:13.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:13.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:13.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:13.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:13.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:13.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:13.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:13.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:13.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:13.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:13.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:13.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:13.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:13.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:13.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:13.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:13.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:13.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:13.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:13.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:13.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:13.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:13.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:13.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:13.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:13.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:13.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:13.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25199 tp_osd_tp root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:13.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root DEL REG 0,18 52505 /[aio] 2024-04-20T21:47:13.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root DEL REG 0,18 52503 /[aio] 2024-04-20T21:47:13.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 0,15 50628 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 1w FIFO 0,14 0t0 51394 pipe 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 2w FIFO 0,14 0t0 51395 pipe 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 3r FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 4w FIFO 0,14 0t0 50463 pipe 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,64,66,69,72,73,79,80,81,82,86,88,89,91] 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 6r FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 7w FIFO 0,14 0t0 44908 pipe 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 8u a_inode 0,15 0 50 [eventpoll:9,59,67,68,70,74,76,77,78,83,87,93,95,102,103] 2024-04-20T21:47:13.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 9r FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 10w FIFO 0,14 0t0 44909 pipe 2024-04-20T21:47:13.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 11u a_inode 0,15 0 50 [eventpoll:12,65,71,75,84,85,90,92,94,96,97,98,99] 2024-04-20T21:47:13.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 12r FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 13w FIFO 0,14 0t0 44910 pipe 2024-04-20T21:47:13.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 14w REG 8,1 1797613621 393326 /var/log/ceph/ceph-osd.1.log 2024-04-20T21:47:13.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 15uW REG 0,25 6 1608 /run/ceph/ceph-osd.1.pid 2024-04-20T21:47:13.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 16r FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 17w FIFO 0,14 0t0 52281 pipe 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 18u unix 0xffff88810a750780 0t0 52282 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 19u IPv4 50468 0t0 TCP *:6800 (LISTEN) 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 20u IPv4 50470 0t0 TCP *:acnet (LISTEN) 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 21u IPv4 50473 0t0 TCP *:6802 (LISTEN) 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 22u IPv4 50477 0t0 TCP *:6803 (LISTEN) 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 23u IPv4 50482 0t0 TCP *:6804 (LISTEN) 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 24u IPv4 50488 0t0 TCP *:6805 (LISTEN) 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 25u IPv4 50495 0t0 TCP *:6806 (LISTEN) 2024-04-20T21:47:13.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 26u IPv4 50503 0t0 TCP *:6807 (LISTEN) 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 27r FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 28w FIFO 0,14 0t0 52288 pipe 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 29r FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 30w FIFO 0,14 0t0 52289 pipe 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 31u sock 0,9 0t0 50625 protocol: UDPv6 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 58u a_inode 0,15 0 50628 [io_uring] 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 59u IPv4 50658 0t0 TCP smithi102.front.sepia.ceph.com:50040->smithi102.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 60r FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 61w FIFO 0,14 0t0 47840 pipe 2024-04-20T21:47:13.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 62r FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 63w FIFO 0,14 0t0 47841 pipe 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 64u IPv4 103246 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40164 (ESTABLISHED) 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 65u IPv4 95282 0t0 TCP smithi102.front.sepia.ceph.com:35822->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 66u IPv4 100147 0t0 TCP smithi102.front.sepia.ceph.com:60202->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 67u IPv4 106298 0t0 TCP smithi102.front.sepia.ceph.com:54554->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 68u IPv4 93924 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:54374 (ESTABLISHED) 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 69u IPv4 93927 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:34310 (ESTABLISHED) 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 70u IPv4 88830 0t0 TCP smithi102.front.sepia.ceph.com:51496->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 71u IPv4 90391 0t0 TCP smithi102.front.sepia.ceph.com:43676->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 72u IPv4 100148 0t0 TCP smithi102.front.sepia.ceph.com:44122->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 73u IPv4 100753 0t0 TCP smithi102.front.sepia.ceph.com:55062->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 74u IPv4 105072 0t0 TCP smithi102.front.sepia.ceph.com:39418->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 75u IPv4 112750 0t0 TCP smithi102.front.sepia.ceph.com:33516->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 76u sock 0,9 0t0 113486 protocol: TCP 2024-04-20T21:47:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 77u IPv4 110015 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:52708 (ESTABLISHED) 2024-04-20T21:47:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 78u IPv4 107188 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:45584 (ESTABLISHED) 2024-04-20T21:47:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 79u IPv4 100754 0t0 TCP smithi102.front.sepia.ceph.com:54156->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 80u IPv4 104371 0t0 TCP smithi102.front.sepia.ceph.com:57888->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 81u IPv4 87013 0t0 TCP smithi102.front.sepia.ceph.com:55712->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 82u IPv4 104008 0t0 TCP smithi102.front.sepia.ceph.com:40164->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 83u IPv4 108886 0t0 TCP smithi102.front.sepia.ceph.com:56258->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 84u IPv4 112877 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:42910 (ESTABLISHED) 2024-04-20T21:47:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 85u IPv4 112962 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:54788 (ESTABLISHED) 2024-04-20T21:47:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 86u IPv4 110780 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:38364 (ESTABLISHED) 2024-04-20T21:47:13.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 87u IPv4 110018 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi102.front.sepia.ceph.com:34132 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 88u IPv4 103249 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:57312 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 89u IPv4 110783 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi102.front.sepia.ceph.com:60686 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 90u IPv4 107907 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:42660 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 91u IPv4 113922 0t0 TCP smithi102.front.sepia.ceph.com:42476->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 92u IPv4 110505 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:57232 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 93u IPv4 107423 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi193.front.sepia.ceph.com:55270 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 94u IPv4 113976 0t0 TCP smithi102.front.sepia.ceph.com:58004->smithi193.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 95u IPv4 108230 0t0 TCP smithi102.front.sepia.ceph.com:6804->smithi193.front.sepia.ceph.com:45260 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 96u IPv4 115795 0t0 TCP smithi102.front.sepia.ceph.com:6800->smithi193.front.sepia.ceph.com:40556 (ESTABLISHED) 2024-04-20T21:47:13.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 97u IPv4 111845 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:41920 (ESTABLISHED) 2024-04-20T21:47:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 98u IPv4 107910 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:51982 (ESTABLISHED) 2024-04-20T21:47:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 99u IPv4 112031 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:44020 (ESTABLISHED) 2024-04-20T21:47:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 102u IPv4 108233 0t0 TCP smithi102.front.sepia.ceph.com:6806->smithi193.front.sepia.ceph.com:40092 (ESTABLISHED) 2024-04-20T21:47:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22371 25200 osd_srv_h root 103u IPv4 107373 0t0 TCP smithi102.front.sepia.ceph.com:6802->smithi102.front.sepia.ceph.com:36624 (ESTABLISHED) 2024-04-20T21:47:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 14w REG 8,1 1891375893 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:13.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 14w REG 8,1 1891309655 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22398 log root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 14w REG 8,1 1891309655 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22400 msgr-work root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 14w REG 8,1 1891309655 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22401 msgr-work root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 14w REG 8,1 1891309944 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22402 msgr-work root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 14w REG 8,1 1891309944 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22408 service root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 14w REG 8,1 1891309944 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22409 admin_soc root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 14w REG 8,1 1891309944 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22455 signal_ha root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 14w REG 8,1 1891309944 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22456 io_contex root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 14w REG 8,1 1891309944 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22457 io_contex root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 14w REG 8,1 1891309944 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22466 OpHistory root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 14w REG 8,1 1891310233 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22467 ceph_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 14w REG 8,1 1891310233 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22513 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 14w REG 8,1 1891310233 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22525 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 14w REG 8,1 1891310233 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22533 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 14w REG 8,1 1891310233 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22541 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 14w REG 8,1 1891314790 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22549 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 14w REG 8,1 1891314790 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22559 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 14w REG 8,1 1891315079 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 22578 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 14w REG 8,1 1891315079 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23173 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 14w REG 8,1 1891315079 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23183 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 14w REG 8,1 1891315079 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23186 safe_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 14w REG 8,1 1891315079 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23188 safe_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 14w REG 8,1 1891315079 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23190 safe_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 14w REG 8,1 1891315079 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23192 safe_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 14w REG 8,1 1891317956 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23194 fn_anonym root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 14w REG 8,1 1891317956 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23336 rocksdb:l root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 14w REG 8,1 1891317956 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23337 rocksdb:l root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 14w REG 8,1 1891317956 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23338 rocksdb:l root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 14w REG 8,1 1891317956 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 23340 rocksdb:h root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 14w REG 8,1 1891317956 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 24614 bstore_ai root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 14w REG 8,1 1891318245 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25470 bstore_ai root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 14w REG 8,1 1891318245 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25651 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 14w REG 8,1 1891318245 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25652 cfin root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 14w REG 8,1 1891318245 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25653 bstore_kv root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 14w REG 8,1 1891318245 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25654 bstore_kv root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 14w REG 8,1 1891318245 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25655 bstore_me root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 14w REG 8,1 1891318245 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25656 ms_dispat root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 14w REG 8,1 1891320077 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25657 ms_local root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 14w REG 8,1 1891320077 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:13.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:13.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:13.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:13.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:13.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:13.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:13.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:13.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:13.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:13.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:13.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:13.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:13.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:13.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:13.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:13.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:13.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:13.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:13.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:13.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:13.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:13.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:13.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25658 safe_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:13.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:13.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:13.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:13.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:13.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:13.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:13.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:13.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:13.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:13.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:13.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:13.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:13.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:13.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:13.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:13.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:13.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:13.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:13.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:13.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:13.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:13.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:13.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:13.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:13.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:13.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:13.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:13.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:13.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:13.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:13.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:13.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:13.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:13.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:13.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:13.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:13.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:13.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:13.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:13.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:13.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:13.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:13.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:13.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:13.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:13.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:13.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:13.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:13.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:13.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:13.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:13.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:13.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:13.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:13.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:13.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:13.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 14w REG 8,1 1891338112 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:13.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:13.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:13.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:13.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:13.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:13.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:13.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:13.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:13.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:13.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25659 safe_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 14w REG 8,1 1891338112 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:14.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25660 ms_dispat root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:14.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:14.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 14w REG 8,1 1891338112 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 99u IPv4 113833 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:55960 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25661 ms_local root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,99,100] 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,102,103] 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 14w REG 8,1 1891339049 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 99 unknown /proc/22372/task/25662/fd/99 (readlink: No such file or directory) 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25662 ms_dispat root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 14w REG 8,1 1891371736 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25663 ms_local root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 14w REG 8,1 1891372025 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25664 ms_dispat root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 14w REG 8,1 1891372025 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25665 ms_local root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 14w REG 8,1 1891372025 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25666 ms_dispat root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 14w REG 8,1 1891372025 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25667 ms_local root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 14w REG 8,1 1891372025 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25668 ms_dispat root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 14w REG 8,1 1891372025 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25669 ms_local root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.618+0000 7fe311a00640 1 Processor -- start 2024-04-20T21:47:14.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.618+0000 7fe311a00640 1 -- start start 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.619+0000 7fe311a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe30c073a10 0x7fe30c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.619+0000 7fe311a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe30c1536d0 con 0x7fe30c14fa50 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.620+0000 7fe311a00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe30c0c76c0 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.620+0000 7fe311a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7fe30c0c7960 con 0x7fe30c073a10 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.620+0000 7fe30aa00640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe30c14fa50 0x7fe30c0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59078/0 (socket says 172.21.15.102:59078) 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.620+0000 7fe30b400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe30c073a10 0x7fe30c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.620+0000 7fe30aa00640 1 -- 172.21.15.102:0/2887082203 learned_addr learned my addr 172.21.15.102:0/2887082203 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.620+0000 7fe30b400640 1 -- 172.21.15.102:0/2887082203 >> v1:172.21.15.193:6789/0 conn(0x7fe30c0c9eb0 legacy=0x7fe30c161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:14.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.620+0000 7fe30b400640 1 -- 172.21.15.102:0/2887082203 >> v1:172.21.15.102:6789/0 conn(0x7fe30c14fa50 legacy=0x7fe30c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.620+0000 7fe30b400640 1 -- 172.21.15.102:0/2887082203 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c0c3d90 con 0x7fe30c073a10 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.621+0000 7fe30b400640 1 --2- 172.21.15.102:0/2887082203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c073a10 0x7fe30c0c4830 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0c7890 tx=0x7fe30402b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc13bf7898b36427 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe309600640 1 -- 172.21.15.102:0/2887082203 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304034070 con 0x7fe30c073a10 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe309600640 1 -- 172.21.15.102:0/2887082203 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe304030890 con 0x7fe30c073a10 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe309600640 1 -- 172.21.15.102:0/2887082203 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304030c60 con 0x7fe30c073a10 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe311a00640 1 -- 172.21.15.102:0/2887082203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c073a10 msgr2=0x7fe30c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe311a00640 1 --2- 172.21.15.102:0/2887082203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c073a10 0x7fe30c0c4830 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0c7890 tx=0x7fe30402b9d0 comp rx=0 tx=0).stop 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe311a00640 1 -- 172.21.15.102:0/2887082203 shutdown_connections 2024-04-20T21:47:14.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe311a00640 1 --2- 172.21.15.102:0/2887082203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c073a10 0x7fe30c0c4830 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:14.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe311a00640 1 -- 172.21.15.102:0/2887082203 >> 172.21.15.102:0/2887082203 conn(0x7fe30c0c9390 msgr2=0x7fe30c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:14.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.622+0000 7fe311a00640 1 -- 172.21.15.102:0/2887082203 shutdown_connections 2024-04-20T21:47:14.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.623+0000 7fe311a00640 1 -- 172.21.15.102:0/2887082203 wait complete. 2024-04-20T21:47:14.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.624+0000 7fe311a00640 1 Processor -- start 2024-04-20T21:47:14.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.624+0000 7fe311a00640 1 -- start start 2024-04-20T21:47:14.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe311a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe30c073a10 0x7fe30c157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:14.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe311a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c0c9eb0 0x7fe30c14a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe311a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe30c14fa50 0x7fe30c142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe311a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c0c4030 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe311a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c0c7c00 con 0x7fe30c073a10 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe311a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7fe30c0c4310 con 0x7fe30c14fa50 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe30a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c0c9eb0 0x7fe30c14a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe30a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c0c9eb0 0x7fe30c14a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44006/0 (socket says 172.21.15.102:44006) 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.625+0000 7fe30a000640 1 -- 172.21.15.102:0/4154404529 learned_addr learned my addr 172.21.15.102:0/4154404529 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe30aa00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe30c073a10 0x7fe30c157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:14.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe30b400640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe30c14fa50 0x7fe30c142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe30a000640 1 -- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe30c14fa50 msgr2=0x7fe30c142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe30a000640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe30c14fa50 0x7fe30c142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe30a000640 1 -- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe30c073a10 msgr2=0x7fe30c157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe30a000640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe30c073a10 0x7fe30c157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe30a000640 1 -- 172.21.15.102:0/4154404529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c14b000 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe30a000640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c0c9eb0 0x7fe30c14a660 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c7c0 tx=0x7fe2f800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.626+0000 7fe302a00640 1 -- 172.21.15.102:0/4154404529 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8016020 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.627+0000 7fe302a00640 1 -- 172.21.15.102:0/4154404529 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f8006d50 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.627+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30c158a50 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.627+0000 7fe302a00640 1 -- 172.21.15.102:0/4154404529 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8003bb0 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.627+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe30c156e20 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.628+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30c1574e0 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.629+0000 7fe302a00640 1 -- 172.21.15.102:0/4154404529 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2f800a850 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.630+0000 7fe302a00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2ec06c8f0 0x7fe2ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.630+0000 7fe302a00640 1 -- 172.21.15.102:0/4154404529 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7fe2f808c070 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.630+0000 7fe30aa00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2ec06c8f0 0x7fe2ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.631+0000 7fe30aa00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2ec06c8f0 0x7fe2ec06edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005880 tx=0x7fe2fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:14.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.635+0000 7fe302a00640 1 -- 172.21.15.102:0/4154404529 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe2f8020020 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 14w REG 8,1 1891372314 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25670 ms_dispat root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 14w REG 8,1 1891372314 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25671 ms_local root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 14w REG 8,1 1891372314 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25672 fn_anonym root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 14w REG 8,1 1891372314 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25673 finisher root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 14w REG 8,1 1891372314 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25674 safe_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 14w REG 8,1 1891372314 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25675 safe_time root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 14w REG 8,1 1891372314 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25676 ceph-osd root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 14w REG 8,1 1891372603 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25677 osd_srv_a root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 14w REG 8,1 1891372603 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25678 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 14w REG 8,1 1891372603 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25679 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 14w REG 8,1 1891372603 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25680 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 14w REG 8,1 1891372603 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25681 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 14w REG 8,1 1891372603 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25682 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 14w REG 8,1 1891372603 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25683 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 14w REG 8,1 1891372892 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25684 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 14w REG 8,1 1891372892 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25685 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.883+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe30c074340 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.886+0000 7fe302a00640 1 -- 172.21.15.102:0/4154404529 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+3553 (secure 0 0 0) 0x7fe2f80594c0 con 0x7fe30c0c9eb0 2024-04-20T21:47:14.591 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:47:14.591 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":109,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:47:07.748327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":108,"state":"up:reconnect","state_seq":456,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:47:14.592 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 109 2024-04-20T21:47:14.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.888+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2ec06c8f0 msgr2=0x7fe2ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.888+0000 7fe311a00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2ec06c8f0 0x7fe2ec06edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005880 tx=0x7fe2fc009040 comp rx=0 tx=0).stop 2024-04-20T21:47:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.888+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c0c9eb0 msgr2=0x7fe30c14a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.888+0000 7fe311a00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c0c9eb0 0x7fe30c14a660 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c7c0 tx=0x7fe2f800cc80 comp rx=0 tx=0).stop 2024-04-20T21:47:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.888+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 shutdown_connections 2024-04-20T21:47:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.889+0000 7fe311a00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe30c14fa50 0x7fe30c142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.889+0000 7fe311a00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe30c0c9eb0 0x7fe30c14a660 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.889+0000 7fe311a00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7fe2ec06c8f0 0x7fe2ec06edb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:14.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.889+0000 7fe311a00640 1 --2- 172.21.15.102:0/4154404529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe30c073a10 0x7fe30c157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.889+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 >> 172.21.15.102:0/4154404529 conn(0x7fe30c0c9390 msgr2=0x7fe30c14bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.889+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 shutdown_connections 2024-04-20T21:47:14.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:12.889+0000 7fe311a00640 1 -- 172.21.15.102:0/4154404529 wait complete. 2024-04-20T21:47:14.594 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:47:14.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 14w REG 8,1 1891372892 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25686 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 14w REG 8,1 1891372892 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25687 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 14w REG 8,1 1891372892 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25688 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 14w REG 8,1 1891372892 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25689 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 14w REG 8,1 1891372892 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25690 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 14w REG 8,1 1891373297 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25691 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 14w REG 8,1 1891375893 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25692 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 14w REG 8,1 1891375893 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25693 tp_osd_tp root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root DEL REG 0,18 53606 /[aio] 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root DEL REG 0,18 53600 /[aio] 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 0,15 53590 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 1w FIFO 0,14 0t0 53301 pipe 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 2w FIFO 0,14 0t0 53302 pipe 2024-04-20T21:47:14.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 3r FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 4w FIFO 0,14 0t0 51430 pipe 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,61,64,68,72,73,74,80,87,89,91,94,97,98] 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 6r FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 7w FIFO 0,14 0t0 53364 pipe 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 8u a_inode 0,15 0 50 [eventpoll:9,66,69,70,76,79,81,84,85,86,90,93,95,100] 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 9r FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 10w FIFO 0,14 0t0 53365 pipe 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 11u a_inode 0,15 0 50 [eventpoll:12,65,67,71,75,77,78,82,83,88,92,96,101,102,103] 2024-04-20T21:47:14.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 12r FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 13w FIFO 0,14 0t0 53366 pipe 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 14w REG 8,1 1891375893 393328 /var/log/ceph/ceph-osd.3.log 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 15uW REG 0,25 6 1612 /run/ceph/ceph-osd.3.pid 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 16r FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 17w FIFO 0,14 0t0 47791 pipe 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 18u unix 0xffff888162172580 0t0 47792 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 19u IPv4 52307 0t0 TCP *:6816 (LISTEN) 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 20u IPv4 52326 0t0 TCP *:6818 (LISTEN) 2024-04-20T21:47:14.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 21u IPv4 52347 0t0 TCP *:6820 (LISTEN) 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 22u IPv4 52371 0t0 TCP *:6823 (LISTEN) 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 23u IPv4 52397 0t0 TCP *:6825 (LISTEN) 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 24u IPv4 52425 0t0 TCP *:6827 (LISTEN) 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 25u IPv4 52455 0t0 TCP *:6829 (LISTEN) 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 26u IPv4 52487 0t0 TCP *:ambit-lm (LISTEN) 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 27r FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 28w FIFO 0,14 0t0 47799 pipe 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 29r FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 30w FIFO 0,14 0t0 47800 pipe 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 31u sock 0,9 0t0 52500 protocol: UDPv6 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-04-20T21:47:14.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 58u a_inode 0,15 0 53590 [io_uring] 2024-04-20T21:47:14.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 59r FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 60w FIFO 0,14 0t0 53617 pipe 2024-04-20T21:47:14.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 61u IPv4 47837 0t0 TCP smithi102.front.sepia.ceph.com:39684->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 62r FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 63w FIFO 0,14 0t0 53618 pipe 2024-04-20T21:47:14.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 64u IPv4 109102 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:54554 (ESTABLISHED) 2024-04-20T21:47:14.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 65u IPv4 113928 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36856 (ESTABLISHED) 2024-04-20T21:47:14.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 66u IPv4 112412 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:34936 (ESTABLISHED) 2024-04-20T21:47:14.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 67u IPv4 113357 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:43480 (ESTABLISHED) 2024-04-20T21:47:14.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 68u IPv4 113925 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:33090 (ESTABLISHED) 2024-04-20T21:47:14.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 69u IPv4 110298 0t0 TCP smithi102.front.sepia.ceph.com:6816->192.168.0.1:35200 (ESTABLISHED) 2024-04-20T21:47:14.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 70u IPv4 110292 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:40270 (ESTABLISHED) 2024-04-20T21:47:14.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 71u IPv4 108997 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:49544 (ESTABLISHED) 2024-04-20T21:47:14.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 72u IPv4 109105 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:33516 (ESTABLISHED) 2024-04-20T21:47:14.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 73u IPv4 107195 0t0 TCP smithi102.front.sepia.ceph.com:54788->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 74u IPv4 107351 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi102.front.sepia.ceph.com:40844 (ESTABLISHED) 2024-04-20T21:47:14.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 75u IPv4 99318 0t0 TCP smithi102.front.sepia.ceph.com:41170->smithi102.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 76u IPv4 111682 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:35448 (ESTABLISHED) 2024-04-20T21:47:14.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 77u IPv4 108977 0t0 TCP smithi102.front.sepia.ceph.com:42208->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 78u IPv4 106191 0t0 TCP smithi102.front.sepia.ceph.com:57508->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 79u IPv4 113774 0t0 TCP smithi102.front.sepia.ceph.com:45584->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 80u IPv4 73272 0t0 TCP smithi102.front.sepia.ceph.com:47458->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 81u IPv4 100269 0t0 TCP smithi102.front.sepia.ceph.com:37232->smithi102.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 82u IPv4 108978 0t0 TCP smithi102.front.sepia.ceph.com:33330->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 83u IPv4 100026 0t0 TCP smithi102.front.sepia.ceph.com:34730->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 84u IPv4 75352 0t0 TCP smithi102.front.sepia.ceph.com:48136->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 85u IPv4 87319 0t0 TCP smithi102.front.sepia.ceph.com:55950->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 86u IPv4 110014 0t0 TCP smithi102.front.sepia.ceph.com:39132->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 87u IPv4 108518 0t0 TCP smithi102.front.sepia.ceph.com:42468->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 88u IPv4 112419 0t0 TCP smithi102.front.sepia.ceph.com:36624->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 89u IPv4 99030 0t0 TCP smithi102.front.sepia.ceph.com:41038->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 90u IPv4 87320 0t0 TCP smithi102.front.sepia.ceph.com:51536->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 91u IPv4 112937 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:33288 (ESTABLISHED) 2024-04-20T21:47:14.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 92u IPv4 113932 0t0 TCP smithi102.front.sepia.ceph.com:6816->smithi193.front.sepia.ceph.com:56934 (ESTABLISHED) 2024-04-20T21:47:14.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 93u IPv4 112415 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:57434 (ESTABLISHED) 2024-04-20T21:47:14.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 94u IPv4 107354 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi102.front.sepia.ceph.com:37896 (ESTABLISHED) 2024-04-20T21:47:14.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 95u IPv4 110295 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:36620 (ESTABLISHED) 2024-04-20T21:47:14.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 96u IPv4 104573 0t0 TCP smithi102.front.sepia.ceph.com:6829->smithi193.front.sepia.ceph.com:39594 (ESTABLISHED) 2024-04-20T21:47:14.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 97u IPv4 113333 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:39398 (ESTABLISHED) 2024-04-20T21:47:14.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 98u IPv4 104576 0t0 TCP smithi102.front.sepia.ceph.com:6825->smithi193.front.sepia.ceph.com:49922 (ESTABLISHED) 2024-04-20T21:47:14.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 100u IPv4 110228 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi193.front.sepia.ceph.com:45360 (ESTABLISHED) 2024-04-20T21:47:14.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 101u sock 0,9 0t0 118878 protocol: TCP 2024-04-20T21:47:14.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 102u IPv4 111835 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:37832 (ESTABLISHED) 2024-04-20T21:47:14.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22372 25694 osd_srv_h root 103u IPv4 112865 0t0 TCP smithi102.front.sepia.ceph.com:6820->smithi102.front.sepia.ceph.com:45464 (ESTABLISHED) 2024-04-20T21:47:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:14.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:14.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:14.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:14.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 14w REG 8,1 1113303607 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:14.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:14.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:14.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:14.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:14.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:14.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:14.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:14.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:14.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:14.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:14.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:14.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:14.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:14.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:14.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:14.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:14.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:14.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:14.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:14.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:14.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:14.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:14.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:14.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:14.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 14w REG 8,1 1113292566 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:14.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:14.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:14.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:14.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:14.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:14.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:14.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:14.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:14.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:14.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:14.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:14.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:14.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:14.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:14.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:14.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:14.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:14.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:14.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:14.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:14.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:14.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:14.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:14.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:14.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22406 log root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:14.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:14.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:14.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 14w REG 8,1 1113292566 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:14.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:14.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:14.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:14.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:14.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:14.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:14.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:14.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:14.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:14.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:14.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:14.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:14.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22410 msgr-work root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:14.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 14w REG 8,1 1113292566 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:14.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:14.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:14.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:14.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:14.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:14.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:14.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:14.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:14.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:14.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:14.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:14.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:14.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:14.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:14.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:14.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:14.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:14.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:14.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:14.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:14.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:14.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:14.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:14.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:14.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:14.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:14.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22411 msgr-work root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:14.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:14.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:14.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 14w REG 8,1 1113292566 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:14.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:14.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:14.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:14.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:14.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:14.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:14.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:14.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:14.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:14.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:14.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:14.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:14.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:14.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:14.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:14.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:14.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:14.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22412 msgr-work root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 14w REG 8,1 1113292566 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:14.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:14.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:14.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:14.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:14.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:14.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:14.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:14.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:14.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:14.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:14.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:14.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:14.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:14.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:14.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:14.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:14.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:14.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:14.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:14.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:14.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:14.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:14.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:14.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:14.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:14.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22418 service root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:14.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:14.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:14.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:14.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:14.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 14w REG 8,1 1113292566 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:14.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:14.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:14.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:14.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:14.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:14.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:14.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:14.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:14.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:14.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:14.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:14.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:14.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:14.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:14.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:14.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:14.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:14.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:14.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:14.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:14.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:14.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:14.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22419 admin_soc root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:14.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:14.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:14.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:14.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:14.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:14.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:14.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:14.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:14.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:14.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:14.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:14.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:14.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:14.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:14.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:14.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:14.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:14.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:14.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:14.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:14.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:14.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:14.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:14.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:14.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:14.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:14.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:14.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:14.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:14.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:14.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:14.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 14w REG 8,1 1113292854 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:14.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:14.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:14.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:14.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:14.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:14.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:14.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:14.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:14.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:14.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:14.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:14.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:14.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:14.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:14.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:14.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:14.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:14.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:14.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:14.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:14.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:14.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:14.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:14.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:14.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:14.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:14.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:14.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:14.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:14.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:14.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:14.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:14.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:14.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:14.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:14.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:14.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:14.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:14.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:14.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:14.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:14.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:14.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:14.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22452 signal_ha root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:14.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:14.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:14.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:14.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:14.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:14.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:14.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:14.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:14.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:14.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:14.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:14.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:14.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:14.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:14.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:14.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:14.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:14.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:14.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 14w REG 8,1 1113292854 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22453 io_contex root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 14w REG 8,1 1113292854 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.040 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22454 io_contex root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 14w REG 8,1 1113292854 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22469 OpHistory root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 14w REG 8,1 1113292854 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22470 ceph_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 14w REG 8,1 1113292854 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22515 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 14w REG 8,1 1113292854 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22527 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 14w REG 8,1 1113293142 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22535 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 14w REG 8,1 1113293142 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22543 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 14w REG 8,1 1113293142 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22551 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 14w REG 8,1 1113295732 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22553 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 14w REG 8,1 1113295732 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22561 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.244 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 14w REG 8,1 1113295732 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22760 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 14w REG 8,1 1113295732 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22761 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 14w REG 8,1 1113296020 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22762 safe_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 14w REG 8,1 1113296020 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22763 safe_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 14w REG 8,1 1113296020 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22764 safe_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 14w REG 8,1 1113298618 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22765 safe_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 14w REG 8,1 1113298618 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.409 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.410 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.411 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.412 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.413 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.414 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.415 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.416 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22766 fn_anonym root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.421 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.422 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 14w REG 8,1 1113298618 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22769 rocksdb:l root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 14w REG 8,1 1113298618 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22770 rocksdb:l root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 14w REG 8,1 1113298618 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22771 rocksdb:l root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 14w REG 8,1 1113298906 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 22772 rocksdb:h root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 14w REG 8,1 1113298906 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24249 bstore_ai root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 14w REG 8,1 1113298906 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 24981 bstore_ai root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 14w REG 8,1 1113298906 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25201 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 14w REG 8,1 1113298906 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25202 cfin root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 14w REG 8,1 1113298906 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25203 bstore_kv root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 14w REG 8,1 1113298906 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25204 bstore_kv root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 14w REG 8,1 1113302167 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25205 bstore_me root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 14w REG 8,1 1113302167 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25206 ms_dispat root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.726 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.727 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.728 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.729 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.730 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.731 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.732 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.733 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.734 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.735 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.736 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.737 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 14w REG 8,1 1113302167 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25207 ms_local root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 14w REG 8,1 1113302167 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25208 safe_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 14w REG 8,1 1113302167 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25209 safe_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 14w REG 8,1 1113302167 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25210 ms_dispat root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 14w REG 8,1 1113302167 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25211 ms_local root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.856 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 14w REG 8,1 1113302455 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25212 ms_dispat root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 14w REG 8,1 1113302455 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25213 ms_local root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 14w REG 8,1 1113302455 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.925 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.926 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.927 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.928 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.929 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.930 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.931 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25214 ms_dispat root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.933 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.934 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.935 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.936 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.937 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.939 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.940 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.941 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.942 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.943 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 14w REG 8,1 1113302455 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.944 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.945 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.946 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.947 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.948 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.949 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.950 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.951 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.952 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.953 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.954 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.955 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.956 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.957 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25215 ms_local root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.958 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.959 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.960 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.961 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.962 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.963 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.964 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.965 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.966 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.967 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.968 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.969 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.970 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 14w REG 8,1 1113302455 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.972 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.973 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.974 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.975 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:15.976 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:15.977 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:15.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:15.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:15.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:15.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:15.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:15.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:15.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:15.978 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:15.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:15.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:15.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:15.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:15.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:15.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:15.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:15.979 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:15.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:15.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:15.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:15.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:15.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:15.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:15.980 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:15.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:15.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:15.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:15.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:15.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25216 ms_dispat root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:15.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:15.981 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:15.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:15.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:15.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:15.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:15.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:15.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:15.982 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:15.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:15.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:15.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:15.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:15.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:15.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:15.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:15.983 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:15.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:15.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:15.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:15.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:15.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:15.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:15.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:15.984 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:15.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:15.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:15.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:15.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:15.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:15.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:15.985 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:15.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:15.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:15.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:15.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:15.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:15.986 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:15.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:15.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:15.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:15.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:15.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:15.987 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:15.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:15.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:15.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:15.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:15.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:15.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:15.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:15.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:15.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:15.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:15.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:15.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:15.989 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:15.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:15.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:15.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:15.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:15.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:15.990 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:15.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:15.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:15.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:15.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:15.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:15.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:15.991 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:15.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:15.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:15.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:15.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:15.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:15.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:15.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:15.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:15.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:15.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:15.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:15.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:15.993 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:15.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:15.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:15.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:15.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:15.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:15.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:15.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 14w REG 8,1 1113302455 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:15.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:15.995 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:15.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:15.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:15.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:15.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:15.996 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:15.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:15.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:15.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:15.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:15.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.997 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:15.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:15.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:15.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:15.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:15.998 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:15.999 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.000 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.001 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.002 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.003 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.004 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.005 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.006 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.007 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25217 ms_local root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.008 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.010 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.011 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.012 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.013 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.014 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.015 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.016 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.017 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.018 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.019 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 14w REG 8,1 1113302455 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.022 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.023 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.024 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.025 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.026 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.027 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.028 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.029 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.030 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.031 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25218 ms_dispat root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.032 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.033 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.035 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.036 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.037 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.038 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.041 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.042 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.043 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.045 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.046 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.047 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 14w REG 8,1 1113302455 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.048 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.049 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.050 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.051 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.052 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.053 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.054 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.055 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.056 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.057 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.058 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25219 ms_local root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.059 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.060 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.061 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.062 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.063 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.064 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.065 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.066 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.068 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.069 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.070 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.072 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.073 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 14w REG 8,1 1113302743 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.074 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.075 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.076 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.077 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.078 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.079 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.080 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.081 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.082 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.083 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.085 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.086 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25220 ms_dispat root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.087 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.088 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.089 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.090 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.091 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.092 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.093 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.094 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.095 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.096 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.097 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.098 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 14w REG 8,1 1113302743 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.099 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.100 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.101 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.102 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.103 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.104 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.105 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.106 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.107 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.108 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.109 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.111 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25221 ms_local root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.112 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.113 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.114 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.115 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.116 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.117 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.118 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.119 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.120 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.121 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.122 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.123 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.124 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 14w REG 8,1 1113302743 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.125 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.126 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.127 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.128 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.129 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.130 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.131 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.132 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.133 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.135 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25222 fn_anonym root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.136 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.137 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.138 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.139 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.141 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.142 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.143 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.144 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.145 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.146 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.147 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.148 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.149 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.150 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 14w REG 8,1 1113302743 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.151 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.152 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.153 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.154 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.155 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.156 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.157 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.158 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.159 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.160 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.161 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.162 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25223 finisher root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.163 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.165 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.166 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.167 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.168 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.170 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.171 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.172 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.173 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.174 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.175 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.176 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 14w REG 8,1 1113302743 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.177 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.178 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.179 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.180 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.181 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.182 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.185 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.186 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.187 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.188 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25224 safe_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.190 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.191 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.192 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.193 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.194 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.195 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.197 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.198 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.199 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 14w REG 8,1 1113302743 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.207 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.208 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.209 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.210 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.211 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.212 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.213 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25225 safe_time root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.214 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.215 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.216 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.217 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.218 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.219 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.220 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.222 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.223 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.224 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.225 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.226 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.227 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.228 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 14w REG 8,1 1113302743 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.230 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.231 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.232 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.233 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.235 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.236 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.237 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.238 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.239 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25226 ceph-osd root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.240 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.241 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.242 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.243 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.245 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.246 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.248 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.249 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.250 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.251 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.254 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 14w REG 8,1 1113303031 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.255 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.256 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.257 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.258 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.259 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.260 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.261 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.262 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.263 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.264 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.265 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.266 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.267 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25227 osd_srv_a root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.268 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.269 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.270 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.271 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.272 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.273 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.274 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.275 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.276 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 14w REG 8,1 1113303031 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.280 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.281 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.282 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.283 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.284 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.285 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.286 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.287 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.288 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.289 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.290 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.291 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25228 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.292 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.293 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.294 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.295 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.296 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.297 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.299 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.300 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.301 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.303 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.304 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.305 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.306 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 14w REG 8,1 1113303031 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.307 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.308 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.309 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.311 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.312 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.313 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.314 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.315 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.316 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.317 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25229 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.318 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.319 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.320 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.321 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.322 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.323 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.324 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.325 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.326 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.327 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.328 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.329 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 14w REG 8,1 1113303031 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.331 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.332 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.334 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.335 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.336 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.337 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.338 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.339 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.340 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.341 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.342 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.343 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25230 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.344 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.345 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.346 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.347 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.348 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.349 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.350 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.351 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.352 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.353 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.355 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.356 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 14w REG 8,1 1113303031 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.357 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.358 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.359 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.360 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.361 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.362 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.363 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.364 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.365 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.366 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.367 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25231 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.368 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.369 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.370 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.371 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.372 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.373 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.374 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.375 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.376 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.377 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.378 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.379 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.380 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.381 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.382 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 14w REG 8,1 1113303031 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.383 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.384 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.385 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.388 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.389 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.390 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.391 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.393 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25232 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.395 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.396 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.397 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.398 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.399 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.400 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.401 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.402 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.405 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.406 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 14w REG 8,1 1113303031 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.408 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.423 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.424 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.425 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.426 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.427 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.428 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.430 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25233 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.436 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.437 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.438 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.439 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.440 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.441 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.442 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.443 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.444 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.445 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.446 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.447 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 14w REG 8,1 1113303319 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.449 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.451 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.452 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.453 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.454 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.455 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.456 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.457 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.458 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.459 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.460 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25234 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.461 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.463 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.464 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.465 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.466 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.467 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.468 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.469 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.471 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.472 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 14w REG 8,1 1113303319 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.474 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.475 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.476 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.477 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.479 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.480 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.481 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.483 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.484 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25235 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.485 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.486 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.487 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.489 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.490 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.491 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.492 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.493 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.494 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.495 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.496 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.497 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 14w REG 8,1 1113303319 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.499 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.500 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.501 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.502 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.503 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.504 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.505 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.506 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.507 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.508 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.509 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25236 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.511 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.512 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.513 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.514 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.515 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.516 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.517 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.518 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.519 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.520 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.521 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.522 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.523 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 14w REG 8,1 1113303319 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.524 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.525 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.526 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.527 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.528 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.529 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.530 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.531 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.533 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.534 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.535 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.536 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25237 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.537 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.538 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.539 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.540 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.541 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.542 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.543 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.544 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.545 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.546 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.547 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.548 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 14w REG 8,1 1113303319 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.549 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.550 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.551 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.552 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.553 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.554 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.555 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.556 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.557 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.558 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.559 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.560 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25238 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.561 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.562 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.563 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.564 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.565 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.566 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.567 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.568 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.569 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.570 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.571 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.572 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.573 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.574 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.575 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 14w REG 8,1 1113303319 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.576 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.577 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.578 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.579 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.580 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.581 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.582 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.584 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.585 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.586 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25239 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.587 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.588 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.589 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.590 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.591 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.592 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.593 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.594 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.595 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.596 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.597 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.599 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.600 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 14w REG 8,1 1113303319 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.601 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.602 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.609 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.610 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.611 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.612 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.613 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25240 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.614 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.615 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.616 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.618 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.619 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.620 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.621 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.622 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.623 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.624 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.625 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 14w REG 8,1 1113303319 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.626 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.627 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.628 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.629 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.630 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.632 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.633 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.634 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.635 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.636 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.637 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.638 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25241 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.639 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.640 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.641 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.642 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.643 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.644 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.645 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.647 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.648 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.649 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.653 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.654 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.655 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.656 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 14w REG 8,1 1113303607 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.657 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.658 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.659 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.660 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.661 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.662 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.663 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.664 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.665 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.666 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.668 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25242 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.670 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.671 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.672 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.673 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.674 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.675 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.676 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.677 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.678 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.679 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.680 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.681 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 14w REG 8,1 1113303607 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.684 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.685 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.686 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.687 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.688 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.689 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.690 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.693 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.694 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.695 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.696 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25243 tp_osd_tp root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root txt REG 8,1 25100696 63388 /usr/bin/ceph-osd 2024-04-20T21:47:16.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 74504 403821 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2024-04-20T21:47:16.697 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 86632 403860 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2024-04-20T21:47:16.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 124736 403800 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2024-04-20T21:47:16.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 41176 403806 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2024-04-20T21:47:16.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 95312 403836 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2024-04-20T21:47:16.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 45176 403857 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2024-04-20T21:47:16.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 37064 403815 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2024-04-20T21:47:16.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 1338648 403845 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2024-04-20T21:47:16.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 74928 403803 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2024-04-20T21:47:16.698 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 83008 403818 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2024-04-20T21:47:16.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 70536 403809 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2024-04-20T21:47:16.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 208184 403812 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2024-04-20T21:47:16.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 69736 62643 /usr/lib64/liboath.so.0.1.3 2024-04-20T21:47:16.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 70432 403833 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2024-04-20T21:47:16.699 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 49424 403824 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2024-04-20T21:47:16.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 120512 403848 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2024-04-20T21:47:16.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 864224 403842 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2024-04-20T21:47:16.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 409312 403839 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2024-04-20T21:47:16.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 36976 403863 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2024-04-20T21:47:16.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 32520 403830 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2024-04-20T21:47:16.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 32664 403768 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2024-04-20T21:47:16.700 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root DEL REG 0,18 44981 /[aio] 2024-04-20T21:47:16.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root DEL REG 0,18 50656 /[aio] 2024-04-20T21:47:16.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 1493800 403790 /usr/lib64/ceph/erasure-code/libec_lrc.so 2024-04-20T21:47:16.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 357488 403786 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2024-04-20T21:47:16.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 74680 403796 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2024-04-20T21:47:16.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 0,15 50645 anon_inode:[io_uring] (stat: No such file or directory) 2024-04-20T21:47:16.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.702 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.703 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.704 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.705 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.706 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.707 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-20T21:47:16.708 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 49744 8296 /usr/lib64/libsnappy.so.1.1.8 2024-04-20T21:47:16.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 16448 7362 /usr/lib64/libaio.so.1.0.1 2024-04-20T21:47:16.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 270800 7447 /usr/lib64/libfuse.so.2.9.9 2024-04-20T21:47:16.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.709 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 1w FIFO 0,14 0t0 51398 pipe 2024-04-20T21:47:16.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 2w FIFO 0,14 0t0 51399 pipe 2024-04-20T21:47:16.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 3r FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 4w FIFO 0,14 0t0 44920 pipe 2024-04-20T21:47:16.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 5u a_inode 0,15 0 50 [eventpoll:6,19,20,21,22,23,24,25,26,66,68,69,72,76,77,78,89,92,96,98,99,102] 2024-04-20T21:47:16.710 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 6r FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 7w FIFO 0,14 0t0 44922 pipe 2024-04-20T21:47:16.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 8u a_inode 0,15 0 50 [eventpoll:9,65,67,74,75,81,83,84,86,87,91,93,94] 2024-04-20T21:47:16.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 9r FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 10w FIFO 0,14 0t0 44923 pipe 2024-04-20T21:47:16.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 11u a_inode 0,15 0 50 [eventpoll:12,60,64,70,71,73,79,80,82,85,88,90,100,103] 2024-04-20T21:47:16.711 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 12r FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 13w FIFO 0,14 0t0 44924 pipe 2024-04-20T21:47:16.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 14w REG 8,1 1113303607 393327 /var/log/ceph/ceph-osd.2.log 2024-04-20T21:47:16.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 15uW REG 0,25 6 1614 /run/ceph/ceph-osd.2.pid 2024-04-20T21:47:16.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 16r FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 17w FIFO 0,14 0t0 44926 pipe 2024-04-20T21:47:16.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 18u unix 0xffff88817bc6cb00 0t0 44927 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.712 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 19u IPv4 53401 0t0 TCP *:pentbox-sim (LISTEN) 2024-04-20T21:47:16.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 20u IPv4 53421 0t0 TCP *:6819 (LISTEN) 2024-04-20T21:47:16.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 21u IPv4 53443 0t0 TCP *:6821 (LISTEN) 2024-04-20T21:47:16.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 22u IPv4 53466 0t0 TCP *:6822 (LISTEN) 2024-04-20T21:47:16.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 23u IPv4 53491 0t0 TCP *:6824 (LISTEN) 2024-04-20T21:47:16.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 24u IPv4 53518 0t0 TCP *:6826 (LISTEN) 2024-04-20T21:47:16.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 25u IPv4 53547 0t0 TCP *:6828 (LISTEN) 2024-04-20T21:47:16.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 26u IPv4 53578 0t0 TCP *:6830 (LISTEN) 2024-04-20T21:47:16.713 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 27r FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 28w FIFO 0,14 0t0 44937 pipe 2024-04-20T21:47:16.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 29r FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 30w FIFO 0,14 0t0 44938 pipe 2024-04-20T21:47:16.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 31u sock 0,9 0t0 50642 protocol: UDPv6 2024-04-20T21:47:16.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2024-04-20T21:47:16.714 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-04-20T21:47:16.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.715 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.716 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.717 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-04-20T21:47:16.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 58u a_inode 0,15 0 50645 [io_uring] 2024-04-20T21:47:16.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 59r FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 60u IPv4 95075 0t0 TCP smithi102.front.sepia.ceph.com:50902->smithi193.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-04-20T21:47:16.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 61w FIFO 0,14 0t0 48643 pipe 2024-04-20T21:47:16.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 62r FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 63w FIFO 0,14 0t0 48644 pipe 2024-04-20T21:47:16.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 64u IPv4 108617 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39958 (ESTABLISHED) 2024-04-20T21:47:16.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 65u IPv4 113873 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:58772 (ESTABLISHED) 2024-04-20T21:47:16.719 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 66u IPv4 113799 0t0 TCP smithi102.front.sepia.ceph.com:34436->smithi102.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 67u IPv4 113876 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:59782 (ESTABLISHED) 2024-04-20T21:47:16.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 68u IPv4 110167 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:50900 (ESTABLISHED) 2024-04-20T21:47:16.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 69u IPv4 113800 0t0 TCP smithi102.front.sepia.ceph.com:58452->smithi102.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 70u IPv4 104812 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:60202 (ESTABLISHED) 2024-04-20T21:47:16.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 71u IPv4 109664 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48260 (ESTABLISHED) 2024-04-20T21:47:16.720 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 72u IPv4 104437 0t0 TCP smithi102.front.sepia.ceph.com:38364->smithi102.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-04-20T21:47:16.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 73u IPv4 104815 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:44122 (ESTABLISHED) 2024-04-20T21:47:16.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 74u IPv4 107890 0t0 TCP smithi102.front.sepia.ceph.com:60686->smithi102.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-04-20T21:47:16.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 75u IPv4 111483 0t0 TCP smithi102.front.sepia.ceph.com:40844->smithi102.front.sepia.ceph.com:6829 (ESTABLISHED) 2024-04-20T21:47:16.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 76u IPv4 113124 0t0 TCP smithi102.front.sepia.ceph.com:37236->smithi193.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-04-20T21:47:16.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 77u IPv4 111620 0t0 TCP smithi102.front.sepia.ceph.com:34350->smithi193.front.sepia.ceph.com:6809 (ESTABLISHED) 2024-04-20T21:47:16.721 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 78u IPv4 113125 0t0 TCP smithi102.front.sepia.ceph.com:43740->smithi193.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-04-20T21:47:16.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 79u IPv4 113225 0t0 TCP smithi102.front.sepia.ceph.com:41278->smithi193.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-04-20T21:47:16.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 80u IPv4 112864 0t0 TCP smithi102.front.sepia.ceph.com:45464->smithi102.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-04-20T21:47:16.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 81u IPv4 111484 0t0 TCP smithi102.front.sepia.ceph.com:37896->smithi102.front.sepia.ceph.com:6825 (ESTABLISHED) 2024-04-20T21:47:16.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 82u IPv4 113226 0t0 TCP smithi102.front.sepia.ceph.com:38040->smithi193.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-04-20T21:47:16.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 83u IPv4 105989 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:56154 (ESTABLISHED) 2024-04-20T21:47:16.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 84u IPv4 112463 0t0 TCP smithi102.front.sepia.ceph.com:39862->smithi193.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-04-20T21:47:16.722 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 85u IPv4 105992 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:42326 (ESTABLISHED) 2024-04-20T21:47:16.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 86u IPv4 112462 0t0 TCP smithi102.front.sepia.ceph.com:41004->smithi193.front.sepia.ceph.com:6821 (ESTABLISHED) 2024-04-20T21:47:16.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 87u IPv4 110170 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:42842 (ESTABLISHED) 2024-04-20T21:47:16.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 88u IPv4 108019 0t0 TCP smithi102.front.sepia.ceph.com:37160->smithi102.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-04-20T21:47:16.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 89u IPv4 107484 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:38378 (ESTABLISHED) 2024-04-20T21:47:16.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 90u IPv4 105626 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi102.front.sepia.ceph.com:41170 (ESTABLISHED) 2024-04-20T21:47:16.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 91u IPv4 101351 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi102.front.sepia.ceph.com:37232 (ESTABLISHED) 2024-04-20T21:47:16.723 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 92u IPv4 100088 0t0 TCP smithi102.front.sepia.ceph.com:46882->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 93u IPv4 108620 0t0 TCP smithi102.front.sepia.ceph.com:6824->smithi193.front.sepia.ceph.com:48250 (ESTABLISHED) 2024-04-20T21:47:16.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 94u IPv4 111535 0t0 TCP smithi102.front.sepia.ceph.com:pentbox-sim->smithi193.front.sepia.ceph.com:34940 (ESTABLISHED) 2024-04-20T21:47:16.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 96u IPv4 111621 0t0 TCP smithi102.front.sepia.ceph.com:54714->smithi193.front.sepia.ceph.com:6805 (ESTABLISHED) 2024-04-20T21:47:16.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 98u IPv4 112646 0t0 TCP smithi102.front.sepia.ceph.com:57186->smithi193.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-04-20T21:47:16.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 99u IPv4 109667 0t0 TCP smithi102.front.sepia.ceph.com:6828->smithi193.front.sepia.ceph.com:39970 (ESTABLISHED) 2024-04-20T21:47:16.724 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 100u IPv4 110487 0t0 TCP smithi102.front.sepia.ceph.com:6821->smithi193.front.sepia.ceph.com:55418 (ESTABLISHED) 2024-04-20T21:47:16.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 102u IPv4 112030 0t0 TCP smithi102.front.sepia.ceph.com:44020->smithi102.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.725 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd 22381 25244 osd_srv_h root 103u IPv4 106788 0t0 TCP smithi102.front.sepia.ceph.com:56236->smithi193.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-04-20T21:47:16.725 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 26273 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.725 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 26273 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.725 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 26273 root txt unknown /proc/26273/exe 2024-04-20T21:47:16.725 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 26438 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.725 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 26438 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.725 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 26438 root txt unknown /proc/26438/exe 2024-04-20T21:47:16.726 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 26463 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.726 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 26463 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.726 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 26463 root txt unknown /proc/26463/exe 2024-04-20T21:47:16.726 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 27218 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.726 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 27218 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.726 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 27218 root txt unknown /proc/27218/exe 2024-04-20T21:47:16.727 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.727 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.727 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root txt REG 8,1 185464 9243 /usr/bin/sudo 2024-04-20T21:47:16.727 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:16.727 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:16.727 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.728 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:16.728 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.728 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:16.728 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:16.728 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:16.728 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:16.728 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:16.729 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:16.729 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:16.729 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:16.729 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:16.729 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.729 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.729 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:16.730 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:16.730 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.730 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.730 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.730 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.730 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.731 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.731 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.731 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.731 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.731 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.731 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.731 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:16.731 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 472552 47567 /usr/libexec/sudo/sudoers.so 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:16.732 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:16.733 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:16.733 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:16.733 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:16.733 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:16.733 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:16.733 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:16.733 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:16.733 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:16.734 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:16.734 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 128024 47563 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-04-20T21:47:16.734 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.734 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:16.734 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:16.734 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:16.734 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.734 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:16.735 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root 0r FIFO 0,14 0t0 49881 pipe 2024-04-20T21:47:16.735 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.735 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.735 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root 3r FIFO 0,14 0t0 48809 pipe 2024-04-20T21:47:16.735 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root 4w FIFO 0,14 0t0 48809 pipe 2024-04-20T21:47:16.735 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root 5u netlink 0t0 53727 AUDIT 2024-04-20T21:47:16.735 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 28019 root 6u unix 0xffff888105634b00 0t0 53728 type=DGRAM (CONNECTED) 2024-04-20T21:47:16.735 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.736 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.736 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root txt REG 8,1 15624 7981 /usr/bin/python3.9 2024-04-20T21:47:16.736 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-20T21:47:16.736 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 76792 7184 /usr/lib64/libbz2.so.1.0.8 2024-04-20T21:47:16.736 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.736 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 67640 39982 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 23912 36213 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 20120 39981 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 56520 36214 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 37712 36223 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 3486536 11950 /usr/lib64/libpython3.9.so.1.0 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 23952 36218 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.737 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 28176 39965 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.738 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 46688 36210 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.738 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 29336 39960 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.738 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 41880 36225 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2024-04-20T21:47:16.738 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.738 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:16.738 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.738 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root 0r FIFO 0,14 0t0 49881 pipe 2024-04-20T21:47:16.738 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.739 INFO:teuthology.orchestra.run.smithi102.stdout:daemon-he 28083 root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.741 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.742 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.743 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.744 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.745 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.747 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.748 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.750 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.751 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.752 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.753 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.754 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.755 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.757 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.760 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.761 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.762 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28109 log root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.763 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.764 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.765 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.766 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.767 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.768 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.769 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.770 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.771 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.772 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28111 msgr-work root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.773 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.774 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.775 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.776 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.777 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.778 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.779 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.780 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.781 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.782 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.783 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.784 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28112 msgr-work root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.785 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.786 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.788 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.789 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.790 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.791 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.793 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.794 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.795 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.796 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28113 msgr-work root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.797 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.798 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.799 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.800 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.801 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.802 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.803 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.804 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.805 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.806 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.807 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28127 service root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.808 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.809 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.810 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.811 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.812 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.813 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.814 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.815 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.816 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.818 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28128 admin_soc root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.819 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.820 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.821 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.822 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.823 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.824 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.825 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.826 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.827 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.828 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.829 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.830 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.831 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28139 signal_ha root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.832 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.833 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.835 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.837 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.838 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.839 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.840 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.841 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28140 io_contex root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.842 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.843 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.844 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.845 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.846 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.847 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.848 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.849 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.851 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.852 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28141 io_contex root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.854 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.855 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.857 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.858 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.859 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.860 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.861 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.862 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.863 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.866 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.867 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.868 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28142 ms_dispat root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.869 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.870 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.871 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.872 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.873 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.874 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.875 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.876 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.877 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.878 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28143 ms_local root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.879 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.880 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.881 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.882 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.883 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.884 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.885 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.886 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.887 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.888 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28144 safe_time root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.889 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.890 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.892 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.893 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.896 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.898 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.899 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.900 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28145 safe_time root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.901 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.902 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.903 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.904 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.905 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.906 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.907 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.908 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.909 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.910 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.911 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.912 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28146 ceph-mds root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.913 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root txt REG 8,1 7612712 63401 /usr/bin/ceph-mds 2024-04-20T21:47:16.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 45480 9348 /usr/lib64/liburcu-cds.so.6.1.0 2024-04-20T21:47:16.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 41360 9346 /usr/lib64/liburcu-bp.so.6.1.0 2024-04-20T21:47:16.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 53816 7480 /usr/lib64/libnuma.so.1.0.0 2024-04-20T21:47:16.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 53272 62306 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2024-04-20T21:47:16.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.914 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 1591920 7217 /usr/lib64/libunistring.so.2.1.0 2024-04-20T21:47:16.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 138712 7373 /usr/lib64/libbrotlicommon.so.1.0.9 2024-04-20T21:47:16.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.915 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 53048 7375 /usr/lib64/libbrotlidec.so.1.0.9 2024-04-20T21:47:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 483832 8964 /usr/lib64/libssh.so.4.9.4 2024-04-20T21:47:16.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 130952 7255 /usr/lib64/libidn2.so.0.3.7 2024-04-20T21:47:16.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 168672 8859 /usr/lib64/libnghttp2.so.14.20.1 2024-04-20T21:47:16.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 617664 7302 /usr/lib64/libnl-route-3.so.200.26.0 2024-04-20T21:47:16.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 2314576 7174 /usr/lib64/libstdc++.so.6.0.29 2024-04-20T21:47:16.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 146376 7294 /usr/lib64/libnl-3.so.200.26.0 2024-04-20T21:47:16.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 639224 62273 /usr/lib64/libthrift-0.15.0.so 2024-04-20T21:47:16.917 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 666496 8968 /usr/lib64/libcurl.so.4.7.0 2024-04-20T21:47:16.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 112536 10463 /usr/lib64/librdmacm.so.1.3.48.0 2024-04-20T21:47:16.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 137720 8081 /usr/lib64/libibverbs.so.1.14.48.0 2024-04-20T21:47:16.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 10173464 393503 /usr/lib64/ceph/libceph-common.so.2 2024-04-20T21:47:16.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 77488 7371 /usr/lib64/libpsl.so.5.3.3 2024-04-20T21:47:16.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 200248 7607 /usr/lib64/libudev.so.1.7.5 2024-04-20T21:47:16.918 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 326760 62735 /usr/lib64/libtcmalloc.so.4.5.9 2024-04-20T21:47:16.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-20T21:47:16.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 61880 62725 /usr/lib64/libunwind.so.8.0.1 2024-04-20T21:47:16.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 287296 7233 /usr/lib64/liblua-5.4.so 2024-04-20T21:47:16.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 24248 9350 /usr/lib64/liburcu-common.so.6.1.0 2024-04-20T21:47:16.919 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-04-20T21:47:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 1w FIFO 0,14 0t0 49882 pipe 2024-04-20T21:47:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 2w FIFO 0,14 0t0 49883 pipe 2024-04-20T21:47:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 3r FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 4w FIFO 0,14 0t0 49936 pipe 2024-04-20T21:47:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 5u a_inode 0,15 0 50 [eventpoll:6,19,20] 2024-04-20T21:47:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 6r FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.920 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 7w FIFO 0,14 0t0 49938 pipe 2024-04-20T21:47:16.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 8u a_inode 0,15 0 50 [eventpoll:9,25] 2024-04-20T21:47:16.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 9r FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 10w FIFO 0,14 0t0 49939 pipe 2024-04-20T21:47:16.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 11u a_inode 0,15 0 50 [eventpoll:12,30] 2024-04-20T21:47:16.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 12r FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 13w FIFO 0,14 0t0 49940 pipe 2024-04-20T21:47:16.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 14w REG 8,1 563838 393339 /var/log/ceph/ceph-mds.e.log 2024-04-20T21:47:16.921 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 15uW REG 0,25 6 1624 /run/ceph/ceph-mds.e.pid 2024-04-20T21:47:16.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 16r FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 17w FIFO 0,14 0t0 49943 pipe 2024-04-20T21:47:16.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 18u unix 0xffff88815c743c00 0t0 49944 /var/run/ceph/ceph-mds.e.asok type=STREAM (LISTEN) 2024-04-20T21:47:16.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 19u IPv4 52635 0t0 TCP *:6836 (LISTEN) 2024-04-20T21:47:16.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 20u IPv4 52673 0t0 TCP *:6837 (LISTEN) 2024-04-20T21:47:16.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 21r FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.922 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 22w FIFO 0,14 0t0 52674 pipe 2024-04-20T21:47:16.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 23r FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 24w FIFO 0,14 0t0 52675 pipe 2024-04-20T21:47:16.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 25u IPv4 109445 0t0 TCP smithi102.front.sepia.ceph.com:38566->smithi193.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-04-20T21:47:16.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 26r FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 27w FIFO 0,14 0t0 52677 pipe 2024-04-20T21:47:16.923 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 28r FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 29w FIFO 0,14 0t0 52678 pipe 2024-04-20T21:47:16.924 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds 28092 28251 safe_time root 30u IPv4 113304 0t0 TCP smithi102.front.sepia.ceph.com:47282->smithi193.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-04-20T21:47:16.924 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32586 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.924 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32586 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.924 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32586 root txt unknown /proc/32586/exe 2024-04-20T21:47:16.924 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32601 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32601 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32601 root txt unknown /proc/32601/exe 2024-04-20T21:47:16.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32602 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32602 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/R 32602 root txt unknown /proc/32602/exe 2024-04-20T21:47:16.925 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 36709 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 36709 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 36709 root txt unknown /proc/36709/exe 2024-04-20T21:47:16.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 37096 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 37096 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 37096 root txt unknown /proc/37096/exe 2024-04-20T21:47:16.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 39662 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.926 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 39662 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 39662 root txt unknown /proc/39662/exe 2024-04-20T21:47:16.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 40012 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 40012 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 40012 root txt unknown /proc/40012/exe 2024-04-20T21:47:16.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 40032 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.927 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 40032 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 40032 root txt unknown /proc/40032/exe 2024-04-20T21:47:16.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 40930 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 40930 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 40930 root txt unknown /proc/40930/exe 2024-04-20T21:47:16.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 41038 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.928 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 41038 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 41038 root txt unknown /proc/41038/exe 2024-04-20T21:47:16.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 41148 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 41148 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 41148 root txt unknown /proc/41148/exe 2024-04-20T21:47:16.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 41173 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.929 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 41173 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 41173 root txt unknown /proc/41173/exe 2024-04-20T21:47:16.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 42049 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 42049 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 42049 root txt unknown /proc/42049/exe 2024-04-20T21:47:16.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 42071 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 42071 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.930 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 42071 root txt unknown /proc/42071/exe 2024-04-20T21:47:16.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 42130 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 42130 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 42130 root txt unknown /proc/42130/exe 2024-04-20T21:47:16.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 44557 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 44557 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.931 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 44557 root txt unknown /proc/44557/exe 2024-04-20T21:47:16.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 45125 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 45125 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 45125 root txt unknown /proc/45125/exe 2024-04-20T21:47:16.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 45126 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 45126 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 45126 root txt unknown /proc/45126/exe 2024-04-20T21:47:16.932 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 45272 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 45272 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 45272 root txt unknown /proc/45272/exe 2024-04-20T21:47:16.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46058 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46058 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46058 root txt unknown /proc/46058/exe 2024-04-20T21:47:16.933 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46246 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46246 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46246 root txt unknown /proc/46246/exe 2024-04-20T21:47:16.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46247 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46247 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46247 root txt unknown /proc/46247/exe 2024-04-20T21:47:16.934 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46266 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46266 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46266 root txt unknown /proc/46266/exe 2024-04-20T21:47:16.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46267 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46267 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46267 root txt unknown /proc/46267/exe 2024-04-20T21:47:16.935 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46268 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46268 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46268 root txt unknown /proc/46268/exe 2024-04-20T21:47:16.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46269 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46269 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 46269 root txt unknown /proc/46269/exe 2024-04-20T21:47:16.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46273 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.936 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46273 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46273 root txt unknown /proc/46273/exe 2024-04-20T21:47:16.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46274 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46274 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 46274 root txt unknown /proc/46274/exe 2024-04-20T21:47:16.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46277 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.937 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46277 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.938 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46277 root txt unknown /proc/46277/exe 2024-04-20T21:47:16.938 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46278 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.938 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46278 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.938 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 46278 root txt unknown /proc/46278/exe 2024-04-20T21:47:16.938 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 46279 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.938 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 46279 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.939 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 46279 root txt unknown /proc/46279/exe 2024-04-20T21:47:16.939 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 46282 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.939 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 46282 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.939 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 46282 root txt unknown /proc/46282/exe 2024-04-20T21:47:16.939 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 46304 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.939 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 46304 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.939 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 46304 root txt unknown /proc/46304/exe 2024-04-20T21:47:16.940 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46308 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.940 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46308 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.940 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46308 root txt unknown /proc/46308/exe 2024-04-20T21:47:16.940 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46310 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.940 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46310 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.940 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46310 root txt unknown /proc/46310/exe 2024-04-20T21:47:16.941 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46333 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.941 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46333 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.941 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/3 46333 root txt unknown /proc/46333/exe 2024-04-20T21:47:16.941 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46334 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.941 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46334 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.941 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 46334 root txt unknown /proc/46334/exe 2024-04-20T21:47:16.941 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 46336 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.942 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 46336 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.942 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/7 46336 root txt unknown /proc/46336/exe 2024-04-20T21:47:16.942 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 47487 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.942 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 47487 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.942 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 47487 root txt unknown /proc/47487/exe 2024-04-20T21:47:16.943 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 47509 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.943 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 47509 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.943 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 47509 root txt unknown /proc/47509/exe 2024-04-20T21:47:16.943 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 48575 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.943 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 48575 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.943 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 48575 root txt unknown /proc/48575/exe 2024-04-20T21:47:16.943 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 48576 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.944 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 48576 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.944 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 48576 root txt unknown /proc/48576/exe 2024-04-20T21:47:16.944 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 48578 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.944 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 48578 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.944 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 48578 root txt unknown /proc/48578/exe 2024-04-20T21:47:16.944 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 48579 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.945 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 48579 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.945 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/u 48579 root txt unknown /proc/48579/exe 2024-04-20T21:47:16.945 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48580 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.945 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48580 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.945 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48580 root txt unknown /proc/48580/exe 2024-04-20T21:47:16.945 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48586 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.945 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48586 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.945 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48586 root txt unknown /proc/48586/exe 2024-04-20T21:47:16.946 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48587 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.946 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48587 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.946 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 48587 root txt unknown /proc/48587/exe 2024-04-20T21:47:16.946 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 49601 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.946 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 49601 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.946 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/5 49601 root txt unknown /proc/49601/exe 2024-04-20T21:47:16.946 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50603 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.946 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50603 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50603 root txt unknown /proc/50603/exe 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 50604 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 50604 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/0 50604 root txt unknown /proc/50604/exe 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50605 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50605 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50605 root txt unknown /proc/50605/exe 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50606 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.947 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50606 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.948 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50606 root txt unknown /proc/50606/exe 2024-04-20T21:47:16.948 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50607 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.948 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50607 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.948 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50607 root txt unknown /proc/50607/exe 2024-04-20T21:47:16.948 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50608 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.948 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50608 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.948 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50608 root txt unknown /proc/50608/exe 2024-04-20T21:47:16.948 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 50609 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 50609 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/1 50609 root txt unknown /proc/50609/exe 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 50638 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 50638 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/4 50638 root txt unknown /proc/50638/exe 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50639 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50639 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50639 root txt unknown /proc/50639/exe 2024-04-20T21:47:16.949 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50640 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.950 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50640 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.950 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50640 root txt unknown /proc/50640/exe 2024-04-20T21:47:16.950 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50641 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.950 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50641 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.950 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50641 root txt unknown /proc/50641/exe 2024-04-20T21:47:16.950 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50642 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.950 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50642 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50642 root txt unknown /proc/50642/exe 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50643 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50643 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50643 root txt unknown /proc/50643/exe 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50644 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50644 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50644 root txt unknown /proc/50644/exe 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50645 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.951 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50645 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/2 50645 root txt unknown /proc/50645/exe 2024-04-20T21:47:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50646 root cwd DIR 8,1 4096 2 / 2024-04-20T21:47:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50646 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:kworker/6 50646 root txt unknown /proc/50646/exe 2024-04-20T21:47:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-04-20T21:47:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:16.953 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.953 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:16.953 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:16.953 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:16.953 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:16.953 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:16.953 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:16.953 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:16.954 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:16.954 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:16.954 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-04-20T21:47:16.954 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:16.954 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:16.954 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.954 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:16.954 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu 0r FIFO 0,14 0t0 115785 pipe 2024-04-20T21:47:16.955 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu 1w FIFO 0,14 0t0 115786 pipe 2024-04-20T21:47:16.955 INFO:teuthology.orchestra.run.smithi102.stdout:bash 53403 ubuntu 2w FIFO 0,14 0t0 115787 pipe 2024-04-20T21:47:16.955 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.955 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.955 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root txt REG 8,1 185464 9243 /usr/bin/sudo 2024-04-20T21:47:16.955 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 353408 7603 /usr/lib64/libnss_systemd.so.2 2024-04-20T21:47:16.955 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-20T21:47:16.955 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-20T21:47:16.956 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-20T21:47:16.956 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 514368 33785 /usr/lib64/security/pam_systemd.so 2024-04-20T21:47:16.956 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.956 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.956 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-20T21:47:16.956 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 19712 8450 /usr/lib64/libpam_misc.so.0.82.1 2024-04-20T21:47:16.956 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 19520 30102 /usr/lib64/security/pam_succeed_if.so 2024-04-20T21:47:16.956 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 27648 30081 /usr/lib64/security/pam_limits.so 2024-04-20T21:47:16.957 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 48392 7711 /usr/lib64/libcrack.so.2.9.0 2024-04-20T21:47:16.957 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 32608 8444 /usr/lib64/libpwquality.so.1.0.2 2024-04-20T21:47:16.957 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 15464 30079 /usr/lib64/security/pam_keyinit.so 2024-04-20T21:47:16.957 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 15840 30061 /usr/lib64/security/pam_pwquality.so 2024-04-20T21:47:16.957 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 56824 30107 /usr/lib64/security/pam_unix.so 2024-04-20T21:47:16.957 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.957 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.957 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-20T21:47:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 688312 7492 /usr/lib64/libssl.so.3.0.7 2024-04-20T21:47:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 128416 8898 /usr/lib64/libsasl2.so.3.0.0 2024-04-20T21:47:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 366424 7894 /usr/lib64/libevent-2.1.so.7.0.1 2024-04-20T21:47:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-20T21:47:16.959 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.959 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 70400 8902 /usr/lib64/liblber.so.2.0.200 2024-04-20T21:47:16.959 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 417712 8906 /usr/lib64/libldap_r.so.2.0.200 2024-04-20T21:47:16.959 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-20T21:47:16.959 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 15040 30067 /usr/lib64/security/pam_deny.so 2024-04-20T21:47:16.959 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-20T21:47:16.959 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-20T21:47:16.959 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 472552 47567 /usr/libexec/sudo/sudoers.so 2024-04-20T21:47:16.960 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:16.960 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.960 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.960 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 15304 30071 /usr/lib64/security/pam_faildelay.so 2024-04-20T21:47:16.960 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 19568 30069 /usr/lib64/security/pam_env.so 2024-04-20T21:47:16.960 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-20T21:47:16.961 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-20T21:47:16.961 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-20T21:47:16.961 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-20T21:47:16.961 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-20T21:47:16.961 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-20T21:47:16.961 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-20T21:47:16.962 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-20T21:47:16.962 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-20T21:47:16.962 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 128024 47563 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-04-20T21:47:16.962 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.962 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-20T21:47:16.962 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-20T21:47:16.963 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:16.963 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.963 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-20T21:47:16.963 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root 0r FIFO 0,14 0t0 115785 pipe 2024-04-20T21:47:16.963 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root 1w FIFO 0,14 0t0 115786 pipe 2024-04-20T21:47:16.963 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root 2w FIFO 0,14 0t0 115787 pipe 2024-04-20T21:47:16.964 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root 3r FIFO 0,14 0t0 116882 pipe 2024-04-20T21:47:16.964 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root 4w FIFO 0,14 0t0 116882 pipe 2024-04-20T21:47:16.964 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root 5u netlink 0t0 116888 AUDIT 2024-04-20T21:47:16.964 INFO:teuthology.orchestra.run.smithi102.stdout:sudo 53419 root 6u unix 0xffff888162353480 0t0 116889 type=DGRAM (CONNECTED) 2024-04-20T21:47:16.964 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.964 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.965 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-20T21:47:16.965 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.965 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:16.965 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.965 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.965 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.966 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.966 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.966 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.966 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.966 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.966 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.967 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.967 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-20T21:47:16.967 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.967 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:16.967 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.967 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root 0r FIFO 0,14 0t0 115785 pipe 2024-04-20T21:47:16.968 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root 1w FIFO 0,14 0t0 115786 pipe 2024-04-20T21:47:16.968 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root 2w FIFO 0,14 0t0 115787 pipe 2024-04-20T21:47:16.968 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root 3r DIR 0,21 0 1 /proc 2024-04-20T21:47:16.968 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root 4r DIR 0,21 7 114124 /proc/53421/fd 2024-04-20T21:47:16.968 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root 5w FIFO 0,14 0t0 114129 pipe 2024-04-20T21:47:16.968 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53421 root 6r FIFO 0,14 0t0 114130 pipe 2024-04-20T21:47:16.969 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-20T21:47:16.969 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root rtd DIR 8,1 4096 2 / 2024-04-20T21:47:16.969 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-20T21:47:16.969 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-20T21:47:16.969 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-20T21:47:16.969 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-20T21:47:16.969 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-20T21:47:16.970 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-20T21:47:16.970 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-20T21:47:16.970 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-20T21:47:16.970 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-20T21:47:16.970 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-20T21:47:16.970 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-20T21:47:16.971 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-20T21:47:16.971 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-20T21:47:16.971 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-20T21:47:16.971 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-20T21:47:16.971 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-20T21:47:16.971 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-20T21:47:16.972 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root 4r FIFO 0,14 0t0 114129 pipe 2024-04-20T21:47:16.972 INFO:teuthology.orchestra.run.smithi102.stdout:lsof 53440 root 7w FIFO 0,14 0t0 114130 pipe 2024-04-20T21:47:16.972 INFO:teuthology.orchestra.run.smithi102.stdout:USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 2024-04-20T21:47:16.972 INFO:teuthology.orchestra.run.smithi102.stdout:root 2 0.0 0.0 0 0 ? S 21:10 0:00 [kthreadd] 2024-04-20T21:47:16.972 INFO:teuthology.orchestra.run.smithi102.stdout:root 3 0.0 0.0 0 0 ? S 21:10 0:00 \_ [pool_workqueue_release] 2024-04-20T21:47:16.972 INFO:teuthology.orchestra.run.smithi102.stdout:root 4 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-rcu_g] 2024-04-20T21:47:16.973 INFO:teuthology.orchestra.run.smithi102.stdout:root 5 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-slub_] 2024-04-20T21:47:16.973 INFO:teuthology.orchestra.run.smithi102.stdout:root 6 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-netns] 2024-04-20T21:47:16.973 INFO:teuthology.orchestra.run.smithi102.stdout:root 9 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/0:0H-events_highpri] 2024-04-20T21:47:16.973 INFO:teuthology.orchestra.run.smithi102.stdout:root 11 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-mm_pe] 2024-04-20T21:47:16.973 INFO:teuthology.orchestra.run.smithi102.stdout:root 12 0.0 0.0 0 0 ? I 21:10 0:00 \_ [rcu_tasks_kthread] 2024-04-20T21:47:16.973 INFO:teuthology.orchestra.run.smithi102.stdout:root 13 0.0 0.0 0 0 ? I 21:10 0:00 \_ [rcu_tasks_rude_kthread] 2024-04-20T21:47:16.973 INFO:teuthology.orchestra.run.smithi102.stdout:root 14 0.0 0.0 0 0 ? I 21:10 0:00 \_ [rcu_tasks_trace_kthread] 2024-04-20T21:47:16.974 INFO:teuthology.orchestra.run.smithi102.stdout:root 15 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ksoftirqd/0] 2024-04-20T21:47:16.974 INFO:teuthology.orchestra.run.smithi102.stdout:root 16 0.1 0.0 0 0 ? I 21:10 0:02 \_ [rcu_preempt] 2024-04-20T21:47:16.974 INFO:teuthology.orchestra.run.smithi102.stdout:root 17 0.0 0.0 0 0 ? S 21:10 0:00 \_ [rcu_exp_par_gp_kthread_worker/0] 2024-04-20T21:47:16.974 INFO:teuthology.orchestra.run.smithi102.stdout:root 18 0.0 0.0 0 0 ? S 21:10 0:00 \_ [rcu_exp_gp_kthread_worker] 2024-04-20T21:47:16.974 INFO:teuthology.orchestra.run.smithi102.stdout:root 19 0.0 0.0 0 0 ? S 21:10 0:00 \_ [migration/0] 2024-04-20T21:47:16.974 INFO:teuthology.orchestra.run.smithi102.stdout:root 20 0.0 0.0 0 0 ? S 21:10 0:00 \_ [cpuhp/0] 2024-04-20T21:47:16.975 INFO:teuthology.orchestra.run.smithi102.stdout:root 21 0.0 0.0 0 0 ? S 21:10 0:00 \_ [cpuhp/1] 2024-04-20T21:47:16.975 INFO:teuthology.orchestra.run.smithi102.stdout:root 22 0.0 0.0 0 0 ? S 21:10 0:00 \_ [migration/1] 2024-04-20T21:47:16.975 INFO:teuthology.orchestra.run.smithi102.stdout:root 23 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ksoftirqd/1] 2024-04-20T21:47:16.975 INFO:teuthology.orchestra.run.smithi102.stdout:root 25 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/1:0H-events_highpri] 2024-04-20T21:47:16.975 INFO:teuthology.orchestra.run.smithi102.stdout:root 26 0.0 0.0 0 0 ? S 21:10 0:00 \_ [cpuhp/2] 2024-04-20T21:47:16.975 INFO:teuthology.orchestra.run.smithi102.stdout:root 27 0.0 0.0 0 0 ? S 21:10 0:00 \_ [migration/2] 2024-04-20T21:47:16.976 INFO:teuthology.orchestra.run.smithi102.stdout:root 28 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ksoftirqd/2] 2024-04-20T21:47:16.976 INFO:teuthology.orchestra.run.smithi102.stdout:root 30 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/2:0H-events_highpri] 2024-04-20T21:47:16.976 INFO:teuthology.orchestra.run.smithi102.stdout:root 31 0.0 0.0 0 0 ? S 21:10 0:00 \_ [cpuhp/3] 2024-04-20T21:47:16.976 INFO:teuthology.orchestra.run.smithi102.stdout:root 32 0.0 0.0 0 0 ? S 21:10 0:00 \_ [migration/3] 2024-04-20T21:47:16.976 INFO:teuthology.orchestra.run.smithi102.stdout:root 33 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ksoftirqd/3] 2024-04-20T21:47:16.976 INFO:teuthology.orchestra.run.smithi102.stdout:root 35 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/3:0H-events_highpri] 2024-04-20T21:47:16.977 INFO:teuthology.orchestra.run.smithi102.stdout:root 36 0.0 0.0 0 0 ? S 21:10 0:00 \_ [cpuhp/4] 2024-04-20T21:47:16.977 INFO:teuthology.orchestra.run.smithi102.stdout:root 37 0.0 0.0 0 0 ? S 21:10 0:00 \_ [migration/4] 2024-04-20T21:47:16.977 INFO:teuthology.orchestra.run.smithi102.stdout:root 38 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ksoftirqd/4] 2024-04-20T21:47:16.977 INFO:teuthology.orchestra.run.smithi102.stdout:root 40 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/4:0H-events_highpri] 2024-04-20T21:47:16.977 INFO:teuthology.orchestra.run.smithi102.stdout:root 41 0.0 0.0 0 0 ? S 21:10 0:00 \_ [cpuhp/5] 2024-04-20T21:47:16.977 INFO:teuthology.orchestra.run.smithi102.stdout:root 42 0.0 0.0 0 0 ? S 21:10 0:00 \_ [migration/5] 2024-04-20T21:47:16.978 INFO:teuthology.orchestra.run.smithi102.stdout:root 43 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ksoftirqd/5] 2024-04-20T21:47:16.978 INFO:teuthology.orchestra.run.smithi102.stdout:root 45 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/5:0H-events_highpri] 2024-04-20T21:47:16.978 INFO:teuthology.orchestra.run.smithi102.stdout:root 46 0.0 0.0 0 0 ? S 21:10 0:00 \_ [cpuhp/6] 2024-04-20T21:47:16.978 INFO:teuthology.orchestra.run.smithi102.stdout:root 47 0.0 0.0 0 0 ? S 21:10 0:00 \_ [migration/6] 2024-04-20T21:47:16.978 INFO:teuthology.orchestra.run.smithi102.stdout:root 48 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ksoftirqd/6] 2024-04-20T21:47:16.978 INFO:teuthology.orchestra.run.smithi102.stdout:root 50 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/6:0H-kblockd] 2024-04-20T21:47:16.978 INFO:teuthology.orchestra.run.smithi102.stdout:root 51 0.0 0.0 0 0 ? S 21:10 0:00 \_ [cpuhp/7] 2024-04-20T21:47:16.979 INFO:teuthology.orchestra.run.smithi102.stdout:root 52 0.0 0.0 0 0 ? S 21:10 0:00 \_ [migration/7] 2024-04-20T21:47:16.979 INFO:teuthology.orchestra.run.smithi102.stdout:root 53 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ksoftirqd/7] 2024-04-20T21:47:16.979 INFO:teuthology.orchestra.run.smithi102.stdout:root 55 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/7:0H-events_highpri] 2024-04-20T21:47:16.979 INFO:teuthology.orchestra.run.smithi102.stdout:root 56 0.0 0.0 0 0 ? S 21:10 0:00 \_ [kdevtmpfs] 2024-04-20T21:47:16.979 INFO:teuthology.orchestra.run.smithi102.stdout:root 57 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-inet_] 2024-04-20T21:47:16.979 INFO:teuthology.orchestra.run.smithi102.stdout:root 58 0.0 0.0 0 0 ? S 21:10 0:00 \_ [kauditd] 2024-04-20T21:47:16.980 INFO:teuthology.orchestra.run.smithi102.stdout:root 59 0.0 0.0 0 0 ? S 21:10 0:00 \_ [khungtaskd] 2024-04-20T21:47:16.980 INFO:teuthology.orchestra.run.smithi102.stdout:root 61 0.0 0.0 0 0 ? S 21:10 0:00 \_ [oom_reaper] 2024-04-20T21:47:16.980 INFO:teuthology.orchestra.run.smithi102.stdout:root 63 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-write] 2024-04-20T21:47:16.980 INFO:teuthology.orchestra.run.smithi102.stdout:root 64 0.0 0.0 0 0 ? S 21:10 0:00 \_ [kcompactd0] 2024-04-20T21:47:16.980 INFO:teuthology.orchestra.run.smithi102.stdout:root 65 0.0 0.0 0 0 ? SN 21:10 0:00 \_ [ksmd] 2024-04-20T21:47:16.980 INFO:teuthology.orchestra.run.smithi102.stdout:root 66 0.0 0.0 0 0 ? SN 21:10 0:00 \_ [khugepaged] 2024-04-20T21:47:16.980 INFO:teuthology.orchestra.run.smithi102.stdout:root 67 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-crypt] 2024-04-20T21:47:16.980 INFO:teuthology.orchestra.run.smithi102.stdout:root 68 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-kinte] 2024-04-20T21:47:16.981 INFO:teuthology.orchestra.run.smithi102.stdout:root 69 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-kbloc] 2024-04-20T21:47:16.981 INFO:teuthology.orchestra.run.smithi102.stdout:root 70 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-blkcg] 2024-04-20T21:47:16.981 INFO:teuthology.orchestra.run.smithi102.stdout:root 78 0.0 0.0 0 0 ? S 21:10 0:00 \_ [irq/9-acpi] 2024-04-20T21:47:16.981 INFO:teuthology.orchestra.run.smithi102.stdout:root 79 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-ata_s] 2024-04-20T21:47:16.981 INFO:teuthology.orchestra.run.smithi102.stdout:root 80 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-md] 2024-04-20T21:47:16.981 INFO:teuthology.orchestra.run.smithi102.stdout:root 81 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-md_bi] 2024-04-20T21:47:16.981 INFO:teuthology.orchestra.run.smithi102.stdout:root 82 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-edac-] 2024-04-20T21:47:16.981 INFO:teuthology.orchestra.run.smithi102.stdout:root 83 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-devfr] 2024-04-20T21:47:16.982 INFO:teuthology.orchestra.run.smithi102.stdout:root 84 0.0 0.0 0 0 ? S 21:10 0:00 \_ [watchdogd] 2024-04-20T21:47:16.982 INFO:teuthology.orchestra.run.smithi102.stdout:root 85 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/0:1H-xfs-log/dm-4] 2024-04-20T21:47:16.982 INFO:teuthology.orchestra.run.smithi102.stdout:root 86 0.0 0.0 0 0 ? S 21:10 0:00 \_ [kswapd0] 2024-04-20T21:47:16.982 INFO:teuthology.orchestra.run.smithi102.stdout:root 88 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-kthro] 2024-04-20T21:47:16.982 INFO:teuthology.orchestra.run.smithi102.stdout:root 92 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-acpi_] 2024-04-20T21:47:16.982 INFO:teuthology.orchestra.run.smithi102.stdout:root 93 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_0] 2024-04-20T21:47:16.982 INFO:teuthology.orchestra.run.smithi102.stdout:root 94 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.982 INFO:teuthology.orchestra.run.smithi102.stdout:root 95 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_1] 2024-04-20T21:47:16.983 INFO:teuthology.orchestra.run.smithi102.stdout:root 96 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.983 INFO:teuthology.orchestra.run.smithi102.stdout:root 97 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_2] 2024-04-20T21:47:16.983 INFO:teuthology.orchestra.run.smithi102.stdout:root 98 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.983 INFO:teuthology.orchestra.run.smithi102.stdout:root 99 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_3] 2024-04-20T21:47:16.983 INFO:teuthology.orchestra.run.smithi102.stdout:root 100 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.983 INFO:teuthology.orchestra.run.smithi102.stdout:root 104 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_4] 2024-04-20T21:47:16.983 INFO:teuthology.orchestra.run.smithi102.stdout:root 105 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.983 INFO:teuthology.orchestra.run.smithi102.stdout:root 106 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_5] 2024-04-20T21:47:16.984 INFO:teuthology.orchestra.run.smithi102.stdout:root 107 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.984 INFO:teuthology.orchestra.run.smithi102.stdout:root 108 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_6] 2024-04-20T21:47:16.984 INFO:teuthology.orchestra.run.smithi102.stdout:root 109 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.984 INFO:teuthology.orchestra.run.smithi102.stdout:root 110 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_7] 2024-04-20T21:47:16.984 INFO:teuthology.orchestra.run.smithi102.stdout:root 111 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.984 INFO:teuthology.orchestra.run.smithi102.stdout:root 112 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_8] 2024-04-20T21:47:16.984 INFO:teuthology.orchestra.run.smithi102.stdout:root 113 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.984 INFO:teuthology.orchestra.run.smithi102.stdout:root 114 0.0 0.0 0 0 ? S 21:10 0:00 \_ [scsi_eh_9] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 115 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-scsi_] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 116 0.5 0.0 0 0 ? I 21:10 0:11 \_ [kworker/u32:6-xfs-cil/dm-0] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 117 0.4 0.0 0 0 ? I 21:10 0:09 \_ [kworker/u32:7-ceph-inode] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 119 0.2 0.0 0 0 ? I 21:10 0:06 \_ [kworker/u32:9-ixgbe] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 128 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/6:1H-kblockd] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 129 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-dm_bu] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 130 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-mld] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 131 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/2:1H-xfs-log/dm-4] 2024-04-20T21:47:16.985 INFO:teuthology.orchestra.run.smithi102.stdout:root 132 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-ipv6_] 2024-04-20T21:47:16.986 INFO:teuthology.orchestra.run.smithi102.stdout:root 141 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/u33:0] 2024-04-20T21:47:16.986 INFO:teuthology.orchestra.run.smithi102.stdout:root 250 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/3:1H-xfs-log/dm-4] 2024-04-20T21:47:16.986 INFO:teuthology.orchestra.run.smithi102.stdout:root 252 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/1:1H-kblockd] 2024-04-20T21:47:16.986 INFO:teuthology.orchestra.run.smithi102.stdout:root 255 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/7:1H-xfs-log/dm-0] 2024-04-20T21:47:16.986 INFO:teuthology.orchestra.run.smithi102.stdout:root 261 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/5:1H-xfs-log/dm-2] 2024-04-20T21:47:16.986 INFO:teuthology.orchestra.run.smithi102.stdout:root 365 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/4:1H-kblockd] 2024-04-20T21:47:16.986 INFO:teuthology.orchestra.run.smithi102.stdout:root 411 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-nvme-] 2024-04-20T21:47:16.986 INFO:teuthology.orchestra.run.smithi102.stdout:root 412 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-nvme-] 2024-04-20T21:47:16.987 INFO:teuthology.orchestra.run.smithi102.stdout:root 413 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-nvme-] 2024-04-20T21:47:16.987 INFO:teuthology.orchestra.run.smithi102.stdout:root 435 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-ixgbe] 2024-04-20T21:47:16.987 INFO:teuthology.orchestra.run.smithi102.stdout:root 448 0.0 0.0 0 0 ? S 21:10 0:00 \_ [jbd2/sda1-8] 2024-04-20T21:47:16.987 INFO:teuthology.orchestra.run.smithi102.stdout:root 449 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-ext4-] 2024-04-20T21:47:16.987 INFO:teuthology.orchestra.run.smithi102.stdout:root 532 0.0 0.0 0 0 ? S 21:10 0:00 \_ [ext4lazyinit] 2024-04-20T21:47:16.987 INFO:teuthology.orchestra.run.smithi102.stdout:root 537 0.2 0.0 0 0 ? I 21:10 0:04 \_ [kworker/6:8-dio/dm-1] 2024-04-20T21:47:16.987 INFO:teuthology.orchestra.run.smithi102.stdout:root 598 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-ipmi-] 2024-04-20T21:47:16.987 INFO:teuthology.orchestra.run.smithi102.stdout:root 615 0.0 0.0 0 0 ? SN 21:10 0:00 \_ [kipmi0] 2024-04-20T21:47:16.988 INFO:teuthology.orchestra.run.smithi102.stdout:root 628 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-rpcio] 2024-04-20T21:47:16.988 INFO:teuthology.orchestra.run.smithi102.stdout:root 629 0.0 0.0 0 0 ? I< 21:10 0:00 \_ [kworker/R-xprti] 2024-04-20T21:47:16.988 INFO:teuthology.orchestra.run.smithi102.stdout:root 8172 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-kdmfl] 2024-04-20T21:47:16.988 INFO:teuthology.orchestra.run.smithi102.stdout:root 8202 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-kdmfl] 2024-04-20T21:47:16.988 INFO:teuthology.orchestra.run.smithi102.stdout:root 8231 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-kdmfl] 2024-04-20T21:47:16.988 INFO:teuthology.orchestra.run.smithi102.stdout:root 8260 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-kdmfl] 2024-04-20T21:47:16.988 INFO:teuthology.orchestra.run.smithi102.stdout:root 8289 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-kdmfl] 2024-04-20T21:47:16.988 INFO:teuthology.orchestra.run.smithi102.stdout:root 8453 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfsal] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8454 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfs_m] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8455 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8456 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8457 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfs-r] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8458 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8459 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfs-i] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8460 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfs-l] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8461 0.0 0.0 0 0 ? I< 21:12 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:root 8462 0.0 0.0 0 0 ? S 21:12 0:00 \_ [xfsaild/dm-4] 2024-04-20T21:47:16.990 INFO:teuthology.orchestra.run.smithi102.stdout:root 16409 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.990 INFO:teuthology.orchestra.run.smithi102.stdout:root 16410 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.990 INFO:teuthology.orchestra.run.smithi102.stdout:root 16411 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-r] 2024-04-20T21:47:16.990 INFO:teuthology.orchestra.run.smithi102.stdout:root 16412 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.990 INFO:teuthology.orchestra.run.smithi102.stdout:root 16413 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-i] 2024-04-20T21:47:16.990 INFO:teuthology.orchestra.run.smithi102.stdout:root 16414 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-l] 2024-04-20T21:47:16.990 INFO:teuthology.orchestra.run.smithi102.stdout:root 16415 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.990 INFO:teuthology.orchestra.run.smithi102.stdout:root 16416 0.0 0.0 0 0 ? S 21:15 0:00 \_ [xfsaild/dm-0] 2024-04-20T21:47:16.991 INFO:teuthology.orchestra.run.smithi102.stdout:root 16528 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.991 INFO:teuthology.orchestra.run.smithi102.stdout:root 16529 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.991 INFO:teuthology.orchestra.run.smithi102.stdout:root 16530 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-r] 2024-04-20T21:47:16.991 INFO:teuthology.orchestra.run.smithi102.stdout:root 16531 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.991 INFO:teuthology.orchestra.run.smithi102.stdout:root 16532 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-i] 2024-04-20T21:47:16.991 INFO:teuthology.orchestra.run.smithi102.stdout:root 16533 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-l] 2024-04-20T21:47:16.991 INFO:teuthology.orchestra.run.smithi102.stdout:root 16534 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.991 INFO:teuthology.orchestra.run.smithi102.stdout:root 16535 0.0 0.0 0 0 ? S 21:15 0:00 \_ [xfsaild/dm-1] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16643 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16644 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16645 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-r] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16646 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16647 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-i] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16648 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-l] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16649 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16650 0.0 0.0 0 0 ? S 21:15 0:00 \_ [xfsaild/dm-2] 2024-04-20T21:47:16.992 INFO:teuthology.orchestra.run.smithi102.stdout:root 16727 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.993 INFO:teuthology.orchestra.run.smithi102.stdout:root 16728 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.993 INFO:teuthology.orchestra.run.smithi102.stdout:root 16729 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-r] 2024-04-20T21:47:16.993 INFO:teuthology.orchestra.run.smithi102.stdout:root 16730 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-b] 2024-04-20T21:47:16.993 INFO:teuthology.orchestra.run.smithi102.stdout:root 16731 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-i] 2024-04-20T21:47:16.993 INFO:teuthology.orchestra.run.smithi102.stdout:root 16732 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-l] 2024-04-20T21:47:16.993 INFO:teuthology.orchestra.run.smithi102.stdout:root 16733 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-xfs-c] 2024-04-20T21:47:16.993 INFO:teuthology.orchestra.run.smithi102.stdout:root 16734 0.0 0.0 0 0 ? S 21:15 0:00 \_ [xfsaild/dm-3] 2024-04-20T21:47:16.993 INFO:teuthology.orchestra.run.smithi102.stdout:root 16780 0.0 0.0 0 0 ? I< 21:15 0:00 \_ [kworker/R-dio/d] 2024-04-20T21:47:16.994 INFO:teuthology.orchestra.run.smithi102.stdout:root 17942 0.0 0.0 0 0 ? I< 21:16 0:00 \_ [kworker/R-dio/d] 2024-04-20T21:47:16.994 INFO:teuthology.orchestra.run.smithi102.stdout:root 19104 0.0 0.0 0 0 ? I< 21:16 0:00 \_ [kworker/R-dio/d] 2024-04-20T21:47:16.994 INFO:teuthology.orchestra.run.smithi102.stdout:root 20268 0.0 0.0 0 0 ? I< 21:16 0:00 \_ [kworker/R-dio/d] 2024-04-20T21:47:16.994 INFO:teuthology.orchestra.run.smithi102.stdout:root 26273 0.0 0.0 0 0 ? I 21:16 0:01 \_ [kworker/1:90-dio/dm-1] 2024-04-20T21:47:16.994 INFO:teuthology.orchestra.run.smithi102.stdout:root 26438 0.0 0.0 0 0 ? I 21:16 0:00 \_ [kworker/6:23-dio/dm-1] 2024-04-20T21:47:16.994 INFO:teuthology.orchestra.run.smithi102.stdout:root 26463 0.1 0.0 0 0 ? I 21:16 0:03 \_ [kworker/6:36-mm_percpu_wq] 2024-04-20T21:47:16.994 INFO:teuthology.orchestra.run.smithi102.stdout:root 27218 0.2 0.0 0 0 ? I 21:16 0:03 \_ [kworker/2:168-dio/dm-0] 2024-04-20T21:47:16.995 INFO:teuthology.orchestra.run.smithi102.stdout:root 32586 0.0 0.0 0 0 ? I< 21:17 0:00 \_ [kworker/R-ceph-] 2024-04-20T21:47:16.995 INFO:teuthology.orchestra.run.smithi102.stdout:root 32601 0.0 0.0 0 0 ? I< 21:17 0:00 \_ [kworker/R-ceph-] 2024-04-20T21:47:16.995 INFO:teuthology.orchestra.run.smithi102.stdout:root 32602 0.0 0.0 0 0 ? I< 21:17 0:00 \_ [kworker/R-ceph-] 2024-04-20T21:47:16.995 INFO:teuthology.orchestra.run.smithi102.stdout:root 36709 0.1 0.0 0 0 ? I 21:21 0:01 \_ [kworker/5:0-dio/dm-1] 2024-04-20T21:47:16.995 INFO:teuthology.orchestra.run.smithi102.stdout:root 37096 0.2 0.0 0 0 ? I 21:22 0:03 \_ [kworker/7:0-dio/dm-3] 2024-04-20T21:47:16.995 INFO:teuthology.orchestra.run.smithi102.stdout:root 39662 0.4 0.0 0 0 ? I 21:25 0:05 \_ [kworker/u32:1-ceph-inode] 2024-04-20T21:47:16.995 INFO:teuthology.orchestra.run.smithi102.stdout:root 40012 0.1 0.0 0 0 ? I 21:25 0:01 \_ [kworker/4:0-dio/dm-0] 2024-04-20T21:47:16.996 INFO:teuthology.orchestra.run.smithi102.stdout:root 40032 0.2 0.0 0 0 ? I 21:25 0:03 \_ [kworker/1:1-dio/dm-3] 2024-04-20T21:47:16.996 INFO:teuthology.orchestra.run.smithi102.stdout:root 40930 0.1 0.0 0 0 ? I 21:27 0:01 \_ [kworker/5:2-dio/dm-2] 2024-04-20T21:47:16.996 INFO:teuthology.orchestra.run.smithi102.stdout:root 41038 0.0 0.0 0 0 ? I 21:28 0:00 \_ [kworker/7:2-dio/dm-3] 2024-04-20T21:47:16.996 INFO:teuthology.orchestra.run.smithi102.stdout:root 41148 0.2 0.0 0 0 ? I 21:28 0:02 \_ [kworker/0:24-dio/dm-0] 2024-04-20T21:47:16.996 INFO:teuthology.orchestra.run.smithi102.stdout:root 41173 0.0 0.0 0 0 ? I 21:28 0:00 \_ [kworker/2:6-dio/dm-1] 2024-04-20T21:47:16.996 INFO:teuthology.orchestra.run.smithi102.stdout:root 42049 0.0 0.0 0 0 ? I 21:29 0:00 \_ [kworker/3:4-events] 2024-04-20T21:47:16.997 INFO:teuthology.orchestra.run.smithi102.stdout:root 42071 0.1 0.0 0 0 ? I 21:29 0:01 \_ [kworker/4:3-dio/dm-0] 2024-04-20T21:47:16.997 INFO:teuthology.orchestra.run.smithi102.stdout:root 42130 0.3 0.0 0 0 ? I 21:31 0:03 \_ [kworker/u32:0-ceph-inode] 2024-04-20T21:47:16.997 INFO:teuthology.orchestra.run.smithi102.stdout:root 44557 0.0 0.0 0 0 ? I 21:34 0:00 \_ [kworker/1:0-events] 2024-04-20T21:47:16.997 INFO:teuthology.orchestra.run.smithi102.stdout:root 45125 0.1 0.0 0 0 ? I 21:35 0:00 \_ [kworker/2:0-dio/dm-0] 2024-04-20T21:47:16.997 INFO:teuthology.orchestra.run.smithi102.stdout:root 45126 0.0 0.0 0 0 ? I 21:35 0:00 \_ [kworker/2:1-dio/dm-1] 2024-04-20T21:47:16.997 INFO:teuthology.orchestra.run.smithi102.stdout:root 45272 0.2 0.0 0 0 ? I 21:35 0:01 \_ [kworker/4:1-mm_percpu_wq] 2024-04-20T21:47:16.998 INFO:teuthology.orchestra.run.smithi102.stdout:root 46058 0.0 0.0 0 0 ? I 21:36 0:00 \_ [kworker/5:1-dio/dm-2] 2024-04-20T21:47:16.998 INFO:teuthology.orchestra.run.smithi102.stdout:root 46246 0.5 0.0 0 0 ? I 21:36 0:03 \_ [kworker/u32:2-flush-253:4] 2024-04-20T21:47:16.998 INFO:teuthology.orchestra.run.smithi102.stdout:root 46247 0.0 0.0 0 0 ? I 21:36 0:00 \_ [kworker/0:0-dio/dm-0] 2024-04-20T21:47:16.998 INFO:teuthology.orchestra.run.smithi102.stdout:root 46266 0.1 0.0 0 0 ? I 21:37 0:00 \_ [kworker/5:3-dio/dm-1] 2024-04-20T21:47:16.998 INFO:teuthology.orchestra.run.smithi102.stdout:root 46267 0.2 0.0 0 0 ? I 21:37 0:01 \_ [kworker/0:1-mm_percpu_wq] 2024-04-20T21:47:16.998 INFO:teuthology.orchestra.run.smithi102.stdout:root 46268 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/1:2-dio/dm-0] 2024-04-20T21:47:16.998 INFO:teuthology.orchestra.run.smithi102.stdout:root 46269 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/5:4-ceph-msgr] 2024-04-20T21:47:16.999 INFO:teuthology.orchestra.run.smithi102.stdout:root 46273 0.3 0.0 0 0 ? I 21:37 0:02 \_ [kworker/u32:3-ceph-inode] 2024-04-20T21:47:16.999 INFO:teuthology.orchestra.run.smithi102.stdout:root 46274 0.5 0.0 0 0 ? I 21:37 0:03 \_ [kworker/u32:4-ceph-inode] 2024-04-20T21:47:16.999 INFO:teuthology.orchestra.run.smithi102.stdout:root 46277 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/1:3-dio/dm-0] 2024-04-20T21:47:16.999 INFO:teuthology.orchestra.run.smithi102.stdout:root 46278 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/1:4-ceph-msgr] 2024-04-20T21:47:16.999 INFO:teuthology.orchestra.run.smithi102.stdout:root 46279 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/7:1-events] 2024-04-20T21:47:16.999 INFO:teuthology.orchestra.run.smithi102.stdout:root 46282 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/6:0-dio/dm-1] 2024-04-20T21:47:16.999 INFO:teuthology.orchestra.run.smithi102.stdout:root 46304 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/4:2-dio/dm-0] 2024-04-20T21:47:17.000 INFO:teuthology.orchestra.run.smithi102.stdout:root 46308 0.1 0.0 0 0 ? I 21:37 0:00 \_ [kworker/3:3-dio/dm-0] 2024-04-20T21:47:17.000 INFO:teuthology.orchestra.run.smithi102.stdout:root 46310 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/3:6-dio/dm-1] 2024-04-20T21:47:17.000 INFO:teuthology.orchestra.run.smithi102.stdout:root 46333 0.0 0.0 0 0 ? I 21:37 0:00 \_ [kworker/3:7-dio/dm-0] 2024-04-20T21:47:17.000 INFO:teuthology.orchestra.run.smithi102.stdout:root 46334 0.0 0.0 0 0 ? I 21:38 0:00 \_ [kworker/0:2-dio/dm-1] 2024-04-20T21:47:17.000 INFO:teuthology.orchestra.run.smithi102.stdout:root 46336 0.0 0.0 0 0 ? I 21:38 0:00 \_ [kworker/7:5-events_freezable] 2024-04-20T21:47:17.000 INFO:teuthology.orchestra.run.smithi102.stdout:root 47487 0.0 0.0 0 0 ? I 21:39 0:00 \_ [kworker/4:4-dio/dm-3] 2024-04-20T21:47:17.001 INFO:teuthology.orchestra.run.smithi102.stdout:root 47509 0.0 0.0 0 0 ? I 21:39 0:00 \_ [kworker/2:2-dio/dm-1] 2024-04-20T21:47:17.001 INFO:teuthology.orchestra.run.smithi102.stdout:root 48575 0.0 0.0 0 0 ? I 21:40 0:00 \_ [kworker/4:5-dio/dm-0] 2024-04-20T21:47:17.001 INFO:teuthology.orchestra.run.smithi102.stdout:root 48576 0.0 0.0 0 0 ? I 21:40 0:00 \_ [kworker/1:5-dio/dm-0] 2024-04-20T21:47:17.001 INFO:teuthology.orchestra.run.smithi102.stdout:root 48578 0.7 0.0 0 0 ? I 21:40 0:02 \_ [kworker/u32:8-flush-253:0] 2024-04-20T21:47:17.001 INFO:teuthology.orchestra.run.smithi102.stdout:root 48579 0.1 0.0 0 0 ? I 21:40 0:00 \_ [kworker/u32:10-xfs-cil/dm-4] 2024-04-20T21:47:17.001 INFO:teuthology.orchestra.run.smithi102.stdout:root 48580 0.0 0.0 0 0 ? I 21:40 0:00 \_ [kworker/6:1-dio/dm-1] 2024-04-20T21:47:17.001 INFO:teuthology.orchestra.run.smithi102.stdout:root 48586 0.0 0.0 0 0 ? I 21:40 0:00 \_ [kworker/6:2-dio/dm-3] 2024-04-20T21:47:17.002 INFO:teuthology.orchestra.run.smithi102.stdout:root 48587 0.0 0.0 0 0 ? I 21:40 0:00 \_ [kworker/6:3-dio/dm-3] 2024-04-20T21:47:17.002 INFO:teuthology.orchestra.run.smithi102.stdout:root 49601 0.1 0.0 0 0 ? I 21:41 0:00 \_ [kworker/5:5-mm_percpu_wq] 2024-04-20T21:47:17.002 INFO:teuthology.orchestra.run.smithi102.stdout:root 50603 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/6:4-dio/dm-3] 2024-04-20T21:47:17.002 INFO:teuthology.orchestra.run.smithi102.stdout:root 50604 0.1 0.0 0 0 ? I 21:44 0:00 \_ [kworker/0:3-dio/dm-0] 2024-04-20T21:47:17.002 INFO:teuthology.orchestra.run.smithi102.stdout:root 50605 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:3-dio/dm-1] 2024-04-20T21:47:17.002 INFO:teuthology.orchestra.run.smithi102.stdout:root 50606 0.1 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:4-dio/dm-1] 2024-04-20T21:47:17.003 INFO:teuthology.orchestra.run.smithi102.stdout:root 50607 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:5-dio/dm-1] 2024-04-20T21:47:17.003 INFO:teuthology.orchestra.run.smithi102.stdout:root 50608 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:7-dio/dm-1] 2024-04-20T21:47:17.003 INFO:teuthology.orchestra.run.smithi102.stdout:root 50609 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/1:6] 2024-04-20T21:47:17.003 INFO:teuthology.orchestra.run.smithi102.stdout:root 50638 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/4:6] 2024-04-20T21:47:17.003 INFO:teuthology.orchestra.run.smithi102.stdout:root 50639 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:8-dio/dm-1] 2024-04-20T21:47:17.003 INFO:teuthology.orchestra.run.smithi102.stdout:root 50640 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:9-dio/dm-1] 2024-04-20T21:47:17.003 INFO:teuthology.orchestra.run.smithi102.stdout:root 50641 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:10-dio/dm-1] 2024-04-20T21:47:17.004 INFO:teuthology.orchestra.run.smithi102.stdout:root 50642 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:11-dio/dm-1] 2024-04-20T21:47:17.004 INFO:teuthology.orchestra.run.smithi102.stdout:root 50643 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:12-events] 2024-04-20T21:47:17.004 INFO:teuthology.orchestra.run.smithi102.stdout:root 50644 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:13] 2024-04-20T21:47:17.004 INFO:teuthology.orchestra.run.smithi102.stdout:root 50645 0.0 0.0 0 0 ? I 21:44 0:00 \_ [kworker/2:14] 2024-04-20T21:47:17.004 INFO:teuthology.orchestra.run.smithi102.stdout:root 50646 0.2 0.0 0 0 ? I 21:44 0:00 \_ [kworker/6:5-dio/dm-2] 2024-04-20T21:47:17.004 INFO:teuthology.orchestra.run.smithi102.stdout:root 1 0.2 0.0 176384 20044 ? Ss 21:10 0:06 /usr/lib/systemd/systemd --switched-root --system --deserialize 31 2024-04-20T21:47:17.005 INFO:teuthology.orchestra.run.smithi102.stdout:root 526 0.1 0.0 27352 15104 ? Ss 21:10 0:02 /usr/lib/systemd/systemd-journald 2024-04-20T21:47:17.005 INFO:teuthology.orchestra.run.smithi102.stdout:root 546 0.0 0.0 37040 14932 ? Ss 21:10 0:00 /usr/lib/systemd/systemd-udevd 2024-04-20T21:47:17.005 INFO:teuthology.orchestra.run.smithi102.stdout:rpc 614 0.0 0.0 13248 5632 ? Ss 21:10 0:00 /usr/bin/rpcbind -w -f 2024-04-20T21:47:17.005 INFO:teuthology.orchestra.run.smithi102.stdout:root 620 0.0 0.0 91916 2172 ? S sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2024-04-20T21:47:17.014 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:47:17.093 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:47:17.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:47:17.094 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:47:17.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.205+0000 7f8bd2600640 1 Processor -- start 2024-04-20T21:47:17.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.206+0000 7f8bd2600640 1 -- start start 2024-04-20T21:47:17.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bd2600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc14fa50 0x7f8bcc0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:17.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bd2600640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8bcc073a10 0x7f8bcc158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:17.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bd2600640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bcc1536d0 con 0x7f8bcc0c9eb0 2024-04-20T21:47:17.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bd2600640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc159500 con 0x7f8bcc073a10 2024-04-20T21:47:17.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bd2600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0c76c0 con 0x7f8bcc14fa50 2024-04-20T21:47:17.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bcbe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8bcc073a10 0x7f8bcc158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:17.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bcaa00640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f8bcc0c9eb0 0x7f8bcc0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.102:48412/0 (socket says 172.21.15.102:48412) 2024-04-20T21:47:17.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bcaa00640 1 -- 172.21.15.102:0/596874044 learned_addr learned my addr 172.21.15.102:0/596874044 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-04-20T21:47:17.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bcb400640 1 --2- 172.21.15.102:0/596874044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc14fa50 0x7f8bcc0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:17.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bca000640 1 -- 172.21.15.102:0/596874044 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272659536 0 0) 0x7f8bb8001110 con 0x7f8bcc0c9eb0 2024-04-20T21:47:17.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.207+0000 7f8bca000640 1 -- 172.21.15.102:0/596874044 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bb0003520 con 0x7f8bcc0c9eb0 2024-04-20T21:47:17.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.208+0000 7f8bcbe00640 1 -- 172.21.15.102:0/596874044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc14fa50 msgr2=0x7f8bcc0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:47:17.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.208+0000 7f8bcbe00640 1 --2- 172.21.15.102:0/596874044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc14fa50 0x7f8bcc0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.208+0000 7f8bcbe00640 1 -- 172.21.15.102:0/596874044 >> v1:172.21.15.193:6789/0 conn(0x7f8bcc0c9eb0 legacy=0x7f8bcc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:17.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.208+0000 7f8bcbe00640 1 -- 172.21.15.102:0/596874044 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc0c7960 con 0x7f8bcc073a10 2024-04-20T21:47:17.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.209+0000 7f8bcbe00640 1 --2- 172.21.15.102:0/596874044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc073a10 0x7f8bcc158fc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009d90 tx=0x7f8bc402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a5ca5658777ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:17.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.209+0000 7f8bca000640 1 -- 172.21.15.102:0/596874044 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4039070 con 0x7f8bcc073a10 2024-04-20T21:47:17.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.209+0000 7f8bca000640 1 -- 172.21.15.102:0/596874044 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc402cce0 con 0x7f8bcc073a10 2024-04-20T21:47:17.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.209+0000 7f8bca000640 1 -- 172.21.15.102:0/596874044 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4034640 con 0x7f8bcc073a10 2024-04-20T21:47:17.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.209+0000 7f8bd2600640 1 -- 172.21.15.102:0/596874044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc073a10 msgr2=0x7f8bcc158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:17.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.209+0000 7f8bd2600640 1 --2- 172.21.15.102:0/596874044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc073a10 0x7f8bcc158fc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009d90 tx=0x7f8bc402bf30 comp rx=0 tx=0).stop 2024-04-20T21:47:17.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.210+0000 7f8bd2600640 1 -- 172.21.15.102:0/596874044 shutdown_connections 2024-04-20T21:47:17.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.210+0000 7f8bd2600640 1 --2- 172.21.15.102:0/596874044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc073a10 0x7f8bcc158fc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.210+0000 7f8bd2600640 1 --2- 172.21.15.102:0/596874044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc14fa50 0x7f8bcc0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.210+0000 7f8bd2600640 1 -- 172.21.15.102:0/596874044 >> 172.21.15.102:0/596874044 conn(0x7f8bcc0c9390 msgr2=0x7f8bcc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:17.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.210+0000 7f8bd2600640 1 -- 172.21.15.102:0/596874044 shutdown_connections 2024-04-20T21:47:17.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.210+0000 7f8bd2600640 1 -- 172.21.15.102:0/596874044 wait complete. 2024-04-20T21:47:17.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.211+0000 7f8bd2600640 1 Processor -- start 2024-04-20T21:47:17.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.211+0000 7f8bd2600640 1 -- start start 2024-04-20T21:47:17.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.212+0000 7f8bd2600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bcc073a10 0x7f8bcc1e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:17.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.212+0000 7f8bd2600640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc0c9eb0 0x7f8bcc1e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:17.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.212+0000 7f8bd2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc14fa50 0x7f8bcc155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:17.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.212+0000 7f8bd2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0c7c00 con 0x7f8bcc14fa50 2024-04-20T21:47:17.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.212+0000 7f8bd2600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0c3d90 con 0x7f8bcc073a10 2024-04-20T21:47:17.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.212+0000 7f8bd2600640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0c4030 con 0x7f8bcc0c9eb0 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bcc073a10 0x7f8bcc1e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bcc073a10 0x7f8bcc1e23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.102:57358/0 (socket says 172.21.15.102:57358) 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 -- 172.21.15.102:0/3753796978 learned_addr learned my addr 172.21.15.102:0/3753796978 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcaa00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc0c9eb0 0x7f8bcc1e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcbe00640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc14fa50 0x7f8bcc155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 -- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc0c9eb0 msgr2=0x7f8bcc1e4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc0c9eb0 0x7f8bcc1e4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 -- 172.21.15.102:0/3753796978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc14fa50 msgr2=0x7f8bcc155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:17.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc14fa50 0x7f8bcc155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 -- 172.21.15.102:0/3753796978 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc0cbc00 con 0x7f8bcc073a10 2024-04-20T21:47:17.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.213+0000 7f8bcb400640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bcc073a10 0x7f8bcc1e23d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00e8d0 tx=0x7f8bbc00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:17.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.214+0000 7f8bc3400640 1 -- 172.21.15.102:0/3753796978 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc017070 con 0x7f8bcc073a10 2024-04-20T21:47:17.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.214+0000 7f8bc3400640 1 -- 172.21.15.102:0/3753796978 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bbc00c940 con 0x7f8bcc073a10 2024-04-20T21:47:17.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.214+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc158360 con 0x7f8bcc073a10 2024-04-20T21:47:17.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.214+0000 7f8bc3400640 1 -- 172.21.15.102:0/3753796978 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00cd30 con 0x7f8bcc073a10 2024-04-20T21:47:17.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.214+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc0cc180 con 0x7f8bcc073a10 2024-04-20T21:47:17.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.215+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc074340 con 0x7f8bcc073a10 2024-04-20T21:47:17.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.216+0000 7f8bc3400640 1 -- 172.21.15.102:0/3753796978 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bbc01a070 con 0x7f8bcc073a10 2024-04-20T21:47:17.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.217+0000 7f8bc3400640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bac06c8f0 0x7f8bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:17.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.217+0000 7f8bc3400640 1 -- 172.21.15.102:0/3753796978 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 8369+0+0 (secure 0 0 0) 0x7f8bbc08c870 con 0x7f8bcc073a10 2024-04-20T21:47:17.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.217+0000 7f8bcaa00640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bac06c8f0 0x7f8bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:47:17.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.218+0000 7f8bcaa00640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bac06c8f0 0x7f8bac06edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc14edd0 tx=0x7f8bb8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:47:17.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.222+0000 7f8bc3400640 1 -- 172.21.15.102:0/3753796978 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8bbc059b60 con 0x7f8bcc073a10 2024-04-20T21:47:17.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.461+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bcc0c7320 con 0x7f8bcc073a10 2024-04-20T21:47:17.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.462+0000 7f8bc3400640 1 -- 172.21.15.102:0/3753796978 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+3553 (secure 0 0 0) 0x7f8bbc05de60 con 0x7f8bcc073a10 2024-04-20T21:47:17.460 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-20T21:47:17.460 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":109,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4172033112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4172033112},{"type":"v1","addr":"172.21.15.102:6837","nonce":4172033112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:16:54.017441+0000","modified":"2024-04-20T21:47:07.748327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":108,"state":"up:reconnect","state_seq":456,"addr":"172.21.15.193:6837/72219413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":72219413},{"type":"v1","addr":"172.21.15.193:6837","nonce":72219413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T21:47:17.461 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 109 2024-04-20T21:47:17.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.464+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bac06c8f0 msgr2=0x7f8bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:17.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.464+0000 7f8bd2600640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bac06c8f0 0x7f8bac06edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc14edd0 tx=0x7f8bb8009040 comp rx=0 tx=0).stop 2024-04-20T21:47:17.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.464+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bcc073a10 msgr2=0x7f8bcc1e23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:47:17.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bcc073a10 0x7f8bcc1e23d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00e8d0 tx=0x7f8bbc00ed90 comp rx=0 tx=0).stop 2024-04-20T21:47:17.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 shutdown_connections 2024-04-20T21:47:17.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bcc14fa50 0x7f8bcc155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bcc0c9eb0 0x7f8bcc1e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:6832/2803621564,v1:172.21.15.193:6833/2803621564] conn(0x7f8bac06c8f0 0x7f8bac06edb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 --2- 172.21.15.102:0/3753796978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bcc073a10 0x7f8bcc1e23d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:17.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 >> 172.21.15.102:0/3753796978 conn(0x7f8bcc0c9390 msgr2=0x7f8bcc06d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:47:17.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 shutdown_connections 2024-04-20T21:47:17.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:17.465+0000 7f8bd2600640 1 -- 172.21.15.102:0/3753796978 wait complete. 2024-04-20T21:47:17.482 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T21:47:18.691 INFO:tasks.ceph.osd.0:Sent signal 15 2024-04-20T21:47:18.691 INFO:tasks.ceph.osd.1:Sent signal 15 2024-04-20T21:47:18.692 INFO:tasks.ceph.osd.2:Sent signal 15 2024-04-20T21:47:18.692 INFO:tasks.ceph.osd.3:Sent signal 15 2024-04-20T21:47:18.692 INFO:tasks.ceph.osd.4:Sent signal 15 2024-04-20T21:47:18.692 INFO:tasks.ceph.osd.5:Sent signal 15 2024-04-20T21:47:18.693 INFO:tasks.ceph.osd.6:Sent signal 15 2024-04-20T21:47:18.693 INFO:tasks.ceph.osd.7:Sent signal 15 2024-04-20T21:47:18.693 INFO:tasks.ceph.mds.e:Sent signal 15 2024-04-20T21:47:18.694 INFO:tasks.ceph.mds.d:Sent signal 15 2024-04-20T21:47:18.694 INFO:tasks.ceph.mon.a:Sent signal 15 2024-04-20T21:47:18.694 INFO:tasks.ceph.mon.b:Sent signal 15 2024-04-20T21:47:18.694 INFO:tasks.ceph.mon.c:Sent signal 15 2024-04-20T21:47:18.695 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-04-20T21:47:18.695 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-04-20T21:47:18.696 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-20T21:47:18.694+0000 7f088d800640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22356) UID: 0 2024-04-20T21:47:18.696 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-20T21:47:18.694+0000 7f088d800640 -1 osd.0 66 *** Got signal Terminated *** 2024-04-20T21:47:18.696 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-04-20T21:47:18.695+0000 7f088d800640 -1 osd.0 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:47:18.697 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-20T21:47:18.695+0000 7f9ab4c00640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22358) UID: 0 2024-04-20T21:47:18.697 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-20T21:47:18.695+0000 7f9ab4c00640 -1 osd.1 66 *** Got signal Terminated *** 2024-04-20T21:47:18.697 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-04-20T21:47:18.695+0000 7f9ab4c00640 -1 osd.1 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:47:18.697 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-20T21:47:18.695+0000 7ff6e3e00640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22362) UID: 0 2024-04-20T21:47:18.698 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-20T21:47:18.695+0000 7ff6e3e00640 -1 osd.2 66 *** Got signal Terminated *** 2024-04-20T21:47:18.698 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-04-20T21:47:18.695+0000 7ff6e3e00640 -1 osd.2 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:47:18.698 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-20T21:47:18.695+0000 7f5304000640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22363) UID: 0 2024-04-20T21:47:18.698 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-20T21:47:18.695+0000 7f5304000640 -1 osd.3 66 *** Got signal Terminated *** 2024-04-20T21:47:18.698 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-04-20T21:47:18.695+0000 7f5304000640 -1 osd.3 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:47:18.699 INFO:tasks.ceph.mds.e.smithi102.stderr:2024-04-20T21:47:18.697+0000 7f382e000640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i e (PID: 28083) UID: 0 2024-04-20T21:47:18.699 INFO:tasks.ceph.mds.e.smithi102.stderr:2024-04-20T21:47:18.697+0000 7f382e000640 -1 mds.e *** got signal Terminated *** 2024-04-20T21:47:18.699 INFO:tasks.ceph.mon.a.smithi102.stderr:2024-04-20T21:47:18.698+0000 7fac15400640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 22087) UID: 0 2024-04-20T21:47:18.700 INFO:tasks.ceph.mon.a.smithi102.stderr:2024-04-20T21:47:18.698+0000 7fac15400640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-20T21:47:18.700 INFO:tasks.ceph.mon.a.smithi102.stderr:2024-04-20T21:47:18.700+0000 7fac0e600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:47:18.700 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f4aec7d8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 23859) UID: 0 2024-04-20T21:47:18.701 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f4aec7d8640 -1 osd.4 66 *** Got signal Terminated *** 2024-04-20T21:47:18.701 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f4aec7d8640 -1 osd.4 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:47:18.701 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f5fa9b4e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 23860) UID: 0 2024-04-20T21:47:18.701 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f5fa9b4e640 -1 osd.5 66 *** Got signal Terminated *** 2024-04-20T21:47:18.701 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f5fa9b4e640 -1 osd.5 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:47:18.702 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f5a2bdbf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 23865) UID: 0 2024-04-20T21:47:18.702 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f5a2bdbf640 -1 osd.6 66 *** Got signal Terminated *** 2024-04-20T21:47:18.702 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f5a2bdbf640 -1 osd.6 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:47:18.703 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f668f4ed640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 23867) UID: 0 2024-04-20T21:47:18.703 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f668f4ed640 -1 osd.7 66 *** Got signal Terminated *** 2024-04-20T21:47:18.703 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-20T21:47:18.695+0000 7f668f4ed640 -1 osd.7 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:47:18.704 INFO:tasks.ceph.mds.d.smithi193.stderr:2024-04-20T21:47:18.696+0000 7f0909732640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 29681) UID: 0 2024-04-20T21:47:18.704 INFO:tasks.ceph.mds.d.smithi193.stderr:2024-04-20T21:47:18.696+0000 7f0909732640 -1 mds.d *** got signal Terminated *** 2024-04-20T21:47:18.704 INFO:tasks.ceph.mon.b.smithi193.stderr:2024-04-20T21:47:18.696+0000 7fe1fd873640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 23280) UID: 0 2024-04-20T21:47:18.704 INFO:tasks.ceph.mon.b.smithi193.stderr:2024-04-20T21:47:18.696+0000 7fe1fd873640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-04-20T21:47:18.705 INFO:tasks.ceph.mon.c.smithi193.stderr:2024-04-20T21:47:18.697+0000 7fd742f80640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 23282) UID: 0 2024-04-20T21:47:18.705 INFO:tasks.ceph.mon.c.smithi193.stderr:2024-04-20T21:47:18.697+0000 7fd742f80640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-04-20T21:47:18.705 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:47:18.699+0000 7f96a2b5c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 23285) UID: 0 2024-04-20T21:47:18.705 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-20T21:47:18.699+0000 7f96a2b5c640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-20T21:47:18.899 INFO:tasks.ceph.mgr.y.smithi102.stderr:daemon-helper: command crashed with signal 15 2024-04-20T21:47:19.483 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:47:19.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.687+0000 7f9367a00640 1 Processor -- start 2024-04-20T21:47:19.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.687+0000 7f9367a00640 1 -- start start 2024-04-20T21:47:19.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9367a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 0x7f936015be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:19.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9367a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 0x7f9360073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:19.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9367a00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f93600c5cc0 2024-04-20T21:47:19.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9367a00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93600c60c0 2024-04-20T21:47:19.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9367a00640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9360155560 2024-04-20T21:47:19.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93600c5cc0 legacy=0x7f93600ccd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:19.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 msgr2=0x7f936015be30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:19.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 msgr2=0x7f9360073230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:19.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 0x7f936015be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:19.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.689+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 0x7f9360073230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:19.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.890+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 msgr2=0x7f9360073230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:19.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.890+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93600c5cc0 legacy=0x7f93600ccd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:19.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.890+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 0x7f9360073230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:19.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.890+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 msgr2=0x7f936015be30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:19.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:19.890+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 0x7f936015be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:20.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:20.291+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 msgr2=0x7f9360073230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:20.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:20.291+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93600c5cc0 legacy=0x7f93600ccd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:20.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:20.291+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 msgr2=0x7f936015be30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:20.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:20.291+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 0x7f9360073230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:20.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:20.291+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 0x7f936015be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:21.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:21.092+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 msgr2=0x7f9360073230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:21.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:21.092+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93600c5cc0 legacy=0x7f93600ccd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:21.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:21.092+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 msgr2=0x7f936015be30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:21.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:21.092+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 0x7f9360073230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:21.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:21.092+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 0x7f936015be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:22.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.689+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 msgr2=0x7f936015be30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:22.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.689+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9360155560 0x7f936015be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:22.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.689+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 msgr2=0x7f9360073230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:22.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.689+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93600c60c0 0x7f9360073230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:22.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.689+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93600c5cc0 legacy=0x7f93600ccd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:22.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.689+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:22.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.689+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f93400036c0 2024-04-20T21:47:22.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.689+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340007400 2024-04-20T21:47:22.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.690+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f93400036c0 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:22.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.690+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:22.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.690+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.891+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f93400036c0 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.891+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:22.891+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:23.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:23.292+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f93400036c0 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:23.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:23.292+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:23.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:23.292+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:24.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:24.093+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:24.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:24.093+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f93400036c0 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:24.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:24.093+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:25.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f93400036c0 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:47:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340009f80 2024-04-20T21:47:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f934000a360 2024-04-20T21:47:25.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340007400 2024-04-20T21:47:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:25.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.690+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:25.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.891+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:25.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.891+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.891+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.891+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:25.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:25.891+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:26.292+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:26.292+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:26.292+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:26.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:26.292+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:26.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:26.292+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:27.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:27.093+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:27.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:27.093+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:27.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:27.093+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:27.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:27.093+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:27.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:27.093+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.690+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.690+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.690+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.690+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.690+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.690+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 0x7f93400136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.690+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f934000d230 2024-04-20T21:47:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.690+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93400112f0 2024-04-20T21:47:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.691+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d230 legacy=0x7f934000d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.691+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.691+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:28.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.892+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:28.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.892+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d230 legacy=0x7f934000d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:28.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:28.892+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:29.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:29.293+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:29.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:29.293+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d230 legacy=0x7f934000d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:29.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:29.293+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:30.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:30.094+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d230 legacy=0x7f934000d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:30.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:30.094+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:30.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:30.095+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:31.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:31.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400112f0 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:31.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d230 legacy=0x7f934000d640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:31.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:47:31.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:31.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:31.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340009f80 2024-04-20T21:47:31.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f934000a360 2024-04-20T21:47:31.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340007400 2024-04-20T21:47:31.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:31.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:31.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:31.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:31.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.691+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:31.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.892+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:31.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.892+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:31.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.892+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:31.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.892+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:31.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:31.892+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:32.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:32.293+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:32.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:32.293+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:32.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:32.293+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:32.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:32.293+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:32.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:32.293+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:32.565 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:47:32.573 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:47:33.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:33.094+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:33.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:33.094+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:33.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:33.094+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:33.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:33.095+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:33.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:33.095+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:34.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:34.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:34.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:34.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a360 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:34.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009f80 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:34.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 0x7f93400146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:34.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f934000d640 2024-04-20T21:47:34.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93400109f0 2024-04-20T21:47:34.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 msgr2=0x7f93400146f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:34.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.691+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d640 legacy=0x7f93400102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:34.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.692+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 0x7f93400146f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:34.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.893+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d640 legacy=0x7f93400102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:34.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.893+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 msgr2=0x7f93400146f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:34.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:34.893+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 0x7f93400146f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:35.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:35.294+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d640 legacy=0x7f93400102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:35.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:35.294+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 msgr2=0x7f93400146f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:35.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:35.294+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 0x7f93400146f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:36.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:36.095+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d640 legacy=0x7f93400102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:36.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:36.095+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 msgr2=0x7f93400146f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:36.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:36.095+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 0x7f93400146f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:37.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 msgr2=0x7f93400146f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:37.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400109f0 0x7f93400146f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:37.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000d640 legacy=0x7f93400102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:37.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:37.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f934000a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340010dd0 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340007400 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340009f80 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340016c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f9365a00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:37.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.692+0000 7f9365a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:37.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.893+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:37.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.893+0000 7f9365a00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:37.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.893+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340016c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:37.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.893+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:37.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:37.893+0000 7f9365a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:38.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:38.294+0000 7f9365a00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:38.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:38.294+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:38.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:38.294+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340016c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:38.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:38.294+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:38.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:38.294+0000 7f9365a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:39.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:39.095+0000 7f9365a00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:39.095+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340016c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:39.095+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:39.095+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:39.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:39.095+0000 7f9365a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:40.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:40.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f93400098c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f93400098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340016c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340006270 2024-04-20T21:47:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93400066d0 2024-04-20T21:47:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:40.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.692+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:40.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.893+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:40.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.893+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:40.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:40.893+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:41.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:41.294+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:41.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:41.294+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:41.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:41.294+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:42.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:42.095+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:42.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:42.095+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:42.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:42.095+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:43.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:43.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400066d0 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:43.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:43.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:47:43.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 0x7f9340013ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:43.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:43.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340006270 2024-04-20T21:47:43.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93400136e0 2024-04-20T21:47:43.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340009f80 2024-04-20T21:47:43.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 msgr2=0x7f9340013ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:43.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:43.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 0x7f9340013ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:43.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:43.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.693+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:43.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.894+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 msgr2=0x7f9340013ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:43.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.894+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 0x7f9340013ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:43.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.894+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:43.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.894+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:43.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:43.894+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:44.295+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 msgr2=0x7f9340013ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:44.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:44.295+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 0x7f9340013ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:44.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:44.295+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:44.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:44.295+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:44.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:44.295+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:45.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:45.096+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 msgr2=0x7f9340013ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:45.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:45.096+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 0x7f9340013ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:45.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:45.096+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:45.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:45.096+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:45.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:45.096+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:46.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:46.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:46.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 msgr2=0x7f9340013ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:46.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400136e0 0x7f9340013ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:46.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:46.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:46.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340010dd0 2024-04-20T21:47:46.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340007400 2024-04-20T21:47:46.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:46.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:46.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.693+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:46.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.894+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:46.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.894+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:46.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:46.894+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:47.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:47.295+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:47.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:47.295+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:47.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:47.295+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:48.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:48.096+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:48.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:48.096+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:48.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:48.096+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:49.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:49.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:49.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340010dd0 legacy=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:49.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:47:49.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:49.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340006270 2024-04-20T21:47:49.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340005250 2024-04-20T21:47:49.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340007400 2024-04-20T21:47:49.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:49.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:49.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:49.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.694+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:49.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.895+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:49.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.895+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:49.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.895+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:49.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:49.895+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:50.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:50.296+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:50.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:50.296+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:50.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:50.296+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:50.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:50.296+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:51.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:51.097+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:51.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:51.097+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:51.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:51.097+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:51.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:51.098+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:52.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:53.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:53.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:53.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:53.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:53.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f9340004b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:53.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340005d40 2024-04-20T21:47:53.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340009f80 2024-04-20T21:47:53.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:53.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f9340004b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.694+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:53.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.895+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:53.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.895+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f9340004b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:53.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.895+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:53.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:52.895+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:53.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:53.296+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:53.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:53.296+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f9340004b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:53.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:53.296+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:53.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:53.296+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:54.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:54.097+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:54.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:54.097+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f9340004b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:54.097+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:54.097+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:55.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:55.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340009f80 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f9340004b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f9340004b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f9340006270 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340005250 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340010280 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:55.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:55.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:55.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.694+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:55.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.895+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:55.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.895+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:55.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.895+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:55.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:55.895+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:56.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:56.296+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:56.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:56.296+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:56.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:56.296+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:56.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:56.296+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:57.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:57.098+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:47:57.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:57.098+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:47:57.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:57.098+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:47:57.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:57.098+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:47:58.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:58.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:47:58.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:58.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:47:58.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:58.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:47:58.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340007400 2024-04-20T21:47:58.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340005d40 2024-04-20T21:47:58.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.694+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:58.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.695+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:58.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.695+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:58.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.695+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:47:58.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.896+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:58.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.896+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:58.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.896+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:58.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:58.896+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:47:59.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:59.297+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:47:59.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:59.297+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:47:59.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:59.297+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:47:59.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:47:59.297+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:00.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:00.098+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:00.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:00.098+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:00.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:00.098+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:00.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:00.098+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:01.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340006270 2024-04-20T21:48:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f9340005250 2024-04-20T21:48:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340010280 2024-04-20T21:48:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f9365a00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:01.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:01.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.695+0000 7f9365a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:01.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.896+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:01.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.896+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:01.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.896+0000 7f9365a00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:01.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:01.896+0000 7f9365a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:02.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:02.297+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:02.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:02.297+0000 7f9365a00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:02.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:02.297+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:02.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:02.297+0000 7f9365a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:02.686 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:48:02.694 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:48:03.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:03.098+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:03.098+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:03.098+0000 7f9365a00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:03.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:03.098+0000 7f9365a00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:04.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005250 legacy=0x7f9340005630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340005d40 2024-04-20T21:48:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340007400 2024-04-20T21:48:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.696+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:04.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.897+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:04.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.897+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:04.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.897+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:04.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:04.897+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:05.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:05.298+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:05.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:05.298+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:05.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:05.298+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:05.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:05.298+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:06.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:06.100+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:06.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:06.100+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:06.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:06.100+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:06.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:06.100+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:07.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:07.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:07.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:07.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005d40 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 0x7f9340019360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340006270 2024-04-20T21:48:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340004b40 2024-04-20T21:48:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.696+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340010280 2024-04-20T21:48:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.697+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 msgr2=0x7f9340019360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.697+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 0x7f9340019360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.697+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340003b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.697+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.697+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:07.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.898+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 msgr2=0x7f9340019360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:07.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.898+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 0x7f9340019360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:07.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.898+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340003b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:07.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.898+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:07.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:07.898+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:08.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:08.298+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 msgr2=0x7f9340019360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:08.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:08.298+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 0x7f9340019360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:08.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:08.298+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340003b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:08.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:08.298+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:08.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:08.298+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:09.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:09.099+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 msgr2=0x7f9340019360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:09.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:09.099+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 0x7f9340019360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:09.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:09.099+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340003b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:09.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:09.099+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:09.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:09.099+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:10.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:10.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:10.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 msgr2=0x7f9340019360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:10.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340004b40 0x7f9340019360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:10.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340006270 legacy=0x7f9340003b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:10.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:10.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340004240 2024-04-20T21:48:10.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340007400 2024-04-20T21:48:10.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:10.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004240 legacy=0x7f9340004f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:10.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.696+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:10.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.897+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:10.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.897+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004240 legacy=0x7f9340004f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:10.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:10.897+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:11.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:11.298+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:11.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:11.298+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004240 legacy=0x7f9340004f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:11.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:11.298+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:12.099+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004240 legacy=0x7f9340004f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:12.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:12.099+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:12.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:12.099+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:13.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:13.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:13.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004240 legacy=0x7f9340004f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:13.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:13.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:13.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:13.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:13.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93400061f0 2024-04-20T21:48:13.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340007400 2024-04-20T21:48:13.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.696+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f93400107a0 con 0x7f9340010280 2024-04-20T21:48:13.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.697+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:13.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.697+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:13.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.697+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.697+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.697+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.697+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:13.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.898+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:13.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.898+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:13.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.898+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:13.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.898+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:13.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.898+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:13.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:13.898+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:14.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:14.299+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:14.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:14.299+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:14.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:14.299+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:14.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:14.299+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:14.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:14.299+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:14.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:14.299+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:15.100+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:15.100+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:15.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:15.100+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:15.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:15.100+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:15.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:15.100+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:15.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:15.100+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:16.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 msgr2=0x7f934000a7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:16.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010280 0x7f934000a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:16.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:16.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:16.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:16.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:16.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f93400136e0 2024-04-20T21:48:16.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340004050 2024-04-20T21:48:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.697+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:16.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.898+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:16.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:16.898+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:17.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:17.299+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:17.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:17.299+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:18.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:18.100+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:18.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:18.100+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f93400136e0 2024-04-20T21:48:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93400061f0 2024-04-20T21:48:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340007400 2024-04-20T21:48:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.698+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.697+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.698+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.698+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:19.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.899+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:19.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.899+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:19.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.899+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:19.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.899+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:19.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:19.899+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:20.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:20.300+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:20.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:20.300+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:20.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:20.300+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:20.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:20.300+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:20.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:20.300+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:21.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:21.101+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:21.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:21.101+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:21.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:21.101+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:21.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:21.101+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:21.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:21.101+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:22.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 msgr2=0x7f9340003b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f93400061f0 0x7f9340003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f934000a7f0 2024-04-20T21:48:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340005770 2024-04-20T21:48:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:22.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:22.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.698+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:22.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.899+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:22.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.899+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:22.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:22.899+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:23.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:23.300+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:23.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:23.300+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:23.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:23.300+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:24.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:24.101+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:24.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:24.101+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:24.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:24.101+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:25.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f93400136e0 2024-04-20T21:48:25.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f934000a7f0 2024-04-20T21:48:25.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340007400 2024-04-20T21:48:25.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:25.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:25.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.698+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:25.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.699+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.899+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.900+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.900+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:25.900+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:26.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:26.300+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:26.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:26.301+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:26.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:26.301+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:26.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:26.301+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:27.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:27.101+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:27.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:27.102+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:27.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:27.102+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:27.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:27.102+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:28.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:28.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:28.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:28.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:28.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:28.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:28.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340005770 2024-04-20T21:48:28.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340006150 2024-04-20T21:48:28.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:28.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:28.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:28.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.699+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:28.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.900+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:28.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.900+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:28.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.900+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:28.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:28.900+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:29.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:29.301+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:29.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:29.301+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:29.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:29.301+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:29.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:29.301+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:30.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:30.102+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:30.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:30.102+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:30.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:30.102+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:30.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:30.102+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:31.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:31.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:31.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 msgr2=0x7f93400171f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:31.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340005770 0x7f93400171f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:31.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:31.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:31.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 0x7f9340003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:31.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340006150 2024-04-20T21:48:31.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f934000a7f0 2024-04-20T21:48:31.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340007400 2024-04-20T21:48:31.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:31.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:31.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:31.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:31.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.699+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:31.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.900+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:31.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.900+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:31.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.900+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:31.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.900+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:31.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:31.900+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:32.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:32.301+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:32.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:32.301+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:32.301+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:32.301+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:32.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:32.301+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:32.835 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:48:32.843 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:48:33.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:33.102+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:33.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:33.102+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:33.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:33.102+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:33.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:33.102+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:33.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:33.102+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:34.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:34.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:34.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:34.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f934000a7f0 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:34.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:34.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 0x7f9340005770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:34.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f93400136e0 2024-04-20T21:48:34.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340010220 2024-04-20T21:48:34.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:34.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 msgr2=0x7f9340005770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:34.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.700+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 0x7f9340005770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:34.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.901+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:34.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.901+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 msgr2=0x7f9340005770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:34.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:34.901+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 0x7f9340005770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:35.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:35.302+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:35.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:35.302+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 msgr2=0x7f9340005770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:35.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:35.302+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 0x7f9340005770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:36.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:36.103+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:36.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:36.103+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 msgr2=0x7f9340005770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:36.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:36.103+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 0x7f9340005770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:37.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 msgr2=0x7f9340005770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:37.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340010220 0x7f9340005770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:37.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f93400136e0 legacy=0x7f9340015ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:37.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:37.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:37.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006150 2024-04-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340007400 2024-04-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f93400107a0 con 0x7f93400171f0 2024-04-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.700+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:37.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.701+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:37.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.701+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:37.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.901+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:37.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.901+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:37.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.902+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:37.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.902+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:37.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.902+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:37.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:37.902+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:38.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:38.302+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:38.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:38.302+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:38.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:38.303+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:38.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:38.303+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:38.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:38.303+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:38.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:38.303+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:39.103+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:39.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:39.103+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:39.104+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:39.104+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:39.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:39.104+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:39.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:39.104+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:40.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:40.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:40.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 msgr2=0x7f9340006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:40.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340007400 0x7f9340006ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:40.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:40.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:40.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f934000a7f0 2024-04-20T21:48:40.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f9340005770 2024-04-20T21:48:40.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:40.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.701+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:40.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.902+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:40.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:40.902+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:41.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:41.303+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:41.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:41.303+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:42.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:42.104+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:42.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:42.104+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:43.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:43.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:43.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:43.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:43.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:43.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340005770 2024-04-20T21:48:43.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006150 2024-04-20T21:48:43.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f93400171f0 2024-04-20T21:48:43.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:43.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:43.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:43.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:43.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.701+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:43.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.902+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:43.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.902+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:43.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.902+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:43.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.902+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:43.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:43.902+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:44.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:44.303+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:44.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:44.303+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:44.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:44.303+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:44.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:44.303+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:44.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:44.303+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:45.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:45.105+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:45.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:45.105+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:45.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:45.105+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:45.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:45.105+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:45.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:45.105+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:46.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 msgr2=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:46.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f93400171f0 0x7f93400116d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:46.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:46.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:46.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340013c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:46.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 0x7f9340015340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:46.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f934000a7f0 2024-04-20T21:48:46.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006ae0 2024-04-20T21:48:46.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:46.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 msgr2=0x7f9340015340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:46.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.702+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 0x7f9340015340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:46.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.903+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:46.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.903+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 msgr2=0x7f9340015340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:46.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:46.903+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 0x7f9340015340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:47.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:47.304+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:47.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:47.304+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 msgr2=0x7f9340015340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:47.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:47.304+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 0x7f9340015340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:48.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:48.105+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:48.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:48.105+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 msgr2=0x7f9340015340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:48.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:48.105+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 0x7f9340015340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:49.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 msgr2=0x7f9340015340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:49.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340006ae0 0x7f9340015340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:49.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f934000a7f0 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:49.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:49.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:49.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340009410 2024-04-20T21:48:49.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340005770 2024-04-20T21:48:49.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006150 2024-04-20T21:48:49.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:49.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:49.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340015880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:49.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.702+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:49.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.903+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:49.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.903+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:49.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.903+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:49.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:49.903+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340015880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:50.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:50.304+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:50.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:50.304+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:50.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:50.304+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340015880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:50.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:50.304+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:51.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:51.106+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:51.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:51.106+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:51.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:51.106+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:51.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:51.106+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340015880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:52.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:52.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:52.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340005770 legacy=0x7f9340004050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:52.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340015880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:52.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:52.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 0x7f93400178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:52.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93400174e0 2024-04-20T21:48:52.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340006ae0 2024-04-20T21:48:52.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 msgr2=0x7f93400178c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:52.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:52.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 0x7f93400178c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:52.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.703+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:52.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.904+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:52.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.904+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 msgr2=0x7f93400178c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:52.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.904+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:52.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:52.904+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 0x7f93400178c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:53.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:53.305+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:53.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:53.305+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 msgr2=0x7f93400178c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:53.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:53.305+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:53.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:53.305+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 0x7f93400178c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:54.106+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 msgr2=0x7f93400178c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:54.106+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:54.106+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 0x7f93400178c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:54.106+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:55.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.703+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:55.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.703+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:55.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 msgr2=0x7f93400178c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:55.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400174e0 0x7f93400178c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:55.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:48:55.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:55.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:55.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f9340009410 2024-04-20T21:48:55.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006150 2024-04-20T21:48:55.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340006ae0 2024-04-20T21:48:55.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f93400156f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:55.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:55.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:55.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:55.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.704+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:55.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.905+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:55.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.905+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f93400156f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:55.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.905+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:55.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.905+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:55.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:55.905+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:56.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:56.306+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f93400156f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:56.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:56.306+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:56.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:56.306+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:56.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:56.306+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:56.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:56.306+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:48:57.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:57.106+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f93400156f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:48:57.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:57.106+0000 7f9365a00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:48:57.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:57.106+0000 7f9365a00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:57.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:57.106+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:48:57.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:57.106+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:48:58.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.703+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340016e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:58.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.703+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340016e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:58.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.703+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:58.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.703+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.703+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f93400156f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.703+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.703+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340004050 2024-04-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.704+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f93400116d0 2024-04-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.704+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.704+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.704+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:48:58.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.905+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:58.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.905+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:58.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:58.905+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:48:59.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:59.306+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:48:59.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:59.306+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:48:59.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:48:59.306+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:00.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:00.107+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:00.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:00.107+0000 7f9365000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:00.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:00.107+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:01.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:01.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:01.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340004050 legacy=0x7f9340004430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:01.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:01.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:49:01.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:01.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f9340009410 2024-04-20T21:49:01.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006150 2024-04-20T21:49:01.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f93400116d0 2024-04-20T21:49:01.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:01.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:01.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.704+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:01.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.705+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:01.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.705+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:01.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.905+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:01.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.906+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:01.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.906+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:01.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.906+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:01.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:01.906+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:02.306+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:02.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:02.307+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:02.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:02.307+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:02.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:02.307+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:02.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:02.307+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:02.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:49:02.955 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:49:03.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:03.107+0000 7f935fe00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:03.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:03.108+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:03.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:03.108+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:03.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:03.108+0000 7f9365a00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:03.108+0000 7f9365a00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:04.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.704+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 msgr2=0x7f9340011ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:04.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.704+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f93400116d0 0x7f9340011ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:04.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.704+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:04.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.704+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:04.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.704+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340014000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:04.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.704+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:04.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.705+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340015c60 2024-04-20T21:49:04.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.705+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006ae0 2024-04-20T21:49:04.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.705+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340015c60 legacy=0x7f9340016040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:04.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.705+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:04.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.705+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.906+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.906+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340015c60 legacy=0x7f9340016040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:04.906+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:05.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:05.307+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340015c60 legacy=0x7f9340016040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:05.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:05.307+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:05.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:05.307+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:06.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:06.108+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:06.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:06.108+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340015c60 legacy=0x7f9340016040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:06.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:06.108+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:07.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.705+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:07.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.705+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:07.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.705+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340015c60 legacy=0x7f9340016040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:07.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.705+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:49:07.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.705+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:07.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.705+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340016750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:07.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.705+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f9340009410 2024-04-20T21:49:07.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.705+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936015e7b0 con 0x7f9340006150 2024-04-20T21:49:07.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.706+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006ae0 2024-04-20T21:49:07.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.706+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:07.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.706+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:07.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.706+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:07.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.906+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340016750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:07.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.907+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:07.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.907+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:07.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:07.907+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:08.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:08.307+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340016750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:08.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:08.308+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:08.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:08.308+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:08.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:08.308+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:09.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:09.108+0000 7f9365000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340016750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:09.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:09.109+0000 7f9365a00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:09.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:09.109+0000 7f935fe00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:09.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:09.109+0000 7f935fe00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:10.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:10.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:10.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935e000640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340006150 legacy=0x7f93400116d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:10.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340009410 legacy=0x7f9340016750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:10.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 0x7f9340004050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:10.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 0x7f93400049a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:10.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340003af0 2024-04-20T21:49:10.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340004590 2024-04-20T21:49:10.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 msgr2=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:10.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 0x7f9340004050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:10.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 msgr2=0x7f93400049a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:10.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.706+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 0x7f93400049a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:10.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.907+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 msgr2=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:10.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.907+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 0x7f9340004050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:10.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.907+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 msgr2=0x7f93400049a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:10.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:10.907+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 0x7f93400049a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:11.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:11.308+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 msgr2=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:11.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:11.308+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 0x7f9340004050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:11.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:11.308+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 msgr2=0x7f93400049a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:11.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:11.308+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 0x7f93400049a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:12.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:12.109+0000 7f9365000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 msgr2=0x7f9340004050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:12.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:12.109+0000 7f9365000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 0x7f9340004050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:12.109+0000 7f935fe00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 msgr2=0x7f93400049a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:12.109+0000 7f935fe00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 0x7f93400049a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:13.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 msgr2=0x7f93400049a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:13.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340004590 0x7f93400049a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:13.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 msgr2=0x7f9340004050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:13.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340003af0 0x7f9340004050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f9365a00640 1 -- reap_dead start 2024-04-20T21:49:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600cbea0 con 0x7f9340004050 2024-04-20T21:49:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340006150 2024-04-20T21:49:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.706+0000 7f935e000640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f9360155940 con 0x7f9340006ae0 2024-04-20T21:49:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.707+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:13.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.707+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:13.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.707+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:13.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.707+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340004050 legacy=0x7f9340004450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:13.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.707+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:13.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.908+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:13.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.908+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340004050 legacy=0x7f9340004450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:13.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.908+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:13.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.908+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:13.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:13.908+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:14.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:14.309+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:14.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:14.309+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:14.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:14.309+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340004050 legacy=0x7f9340004450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:14.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:14.309+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:14.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:14.309+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:15.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:15.110+0000 7f935fe00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:15.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:15.110+0000 7f935fe00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:15.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:15.110+0000 7f9365a00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340004050 legacy=0x7f9340004450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:15.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:15.110+0000 7f9365000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:15.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:15.110+0000 7f9365000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:16.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935e000640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 msgr2=0x7f9340015010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:16.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935e000640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9340006ae0 0x7f9340015010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:16.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935e000640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 msgr2=0x7f9340003ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:16.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935e000640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9340006150 0x7f9340003ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:16.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935e000640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f9340004050 legacy=0x7f9340004450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:16.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935e000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 0x7f93400136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:16.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935e000640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9340009b00 con 0x7f9340013c20 2024-04-20T21:49:16.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935e000640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f936015f0c0 con 0x7f9340009410 2024-04-20T21:49:16.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340013c20 legacy=0x7f9340014030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:16.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.707+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:16.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.708+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.908+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340013c20 legacy=0x7f9340014030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.909+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:16.909+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:17.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:17.309+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340013c20 legacy=0x7f9340014030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:17.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:17.310+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:17.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:17.310+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:18.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:18.110+0000 7f935fe00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f9340013c20 legacy=0x7f9340014030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:18.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:18.111+0000 7f9365000640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 msgr2=0x7f93400136e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:18.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:18.111+0000 7f9365000640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9340009410 0x7f93400136e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:19.586 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-20T21:49:19.587 ERROR:tasks.mds_thrash.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/mds_thrash.py", line 124, in _run self.do_thrash() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/mds_thrash.py", line 312, in do_thrash status = self.wait_for_stable(rank, gid) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/mds_thrash.py", line 188, in wait_for_stable status = self.fs.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/cephfs/filesystem.py", line 407, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue().strip() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-20T21:49:19.603 INFO:tasks.mds_thrash:done joining 2024-04-20T21:49:19.603 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-20T21:49:19.614 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-20T21:49:19.614 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-20T21:49:19.614 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:49:19.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-04-20T21:49:19.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:49:19.685 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-04-20T21:49:19.739 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:49:19.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:49:19.740 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:49:19.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:49:19.828 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-04-20T21:49:19.939 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-20T21:49:19.951 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T21:49:19.951 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:49:20.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.128+0000 7f082ce00640 1 Processor -- start 2024-04-20T21:49:20.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.128+0000 7f082ce00640 1 -- start start 2024-04-20T21:49:20.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.129+0000 7f082ce00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 0x7f082806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:20.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.129+0000 7f082ce00640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f082815ea20 2024-04-20T21:49:20.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.130+0000 7f082ce00640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f082815f2b0 2024-04-20T21:49:20.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.130+0000 7f082ce00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f082815ee00 2024-04-20T21:49:20.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.130+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f08280c7ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:20.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.130+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 msgr2=0x7f082806d940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:20.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.130+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 0x7f082806d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:20.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.130+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f082815f2b0 legacy=0x7f0828154e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:20.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.331+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f082815f2b0 legacy=0x7f0828154e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:20.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.331+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 msgr2=0x7f082806d940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:20.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.331+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f08280c7ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:20.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.331+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 0x7f082806d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:20.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.732+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 msgr2=0x7f082806d940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:20.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.732+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f08280c7ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:20.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.732+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f082815f2b0 legacy=0x7f0828154e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:20.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:20.732+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 0x7f082806d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:21.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:21.533+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 msgr2=0x7f082806d940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:21.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:21.533+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 0x7f082806d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:21.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:21.533+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f08280c7ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:21.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:21.533+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f082815f2b0 legacy=0x7f0828154e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:23.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 msgr2=0x7f082806d940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:23.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f082815ee00 0x7f082806d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:23.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f082815f2b0 legacy=0x7f0828154e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:23.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f08280c7ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:23.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 0x7f0808005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:23.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 0x7f0808008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:23.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f08080036a0 2024-04-20T21:49:23.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f08080061a0 2024-04-20T21:49:23.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 msgr2=0x7f0808005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:23.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 msgr2=0x7f0808008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:23.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.130+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 0x7f0808005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:23.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.131+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 0x7f0808008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:23.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.331+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 msgr2=0x7f0808005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:23.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.331+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 0x7f0808005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:23.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.332+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 msgr2=0x7f0808008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:23.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.332+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 0x7f0808008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:23.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.732+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 msgr2=0x7f0808005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:23.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.732+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 0x7f0808005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:23.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.733+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 msgr2=0x7f0808008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:23.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:23.733+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 0x7f0808008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:24.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:24.534+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 msgr2=0x7f0808005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:24.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:24.534+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 0x7f0808005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:24.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:24.534+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 msgr2=0x7f0808008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:24.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:24.534+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 0x7f0808008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:26.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 msgr2=0x7f0808008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:26.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08080061a0 0x7f0808008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:26.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 msgr2=0x7f0808005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:26.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080036a0 0x7f0808005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:26.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:49:26.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 0x7f0808006650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:26.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f082815ea20 2024-04-20T21:49:26.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f0808006b90 2024-04-20T21:49:26.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808003b10 2024-04-20T21:49:26.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 msgr2=0x7f0808006650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:26.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006b90 legacy=0x7f0808007020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:26.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f080800b120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:26.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.131+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 0x7f0808006650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.332+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f080800b120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.332+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006b90 legacy=0x7f0808007020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.332+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 msgr2=0x7f0808006650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.332+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 0x7f0808006650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:26.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.733+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f080800b120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:26.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.733+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006b90 legacy=0x7f0808007020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:26.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.733+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 msgr2=0x7f0808006650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:26.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:26.733+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 0x7f0808006650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:27.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:27.534+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f080800b120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:27.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:27.534+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006b90 legacy=0x7f0808007020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:27.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:27.534+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 msgr2=0x7f0808006650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:27.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:27.534+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 0x7f0808006650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 msgr2=0x7f0808006650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808003b10 0x7f0808006650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006b90 legacy=0x7f0808007020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f082815ea20 legacy=0x7f080800b120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 0x7f0808010c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 0x7f0808013570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800e850 2024-04-20T21:49:29.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808011180 2024-04-20T21:49:29.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:29.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:29.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 msgr2=0x7f0808013570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:29.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.132+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 0x7f0808013570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:29.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.333+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:29.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.333+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:29.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.333+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 msgr2=0x7f0808013570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:29.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.333+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 0x7f0808013570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:29.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.734+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:29.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.734+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:29.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.734+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 msgr2=0x7f0808013570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:29.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:29.734+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 0x7f0808013570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:30.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:30.535+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:30.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:30.535+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:30.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:30.535+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 msgr2=0x7f0808013570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:30.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:30.535+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 0x7f0808013570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:32.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 msgr2=0x7f0808013570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:32.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808011180 0x7f0808013570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:32.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:32.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e850 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:32.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:49:32.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:32.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:32.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f08080070a0 2024-04-20T21:49:32.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808006650 2024-04-20T21:49:32.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808005b00 2024-04-20T21:49:32.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:32.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:32.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:32.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:32.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.132+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:32.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.333+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:32.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.333+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:32.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.333+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:32.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.333+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:32.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.333+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:32.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.734+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:32.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.734+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:32.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.734+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:32.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.734+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:32.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:32.734+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:33.077 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:49:33.085 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:49:33.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:33.535+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:33.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:33.535+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:33.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:33.535+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:33.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:33.535+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:33.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:33.535+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:35.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.132+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.132+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.132+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.132+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.132+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.132+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:35.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.132+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f0808007bd0 2024-04-20T21:49:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.132+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800d100 2024-04-20T21:49:35.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.133+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:35.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.133+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:35.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.133+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:35.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.334+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:35.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.334+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:35.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.334+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:35.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.735+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:35.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.735+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:35.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:35.735+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:36.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:36.536+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:36.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:36.536+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:36.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:36.536+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:38.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:38.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:38.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:38.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:49:38.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:38.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f0808007bd0 2024-04-20T21:49:38.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f08080070a0 2024-04-20T21:49:38.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808005b00 2024-04-20T21:49:38.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:38.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:38.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:38.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.133+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:38.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.334+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:38.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.334+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:38.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.334+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:38.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.334+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:38.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.735+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:38.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.735+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:38.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.735+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:38.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:38.735+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:39.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:39.537+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:39.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:39.537+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:39.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:39.537+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:39.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:39.537+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:41.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 msgr2=0x7f0808010c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:41.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808005b00 0x7f0808010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:41.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:41.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:41.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:41.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:41.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800d100 2024-04-20T21:49:41.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006650 2024-04-20T21:49:41.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:41.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:41.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:41.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.134+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:41.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.335+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:41.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.335+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:41.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.335+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:41.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.335+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:41.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.736+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:41.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.736+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:41.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.736+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:41.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:41.736+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:42.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:42.537+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:42.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:42.537+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:42.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:42.537+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:42.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:42.537+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:44.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.134+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:44.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.134+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006650 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:44.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.134+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:44.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d100 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:44.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:49:44.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:44.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f0808007bd0 2024-04-20T21:49:44.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f08080070a0 2024-04-20T21:49:44.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808005b00 2024-04-20T21:49:44.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:44.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:44.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:44.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.135+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:44.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.336+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:44.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.336+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:44.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.336+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:44.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.336+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:44.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.737+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:44.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.737+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:44.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.737+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:44.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:44.737+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:45.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:45.538+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:45.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:45.538+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:45.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:45.538+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:45.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:45.538+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:47.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.134+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:47.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.134+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005b00 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:47.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.134+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:47.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.134+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:47.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.134+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:47.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.134+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:47.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.134+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808006650 2024-04-20T21:49:47.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.134+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f080800dc90 2024-04-20T21:49:47.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.135+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:47.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.135+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:47.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.135+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:47.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.135+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:47.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.336+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:47.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.336+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:47.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.336+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:47.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.336+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:47.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.737+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:47.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.737+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:47.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.737+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:47.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:47.737+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:48.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:48.538+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:48.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:48.538+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:48.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:48.538+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:48.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:48.538+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:50.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:50.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800dc90 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:50.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f0808006a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:50.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f0808006a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:50.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:49:50.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 0x7f080800c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:50.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:50.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f0808007bd0 2024-04-20T21:49:50.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808006650 2024-04-20T21:49:50.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f080800c0f0 2024-04-20T21:49:50.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 msgr2=0x7f080800c4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:50.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:50.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:50.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 0x7f080800c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:50.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.135+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.336+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.336+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:50.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.336+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:50.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.336+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 msgr2=0x7f080800c4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:50.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.336+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 0x7f080800c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:50.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.737+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:50.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.737+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:50.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.737+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:50.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.737+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 msgr2=0x7f080800c4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:50.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:50.737+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 0x7f080800c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:51.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:51.538+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:51.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:51.538+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:51.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:51.538+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:51.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:51.538+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 msgr2=0x7f080800c4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:51.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:51.538+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 0x7f080800c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:53.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.135+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 msgr2=0x7f080800c4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.135+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800c0f0 0x7f080800c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.135+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.135+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.135+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.135+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.135+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f0808005b00 2024-04-20T21:49:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.135+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f08080070a0 2024-04-20T21:49:53.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.136+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:53.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.136+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:53.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.136+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:53.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.337+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:53.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.337+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:53.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.337+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:53.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.738+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:53.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.738+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:53.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:53.738+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:54.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:54.539+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:54.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:54.539+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:54.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:54.539+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:56.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:56.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08080070a0 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:56.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:56.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:49:56.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:56.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:56.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f0808005b00 2024-04-20T21:49:56.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f08080070a0 2024-04-20T21:49:56.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006650 2024-04-20T21:49:56.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:56.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:56.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:56.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:56.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.136+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:56.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.337+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:56.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.337+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:56.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.337+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:56.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.337+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:56.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.337+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:56.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.738+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:56.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.738+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:56.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.738+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:56.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.738+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:56.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:56.738+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:49:57.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:57.539+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:49:57.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:57.539+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:57.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:57.539+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:49:57.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:57.539+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:49:57.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:57.539+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:49:59.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:59.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:59.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:59.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:49:59.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:49:59.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:49:59.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f0808007bd0 2024-04-20T21:49:59.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808010c40 2024-04-20T21:49:59.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.136+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:59.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.137+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:59.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.137+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:49:59.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.338+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:59.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.338+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:59.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.338+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:49:59.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.739+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:49:59.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.739+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:49:59.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:49:59.739+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:00.540+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:00.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:00.540+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:00.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:00.540+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:02.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:02.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:02.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808007bd0 legacy=0x7f0808007fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f0808015b90 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f0808005b00 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f08080070a0 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808015b90 legacy=0x7f0808017f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:02.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.137+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:02.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.338+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808015b90 legacy=0x7f0808017f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:02.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.338+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:02.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.338+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:02.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.338+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:02.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.739+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808015b90 legacy=0x7f0808017f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.739+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.739+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:02.739+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:03.398 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:50:03.406 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:50:03.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:03.540+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:03.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:03.540+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808015b90 legacy=0x7f0808017f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:03.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:03.540+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:03.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:03.540+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808015b90 legacy=0x7f0808017f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808010c40 2024-04-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006650 2024-04-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:05.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.137+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:05.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.138+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:05.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.138+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:05.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.339+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:05.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.339+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:05.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.339+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:05.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.339+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:05.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.740+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:05.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.740+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:05.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.740+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:05.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:05.740+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:06.541+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:06.541+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:06.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:06.541+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:06.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:06.541+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:08.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:08.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006650 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:08.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:08.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:08.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:08.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:08.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:08.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f0808005b00 2024-04-20T21:50:08.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f08080070a0 2024-04-20T21:50:08.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006630 2024-04-20T21:50:08.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:08.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:08.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:08.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f0826800640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:08.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.138+0000 7f0826800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:08.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.339+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:08.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.339+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:08.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.339+0000 7f0826800640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:08.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.339+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:08.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.339+0000 7f0826800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.740+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.740+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.740+0000 7f0826800640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.740+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:08.740+0000 7f0826800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:09.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:09.541+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:09.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:09.541+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:09.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:09.541+0000 7f0826800640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:09.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:09.541+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:09.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:09.541+0000 7f0826800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:11.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:11.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:11.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:11.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f08080070a0 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:11.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808005b00 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:11.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:11.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f0808008040 2024-04-20T21:50:11.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808010c40 2024-04-20T21:50:11.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:11.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.138+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:11.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.139+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:11.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.340+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:11.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.340+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:11.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.340+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:11.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.741+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:11.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.741+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:11.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:11.741+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:12.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:12.542+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:12.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:12.542+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:12.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:12.542+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:14.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:14.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:14.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:14.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f0808008040 2024-04-20T21:50:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808010c40 2024-04-20T21:50:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006630 2024-04-20T21:50:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:14.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:14.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:14.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.139+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:14.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.340+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:14.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.340+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:14.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.340+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:14.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.340+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:14.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.340+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.741+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.741+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.741+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.741+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:14.741+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:15.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:15.542+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:15.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:15.542+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:15.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:15.542+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:15.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:15.542+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:15.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:15.542+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:17.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:17.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:17.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:17.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:17.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:17.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:17.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f08080070a0 2024-04-20T21:50:17.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808005b00 2024-04-20T21:50:17.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:17.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:17.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.139+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:17.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.340+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:17.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.340+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:17.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.340+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.741+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.741+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:17.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:17.741+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:18.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:18.542+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:18.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:18.542+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:18.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:18.542+0000 7f0825e00640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:20.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:20.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:20.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080070a0 legacy=0x7f08080074c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:20.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:20.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:20.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:20.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f0808008040 2024-04-20T21:50:20.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808005b00 2024-04-20T21:50:20.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.139+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006630 2024-04-20T21:50:20.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.140+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:20.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.140+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:20.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.140+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:20.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.140+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:20.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.140+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:20.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.341+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:20.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.341+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:20.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.341+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:20.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.341+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:20.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.341+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:20.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.742+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:20.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.742+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:20.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.742+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:20.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.742+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:20.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:20.742+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:21.543+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:21.543+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:21.543+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:21.543+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:21.543+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:23.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:23.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:23.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:23.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808005b00 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:23.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:23.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:23.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f08080051a0 2024-04-20T21:50:23.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808010c40 2024-04-20T21:50:23.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:23.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080051a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:23.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.140+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:23.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.341+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:23.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.341+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:23.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.341+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080051a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:23.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.742+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:23.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.742+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:23.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:23.742+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080051a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:24.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:24.543+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:24.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:24.543+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:24.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:24.543+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080051a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:26.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f080800d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:26.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f080800d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:26.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080051a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:26.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:26.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:26.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f0808008040 2024-04-20T21:50:26.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f08080077b0 2024-04-20T21:50:26.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808006630 2024-04-20T21:50:26.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:26.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:26.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f0808018bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:26.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.141+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:26.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.342+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:26.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.342+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:26.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.342+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f0808018bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:26.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.342+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:26.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.743+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:26.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.743+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:26.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.743+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f0808018bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:26.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:26.743+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:27.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:27.544+0000 7f0826800640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:27.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:27.544+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:27.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:27.544+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:27.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:27.544+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f0808018bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:29.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:29.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:29.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f0808018bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:29.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:29.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f0808006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:29.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 0x7f08080070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:29.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800d510 2024-04-20T21:50:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808010c40 2024-04-20T21:50:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:29.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.141+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:29.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.342+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:29.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.342+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:29.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.342+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:29.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.342+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:29.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.743+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:29.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.743+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:29.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.743+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:29.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:29.743+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:30.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:30.544+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:30.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:30.544+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:30.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:30.544+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:30.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:30.544+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:32.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808006a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:32.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f0808006a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:32.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:32.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800d510 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:32.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:32.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:32.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f0808008040 2024-04-20T21:50:32.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f08080077b0 2024-04-20T21:50:32.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808003a80 2024-04-20T21:50:32.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:32.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:32.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:32.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.142+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:32.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.343+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:32.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.343+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:32.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.343+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:32.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.343+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:32.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.744+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:32.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.744+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:32.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.744+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:32.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:32.744+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:33.536 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:50:33.544 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:50:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:33.545+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:33.545+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:33.545+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:33.545+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:35.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.141+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:35.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.141+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:35.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.141+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f08080077b0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:35.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.141+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:35.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.141+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:35.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.141+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:35.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.141+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808006630 2024-04-20T21:50:35.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.141+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006a30 2024-04-20T21:50:35.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.142+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:35.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.142+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:35.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.142+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:35.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.142+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:35.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.343+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:35.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.343+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:35.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.343+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:35.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.343+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:35.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.744+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:35.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.744+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:35.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.744+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:35.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:35.744+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:36.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:36.545+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:36.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:36.545+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:36.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:36.545+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:36.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:36.545+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:38.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.141+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:38.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.141+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:38.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.141+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 msgr2=0x7f080800e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.141+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808006630 0x7f080800e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.141+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.141+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f08080048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.141+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808003a80 2024-04-20T21:50:38.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808010c40 2024-04-20T21:50:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f080800d990 con 0x7f0808006a30 2024-04-20T21:50:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:38.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:38.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:38.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f0826800640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:38.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:38.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.142+0000 7f0826800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:38.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.343+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:38.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.343+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:38.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.343+0000 7f0826800640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:38.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.343+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:38.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.343+0000 7f0826800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:38.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.343+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.744+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.744+0000 7f0826800640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.744+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.744+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.744+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:38.744+0000 7f0826800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:39.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:39.545+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:39.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:39.545+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:39.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:39.545+0000 7f0826800640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:39.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:39.545+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:39.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:39.545+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:39.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:39.545+0000 7f0826800640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:41.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:41.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:41.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:41.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808010c40 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:41.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 msgr2=0x7f0808003e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:41.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808003a80 0x7f0808003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:41.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f0808008040 2024-04-20T21:50:41.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f080800e0a0 2024-04-20T21:50:41.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:41.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.142+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:41.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.343+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:41.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.343+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.744+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:41.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:41.744+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:42.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:42.545+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:42.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:42.545+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:44.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.142+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:44.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.142+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808008040 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:44.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.142+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:44.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.142+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:44.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.142+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 0x7f08080070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:44.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.142+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f08080077b0 2024-04-20T21:50:44.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.142+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800e0a0 2024-04-20T21:50:44.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.142+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006a30 2024-04-20T21:50:44.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.143+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f08080077b0 legacy=0x7f0808012570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:44.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.143+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:44.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.143+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:44.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.143+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:44.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.143+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.344+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.344+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f08080077b0 legacy=0x7f0808012570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.344+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:44.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.344+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:44.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.344+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:44.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.745+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:44.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.745+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f08080077b0 legacy=0x7f0808012570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:44.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.745+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:44.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.745+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:44.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:44.745+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:45.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:45.546+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f08080077b0 legacy=0x7f0808012570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:45.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:45.546+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:45.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:45.546+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:45.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:45.546+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:45.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:45.546+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:47.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f08080043a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:47.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f08080043a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:47.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 msgr2=0x7f08080070a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:47.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800e0a0 0x7f08080070a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:47.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f08080077b0 legacy=0x7f0808012570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:47.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 0x7f08080048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:47.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f0808012c80 2024-04-20T21:50:47.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808005580 2024-04-20T21:50:47.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:47.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808012c80 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:47.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.143+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:47.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.344+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808012c80 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:47.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.344+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:47.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.344+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:47.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.745+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808012c80 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:47.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.745+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:47.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:47.745+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:48.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:48.546+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808012c80 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:48.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:48.546+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:48.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:48.546+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:50.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 msgr2=0x7f08080048e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:50.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808005580 0x7f08080048e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:50.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808012c80 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:50.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:50.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:50.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f0808005580 2024-04-20T21:50:50.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806d140 con 0x7f080800e0a0 2024-04-20T21:50:50.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808006a30 2024-04-20T21:50:50.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:50.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.143+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808005580 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:50.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.144+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:50.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.144+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:50.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.345+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:50.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.345+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:50.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.345+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808005580 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:50.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.345+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:50.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.746+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:50.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.746+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808005580 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:50.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.746+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:50.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:50.746+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:51.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:51.547+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:51.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:51.547+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808005580 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:51.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:51.547+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:51.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:51.547+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:53.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:53.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:53.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:53.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f0808005580 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:53.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 0x7f0808005960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:53.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:53.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808010c40 2024-04-20T21:50:53.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808007080 2024-04-20T21:50:53.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808005960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:53.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:53.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 0x7f0808005960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:53.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.144+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:53.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.345+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808005960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:53.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.345+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:53.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.345+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 0x7f0808005960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:53.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.345+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:53.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.746+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808005960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:53.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.746+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:53.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.746+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:53.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:53.746+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 0x7f0808005960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:54.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:54.547+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808005960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:54.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:54.547+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 0x7f0808005960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:54.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:54.547+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:54.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:54.547+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:56.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:56.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:56.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 msgr2=0x7f0808005960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:56.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0808010c40 0x7f0808005960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:56.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:56.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:50:56.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:56.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f080800e0a0 2024-04-20T21:50:56.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800c0d0 2024-04-20T21:50:56.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808006a30 2024-04-20T21:50:56.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:56.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:56.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:56.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:56.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.144+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:56.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.345+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:56.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.345+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:56.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.345+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:56.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.345+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:56.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.345+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.746+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.746+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:56.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.746+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:56.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.746+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:56.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:56.746+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:50:57.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:57.547+0000 7f0826800640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:50:57.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:57.547+0000 7f0825400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:50:57.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:57.547+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:50:57.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:57.547+0000 7f0825400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:57.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:57.547+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:50:59.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 msgr2=0x7f0808004190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:59.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a30 0x7f0808004190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:59.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:59.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:50:59.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f08080046d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:50:59.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:50:59.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f080800c9f0 2024-04-20T21:50:59.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808007080 2024-04-20T21:50:59.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:59.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:59.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.145+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:50:59.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.346+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:59.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.346+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:59.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.346+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:50:59.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.747+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:50:59.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.747+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:50:59.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:50:59.747+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:00.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:00.548+0000 7f0825400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:00.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:00.548+0000 7f0825e00640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:00.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:00.548+0000 7f0825e00640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:02.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:02.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:02.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:51:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f080800e0a0 2024-04-20T21:51:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800c0d0 2024-04-20T21:51:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808007080 2024-04-20T21:51:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:02.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:02.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:02.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:02.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.145+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:02.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.346+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:02.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.346+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:02.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.346+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:02.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.346+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:02.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.346+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:02.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.747+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.747+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.747+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.747+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:02.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:02.747+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:03.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:03.548+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:03.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:03.548+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:03.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:03.548+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:03.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:03.548+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:03.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:03.548+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:03.657 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:03.666 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:05.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:05.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:05.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:05.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:05.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:05.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:05.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f080800c9f0 2024-04-20T21:51:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808006a10 2024-04-20T21:51:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:05.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.146+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:05.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.347+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:05.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.347+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:05.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.347+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:05.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.748+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:05.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.748+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:05.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:05.748+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:06.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:06.550+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:06.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:06.550+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:06.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:06.550+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:08.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:08.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:08.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:08.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:08.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:51:08.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:08.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f080800e0a0 2024-04-20T21:51:08.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800c0d0 2024-04-20T21:51:08.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808007080 2024-04-20T21:51:08.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:08.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:08.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:08.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:08.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.146+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.347+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.347+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.347+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.347+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:08.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.347+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:08.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.748+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:08.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.748+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:08.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.748+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:08.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.748+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:08.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:08.748+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:09.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:09.549+0000 7f0826800640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:09.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:09.549+0000 7f0826800640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:09.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:09.549+0000 7f0825e00640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:09.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:09.549+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:09.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:09.549+0000 7f0825e00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:11.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:11.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:11.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:11.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:11.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f081f400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f080800e0a0 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:11.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:11.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f080800c9f0 2024-04-20T21:51:11.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f0808006a10 2024-04-20T21:51:11.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:11.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:11.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.146+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:11.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.347+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:11.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.347+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:11.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.347+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:11.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.748+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:11.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.748+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:11.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:11.748+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:12.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:12.550+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:12.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:12.550+0000 7f0825400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:12.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:12.550+0000 7f0825400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f081f400640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 msgr2=0x7f080800b0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0808006a10 0x7f080800b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:14.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800c9f0 legacy=0x7f0808003a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f0826800640 1 -- reap_dead start 2024-04-20T21:51:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f081f400640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280c8f20 con 0x7f080800b600 2024-04-20T21:51:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f081f400640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800c0d0 2024-04-20T21:51:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f081f400640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 -- 0x7f0808008d30 con 0x7f0808007080 2024-04-20T21:51:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800b600 legacy=0x7f080801aad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:14.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:14.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.147+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.348+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800b600 legacy=0x7f080801aad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.348+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.348+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.348+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.348+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:14.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.749+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:14.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.749+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800b600 legacy=0x7f080801aad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:14.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.749+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:14.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.749+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:14.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:14.749+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:15.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:15.550+0000 7f0826800640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.102:3300/0 2024-04-20T21:51:15.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:15.550+0000 7f0825e00640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800b600 legacy=0x7f080801aad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.102:6789/0 2024-04-20T21:51:15.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:15.550+0000 7f0826800640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:15.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:15.550+0000 7f0825400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3300/0 2024-04-20T21:51:15.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:15.550+0000 7f0825400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:17.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f081f400640 1 -- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 msgr2=0x7f0808007490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:17.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f081f400640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0808007080 0x7f0808007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:17.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f081f400640 1 -- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 msgr2=0x7f080800c4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:17.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f081f400640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f080800c0d0 0x7f080800c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:51:17.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f081f400640 1 -- >> v1:172.21.15.102:6789/0 conn(0x7f080800b600 legacy=0x7f080801aad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:51:17.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f081f400640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 0x7f080800b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:51:17.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f081f400640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082806cb80 con 0x7f0808006a10 2024-04-20T21:51:17.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f081f400640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 -- 0x7f082806ce60 con 0x7f080800e0a0 2024-04-20T21:51:17.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006a10 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:17.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 msgr2=0x7f080800b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:17.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.147+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 0x7f080800b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:51:17.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.348+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 msgr2=0x7f080800b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:17.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.348+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006a10 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:17.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.348+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 0x7f080800b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:51:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.749+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006a10 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.749+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 msgr2=0x7f080800b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:17.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:17.749+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 0x7f080800b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:51:18.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:18.551+0000 7f0825e00640 1 -- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 msgr2=0x7f080800b9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.193:3301/0 2024-04-20T21:51:18.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:18.551+0000 7f0825400640 1 -- >> v1:172.21.15.193:6789/0 conn(0x7f0808006a10 legacy=0x7f0808005580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.193:6789/0 2024-04-20T21:51:18.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-20T21:51:18.551+0000 7f0825e00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f080800e0a0 0x7f080800b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:51:20.036 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-20T21:51:20.036 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-20T21:51:20.049 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-20T21:51:20.050 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:51:20.050 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:51:20.052 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-04-20T21:51:20.063 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:51:20.063 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:51:20.063 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:51:20.063 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-04-20T21:51:20.073 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:51:20.073 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-20T21:51:20.074 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.074 INFO:tasks.ceph.mds.e:Stopped 2024-04-20T21:51:20.074 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:51:20.074 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:51:20.074 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi193 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2024-04-20T21:51:20.085 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:51:20.085 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:51:20.085 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.085 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:51:20.085 ERROR:tasks.ceph.mds.f:tried to stop a non-running daemon 2024-04-20T21:51:20.087 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-20T21:51:20.087 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-20T21:51:20.088 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.088 INFO:tasks.ceph.osd.0:Stopped 2024-04-20T21:51:20.088 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-20T21:51:20.088 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.088 INFO:tasks.ceph.osd.1:Stopped 2024-04-20T21:51:20.088 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-20T21:51:20.089 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.089 INFO:tasks.ceph.osd.2:Stopped 2024-04-20T21:51:20.089 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-20T21:51:20.089 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.089 INFO:tasks.ceph.osd.3:Stopped 2024-04-20T21:51:20.090 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-20T21:51:20.090 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.090 INFO:tasks.ceph.osd.4:Stopped 2024-04-20T21:51:20.090 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-20T21:51:20.090 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.090 INFO:tasks.ceph.osd.5:Stopped 2024-04-20T21:51:20.090 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-20T21:51:20.091 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.091 INFO:tasks.ceph.osd.6:Stopped 2024-04-20T21:51:20.091 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-20T21:51:20.091 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.091 INFO:tasks.ceph.osd.7:Stopped 2024-04-20T21:51:20.092 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-20T21:51:20.092 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-20T21:51:20.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:51:20.092 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y' 2024-04-20T21:51:20.104 INFO:tasks.ceph.mgr.y:Stopped 2024-04-20T21:51:20.104 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-20T21:51:20.104 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.104 INFO:tasks.ceph.mgr.x:Stopped 2024-04-20T21:51:20.105 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-20T21:51:20.105 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T21:51:20.105 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.107 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T21:51:20.107 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T21:51:20.107 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.107 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T21:51:20.107 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T21:51:20.107 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:51:20.108 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T21:51:20.108 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-20T21:51:20.108 DEBUG:teuthology.orchestra.run.smithi102:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v 'Replacing daemon mds' | head -n 1 2024-04-20T21:51:20.242 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:51:20.242 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-20T21:51:20.458 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:51:20.459 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-20T21:51:20.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:51:20.625 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-20T21:51:20.797 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:51:20.797 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-20T21:51:21.048 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:51:21.049 DEBUG:teuthology.orchestra.run.smithi193:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-20T21:51:21.265 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:51:21.265 DEBUG:teuthology.orchestra.run.smithi193:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-20T21:51:21.328 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:51:21.328 DEBUG:teuthology.orchestra.run.smithi193:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-20T21:51:21.395 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:51:21.395 DEBUG:teuthology.orchestra.run.smithi193:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-20T21:51:21.453 INFO:tasks.ceph:Archiving mon data... 2024-04-20T21:51:21.455 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/data/mon.a.tgz 2024-04-20T21:51:21.455 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-04-20T21:51:21.511 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.mRfqw4jQEo 2024-04-20T21:51:21.512 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.mRfqw4jQEo 2024-04-20T21:51:21.695 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.mRfqw4jQEo 2024-04-20T21:51:21.835 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.mRfqw4jQEo is 402KB 2024-04-20T21:51:21.940 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.mRfqw4jQEo 2024-04-20T21:51:21.995 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/data/mon.b.tgz 2024-04-20T21:51:21.995 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-04-20T21:51:22.013 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.fgD8Ia5o3W 2024-04-20T21:51:22.014 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.fgD8Ia5o3W 2024-04-20T21:51:22.177 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.fgD8Ia5o3W 2024-04-20T21:51:22.260 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.fgD8Ia5o3W is 412KB 2024-04-20T21:51:22.358 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.fgD8Ia5o3W 2024-04-20T21:51:22.372 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/data/mon.c.tgz 2024-04-20T21:51:22.373 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-04-20T21:51:22.427 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.wCNbGaGCl6 2024-04-20T21:51:22.428 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.wCNbGaGCl6 2024-04-20T21:51:22.572 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.wCNbGaGCl6 2024-04-20T21:51:22.653 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.wCNbGaGCl6 is 412KB 2024-04-20T21:51:22.750 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.wCNbGaGCl6 2024-04-20T21:51:22.765 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-20T21:51:22.765 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T21:51:22.773 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T21:51:22.904 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T21:51:22.919 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T21:51:22.919 INFO:tasks.ceph:Archiving crash dumps... 2024-04-20T21:51:22.921 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/remote/smithi102/crash 2024-04-20T21:51:22.921 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T21:51:23.123 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/remote/smithi193/crash 2024-04-20T21:51:23.124 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T21:51:23.199 INFO:tasks.ceph:Shutting down logrotate 2024-04-20T21:51:23.199 DEBUG:teuthology.orchestra.run.smithi102:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:23.207 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:23.321 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:23.406 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:23.430 INFO:tasks.ceph:Compressing logs... 2024-04-20T21:51:23.430 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:51:23.438 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:51:23.496 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23479.log 2024-04-20T21:51:23.496 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T21:51:23.497 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.23479.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-20T21:51:23.497 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.23479.log.gz 2024-04-20T21:51:23.497 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T21:51:23.497 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-20T21:51:23.497 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-20T21:51:23.497 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-20T21:51:23.497 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-20T21:51:23.497 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23730.log 2024-04-20T21:51:23.498 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-20T21:51:23.498 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.23730.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.23730.log.gz 2024-04-20T21:51:23.498 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-20T21:51:23.498 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mgr.x.log: 94.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T21:51:23.498 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23563.log 2024-04-20T21:51:23.499 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mon.c.log: 93.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T21:51:23.499 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-20T21:51:23.500 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.23563.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.23563.log.gz 2024-04-20T21:51:23.514 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-04-20T21:51:23.515 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47635.log 2024-04-20T21:51:23.516 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41502.log 2024-04-20T21:51:23.516 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-04-20T21:51:23.517 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31714.log 2024-04-20T21:51:23.518 INFO:teuthology.orchestra.run.smithi102.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47635.log.gz 2024-04-20T21:51:23.518 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41502.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41502.log.gz 2024-04-20T21:51:23.519 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28728.log 2024-04-20T21:51:23.519 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23650.log 2024-04-20T21:51:23.520 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-04-20T21:51:23.520 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31714.log.gz 2024-04-20T21:51:23.521 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28728.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28728.log.gz 2024-04-20T21:51:23.521 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41015.log 2024-04-20T21:51:23.522 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48042.log 2024-04-20T21:51:23.522 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-04-20T21:51:23.523 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41015.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41015.log.gz 2024-04-20T21:51:23.523 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30861.log 2024-04-20T21:51:23.525 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48752.log 2024-04-20T21:51:23.525 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48042.log.gz 2024-04-20T21:51:23.526 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30861.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30861.log.gz 2024-04-20T21:51:23.526 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35798.log 2024-04-20T21:51:23.527 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38435.log 2024-04-20T21:51:23.527 INFO:teuthology.orchestra.run.smithi102.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48752.log.gz 2024-04-20T21:51:23.528 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.35798.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35798.log.gz 2024-04-20T21:51:23.528 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27974.log 2024-04-20T21:51:23.529 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29827.log 2024-04-20T21:51:23.529 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38435.log.gz 2024-04-20T21:51:23.530 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.27974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29927.log 2024-04-20T21:51:23.530 INFO:teuthology.orchestra.run.smithi102.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.27974.log.gz 2024-04-20T21:51:23.531 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37031.log 2024-04-20T21:51:23.532 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29827.log.gz 2024-04-20T21:51:23.532 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42977.log 88.7% 2024-04-20T21:51:23.533 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29927.log.gz 2024-04-20T21:51:23.535 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23610.log 2024-04-20T21:51:23.535 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.23650.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.23650.log.gz 2024-04-20T21:51:23.535 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37031.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.37031.log.gz 2024-04-20T21:51:23.535 INFO:teuthology.orchestra.run.smithi102.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31213.log 2024-04-20T21:51:23.536 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30695.log 2024-04-20T21:51:23.536 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42977.log.gz 2024-04-20T21:51:23.536 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47677.log 2024-04-20T21:51:23.536 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31213.log.gz 2024-04-20T21:51:23.537 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30695.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30695.log.gzgzip 2024-04-20T21:51:23.537 INFO:teuthology.orchestra.run.smithi102.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.27893.log 2024-04-20T21:51:23.538 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47677.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47677.log.gz 2024-04-20T21:51:23.538 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45102.log 2024-04-20T21:51:23.539 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.27893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34003.log 2024-04-20T21:51:23.539 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27893.log.gz 2024-04-20T21:51:23.540 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45102.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45102.log.gz 2024-04-20T21:51:23.540 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42402.log 2024-04-20T21:51:23.541 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34528.log 2024-04-20T21:51:23.541 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34003.log.gz 2024-04-20T21:51:23.542 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28419.log 2024-04-20T21:51:23.542 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42402.log.gz 2024-04-20T21:51:23.543 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34528.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34528.log.gz 2024-04-20T21:51:23.543 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41856.log 2024-04-20T21:51:23.544 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52412.log 2024-04-20T21:51:23.545 INFO:teuthology.orchestra.run.smithi102.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28419.log.gz 2024-04-20T21:51:23.546 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51795.log 2024-04-20T21:51:23.546 INFO:teuthology.orchestra.run.smithi102.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41856.log.gz 2024-04-20T21:51:23.547 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52412.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52412.log.gz 2024-04-20T21:51:23.547 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53088.log 2024-04-20T21:51:23.548 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23770.log 2024-04-20T21:51:23.548 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.23610.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.23610.log.gz 2024-04-20T21:51:23.549 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35840.log 2024-04-20T21:51:23.549 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51795.log.gz 2024-04-20T21:51:23.549 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53088.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53088.log.gz 2024-04-20T21:51:23.550 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40467.log 2024-04-20T21:51:23.550 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.35840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50752.log 2024-04-20T21:51:23.550 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35840.log.gz 2024-04-20T21:51:23.551 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42851.log 2024-04-20T21:51:23.551 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40467.log.gz 2024-04-20T21:51:23.552 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50752.log: gzip -5 88.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.50752.log.gz 2024-04-20T21:51:23.552 INFO:teuthology.orchestra.run.smithi102.stderr: -- /var/log/ceph/ceph-client.admin.49453.log 2024-04-20T21:51:23.553 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42851.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42851.log.gz 2024-04-20T21:51:23.554 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44661.log 2024-04-20T21:51:23.555 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-04-20T21:51:23.555 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49453.log.gz 2024-04-20T21:51:23.556 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44661.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44661.log.gz 2024-04-20T21:51:23.556 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46566.log 2024-04-20T21:51:23.557 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45376.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-04-20T21:51:23.557 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34223.log 2024-04-20T21:51:23.558 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46566.log: gzip -5 88.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.46566.log.gz -- 2024-04-20T21:51:23.558 INFO:teuthology.orchestra.run.smithi102.stderr: /var/log/ceph/ceph-client.admin.46742.log 2024-04-20T21:51:23.559 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34223.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34223.log.gz 2024-04-20T21:51:23.560 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-04-20T21:51:23.561 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32819.log 2024-04-20T21:51:23.561 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46742.log.gz 2024-04-20T21:51:23.562 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-20T21:51:23.562 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.23770.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.23770.log.gz 2024-04-20T21:51:23.563 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44274.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-04-20T21:51:23.563 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31623.log 2024-04-20T21:51:23.564 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28770.log 2024-04-20T21:51:23.564 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32819.log.gz 2024-04-20T21:51:23.564 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31623.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31623.log.gz 2024-04-20T21:51:23.565 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48355.log 2024-04-20T21:51:23.565 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35966.log 2024-04-20T21:51:23.565 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28770.log.gz 2024-04-20T21:51:23.566 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48355.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.48355.log.gz 2024-04-20T21:51:23.566 INFO:teuthology.orchestra.run.smithi102.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42767.log 2024-04-20T21:51:23.567 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.35966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36686.log 2024-04-20T21:51:23.568 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35966.log.gz 2024-04-20T21:51:23.568 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49556.log 2024-04-20T21:51:23.569 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42767.log.gz 2024-04-20T21:51:23.569 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36686.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36686.log.gz 2024-04-20T21:51:23.570 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47183.log 2024-04-20T21:51:23.571 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49556.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49556.log.gz 2024-04-20T21:51:23.571 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-04-20T21:51:23.572 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31925.log 2024-04-20T21:51:23.572 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47183.log.gz 2024-04-20T21:51:23.573 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23690.log 2024-04-20T21:51:23.574 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41373.log: gzip -5 87.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz -- 2024-04-20T21:51:23.574 INFO:teuthology.orchestra.run.smithi102.stderr: /var/log/ceph/ceph-client.admin.40289.log 2024-04-20T21:51:23.575 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51544.log 2024-04-20T21:51:23.575 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31925.log.gz 2024-04-20T21:51:23.575 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31207.log 2024-04-20T21:51:23.576 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40289.log.gz 2024-04-20T21:51:23.576 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51544.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T21:51:23.577 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51544.log.gz 2024-04-20T21:51:23.578 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31207.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31207.log.gz 2024-04-20T21:51:23.578 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48294.log 2024-04-20T21:51:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49758.log 2024-04-20T21:51:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28938.log 2024-04-20T21:51:23.581 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48294.log.gz 2024-04-20T21:51:23.581 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31493.log 2024-04-20T21:51:23.582 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49758.log.gz 2024-04-20T21:51:23.582 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28854.log 2024-04-20T21:51:23.583 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T21:51:23.583 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31493.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.28938.log.gz 2024-04-20T21:51:23.584 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31493.log.gz 2024-04-20T21:51:23.584 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30875.log 2024-04-20T21:51:23.585 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42512.log 2024-04-20T21:51:23.585 INFO:teuthology.orchestra.run.smithi102.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28854.log.gz 2024-04-20T21:51:23.586 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41772.log 2024-04-20T21:51:23.586 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30875.log.gz 2024-04-20T21:51:23.587 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31451.log 2024-04-20T21:51:23.587 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42512.log.gz 2024-04-20T21:51:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39010.log 2024-04-20T21:51:23.588 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41772.log.gz 2024-04-20T21:51:23.589 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34715.log 2024-04-20T21:51:23.589 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31451.log.gz 2024-04-20T21:51:23.591 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30569.log 2024-04-20T21:51:23.591 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39010.log.gz 2024-04-20T21:51:23.592 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-20T21:51:23.592 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.23690.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.23690.log.gz 2024-04-20T21:51:23.593 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51358.log 2024-04-20T21:51:23.593 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34715.log.gz 2024-04-20T21:51:23.593 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30569.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-20T21:51:23.593 INFO:teuthology.orchestra.run.smithi102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30569.log.gz 2024-04-20T21:51:23.594 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28896.log 2024-04-20T21:51:23.594 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51358.log.gz 2024-04-20T21:51:23.594 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42683.log 2024-04-20T21:51:23.595 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-04-20T21:51:23.595 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28896.log.gz 2024-04-20T21:51:23.596 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47309.log 2024-04-20T21:51:23.597 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42683.log.gz 2024-04-20T21:51:23.598 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52934.log 2024-04-20T21:51:23.598 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-04-20T21:51:23.599 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47309.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47309.log.gz 2024-04-20T21:51:23.599 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30611.log 2024-04-20T21:51:23.600 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23519.log 2024-04-20T21:51:23.600 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44808.log 2024-04-20T21:51:23.601 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52934.log.gz 2024-04-20T21:51:23.601 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46871.log 2024-04-20T21:51:23.601 INFO:teuthology.orchestra.run.smithi102.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30611.log.gz 2024-04-20T21:51:23.602 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-04-20T21:51:23.603 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44808.log.gz 2024-04-20T21:51:23.604 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36989.log 2024-04-20T21:51:23.604 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46871.log.gz 2024-04-20T21:51:23.605 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44706.log 2024-04-20T21:51:23.605 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-04-20T21:51:23.606 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37166.log 2024-04-20T21:51:23.606 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36989.log.gz 2024-04-20T21:51:23.607 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39052.log 2024-04-20T21:51:23.608 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44706.log.gz 2024-04-20T21:51:23.609 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37166.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37166.log.gz 2024-04-20T21:51:23.609 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-04-20T21:51:23.609 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51274.log 2024-04-20T21:51:23.610 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39052.log.gz 2024-04-20T21:51:23.610 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38477.log 2024-04-20T21:51:23.611 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-04-20T21:51:23.612 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48794.log 2024-04-20T21:51:23.612 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51274.log.gz 2024-04-20T21:51:23.613 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38477.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38477.log.gz 2024-04-20T21:51:23.613 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47779.log 2024-04-20T21:51:23.614 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45502.log 2024-04-20T21:51:23.615 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48794.log.gz 2024-04-20T21:51:23.616 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51502.log 2024-04-20T21:51:23.616 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47779.log.gz 2024-04-20T21:51:23.617 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.23519.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23519.log.gz 2024-04-20T21:51:23.618 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37397.log 2024-04-20T21:51:23.618 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45502.log.gz 2024-04-20T21:51:23.619 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51502.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51502.log.gz 2024-04-20T21:51:23.619 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45951.log 2024-04-20T21:51:23.619 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22472.log 2024-04-20T21:51:23.620 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37397.log.gz 2024-04-20T21:51:23.620 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30215.log 2024-04-20T21:51:23.621 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45951.log.gz 2024-04-20T21:51:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.22472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50529.log 2024-04-20T21:51:23.622 INFO:teuthology.orchestra.run.smithi102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22472.log.gz 2024-04-20T21:51:23.623 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49842.log 2024-04-20T21:51:23.623 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30215.log.gz 2024-04-20T21:51:23.624 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50529.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T21:51:23.624 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50529.log.gz 2024-04-20T21:51:23.625 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45681.log 2024-04-20T21:51:23.625 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49842.log.gz 2024-04-20T21:51:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36331.log 2024-04-20T21:51:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31211.log 2024-04-20T21:51:23.628 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45681.log.gz 2024-04-20T21:51:23.629 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36331.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T21:51:23.629 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36331.log.gz 2024-04-20T21:51:23.629 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43452.log 2024-04-20T21:51:23.631 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52850.log 2024-04-20T21:51:23.631 INFO:teuthology.orchestra.run.smithi102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31211.log.gz 2024-04-20T21:51:23.632 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43452.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43452.log.gz 2024-04-20T21:51:23.632 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44027.log 2024-04-20T21:51:23.633 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46608.log 2024-04-20T21:51:23.633 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52850.log.gz 2024-04-20T21:51:23.634 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34399.log 2024-04-20T21:51:23.634 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44027.log.gz 2024-04-20T21:51:23.635 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33071.log 2024-04-20T21:51:23.636 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46608.log.gz 2024-04-20T21:51:23.637 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34399.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34399.log.gz -5 2024-04-20T21:51:23.637 INFO:teuthology.orchestra.run.smithi102.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44526.log 2024-04-20T21:51:23.638 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28317.log 2024-04-20T21:51:23.638 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33071.log.gz 2024-04-20T21:51:23.639 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47225.log 2024-04-20T21:51:23.639 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44526.log.gz 2024-04-20T21:51:23.640 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49884.log 2024-04-20T21:51:23.641 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28317.log.gz 2024-04-20T21:51:23.642 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47225.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47225.log.gz 2024-04-20T21:51:23.642 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46143.log 2024-04-20T21:51:23.643 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30383.log 2024-04-20T21:51:23.643 INFO:teuthology.orchestra.run.smithi102.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49884.log.gz 2024-04-20T21:51:23.644 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44069.log 2024-04-20T21:51:23.644 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46143.log.gz 2024-04-20T21:51:23.645 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43757.log 2024-04-20T21:51:23.645 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30383.log.gz 2024-04-20T21:51:23.646 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44069.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44069.log.gz 2024-04-20T21:51:23.646 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28463.log 2024-04-20T21:51:23.648 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-04-20T21:51:23.648 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43757.log.gz 2024-04-20T21:51:23.649 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50794.log 2024-04-20T21:51:23.649 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28463.log.gz 2024-04-20T21:51:23.650 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28377.log 2024-04-20T21:51:23.650 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-04-20T21:51:23.651 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50794.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50794.log.gz 2024-04-20T21:51:23.651 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-04-20T21:51:23.652 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46784.log 2024-04-20T21:51:23.653 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28377.log.gz 2024-04-20T21:51:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49054.log 2024-04-20T21:51:23.654 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-04-20T21:51:23.655 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50710.log 2024-04-20T21:51:23.655 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46784.log.gz 2024-04-20T21:51:23.656 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53194.log 2024-04-20T21:51:23.656 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49054.log.gz 2024-04-20T21:51:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41670.log 2024-04-20T21:51:23.657 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50710.log.gz 2024-04-20T21:51:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36247.log 2024-04-20T21:51:23.658 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53194.log.gz 2024-04-20T21:51:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49411.log 2024-04-20T21:51:23.660 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41670.log.gz 2024-04-20T21:51:23.660 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36247.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.36247.log.gz /var/log/ceph/ceph-client.admin.52454.log 2024-04-20T21:51:23.661 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-20T21:51:23.661 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42452.log 2024-04-20T21:51:23.662 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49411.log.gz 2024-04-20T21:51:23.663 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30871.log 2024-04-20T21:51:23.663 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52454.log.gz 2024-04-20T21:51:23.664 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31799.log 2024-04-20T21:51:23.664 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42452.log.gz 2024-04-20T21:51:23.665 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30871.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30871.log.gz 2024-04-20T21:51:23.666 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41899.log 2024-04-20T21:51:23.667 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36606.log 2024-04-20T21:51:23.667 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31799.log.gz 2024-04-20T21:51:23.668 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49327.log 2024-04-20T21:51:23.668 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41899.log.gz 2024-04-20T21:51:23.669 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51022.log 2024-04-20T21:51:23.670 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36606.log.gz 2024-04-20T21:51:23.671 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49327.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49327.log.gz 2024-04-20T21:51:23.671 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50322.log 2024-04-20T21:51:23.671 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2024-04-20T21:51:23.672 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51022.log.gz 2024-04-20T21:51:23.673 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49716.log 2024-04-20T21:51:23.673 INFO:teuthology.orchestra.run.smithi102.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50322.log.gz 2024-04-20T21:51:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30527.log 2024-04-20T21:51:23.675 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2024-04-20T21:51:23.675 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49716.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49716.log.gz 2024-04-20T21:51:23.676 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44442.log 2024-04-20T21:51:23.676 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52892.log 2024-04-20T21:51:23.677 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.30527.log.gz 2024-04-20T21:51:23.678 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44442.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-20T21:51:23.678 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44442.log.gz 2024-04-20T21:51:23.679 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44400.log 2024-04-20T21:51:23.679 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.3.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52892.log.gz 2024-04-20T21:51:23.680 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52370.log 2024-04-20T21:51:23.681 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52514.log 2024-04-20T21:51:23.681 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44400.log.gz 2024-04-20T21:51:23.682 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40117.log 2024-04-20T21:51:23.683 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52370.log.gz 2024-04-20T21:51:23.684 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52514.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52514.log.gz 2024-04-20T21:51:23.684 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44976.log 2024-04-20T21:51:23.685 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40163.log 2024-04-20T21:51:23.686 INFO:teuthology.orchestra.run.smithi102.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40117.log.gz 2024-04-20T21:51:23.687 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44976.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44976.log.gz 2024-04-20T21:51:23.687 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36860.log 2024-04-20T21:51:23.688 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41104.log 2024-04-20T21:51:23.689 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40163.log.gz 2024-04-20T21:51:23.690 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42893.log 2024-04-20T21:51:23.690 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36860.log.gz 2024-04-20T21:51:23.691 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41104.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41104.log.gz 2024-04-20T21:51:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45060.log 2024-04-20T21:51:23.692 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41964.log 2024-04-20T21:51:23.693 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42893.log.gz 2024-04-20T21:51:23.694 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45060.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45060.log.gz 2024-04-20T21:51:23.694 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37116.log 2024-04-20T21:51:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-04-20T21:51:23.700 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41964.log.gz 2024-04-20T21:51:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37116.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37116.log.gz 2024-04-20T21:51:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35756.log 2024-04-20T21:51:23.700 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45594.log 2024-04-20T21:51:23.701 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-04-20T21:51:23.701 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.35756.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35756.log.gz 2024-04-20T21:51:23.701 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32735.log 2024-04-20T21:51:23.702 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41239.log 2024-04-20T21:51:23.702 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45594.log.gz 2024-04-20T21:51:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32735.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32735.log.gz 2024-04-20T21:51:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34441.log 2024-04-20T21:51:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30876.log 2024-04-20T21:51:23.704 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41239.log.gz 2024-04-20T21:51:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34441.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34441.log.gz 2024-04-20T21:51:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48397.log 2024-04-20T21:51:23.707 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46931.log 2024-04-20T21:51:23.707 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30876.log.gz 2024-04-20T21:51:23.708 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48397.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 88.1% 2024-04-20T21:51:23.708 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48397.log.gz 2024-04-20T21:51:23.709 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28557.log 2024-04-20T21:51:23.710 INFO:teuthology.orchestra.run.smithi102.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46931.log.gz 2024-04-20T21:51:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39094.log 2024-04-20T21:51:23.712 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28557.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28557.log.gz 2024-04-20T21:51:23.713 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42554.log 2024-04-20T21:51:23.714 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32135.log 2024-04-20T21:51:23.715 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39094.log.gz 2024-04-20T21:51:23.715 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50896.log 2024-04-20T21:51:23.715 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42554.log.gz 2024-04-20T21:51:23.716 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48668.log 2024-04-20T21:51:23.717 INFO:teuthology.orchestra.run.smithi102.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.32135.log.gz 2024-04-20T21:51:23.718 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50896.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50896.log.gz 2024-04-20T21:51:23.718 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36008.log 2024-04-20T21:51:23.719 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2024-04-20T21:51:23.719 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48668.log.gz 2024-04-20T21:51:23.721 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30866.log 2024-04-20T21:51:23.721 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36008.log.gz 2024-04-20T21:51:23.722 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43182.log 2024-04-20T21:51:23.722 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2024-04-20T21:51:23.723 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42234.log 2024-04-20T21:51:23.724 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30866.log.gz 2024-04-20T21:51:23.725 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34906.log 2024-04-20T21:51:23.725 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43182.log.gz 2024-04-20T21:51:23.726 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46650.log 2024-04-20T21:51:23.726 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42234.log.gz 2024-04-20T21:51:23.727 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53152.log 2024-04-20T21:51:23.728 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34906.log.gz 2024-04-20T21:51:23.729 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30653.log 2024-04-20T21:51:23.729 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46650.log.gz 2024-04-20T21:51:23.730 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40331.log 2024-04-20T21:51:23.730 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53152.log.gz 2024-04-20T21:51:23.731 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52556.log 2024-04-20T21:51:23.732 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30653.log.gz 2024-04-20T21:51:23.733 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40331.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-20T21:51:23.733 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40331.log.gz 2024-04-20T21:51:23.734 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52556.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:51:23.734 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52556.log.gz 2024-04-20T21:51:23.735 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48126.log 2024-04-20T21:51:23.737 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52023.log 2024-04-20T21:51:23.738 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53278.log 2024-04-20T21:51:23.738 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48126.log.gz 2024-04-20T21:51:23.740 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52023.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52023.log.gz 2024-04-20T21:51:23.741 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53278.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53278.log.gz 2024-04-20T21:51:23.741 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51837.log 2024-04-20T21:51:23.742 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41544.log 2024-04-20T21:51:23.744 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33197.log 2024-04-20T21:51:23.744 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51837.log.gz 2024-04-20T21:51:23.746 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41544.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41544.log.gz 2024-04-20T21:51:23.747 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33197.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33197.log.gz 2024-04-20T21:51:23.747 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45222.log 2024-04-20T21:51:23.748 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30341.log 2024-04-20T21:51:23.750 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34672.log 2024-04-20T21:51:23.750 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45222.log.gz 2024-04-20T21:51:23.751 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30341.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30341.log.gz 2024-04-20T21:51:23.752 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34672.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34672.log.gz 2024-04-20T21:51:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51981.log 2024-04-20T21:51:23.754 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43060.log 2024-04-20T21:51:23.756 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31578.log 2024-04-20T21:51:23.756 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51981.log.gz 2024-04-20T21:51:23.757 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43060.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43060.log.gz 2024-04-20T21:51:23.758 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31578.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31578.log.gz 2024-04-20T21:51:23.758 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-20T21:51:23.760 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45460.log 2024-04-20T21:51:23.761 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52115.log 2024-04-20T21:51:23.763 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45460.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45460.log.gz 2024-04-20T21:51:23.763 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52115.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52115.log.gz 2024-04-20T21:51:23.764 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52244.log 2024-04-20T21:51:23.764 INFO:teuthology.orchestra.run.smithi102.stderr: 94.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-20T21:51:23.766 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.15203.log 2024-04-20T21:51:23.767 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52244.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33323.log 2024-04-20T21:51:23.767 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52244.log.gz 2024-04-20T21:51:23.768 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.tmp-client.admin.15203.log: 0.0%gzip -- replaced with /var/log/ceph/ceph.tmp-client.admin.15203.log.gz -5 2024-04-20T21:51:23.768 INFO:teuthology.orchestra.run.smithi102.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36905.log 2024-04-20T21:51:23.770 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45418.log 2024-04-20T21:51:23.770 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33323.log.gz 2024-04-20T21:51:23.771 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31409.log 2024-04-20T21:51:23.771 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36905.log.gz 2024-04-20T21:51:23.772 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47456.log 2024-04-20T21:51:23.773 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45418.log.gz 2024-04-20T21:51:23.774 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2024-04-20T21:51:23.774 INFO:teuthology.orchestra.run.smithi102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31409.log.gz 2024-04-20T21:51:23.775 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47456.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47456.log.gz 2024-04-20T21:51:23.776 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33365.log 2024-04-20T21:51:23.776 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gz 2024-04-20T21:51:23.777 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31199.log 2024-04-20T21:51:23.778 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41331.log 2024-04-20T21:51:23.779 INFO:teuthology.orchestra.run.smithi102.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33365.log.gz 2024-04-20T21:51:23.780 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31199.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31199.log.gz 2024-04-20T21:51:23.781 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29969.log 2024-04-20T21:51:23.781 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41331.log.gz 2024-04-20T21:51:23.782 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52976.log 2024-04-20T21:51:23.783 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38579.log 2024-04-20T21:51:23.783 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29969.log.gz 2024-04-20T21:51:23.785 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52976.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52976.log.gz 2024-04-20T21:51:23.785 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48626.log 2024-04-20T21:51:23.786 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38579.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38579.log.gz 2024-04-20T21:51:23.786 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44484.log 2024-04-20T21:51:23.788 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22369.log 2024-04-20T21:51:23.788 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48626.log.gz 2024-04-20T21:51:23.790 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44484.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44484.log.gz 2024-04-20T21:51:23.790 INFO:teuthology.orchestra.run.smithi102.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33155.log 2024-04-20T21:51:23.790 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.22369.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.22369.log.gz 2024-04-20T21:51:23.791 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48962.log 2024-04-20T21:51:23.792 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51190.log 2024-04-20T21:51:23.792 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33155.log.gz 2024-04-20T21:51:23.794 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48000.log 2024-04-20T21:51:23.794 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48962.log.gz 2024-04-20T21:51:23.795 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2024-04-20T21:51:23.795 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51190.log.gz 2024-04-20T21:51:23.796 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29433.log 2024-04-20T21:51:23.796 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48000.log.gz 2024-04-20T21:51:23.798 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30862.log 2024-04-20T21:51:23.798 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gz 2024-04-20T21:51:23.799 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38884.log 2024-04-20T21:51:23.801 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30862.log: 88.6% 92.4% -- replaced with /var/log/ceph/ceph-client.admin.29433.log.gz 2024-04-20T21:51:23.801 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30862.log.gz 2024-04-20T21:51:23.802 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38884.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38884.log.gz 2024-04-20T21:51:23.802 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28812.log 2024-04-20T21:51:23.804 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41197.log 2024-04-20T21:51:23.805 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44316.log 2024-04-20T21:51:23.805 INFO:teuthology.orchestra.run.smithi102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28812.log.gz 2024-04-20T21:51:23.806 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41197.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41197.log.gz 2024-04-20T21:51:23.808 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44316.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44316.log.gz 2024-04-20T21:51:23.808 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49495.log 2024-04-20T21:51:23.810 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28507.log 2024-04-20T21:51:23.811 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49495.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49495.log.gz 2024-04-20T21:51:23.812 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32861.log 2024-04-20T21:51:23.812 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28507.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28507.log.gz 2024-04-20T21:51:23.813 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46482.log 2024-04-20T21:51:23.814 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53460.log 2024-04-20T21:51:23.814 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32861.log.gz 2024-04-20T21:51:23.815 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29785.log 2024-04-20T21:51:23.816 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46482.log.gz 2024-04-20T21:51:23.817 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53236.log 2024-04-20T21:51:23.817 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53460.log.gz 2024-04-20T21:51:23.818 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51460.log 2024-04-20T21:51:23.818 INFO:teuthology.orchestra.run.smithi102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.29785.log.gz 2024-04-20T21:51:23.820 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36205.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53236.log.gz 2024-04-20T21:51:23.820 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-20T21:51:23.821 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51460.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51460.log.gz 2024-04-20T21:51:23.821 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45909.log 2024-04-20T21:51:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51316.log 2024-04-20T21:51:23.823 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36205.log.gz 2024-04-20T21:51:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45909.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45909.log.gz 2024-04-20T21:51:23.825 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-04-20T21:51:23.826 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51316.log.gz 2024-04-20T21:51:23.827 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42360.log 2024-04-20T21:51:23.828 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50668.log 2024-04-20T21:51:23.829 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-04-20T21:51:23.829 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42360.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42360.log.gz 2024-04-20T21:51:23.830 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-04-20T21:51:23.831 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50668.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48920.log 2024-04-20T21:51:23.831 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50668.log.gz 2024-04-20T21:51:23.832 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42809.log 2024-04-20T21:51:23.833 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-04-20T21:51:23.834 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48920.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48920.log.gz 2024-04-20T21:51:23.834 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31672.log 2024-04-20T21:51:23.836 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42809.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42809.log.gz 2024-04-20T21:51:23.836 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44850.log 2024-04-20T21:51:23.837 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31672.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31672.log.gz 2024-04-20T21:51:23.837 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34757.log 2024-04-20T21:51:23.838 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44934.log 2024-04-20T21:51:23.839 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44850.log.gz 2024-04-20T21:51:23.840 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34757.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34757.log.gz 2024-04-20T21:51:23.841 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46700.log 2024-04-20T21:51:23.841 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44934.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44934.log.gz 2024-04-20T21:51:23.842 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41460.log 2024-04-20T21:51:23.843 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46700.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-20T21:51:23.843 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46700.log.gz 2024-04-20T21:51:23.845 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41460.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41460.log.gz 2024-04-20T21:51:23.845 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36100.log 2024-04-20T21:51:23.847 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53320.log 2024-04-20T21:51:23.848 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36100.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36100.log.gz 2024-04-20T21:51:23.852 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37481.log 2024-04-20T21:51:23.852 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53320.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53320.log.gz 2024-04-20T21:51:23.852 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51148.log 2024-04-20T21:51:23.852 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37481.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37481.log.gz 2024-04-20T21:51:23.854 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43266.log 2024-04-20T21:51:23.854 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51148.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51148.log.gz 2024-04-20T21:51:23.855 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40423.log 2024-04-20T21:51:23.856 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43266.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43266.log.gz 2024-04-20T21:51:23.857 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49670.log/var/log/ceph/ceph-client.admin.40423.log: 2024-04-20T21:51:23.857 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40423.log.gz 2024-04-20T21:51:23.858 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30137.log 2024-04-20T21:51:23.860 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49670.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49670.log.gz 2024-04-20T21:51:23.861 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38797.log 2024-04-20T21:51:23.861 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30137.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30137.log.gz 2024-04-20T21:51:23.862 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36289.log 2024-04-20T21:51:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38797.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38797.log.gz 2024-04-20T21:51:23.864 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32093.log 2024-04-20T21:51:23.865 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36289.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36289.log.gz 2024-04-20T21:51:23.866 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2024-04-20T21:51:23.868 INFO:teuthology.orchestra.run.smithi193.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-20T21:51:23.870 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32093.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32093.log.gz 2024-04-20T21:51:23.870 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35924.log 2024-04-20T21:51:23.871 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50112.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz 2024-04-20T21:51:23.871 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33491.log 2024-04-20T21:51:23.873 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.35924.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35924.log.gz 2024-04-20T21:51:23.873 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2024-04-20T21:51:23.873 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33491.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33491.log.gz 2024-04-20T21:51:23.874 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51232.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2024-04-20T21:51:23.875 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34630.log 2024-04-20T21:51:23.877 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40205.log 2024-04-20T21:51:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34630.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34630.log.gz 2024-04-20T21:51:23.878 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48458.log 2024-04-20T21:51:23.879 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40205.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40205.log.gz 2024-04-20T21:51:23.880 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40973.log 2024-04-20T21:51:23.882 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48458.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48458.log.gz 2024-04-20T21:51:23.883 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46356.log 2024-04-20T21:51:23.883 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40973.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40973.log.gz 2024-04-20T21:51:23.884 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32903.log 2024-04-20T21:51:23.885 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46356.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46356.log.gz 2024-04-20T21:51:23.886 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41814.log 2024-04-20T21:51:23.887 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32903.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32903.log.gz 2024-04-20T21:51:23.888 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51605.log/var/log/ceph/ceph-client.admin.41814.log: 2024-04-20T21:51:23.889 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41814.log.gz 2024-04-20T21:51:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31757.log 2024-04-20T21:51:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51605.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51605.log.gz 2024-04-20T21:51:23.892 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35038.log 2024-04-20T21:51:23.893 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31757.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31757.log.gz 2024-04-20T21:51:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43494.log 2024-04-20T21:51:23.895 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.35038.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35038.log.gz 2024-04-20T21:51:23.896 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48084.log 2024-04-20T21:51:23.896 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43494.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43494.log.gz 2024-04-20T21:51:23.897 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32009.log 2024-04-20T21:51:23.898 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48084.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48084.log.gz 2024-04-20T21:51:23.905 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51753.log 2024-04-20T21:51:23.906 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32009.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32009.log.gz 2024-04-20T21:51:23.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51753.log: -- /var/log/ceph/ceph-client.admin.31208.log 2024-04-20T21:51:23.906 INFO:teuthology.orchestra.run.smithi102.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51753.log.gz 2024-04-20T21:51:23.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36373.log 2024-04-20T21:51:23.906 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31208.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31208.log.gz 2024-04-20T21:51:23.906 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43410.log 2024-04-20T21:51:23.908 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36373.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36373.log.gz 2024-04-20T21:51:23.908 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40247.log 2024-04-20T21:51:23.908 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43410.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43410.log.gz 2024-04-20T21:51:23.909 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28089.log 2024-04-20T21:51:23.910 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40247.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40247.log.gz 2024-04-20T21:51:23.911 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-04-20T21:51:23.912 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28089.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28089.log.gz 2024-04-20T21:51:23.914 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41415.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-04-20T21:51:23.914 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30485.log 2024-04-20T21:51:23.916 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50938.log 2024-04-20T21:51:23.917 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30485.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30485.log.gz 2024-04-20T21:51:23.918 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2024-04-20T21:51:23.920 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50938.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50938.log.gz 2024-04-20T21:51:23.920 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44619.log 2024-04-20T21:51:23.921 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51418.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2024-04-20T21:51:23.922 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37634.log 2024-04-20T21:51:23.923 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44619.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44619.log.gz 2024-04-20T21:51:23.924 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41586.log 2024-04-20T21:51:23.925 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37634.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37634.log.gz 2024-04-20T21:51:23.926 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28188.log 2024-04-20T21:51:23.926 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41586.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41586.log.gz 2024-04-20T21:51:23.927 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47394.log 2024-04-20T21:51:23.928 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28188.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28188.log.gz 2024-04-20T21:51:23.929 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49968.log 2024-04-20T21:51:23.930 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47394.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47394.log.gz 2024-04-20T21:51:23.931 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49096.log 2024-04-20T21:51:23.932 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49968.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49968.log.gz 2024-04-20T21:51:23.933 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43631.log 2024-04-20T21:51:23.935 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49096.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49096.log.gz 2024-04-20T21:51:23.939 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47958.log 2024-04-20T21:51:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43631.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz 2024-04-20T21:51:24.006 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34799.log 2024-04-20T21:51:24.007 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47958.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47958.log.gz 2024-04-20T21:51:24.007 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2024-04-20T21:51:24.055 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34799.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34799.log.gz 2024-04-20T21:51:24.055 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51106.log/var/log/ceph/ceph-client.admin.46440.log: 2024-04-20T21:51:24.056 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2024-04-20T21:51:24.056 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40512.log 2024-04-20T21:51:24.056 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51106.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51106.log.gz 2024-04-20T21:51:24.056 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5/var/log/ceph/ceph-client.admin.40512.log: --verbose -- /var/log/ceph/ceph-client.admin.38968.log 2024-04-20T21:51:24.056 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40512.log.gz 2024-04-20T21:51:24.056 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28685.log 2024-04-20T21:51:24.057 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38968.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38968.log.gz 2024-04-20T21:51:24.057 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47916.log/var/log/ceph/ceph-client.admin.28685.log: 2024-04-20T21:51:24.057 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28685.log.gz 2024-04-20T21:51:24.057 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2024-04-20T21:51:24.057 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47916.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47916.log.gz 2024-04-20T21:51:24.057 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47099.log 2024-04-20T21:51:24.057 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49012.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2024-04-20T21:51:24.057 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33449.log 2024-04-20T21:51:24.058 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47099.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47099.log.gz 2024-04-20T21:51:24.058 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22104.log 2024-04-20T21:51:24.058 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33449.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33449.log.gz 2024-04-20T21:51:24.058 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40596.log 2024-04-20T21:51:24.058 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.22104.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.22104.log.gz 2024-04-20T21:51:24.058 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40596.log: /var/log/ceph/ceph-client.admin.38926.log 2024-04-20T21:51:24.058 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40596.log.gz 2024-04-20T21:51:24.058 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28643.log 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38926.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38926.log.gz 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.28643.log: /var/log/ceph/ceph-client.admin.37439.log 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28643.log.gz 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33575.log 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37439.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37439.log.gz 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr:gzip/var/log/ceph/ceph-client.admin.33575.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30299.log 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33575.log.gz 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42318.log 2024-04-20T21:51:24.059 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30299.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30299.log.gz 2024-04-20T21:51:24.060 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42318.log: /var/log/ceph/ceph-client.admin.30874.log 2024-04-20T21:51:24.060 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42318.log.gz 2024-04-20T21:51:24.060 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43350.log 2024-04-20T21:51:24.060 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30874.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30874.log.gz 2024-04-20T21:51:24.060 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43350.log: /var/log/ceph/ceph-client.admin.42599.log 2024-04-20T21:51:24.060 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43350.log.gz 2024-04-20T21:51:24.060 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38713.log 2024-04-20T21:51:24.060 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42599.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42599.log.gz 2024-04-20T21:51:24.061 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38519.log 2024-04-20T21:51:24.061 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38713.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38713.log.gz 2024-04-20T21:51:24.061 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44577.log 2024-04-20T21:51:24.061 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38519.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38519.log.gz 2024-04-20T21:51:24.061 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43985.log 2024-04-20T21:51:24.061 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44577.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44577.log.gz 2024-04-20T21:51:24.061 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43308.log 2024-04-20T21:51:24.061 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43985.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43985.log.gz 2024-04-20T21:51:24.062 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43308.log: /var/log/ceph/ceph-client.admin.42192.log 2024-04-20T21:51:24.062 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43308.log.gz 2024-04-20T21:51:24.062 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31841.log 2024-04-20T21:51:24.062 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42192.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42192.log.gz 2024-04-20T21:51:24.062 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30443.log 2024-04-20T21:51:24.062 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31841.log.gz 2024-04-20T21:51:24.062 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45993.log 2024-04-20T21:51:24.062 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30443.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30443.log.gz 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45993.log: -- /var/log/ceph/ceph-client.admin.29659.log 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45993.log.gz 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42026.log 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29659.log.gz 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42026.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42026.log.gz 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose --/var/log/ceph/ceph-mds.e.log: /var/log/ceph/ceph-client.admin.41712.log 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53802.log 2024-04-20T21:51:24.063 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41712.log.gz 2024-04-20T21:51:24.064 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30877.log 2024-04-20T21:51:24.064 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52328.log 2024-04-20T21:51:24.064 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30877.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.30877.log.gz 2024-04-20T21:51:24.064 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34307.log 2024-04-20T21:51:24.064 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52328.log: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.53802.log.gz 2024-04-20T21:51:24.064 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52328.log.gz 2024-04-20T21:51:24.064 INFO:teuthology.orchestra.run.smithi102.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-20T21:51:24.064 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46973.log 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34307.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34307.log.gz 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38621.log 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46973.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46973.log.gz 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49800.log 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38621.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38621.log.gz 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53563.log 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49800.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49800.log.gz 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47551.log 2024-04-20T21:51:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53563.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53563.log.gz 2024-04-20T21:51:24.066 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-04-20T21:51:24.066 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47551.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47551.log.gz 2024-04-20T21:51:24.066 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36499.log 2024-04-20T21:51:24.066 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34139.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-04-20T21:51:24.066 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45334.log 2024-04-20T21:51:24.066 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36499.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36499.log.gz 2024-04-20T21:51:24.066 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45334.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-04-20T21:51:24.066 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36050.log 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36415.log 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36050.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36050.log.gz 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50280.log 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36415.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36415.log.gz 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37268.log 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50280.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50280.log.gz 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47057.log 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37268.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37268.log.gz 2024-04-20T21:51:24.067 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37208.log 2024-04-20T21:51:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47057.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47057.log.gz 2024-04-20T21:51:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33281.log 2024-04-20T21:51:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37208.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37208.log.gz 2024-04-20T21:51:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-04-20T21:51:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33281.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33281.log.gz 2024-04-20T21:51:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45636.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-04-20T21:51:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50154.log 2024-04-20T21:51:24.068 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40806.log 2024-04-20T21:51:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50154.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50154.log.gz 2024-04-20T21:51:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47141.log 2024-04-20T21:51:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40806.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40806.log.gz 2024-04-20T21:51:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44358.log 2024-04-20T21:51:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47141.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47141.log.gz 2024-04-20T21:51:24.076 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50052.log 2024-04-20T21:51:24.076 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44358.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44358.log.gz 2024-04-20T21:51:24.076 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32051.log 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50052.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50052.log.gz 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32051.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32051.log.gz 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33533.log 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50238.log 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33533.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33533.log.gz 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46078.log 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50238.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50238.log.gz 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45018.log 2024-04-20T21:51:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46078.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46078.log.gz 2024-04-20T21:51:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50010.log 2024-04-20T21:51:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45018.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45018.log.gz 2024-04-20T21:51:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28599.log 2024-04-20T21:51:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50010.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50010.log.gz 2024-04-20T21:51:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28599.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28599.log.gz 2024-04-20T21:51:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36541.log 2024-04-20T21:51:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38393.log 2024-04-20T21:51:24.079 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36541.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36541.log.gz 2024-04-20T21:51:24.079 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51711.log 2024-04-20T21:51:24.082 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38393.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38393.log.gz 2024-04-20T21:51:24.082 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43943.log 2024-04-20T21:51:24.082 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.51711.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51711.log.gz 2024-04-20T21:51:24.083 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45146.log 2024-04-20T21:51:24.083 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43943.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43943.log.gz 2024-04-20T21:51:24.083 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52157.log 2024-04-20T21:51:24.083 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45146.log.gz 2024-04-20T21:51:24.084 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40075.log 2024-04-20T21:51:24.084 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52157.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52157.log.gz 2024-04-20T21:51:24.086 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38755.log 2024-04-20T21:51:24.086 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40075.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40075.log.gz 2024-04-20T21:51:24.087 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38755.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38755.log.gz 2024-04-20T21:51:24.088 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53068.log 2024-04-20T21:51:24.089 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47267.log 2024-04-20T21:51:24.091 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53068.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53068.log.gz 2024-04-20T21:51:24.092 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40740.log 2024-04-20T21:51:24.093 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47267.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47267.log.gz 2024-04-20T21:51:24.093 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34055.log/var/log/ceph/ceph-client.admin.40740.log: 2024-04-20T21:51:24.094 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40740.log.gz 2024-04-20T21:51:24.095 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50364.log 2024-04-20T21:51:24.096 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34055.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34055.log.gz 2024-04-20T21:51:24.097 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46398.log 2024-04-20T21:51:24.099 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50364.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50364.log.gz 2024-04-20T21:51:24.099 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-04-20T21:51:24.100 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46398.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46398.log.gz 2024-04-20T21:51:24.101 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2024-04-20T21:51:24.102 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43586.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-04-20T21:51:24.103 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43799.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2024-04-20T21:51:24.104 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53360.log 2024-04-20T21:51:24.105 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40868.log 2024-04-20T21:51:24.112 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53360.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53360.log.gz 2024-04-20T21:51:24.112 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30053.log 2024-04-20T21:51:24.112 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40868.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40868.log.gz 2024-04-20T21:51:24.112 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46829.log 2024-04-20T21:51:24.112 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30053.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30053.log.gz 2024-04-20T21:51:24.112 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48168.log 2024-04-20T21:51:24.114 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46829.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46829.log.gz 2024-04-20T21:51:24.114 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32987.log 2024-04-20T21:51:24.114 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48168.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48168.log.gz 2024-04-20T21:51:24.115 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32987.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32987.log.gz 2024-04-20T21:51:24.116 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52598.log 2024-04-20T21:51:24.118 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37523.log 2024-04-20T21:51:24.119 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52598.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52598.log.gz 2024-04-20T21:51:24.120 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44892.log 2024-04-20T21:51:24.121 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37523.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37523.log.gz 2024-04-20T21:51:24.121 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52724.log 2024-04-20T21:51:24.122 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44892.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44892.log.gz 2024-04-20T21:51:24.123 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31883.log 2024-04-20T21:51:24.125 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52724.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52724.log.gz 2024-04-20T21:51:24.125 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34097.log 2024-04-20T21:51:24.126 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31883.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31883.log.gz 2024-04-20T21:51:24.127 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-04-20T21:51:24.128 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34097.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34097.log.gz 2024-04-20T21:51:24.129 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43544.log: /var/log/ceph/ceph-client.admin.36457.log 2024-04-20T21:51:24.130 INFO:teuthology.orchestra.run.smithi102.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-04-20T21:51:24.131 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2024-04-20T21:51:24.132 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36457.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36457.log.gz 2024-04-20T21:51:24.133 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29743.log/var/log/ceph/ceph-client.admin.48836.log: 2024-04-20T21:51:24.133 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2024-04-20T21:51:24.134 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52640.log 2024-04-20T21:51:24.136 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29743.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29743.log.gz 2024-04-20T21:51:24.137 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49243.log 2024-04-20T21:51:24.137 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52640.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52640.log.gz 2024-04-20T21:51:24.138 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52286.log 2024-04-20T21:51:24.140 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49243.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49243.log.gz 2024-04-20T21:51:24.140 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37073.log 2024-04-20T21:51:24.142 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52286.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52286.log.gz 2024-04-20T21:51:24.144 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37073.log: -- /var/log/ceph/ceph-client.admin.44232.log 2024-04-20T21:51:24.144 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37073.log.gz 2024-04-20T21:51:24.145 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52766.log 2024-04-20T21:51:24.145 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.44232.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz 2024-04-20T21:51:24.146 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40638.log/var/log/ceph/ceph-client.admin.52766.log: 2024-04-20T21:51:24.147 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52766.log.gz 2024-04-20T21:51:24.148 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46524.log 2024-04-20T21:51:24.149 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40638.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40638.log.gz 2024-04-20T21:51:24.150 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42725.log 2024-04-20T21:51:24.150 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46524.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46524.log.gz 2024-04-20T21:51:24.151 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49926.log 2024-04-20T21:51:24.152 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42725.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42725.log.gz 2024-04-20T21:51:24.153 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43673.log 2024-04-20T21:51:24.154 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49926.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49926.log.gz 2024-04-20T21:51:24.155 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37748.log 2024-04-20T21:51:24.156 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43673.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43673.log.gz 2024-04-20T21:51:24.157 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43841.log 2024-04-20T21:51:24.158 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37748.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37748.log.gz 2024-04-20T21:51:24.159 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31203.log 2024-04-20T21:51:24.160 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43841.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43841.log.gz 2024-04-20T21:51:24.160 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42641.log 2024-04-20T21:51:24.162 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31203.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31203.log.gz 2024-04-20T21:51:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39136.log 2024-04-20T21:51:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42641.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42641.log.gz 2024-04-20T21:51:24.164 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29869.log 2024-04-20T21:51:24.165 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39136.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39136.log.gz 2024-04-20T21:51:24.166 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5/var/log/ceph/ceph-client.admin.29869.log: --verbose -- /var/log/ceph/ceph-client.admin.30011.log 2024-04-20T21:51:24.167 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29869.log.gz 2024-04-20T21:51:24.168 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51669.log 2024-04-20T21:51:24.170 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30011.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30011.log.gz 2024-04-20T21:51:24.171 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37313.log/var/log/ceph/ceph-client.admin.51669.log: 2024-04-20T21:51:24.171 INFO:teuthology.orchestra.run.smithi102.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51669.log.gz 2024-04-20T21:51:24.172 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41281.log 2024-04-20T21:51:24.173 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37313.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37313.log.gz 2024-04-20T21:51:24.174 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5/var/log/ceph/ceph-client.admin.41281.log: --verbose -- /var/log/ceph/ceph-client.admin.41062.log 2024-04-20T21:51:24.175 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41281.log.gz 2024-04-20T21:51:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32945.log 2024-04-20T21:51:24.177 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.41062.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41062.log.gz 2024-04-20T21:51:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32582.log 2024-04-20T21:51:24.178 INFO:teuthology.orchestra.run.smithi102.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32945.log.gz 2024-04-20T21:51:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-04-20T21:51:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.0.32582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.32582.log.gz 2024-04-20T21:51:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38663.log 2024-04-20T21:51:24.183 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45783.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-04-20T21:51:24.183 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-20T21:51:24.184 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38663.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38663.log.gz 2024-04-20T21:51:24.185 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-04-20T21:51:24.188 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.48252.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-04-20T21:51:24.189 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36160.log 2024-04-20T21:51:24.191 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.36160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47593.log 2024-04-20T21:51:24.192 INFO:teuthology.orchestra.run.smithi102.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36160.log.gz 2024-04-20T21:51:24.194 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50854.log 2024-04-20T21:51:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52065.log 2024-04-20T21:51:24.196 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50854.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50854.log.gz 2024-04-20T21:51:24.197 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47593.log.gz 2024-04-20T21:51:24.199 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29701.log 2024-04-20T21:51:24.201 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52065.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52065.log.gz 2024-04-20T21:51:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33029.log 2024-04-20T21:51:24.204 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29701.log.gz 2024-04-20T21:51:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34486.log 2024-04-20T21:51:24.208 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33029.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33029.log.gz 2024-04-20T21:51:24.209 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34486.log: /var/log/ceph/ceph-client.admin.35882.log 2024-04-20T21:51:24.210 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34486.log.gz 2024-04-20T21:51:24.212 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.35882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30095.log 2024-04-20T21:51:24.213 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35882.log.gz 2024-04-20T21:51:24.215 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.30095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44134.log 2024-04-20T21:51:24.216 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45292.log 2024-04-20T21:51:24.218 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4%/var/log/ceph/ceph-client.admin.44134.log: -- replaced with /var/log/ceph/ceph-client.admin.30095.log.gz 2024-04-20T21:51:24.220 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.45292.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44134.log.gz 2024-04-20T21:51:24.220 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36947.log 2024-04-20T21:51:24.221 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33113.log 2024-04-20T21:51:24.223 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7%/var/log/ceph/ceph-client.admin.36947.log: -- replaced with /var/log/ceph/ceph-client.admin.45292.log.gz 2024-04-20T21:51:24.224 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36947.log.gz 2024-04-20T21:51:24.226 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34265.log 2024-04-20T21:51:24.229 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34265.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33113.log.gz 2024-04-20T21:51:24.229 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42150.log 2024-04-20T21:51:24.230 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34265.log.gz 2024-04-20T21:51:24.232 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43883.log 2024-04-20T21:51:24.235 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43883.log: /var/log/ceph/ceph-client.admin.42150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38841.log 2024-04-20T21:51:24.236 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42150.log.gz 2024-04-20T21:51:24.237 INFO:teuthology.orchestra.run.smithi102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz 2024-04-20T21:51:24.240 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32777.log 2024-04-20T21:51:24.240 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38841.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38841.log.gz 2024-04-20T21:51:24.241 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51879.log 2024-04-20T21:51:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49369.log 2024-04-20T21:51:24.244 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7%/var/log/ceph/ceph-client.admin.51879.log: -- replaced with /var/log/ceph/ceph-client.admin.32777.log.gz 2024-04-20T21:51:24.246 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49369.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49285.log 2024-04-20T21:51:24.246 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51879.log.gz 2024-04-20T21:51:24.247 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37355.log 2024-04-20T21:51:24.248 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42935.log 2024-04-20T21:51:24.249 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49369.log.gz 2024-04-20T21:51:24.250 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37355.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49285.log.gz 2024-04-20T21:51:24.252 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37355.log.gz 2024-04-20T21:51:24.253 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43122.log 2024-04-20T21:51:24.256 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.42935.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42935.log.gz 2024-04-20T21:51:24.258 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.43122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52682.log 2024-04-20T21:51:24.258 INFO:teuthology.orchestra.run.smithi102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43122.log.gz 2024-04-20T21:51:24.259 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29008.log 2024-04-20T21:51:24.260 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52682.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52682.log.gz 2024-04-20T21:51:24.262 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2024-04-20T21:51:24.263 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52808.log 2024-04-20T21:51:24.267 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.29008.log: /var/log/ceph/ceph-client.admin.41628.log: 88.6% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29008.log.gz 2024-04-20T21:51:24.268 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2024-04-20T21:51:24.269 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.52808.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49159.log 2024-04-20T21:51:24.272 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.49159.log: -- replaced with /var/log/ceph/ceph-client.admin.52808.log.gz 2024-04-20T21:51:24.272 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48710.log 2024-04-20T21:51:24.274 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39196.log 2024-04-20T21:51:24.276 INFO:teuthology.orchestra.run.smithi102.stderr: 88.8%/var/log/ceph/ceph-client.admin.48710.log: -- replaced with /var/log/ceph/ceph-client.admin.49159.log.gz 2024-04-20T21:51:24.277 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39196.log: 87.8% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48710.log.gz 2024-04-20T21:51:24.278 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39196.log.gz 2024-04-20T21:51:24.278 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47015.log 2024-04-20T21:51:24.281 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52199.log 2024-04-20T21:51:24.282 INFO:teuthology.orchestra.run.smithi102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47015.log.gz 2024-04-20T21:51:24.283 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50196.log 2024-04-20T21:51:24.288 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50196.log: /var/log/ceph/ceph-client.admin.52199.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42276.log 2024-04-20T21:51:24.288 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52199.log.gz 2024-04-20T21:51:24.289 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50196.log.gz 2024-04-20T21:51:24.290 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40698.log 2024-04-20T21:51:24.293 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40698.log: /var/log/ceph/ceph-client.admin.42276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30257.log 2024-04-20T21:51:24.295 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42276.log.gz 2024-04-20T21:51:24.296 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40698.log.gz 2024-04-20T21:51:24.297 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34181.log 2024-04-20T21:51:24.301 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34181.log: /var/log/ceph/ceph-client.admin.30257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49201.log 2024-04-20T21:51:24.303 INFO:teuthology.orchestra.run.smithi102.stderr: 87.6% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30257.log.gz 2024-04-20T21:51:24.303 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34181.log.gz 2024-04-20T21:51:24.305 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50406.log 2024-04-20T21:51:24.308 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50406.log: /var/log/ceph/ceph-client.admin.49201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33407.log 2024-04-20T21:51:24.310 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49201.log.gz 2024-04-20T21:51:24.310 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50406.log.gz 2024-04-20T21:51:24.315 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53502.log 2024-04-20T21:51:24.315 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45741.log 2024-04-20T21:51:24.317 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33407.log: /var/log/ceph/ceph-client.admin.53502.log: 89.3%/var/log/ceph/ceph-client.admin.45741.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53502.log.gz 2024-04-20T21:51:24.318 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31967.log 2024-04-20T21:51:24.318 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33407.log.gz 2024-04-20T21:51:24.319 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45741.log.gz 2024-04-20T21:51:24.321 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34357.log 2024-04-20T21:51:24.325 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.34357.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31967.log.gz 2024-04-20T21:51:24.325 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40554.log 2024-04-20T21:51:24.326 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40381.log 2024-04-20T21:51:24.329 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1%/var/log/ceph/ceph-client.admin.40554.log: -- replaced with /var/log/ceph/ceph-client.admin.34357.log.gz 2024-04-20T21:51:24.332 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.40381.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50467.log 2024-04-20T21:51:24.334 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.50467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33239.log 2024-04-20T21:51:24.335 INFO:teuthology.orchestra.run.smithi102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40554.log.gz 2024-04-20T21:51:24.335 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40381.log.gz 2024-04-20T21:51:24.336 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28230.log 2024-04-20T21:51:24.337 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.33239.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33239.log.gz 2024-04-20T21:51:24.338 INFO:teuthology.orchestra.run.smithi102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50467.log.gz 2024-04-20T21:51:24.339 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31198.log 2024-04-20T21:51:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28230.log: /var/log/ceph/ceph-client.admin.31198.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28230.log.gz 2024-04-20T21:51:24.343 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47719.log 2024-04-20T21:51:24.345 INFO:teuthology.orchestra.run.smithi102.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46035.log 2024-04-20T21:51:24.345 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31198.log.gz 2024-04-20T21:51:24.346 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5/var/log/ceph/ceph-client.admin.47719.log: --verbose -- /var/log/ceph/ceph-client.admin.53622.log 2024-04-20T21:51:24.347 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.46035.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47719.log.gz 2024-04-20T21:51:24.348 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48520.log 2024-04-20T21:51:24.350 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.53622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32263.log 2024-04-20T21:51:24.351 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46035.log.gz 2024-04-20T21:51:24.355 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.48520.log: 88.1%/var/log/ceph/ceph-client.admin.32263.log: gzip -5 95.6% -- replaced with /var/log/ceph/ceph-client.admin.53622.log.gz 2024-04-20T21:51:24.355 INFO:teuthology.orchestra.run.smithi102.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31212.log 2024-04-20T21:51:24.355 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32263.log.gz 2024-04-20T21:51:24.355 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48520.log.gz 2024-04-20T21:51:24.358 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31212.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31212.log.gz 2024-04-20T21:51:24.358 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28275.log 2024-04-20T21:51:24.360 INFO:teuthology.orchestra.run.smithi193.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-20T21:51:24.361 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.28275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47829.log 2024-04-20T21:51:24.364 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.47829.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28275.log.gz 2024-04-20T21:51:24.364 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48878.log 2024-04-20T21:51:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31535.log 2024-04-20T21:51:24.368 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4%/var/log/ceph/ceph-client.admin.48878.log: -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2024-04-20T21:51:24.369 INFO:teuthology.orchestra.run.smithi102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48878.log.gz 2024-04-20T21:51:24.370 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51939.log 2024-04-20T21:51:24.374 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.31535.log: /var/log/ceph/ceph-client.admin.51939.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31535.log.gz 2024-04-20T21:51:24.375 INFO:teuthology.orchestra.run.smithi102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51939.log.gz 2024-04-20T21:51:24.748 INFO:teuthology.orchestra.run.smithi193.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-20T21:51:25.452 INFO:teuthology.orchestra.run.smithi102.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-20T21:51:29.873 INFO:teuthology.orchestra.run.smithi193.stderr: 94.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-20T21:51:30.603 INFO:teuthology.orchestra.run.smithi193.stderr: 94.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-20T21:51:31.407 INFO:teuthology.orchestra.run.smithi193.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-20T21:51:36.752 INFO:teuthology.orchestra.run.smithi193.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-20T21:51:37.816 INFO:teuthology.orchestra.run.smithi193.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-20T21:51:39.672 INFO:teuthology.orchestra.run.smithi193.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-20T21:51:39.934 INFO:teuthology.orchestra.run.smithi102.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-20T21:51:41.629 INFO:teuthology.orchestra.run.smithi102.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-20T21:51:42.750 INFO:teuthology.orchestra.run.smithi193.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-20T21:51:42.752 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-04-20T21:51:42.752 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m19.268s 2024-04-20T21:51:42.752 INFO:teuthology.orchestra.run.smithi193.stderr:user 1m22.918s 2024-04-20T21:51:42.752 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m2.574s 2024-04-20T21:51:45.219 INFO:teuthology.orchestra.run.smithi102.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-20T21:51:46.706 INFO:teuthology.orchestra.run.smithi102.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-20T21:52:22.249 INFO:teuthology.orchestra.run.smithi102.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-20T21:52:48.089 INFO:teuthology.orchestra.run.smithi102.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-20T21:52:48.092 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-20T21:52:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:real 1m24.608s 2024-04-20T21:52:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:user 3m26.592s 2024-04-20T21:52:48.092 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m16.837s 2024-04-20T21:52:48.093 INFO:tasks.ceph:Archiving logs... 2024-04-20T21:52:48.094 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/remote/smithi102/log 2024-04-20T21:52:48.094 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:53:17.346 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/remote/smithi193/log 2024-04-20T21:53:17.348 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:53:28.363 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-20T21:53:28.381 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T21:53:28.999 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-20T21:53:29.004 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T21:53:29.004 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:53:29.020 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:53:29.155 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T21:53:29.178 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T21:53:29.179 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:53:29.187 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:53:29.204 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:53:29.209 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:53:29.209 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-04-20T21:53:29.209 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 -883us[ -883us] +/- 66ms 2024-04-20T21:53:29.210 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv02.front.sepia.ceph.com 3 8 377 36 -587us[ -594us] +/- 66ms 2024-04-20T21:53:29.210 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +1468us[+1461us] +/- 74ms 2024-04-20T21:53:29.210 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:53:29.234 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:53:29.244 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:53:29.244 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-20T21:53:29.245 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 -845us[ -830us] +/- 66ms 2024-04-20T21:53:29.245 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv02.front.sepia.ceph.com 3 8 377 30 -531us[ -515us] +/- 66ms 2024-04-20T21:53:29.245 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +1493us[+1493us] +/- 74ms 2024-04-20T21:53:29.245 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:53:29.246 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T21:53:29.256 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T21:53:29.256 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T21:53:29.281 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:53:29.443 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:53:29.496 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:53:29.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:53:29.587 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:53:29.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:53:29.616 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T21:53:29.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T21:53:29.652 INFO:teuthology.task.internal:Duration was 2570.078307 seconds 2024-04-20T21:53:29.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T21:53:29.693 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T21:53:29.694 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:53:29.701 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:53:29.742 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:53:29.826 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:53:30.268 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T21:53:30.268 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-04-20T21:53:30.268 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:53:30.330 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-04-20T21:53:30.330 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:53:30.355 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T21:53:30.355 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:53:30.360 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:53:30.434 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T21:53:30.434 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:53:30.440 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:53:30.617 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T21:53:30.628 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T21:53:30.628 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:53:30.700 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:53:30.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T21:53:30.735 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:30.741 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:30.789 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-04-20T21:53:30.834 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-04-20T21:53:30.901 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:30.945 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi102.front.sepia.ceph.com, flagging run as failed 2024-04-20T21:53:30.946 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:30.960 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi193.front.sepia.ceph.com, flagging run as failed 2024-04-20T21:53:30.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T21:53:30.971 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T21:53:30.972 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/remote/smithi102 2024-04-20T21:53:30.972 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:54:26.064 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2024-04-20T21:54:26.077 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414/remote/smithi102/coredump/1713649317.48459.core.gz: gzip compressed data, was "1713649317.48459.core", last modified: Sat Apr 20 21:53:31 2024, max compression, original size modulo 2^32 404869120 2024-04-20T21:54:26.078 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2024-04-20T21:54:27.847 INFO:teuthology.task.internal: core looks like: /tmp/tmpe0r6adae: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-04-20T21:54:27.892 INFO:teuthology.task.internal: dump_program: ceph-mds -f --cluster ceph -i a 2024-04-20T21:54:27.892 DEBUG:teuthology.orchestra.run.smithi102:> which 'ceph-mds -f --cluster ceph -i a' 2024-04-20T21:54:27.964 INFO:teuthology.orchestra.run.smithi102.stderr:/usr/bin/which: invalid option -- 'f' 2024-04-20T21:54:27.965 INFO:teuthology.orchestra.run.smithi102.stderr:/usr/bin/which: unrecognized option '--cluster' 2024-04-20T21:54:27.965 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-mds 2024-04-20T21:54:27.965 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph 2024-04-20T21:54:27.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:54:27.966 INFO:teuthology.orchestra.run.smithi102.stderr:/usr/bin/which: no a in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-04-20T21:54:27.967 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 1: "which 'ceph-mds -f --cluster ceph -i a'" 2024-04-20T21:54:27.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T21:54:28.001 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T21:54:28.002 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T21:54:28.048 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T21:54:28.048 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:54:28.060 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:54:28.078 INFO:teuthology.orchestra.run.smithi193.stdout: 262161 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest 2024-04-20T21:54:28.078 INFO:teuthology.orchestra.run.smithi193.stdout: 262162 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest/archive 2024-04-20T21:54:28.079 INFO:teuthology.orchestra.run.smithi193.stdout: 393219 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest/archive/audit 2024-04-20T21:54:28.079 INFO:teuthology.orchestra.run.smithi193.stdout: 393502 60 -rw------- 1 ubuntu ubuntu 57579 Apr 20 21:53 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-20T21:54:28.079 INFO:teuthology.orchestra.run.smithi193.stdout: 262164 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:54:28.079 INFO:teuthology.orchestra.run.smithi193.stdout: 262253 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 20 21:10 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-20T21:54:28.079 INFO:teuthology.orchestra.run.smithi193.stdout: 262173 60 -rw-r--r-- 1 ubuntu ubuntu 59477 Apr 20 21:53 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:54:28.079 INFO:teuthology.orchestra.run.smithi193.stdout: 262174 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 20 21:10 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-20T21:54:28.079 INFO:teuthology.orchestra.run.smithi193.stdout: 262163 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:43 /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:54:28.079 INFO:teuthology.orchestra.run.smithi193.stdout: 262277 104392 -rw------- 1 root root 406962176 Apr 20 21:43 /home/ubuntu/cephtest/archive/coredump/1713649405.29686.core 2024-04-20T21:54:28.080 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-20T21:54:28.112 INFO:teuthology.orchestra.run.smithi102.stdout: 393249 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest 2024-04-20T21:54:28.112 INFO:teuthology.orchestra.run.smithi102.stdout: 393250 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest/archive 2024-04-20T21:54:28.112 INFO:teuthology.orchestra.run.smithi102.stdout: 527061 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest/archive/audit 2024-04-20T21:54:28.112 INFO:teuthology.orchestra.run.smithi102.stdout: 527344 112 -rw------- 1 ubuntu ubuntu 112485 Apr 20 21:53 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-20T21:54:28.112 INFO:teuthology.orchestra.run.smithi102.stdout: 393252 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:54:28.112 INFO:teuthology.orchestra.run.smithi102.stdout: 393318 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 20 21:10 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-20T21:54:28.113 INFO:teuthology.orchestra.run.smithi102.stdout: 393253 88 -rw-r--r-- 1 ubuntu ubuntu 87289 Apr 20 21:53 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:54:28.113 INFO:teuthology.orchestra.run.smithi102.stdout: 393254 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 20 21:10 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-20T21:54:28.113 INFO:teuthology.orchestra.run.smithi102.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:47 /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:54:28.113 INFO:teuthology.orchestra.run.smithi102.stdout: 395254 108060 -rw------- 1 root root 404869120 Apr 20 21:41 /home/ubuntu/cephtest/archive/coredump/1713649317.48459.core 2024-04-20T21:54:28.113 INFO:teuthology.orchestra.run.smithi102.stdout: 395320 113920 -rw------- 1 root root 355807232 Apr 20 21:46 /home/ubuntu/cephtest/archive/coredump/1713649602.51606.core 2024-04-20T21:54:28.113 INFO:teuthology.orchestra.run.smithi102.stdout: 395337 13128 -rw------- 1 ubuntu ubuntu 16371712 Apr 20 21:47 /home/ubuntu/cephtest/archive/coredump/1713649625.51649.core 2024-04-20T21:54:28.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:54:28.115 INFO:teuthology.orchestra.run.smithi102.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-20T21:54:28.115 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-20T21:54:28.130 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T21:54:28.141 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T21:54:28.163 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi102 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-20T21:54:28.163 INFO:teuthology.nuke:Checking targets against current locks 2024-04-20T21:54:28.194 DEBUG:teuthology.nuke:shortname: smithi102 2024-04-20T21:54:28.194 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:54:28.218 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:47.456135', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9QeV7h0MaaSwHaphBUDlmNRalRKbj5cvKLE3E71dxdL1fda695dWpZEM3fByLnalQsaIUmEX/oi2MOCo8wV68='} 2024-04-20T21:54:28.221 DEBUG:teuthology.nuke:shortname: smithi193 2024-04-20T21:54:28.221 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:54:28.245 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:47.454169', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAnUap0WmMc8ZX95t3l0FWneNQgTBlAOPaCQ5F7fbon/6yzNVg802gspoF17WsVJGSrplIgSAwTBiqbCHfDUXYA='} 2024-04-20T21:54:28.267 INFO:teuthology.orchestra.console.smithi102:Power off 2024-04-20T21:54:28.267 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:54:28.291 DEBUG:teuthology.orchestra.console.smithi102:power off output: Chassis Power Control: Down/Off 2024-04-20T21:54:28.291 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:54:28.309 INFO:teuthology.orchestra.console.smithi193:Power off 2024-04-20T21:54:28.309 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:54:28.318 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-04-20T21:54:28.331 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-04-20T21:54:28.331 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:54:28.352 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-20T21:54:32.319 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:54:32.354 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:54:32.430 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-04-20T21:54:32.464 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-20T21:54:36.436 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:54:36.465 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:54:36.548 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-04-20T21:54:36.576 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-04-20T21:54:36.676 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-04-20T21:54:40.550 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:54:40.660 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is off 2024-04-20T21:54:40.761 INFO:teuthology.orchestra.console.smithi102:Power off completed 2024-04-20T21:54:40.867 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 2570.0783071517944 failure_reason: 'Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi102 with status 135: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d4ff550fa1e6496383b27826043ebdfc status: fail success: false 2024-04-20T21:54:40.868 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T21:54:40.954 INFO:teuthology.run:FAIL